Why Playwright and Selenium Projects Fail (5,441-Response Survey + 90 Days of Cost Data)

Liviu Lupei

Founder & Solutions Architect, Endtest · October 29, 2026

By Liviu Lupei · Published October 29, 2026 · Last updated October 29, 2026 · 12-minute read

Disclosure: We make Endtest, a commercial alternative to the approach this article criticizes. That's a conflict of interest, and you should read everything below with it in mind. To offset it, we show our numbers, our method, and the limits of both. The limits get their own section near the end.

The short answer

Why do Playwright and Selenium projects fail? Mostly because maintaining the tests costs more than the team planned for, not because the frameworks are bad. In Endtest's September 2026 survey of 5,441 respondents, 37% said their project couldn't scale (maintenance, cost, bloat), 24% said too few people contributed, and 11% said the product changed faster than the tests. In a 90-day side-by-side at one customer, maintaining Playwright tests with Claude took 162 engineer-hours. Maintaining the same list of tests in Endtest took 18.

Key findings

• 37% of respondents said their project couldn't scale: excessive maintenance, high costs, bloated architecture, or technical limits.
• 72% named scalability, too few contributors, or the product outrunning the tests. Those are operational problems, not framework problems.
• 94% said they hadn't treated their Selenium or Playwright project like a real engineering project from the start. No cost estimate, no ROI calculation, no proof of concept against other tools.
• 81% said the failure changed how they'd evaluate test automation next time.
• In a 90-day test with matched concurrency (1 slot each), Playwright maintenance hours rose every sprint, from 3 to 64. Endtest's went from 0 to 7.
• From sprint 2 on, the Playwright-to-Endtest ratio stayed between 8x and 9.3x.
• Tooling was the small part. Engineer time was over 90% of the Playwright total at a $100/hour rate.

Who we asked, and how

In September 2026 we sent a survey to a representative of each Endtest customer. 5,441 people responded. It mixed multiple-choice questions with free-text answers and asked about previous Selenium and Playwright projects.

Two things to know. The sample is self-selected: only people who chose to answer. And it's mostly Endtest customers, which means people who, at some point, went looking for something other than a code-first framework. We tried surveying people outside Endtest. Getting reliable answers was hard, so we don't have that data.

Read the percentages as what this group reported. They are not the failure rate of Playwright or Selenium projects in general.

Why do Playwright and Selenium projects fail?

One caveat on the 7%. These are the reasons people gave for failure. They are not the share of projects that had flaky tests. A project that died of maintenance costs probably had flaky tests too.

The mistake that happens before the first test

Look at the 94% again. Most teams picked a language and a framework and started typing.

Think about how that sounds. No estimate, no owner, no success criteria, no comparison of options, no idea what maintenance will cost. Your engineering manager would have questions.

Then the demo works. A browser opens, clicks around, reports green. Everyone's happy.

But the demo answers the easiest question: can we automate this workflow? It doesn't answer the ones that matter. Can we maintain 500 of these as the product changes? Can anyone besides the original author? Will the value exceed what we spend creating, running, and maintaining them?

That last one is a basic ROI question. It shouldn't take a failed six-month project to ask it.

1. The project couldn't scale (37%)

A free framework is not free test automation.

Playwright and Selenium cost nothing to download. Running them costs something: CI minutes or a browser cloud, storage for screenshots, videos, and traces, email and SMS testing services, secrets and test data management, and now AI tokens. We priced a real example below.

But those line items aren't the big number. The big number is hours. Every UI change means someone investigates whether the failure is a real bug, a broken locator, an environment problem, or an outdated expectation. Then someone fixes it and verifies the fix still checks the right thing.

AI doesn't remove that. Claude can read your codebase and your docs, but it doesn't necessarily know how the app is supposed to behave. Specs are incomplete. Sometimes the intended behavior lives in one person's head. Sometimes the implementation has a bug. An AI that can't tell the difference will happily update your test to match the bug. The suite goes green. You've automated not finding the defect.

There's a second trap. AI generates code faster than a team can agree on conventions for it. Ask for tests one at a time with no shared structure and you get five different login flows, mixed page objects and raw selectors, assertions that check a button is visible instead of checking an outcome. Hundreds of tests, no coherent system.

Endtest stores tests as structured, human-readable steps rather than as Playwright or Selenium code. The person maintaining a test needs to understand the app, not a test runner's architecture. AI-generated Endtest tests are saved in the same format, so they don't turn into another pile of scripts.

2. Too few people contributed (24%)

Nobody wants another repository to maintain.

One enthusiastic engineer builds the framework. Everyone else already has sprint commitments, and now they're asked to learn a test framework's architecture and maintain code they didn't write. So the project stays dependent on one person. When that person gets promoted, burns out, or leaves, the suite loses momentum fast.

That's an organizational risk, not a technical achievement. If only one person can work on your tests, you don't have a testing strategy. You have a bus-factor problem.

Readable steps lower the barrier: a manual QA engineer can update a test, a developer can investigate a failing step, a product manager can check that the test validates the right behavior. Endtest adds role-based Collaborators (Owner, Editor, Viewer) and Test Case Version History, so a removed assertion can be found and restored without Git archaeology.

3. The product moved faster than the tests (11%)

Picture a two-week sprint. Week one, the payment flow isn't stable enough to test. Week two, the UI changes three times. Friday, the feature ships. The automated tests don't. The team tests manually and moves the automation to next sprint.

Next sprint, there are new features and changes to the old flow. The backlog grows. After a few sprints you have plenty of tests and not enough coverage of what shipped last week. You're automating yesterday's app.

A test that arrives two sprints late still has regression value. But it can't go back in time and catch the bug it would have caught before release.

AI makes this sharper, not softer. Developers use AI to ship faster too, so the amount of behavior to validate grows. Generating tests faster doesn't fix a process where testing isn't part of the definition of done. Endtest Bot is our attempt at the creation side: an autonomous agent that explores your app and creates tests as regular, editable Endtest tests. It's in beta, you can restrict what it's allowed to do, and generated tests are executed before they're added to a suite. It doesn't decide which behaviors matter. You still do.

4. The tests were flaky, irrelevant, or inaccurate (7%)

A flaky test makes noise. An inaccurate test makes false confidence. The second is worse.

Flakiness has familiar causes: dynamic CSS classes, clicking before an element is ready, slow async calls, parallel tests editing the same record, third-party services, leftover state. Playwright's auto-waiting and Selenium's explicit waits help. Neither removes every source of nondeterminism, and raising timeouts or retry counts can hide a problem instead of fixing it. Google wrote about this years ago. It isn't specific to any framework.

Inaccurate tests are sneakier. A test clicks Submit, sees a confirmation message, and passes. But the backend failed to create the order, or saved the wrong amount. Green dashboard, broken business flow. AI-generated tests are prone to this when assertions come only from what the assistant saw during one successful run.

One design choice matters here: where AI sits. Letting an agent interpret your instructions fresh on every run adds a variable to every execution. In Endtest, AI can create a test, but running it doesn't require an LLM. The test is stored as fixed steps and executed the same way each time. AI features such as AI Assertions, AI Variables, and AI-powered Self-Healing are optional. No platform can promise identical results when networks, data, and external services vary. But variability from AI interpretation isn't a mandatory part of ordinary regression runs.

5. The team kept switching tools (5%)

Selenium with Java. Then Selenium with Python. Then Cypress, until a multi-tab workflow hit its limits. Then Playwright. Then, in 2026, "let Codex regenerate the whole suite." A few hundred AI-written tests later: inconsistent fixtures, duplicated flows, assertions that don't assert much.

Changing the framework helps when the framework is the constraint. It does almost nothing when the constraint is ownership, test design, or engineering time. The new repo looks clean for a while. Then the same patterns come back.

If you already have hundreds of tests you don't want to throw away, AI Test Import converts exported Selenium or Playwright tests into native Endtest steps. Complex custom logic may need manual review.

6. Layoffs, reorganizations, and other reasons (16%)

Acquisitions, discontinued products, new leadership consolidating tools, lost budgets. And one pattern that deserves a mention: QA is an easy cut. "Can we ship without a dedicated QA team?" Technically, yes. So the people who maintained the suite go, the repo stays, the tests keep running for a while, and nobody owns the failures.

Tests stored as readable steps survive that better, because a new person can open one and see what it does. They don't have to reverse-engineer someone's helper functions.

How much time does it take to maintain Playwright tests? A 90-day side-by-side

The survey tells you what people reported. We wanted a measurement too. One Endtest customer ran a 90-day proof of concept comparing Endtest against Playwright with Claude, and gave us the numbers.

The setup

• Same work: The team already had a list of tests to automate. They worked through it one by one, in both tools.
• Duration: 90 days, six two-week sprints.
• Concurrency: one slot each (one BrowserStack parallel session for Playwright, one slot on Endtest).
• Who: an Endtest customer in the insurance industry (we can't name them). A senior, US-based software engineer did the work. The hourly rate wasn't disclosed.
• What counts as maintenance: keeping tests passing and relevant. For Playwright, that included updating tests after UI changes, fixing failures, tuning the framework, getting tests to run on additional browsers, and back-and-forth with Claude to add missing details and correct its assumptions.
• How hours were measured: logged in Jira. These are self-reported estimates ("about 3 hours today on the Playwright tests"), not stopwatch time.
• Result by day 90: 622 tests in Endtest, 338 in Playwright.

Maintenance hours per sprint

Maintenance hours per two-week sprint, from hours logged in Jira. Data: maintenance hours by sprint (CSV).

What the numbers show

1. It isn't setup noise. If framework setup were the story, Playwright hours would start high and fall. They rose every sprint: 3, 8, 16, 28, 43, 64. The increases were 5, 8, 12, 15, and 21 hours. Endtest's were 1, 1, 1, 2, and 2.
2. The gap held. From sprint 2 on, the ratio stayed between 8x and 9.3x.
3. Sprint 6 was heavy. 64 hours is about 80% of one full-time engineer's two weeks (roughly 80 hours), spent on maintenance alone.
4. Per test, the gap is wider. Dividing total hours by final test counts gives roughly 0.48 hours per Playwright test against 0.03 per Endtest test. That's about 16x. It's a rough figure, because the suites grew during the period.
5. Maintenance eats creation. Hours spent fixing tests are hours not spent writing new ones. The 622 versus 338 gap is probably linked to the maintenance gap.

We can't tell you what sprint 7 looks like. The Playwright curve was still climbing when the test ended. We aren't going to extrapolate it.

What did the tooling cost?

Three honest notes. First, Claude's tokens were subsidized, so the real figure for most teams could be higher. Second, if you remove the AI tokens, the rest of the Playwright stack cost $187 a month against Endtest's $225. On infrastructure alone it's close to a wash at this scale. The difference is hours.

Third, know what the $225 buys. Endtest's pricing includes unlimited test executions, test creation, users, and AI usage on every plan. Email and SMS testing are available starting with the Pro plan, and this customer's subscription was Pro with 1 parallel slot, at $225 a month. That plan keeps test results for 90 days and covers Chrome, Firefox, Edge, and Internet Explorer. Apple Safari is on Enterprise plans, which are custom-priced. If you need real Safari, your numbers will differ.

What did it cost in total?

We don't know the customer's hourly rate, so here are three illustrative ones. Labor is maintenance hours only. Tooling is three months of the table above.

At every rate, Endtest was cheaper on both labor and tooling, so there's no break-even point in this data. Tooling was under 10% of the Playwright total at $100 an hour. Engineering time is the cost.

Your numbers will differ. Plug in your own rate, suite size, and sprint length.

Why AI didn't change the economics

AI made starting easy. It didn't make sustaining easy.

Generating a Playwright test takes minutes now. Maintaining 338 of them took 162 hours in our example, and a lot of those hours were conversations: giving the model context it didn't have, correcting what it assumed, fine-tuning the result. That's how most teams actually use AI for maintenance, and it's the cost the demos leave out.

AI is an accelerator. Give it a team with clear ownership, conventions, and stable test data and it speeds things up. Give it a team with none of that and you get hundreds of extra tests nobody has verified. The first team uses AI to improve a working process. The second uses it to scale a broken one.

Do real browsers matter for test automation?

Sometimes, yes. But the evidence is more mixed than a vendor would like.

Headless browsers in Linux containers are fast, cheap, and fine for a large share of tests. We're not saying they're bad. Engines, operating systems, and browser builds do differ, though, and some problems only appear in the combination your users actually run. Here are the examples we found, including the ones that don't flatter our argument.

• A web UI that never finished loading. In a September 2026 bug report on the deepseek-harness web UI, session history hung forever in Firefox-engine browsers while the same session loaded fine in Chromium. JavaScript engines format native function source differently, and the app's check assumed Chrome's format. A commenter reported Safari was affected too. The reporter reproduced it in headless Firefox on Linux, so this is a case for testing beyond Chromium, not for real machines specifically.
• Playwright's WebKit differs by platform. In issue #11009 (2021), certain MP4 video crashed Playwright's WebKit on macOS while Linux CI ran fine and regular Safari didn't crash. In issue #4959 (also 2021), macOS WebKit showed a camera and microphone permission dialog, while on Linux the same API wasn't available at all. Both are old, and both were problems in the test tool's browser builds, not in the apps being tested. They show that WebKit on Linux, WebKit on macOS, and Safari are three different things.

That cuts both ways. A different environment can hide a real defect, and it can also produce failures your users will never see. The lesson isn't that real browsers always win. Test on what your customers use, and know exactly what your tests are running on.

Safari is the clearest case. Playwright's WebKit support uses a patched WebKit build, which is Safari's engine but not the branded Safari browser. Selenium can drive real Safari through SafariDriver, if you have a macOS environment. If your customers use Safari, find out which one your tests are actually running.

Endtest runs web tests on real browsers on real Windows and macOS machines, so you don't provision or maintain those environments yourself. On the current pricing page, Chrome, Firefox, and Edge are on the entry plan, Pro adds Internet Explorer, and Apple Safari is on Enterprise plans.

How do you avoid a failed test automation project?

1. Run a proof of concept on hard scenarios. Not a login test. Use email verification, SMS, multiple tabs, multiple users, file uploads, and real cross-browser needs.
2. Change the app, then measure. Deliberately break things. Time how long it takes to find and fix each failure.
3. Have someone else do the fixing. If only the original author can update the tests, you've learned something important.
4. Compare more than one approach on the same scenarios. Selenium, Playwright, and a managed platform, against the same workflows.
5. Estimate total cost and ROI first. Setup effort, monthly maintenance hours, infrastructure, external services, AI usage, support. A useful formula: Annual testing ROI = (annual value of testing benefits − annual total cost of testing) / annual total cost of testing × 100.
6. Make ownership explicit. Someone is responsible for the health of the suite. It shouldn't be one person.
7. Plan tests with the sprint. Put important regression coverage in the definition of done.
8. Measure outcomes, not test counts. Regressions caught before production, maintenance hours, false positives, execution time, and whether engineers trust a red build.

Can you leave Endtest? What exporting your tests looks like

Ask this before adopting any platform.

You can export an Endtest test as a JSON file, and the file contains all the data from each step. You can also pull everything out through the Endtest MCP: it can return a whole suite, with every test case and each step's action type, locators, and parameters. Stored secrets come back as references, not values. The export docs are clear on the limit: exported JSON can't be executed, and the option to export in an executable format has been removed. To run your tests somewhere else, you'd translate the steps and locators into another framework or tool. An AI assistant makes that translation much easier than it used to be, but it's still work.

So, plainly: your test definitions are portable, and your test execution isn't. That's a real trade-off, and it comes with any managed platform. With a Playwright suite you own the code, and you also own everything around it. Which of those you'd rather own is a decision for your team.

When are Playwright and Selenium still the right choice?

More often than a vendor blog will admit.

Both are mature, capable frameworks. Playwright has excellent browser APIs, auto-waiting, tracing, and a well-integrated test runner. Selenium has a huge ecosystem and broad language support. Staying with them can be the right call if you have experienced automation engineers, clear ownership, a maintainable architecture, and real engineering capacity. It's also right if you need deep, code-level control or a fully open-source stack.

Reconsider if you're constantly rewriting tests, falling behind releases, or spending more time maintaining the framework than validating the product. And even then, the fix isn't always a new tool. Sometimes it's better conventions, fewer and more meaningful tests, or actual time set aside for testing.

Frequently asked questions

Why do Playwright projects fail?

Mostly because of maintenance cost and ownership, not the framework. In Endtest's September 2026 survey of 5,441 respondents, 37% said their Selenium or Playwright project couldn't scale (maintenance, cost, bloat, or technical limits), 24% cited too few contributors or no ownership, and 11% said the product changed faster than the tests.

How much time does it take to maintain Playwright tests?

It depends on the suite, but here is one measured example. A team maintaining 338 Playwright tests with Claude's help logged 162 hours over six two-week sprints, rising from 3 hours in sprint 1 to 64 in sprint 6. The same team logged 18 hours maintaining 622 Endtest tests. It's one customer, so treat it as a data point, not a benchmark.

Is Playwright really free?

The library is free. Running it isn't. A realistic stack adds CI or a browser cloud, artifact storage, email and SMS testing services, and AI tokens. In our 90-day example those cost $507 per month, but engineer time was the larger cost: 162 logged maintenance hours.

Does AI like Claude or Codex reduce test maintenance?

It speeds up writing tests, but maintenance still needs a human who knows the intended behavior. In our example, Playwright maintenance included a lot of back-and-forth with Claude to add details and correct assumptions, and hours still rose every sprint. AI can also update a test to match a bug if nobody checks the expected outcome.

Does Playwright test real Safari?

No. Playwright uses a patched WebKit build, which is Safari's engine but not the branded Safari browser. Selenium can automate real Safari through SafariDriver on macOS. Teams that need to validate real Safari behavior should plan for that.

Should I switch from Playwright or Selenium to Endtest?

Not automatically. If you have experienced automation engineers, clear ownership, and capacity, staying can be right. If your team keeps rewriting tests or maintenance hours keep growing, run a proof of concept with the same real scenarios on each option and measure maintenance hours after changing the app.

Can I export my tests from Endtest?

Yes. You can export a test as a JSON file that contains all the data from each step, and the Endtest MCP can return whole suites with every test case, step, and locator. The JSON isn't executable, so running the tests in another tool means translating the steps and locators, which an AI assistant can help with. Your test definitions stay yours, but you'd need to rebuild execution elsewhere.

How was the survey run?

In September 2026 we sent it to a representative of each Endtest customer and received 5,441 responses, with multiple-choice and free-text answers. It's self-selected and mostly customers, so it describes that group, not all Playwright or Selenium users.

What this data does not prove

We'd rather tell you than have you find out.

• The survey is self-selected and mostly customers. It can't tell you how often Playwright or Selenium projects fail in general. Answers are also retrospective, and people misremember.
• We sell one of the options. That's a conflict of interest, however carefully we write.
• The cost test is one customer, one app, 90 days. Other teams, applications, and workflows will see different numbers.
• Playwright results reflect how this team worked with Claude. Stricter conventions, shared page objects, and better prompts might lower the hours. We don't know by how much.
• Hours are self-reported in Jira. They're estimates logged by the engineer, not timed. A gap of about 9x is hard to explain with logging error alone, but the exact figures aren't precise.
• Creation time isn't broken out. It was tracked, but a mix of AI-generated and AI-assisted tests made a clean breakdown impractical.
• The Endtest plan was Pro with 1 parallel slot, at $225 a month. It covers Chrome, Firefox, Edge, and Internet Explorer and keeps results for 90 days. Pro is normally sold with at least 2 slots ($450 a month on the current pricing page). Real Safari is on Enterprise plans with custom pricing, so teams that need it will see different costs. We don't have a browser-by-browser coverage comparison between the two setups.
• The hourly rates are illustrative. The customer's real rate wasn't disclosed. The Claude token cost was subsidized and may be higher for you.
• We don't have defect-detection data. The numbers cover maintenance effort, test counts, and tooling cost. Don't read them as a claim that one suite catches more bugs.
• Other customers show similar patterns with different setups. Those aren't comparable enough to combine, so we haven't included them, and you can't verify them here.

If you're evaluating tools, run your own proof of concept. That's the one number you can fully trust.

About this data and how to cite it

Survey: Endtest customer survey, September 2026, n = 5,441 respondents (multiple-choice and free-text responses). Cost test: single-customer (insurance industry), 90-day, six-sprint proof of concept comparing Endtest with Playwright and Claude, hours logged in Jira.

Download the aggregated data: survey results (CSV), maintenance hours by sprint (CSV), and monthly tooling costs (CSV).

Suggested citation: Lupei, L. (2026, October 29). Why Playwright and Selenium Projects Fail. Endtest.

Sources and further reading

• Playwright documentation: browsers
• Flaky tests at Google and how we mitigate them
• Endtest pricing
• Endtest cross-browser testing
• Endtest AI Test Import
• Endtest Bot