A suite of tests which passed in March may fail in September with no changes to the app. The Appium suite is one year old, takes hours to run and is ignored when it turns red, and manual regression goes back in the background. The typical failure of mobile QA automation is by design and the solution lies in architecture. The diagnosis is here, and the structure that endures device and OS churn.
Why Mobile Automation Suites Decay Faster Than Web Suites
Web suites are based on slow-changing browser engines. Mobile suites are essentially playing catch-up with the ever-changing nature of iOS and Android releases, manufacturer skins and screen sizes, which all affect how an element will render and when it will be tappable. Add the state web tests, including permission dialogs, push prompts, keyboard overlays, network switches, backgrounding and biometric prompts.
Longer tooling chain as well. Each of the Appium server, the UiAutomator2 or XCUITest drivers, the device farm, and continuous integration (CI) agents can fail in a variety of ways, and teams regularly record the failures as app bugs.
Each team goes through the lifecycle again. Quickly build out, increasing flakiness, retries, red builds no one reads, then manual regression. It is at that point that it is appropriate to switch frameworks, but not change the design, since it will just be a new stack with the same problems.
The Design Mistakes Behind Flaky Mobile Tests
The majority of flaky suites started out as manual test cases, which were then automated. This leads to a 40-step walk in which if one step is broken, the rest of the steps fail the test and are hidden.
Locators aggravate it. XPath-based views that are wrapped in a new container are broken by the developer, and text based views are broken with every copy change and locale. The issue is that there are no accessibility IDs. Then there's the hard-coded sleeps, which waste even more time on a fast device, and are still too brief on a slow emulator, making the suite slower and flakier.
State leaks complete the picture: shared accounts, un-cleaned up carts, apps that are not reset between runs, and order-dependent tests. Generating test data via UI, rather than APIs, increases the number of run times and failure points. A common example is a checkout test that fails on a small Android device due to a promo banner being pushed down past the fold, below the "Place order" button.
Rebalancing What Gets Tested at Which Layer
A healthy mobile pyramid looks like this: unit tests at the bottom, API and contract tests, thin layer of native component tests (Espresso and XCUITest), and a thin layer of cross-platform end-to-end tests on top. Typically, decaying suites are inverted.
Tests at the Appium level should be limited to few critical journeys: Login, Purchase, Onboarding. Form validation, pricing rules, eligibility logic should be at the API level, and they should be able to be executed in seconds and not require rendering.
On single-platform UI checks, native frameworks will be the winner when it comes to speed and stability, as they will be synced with the UI thread of the application. It's two codebases, and only works if each platform team owns their tests. The following heuristic resolves most layer discussions: A failing test should refer to one cause. If not, the test is too long or is located at the wrong layer.
Building a Test Architecture That Survives Device and OS Churn
The Screen Object or Screenplay pattern provides a common interface for the tests while implementing the iOS and Android versions is done in different classes, so a change in the login screen translates to a change in one class rather than 60 tests.
Locators must have a contract with developers. Accessibility IDs are part of the definition of done, and are subject to the same pull request review as any other code. Environments should be deterministic: stubbed backends for UI tests, feature flags that are switched on or off, app state should be set through deep links or test hooks rather than three screens into the app to reach the one being tested.
Construct the device matrix based on user analytics, not all of the devices and OS versions that are offered by the farm. From the start, design for parallel execution, have separate tests, separate accounts and no shared fixtures.
Keeping the Suite Trustworthy and Owned Over Time
There is a need for a policy for flaky tests. Place in a set time limit, monitor flake rate as a measure, fix or delete. Intermittent defects will be missed when users use the real product.
Monitor pass rate on same code, mean time to diagnose failure, mean duration of a suite and frequency of releases shipping with red tests. Screenshots, video, device logs and Appium logs automatically attach to every failure in minutes, instead of hours.
Automation code deserves the same review standard as product code, and developers own testability alongside QA. Some teams have deep product knowledge but have never rebuilt a mobile framework. When a suite has decayed past incremental fixes, bringing in professional Appium testing partners to restructure the architecture and hand back something the internal team can maintain is a legitimate decision, not an admission of failure.
Conclusion
Run the 10 flakiest tests for the past month of CI runs before rewriting anything. Typically reveal the majority of the issues listed above: brittle locators, sleeps, leaked state, and journeys at the incorrect layer. The pattern behind each one is more important to fix than any retry setting in order to restore more trust in the suite.
