The Shopify A/B testing app landscape in 2026: 137 apps, 385 reviews, and what merchants actually complain about

What we do not know, stated plainly

• Shopify does not publish install counts. Review count is the only proxy available and it systematically understates real usage — most happy merchants never review anything.
• Market share, revenue and growth rate cannot be derived from this data, and have not been. Any number in this post that looks like traction is a review count, not a customer count.
• 599 of the 1,229 reported results were collected. We sampled the tail and stopped when it turned into address validators and stock counters. That cut is our editorial decision, not Shopify's.
• The tiering below (core / enterprise / adjacent / noise) is our editorial call. Shopify does not categorise the category this way.
• Per-app strengths and weaknesses were derived from at most six positive and six negative reviews per app. For the leaders that is close to every review at three stars or below; for an app with four reviews it has no statistical value and is not presented as if it does.

The 61-app core set is the real category, and it is what the rest of this post is about. The 13 enterprise listings are a separate phenomenon worth naming: AB Tasty, Kameleoon, Dynamic Yield, Intellimize and Omniconvert all have Shopify listings, and almost none of them have reviews. That is presence, not traction — those companies sell through a sales team, and the App Store listing is a connector, not a storefront.

Price, shipping and margin

The largest group, and the one that separates this category from web A/B testing generally. Ten of the fifteen test a commercial variable rather than a piece of a page.

• Intelligems — the widest test surface in the set: price, shipping, content, checkout and post-purchase, reported on profit per visitor rather than raw conversion rate. Most reviews of any app here and the most enterprise-leaning shape.
• Elevate — a price and shipping specialist, listed since December 2020, carrying 4.9★ over 127 reviews with 2.4% of them at one or two stars. That is the cleanest published distribution at that volume in the set.
• ABConvert — price, shipping, theme and URL testing from $99, the same entry anchor as Shoplift. What differs is the meter: you are billed on test orders rather than on site traffic, with 1,000 included on Starter and $59 per additional thousand, falling to $24 per thousand on the $599 Pro plan. It competes on how you are billed, not on being cheap.
• ABsolutely — price, shipping, theme and template testing; 5.0★ across 32 reviews, which is a small enough base that it says more about early customers than about the product.
• Eraya — a free plan and a $59 ceiling, listed October 2025. Too new to judge on 19 reviews, and priced to be tried.

Theme and template

Swapping a whole theme or template against another. This is the capability a snippet cannot reach — it needs to work inside the theme — and it is also the capability merchants write about least: one positive review in 312 names it.

• Shoplift — tests are built inside Shopify's theme customizer, results read Bayesian, and Lift Assist auto-creates experiments. Priced per monthly unique visitor, with the highest listed ceiling in the set at $999.
• Essential CRO — theme and template split testing, listed June 2025, 5.0★ over 24 reviews and no plan prices published on the listing we read.
• Shogun — pages, themes, templates and price testing, backed by an established page-builder brand, and carrying the weakest rating of the funded cohort: 4.2★ with 12.8% of reviews at one or two stars. Distribution on its own does not carry a testing product.
• Theme Scientist — the oldest listing in the set (March 2020) and the cheapest theme tester at $9.99, at 3.5★ over 10 reviews.

Content and layout, without a developer

The group closest to what Looplift does, and the group whose praise is the most uniform: install it, change the page, no developer.

• Visually — a no-code visual editor bundled with personalisation, recommendations and upsells; free on development stores and metered by order volume above that. Client-side visual editing is both its reach and its exposure.
• GemX — page and funnel optimisation at $49–$99, listed September 2024.
• CustomFit AI — no-code personalisation plus testing, free on development stores and priced by monthly visitors from $79.

The cheap end, and the cautionary end

Three listings are worth naming for what they demonstrate rather than what they sell.

• ABboost — $4.99 to $29.99, the lowest ceiling in the set, listed November 2025.
• Split — 2.8★ over 13 reviews, with 38% of them at one or two stars: the clearest published example in this category of price testing going wrong on a live store.
• A/B: Product and Price Testing — 3.2★ over 11 reviews, the same lesson from a different vendor. Merchants say so publicly when a test touches money and misfires, which is the strongest argument in this dataset for keeping commercial variables out of a testing tool's reach.

Two things to take from the shape rather than the rows. Ten of the fifteen test price or shipping, which means the category's centre of gravity is server-side commerce logic, not storefront presentation. And a rating here is not a usage count: Shopify publishes no install figures, so a 5.0★ over 19 reviews and a 4.8★ over 161 are not the same evidence, and neither is evidence of market share.

Looplift is not in this table. It is not listed on the Shopify App Store, so it has no rating, no review count and no badge — which is the honest position to be in while publishing a table like this one.

Thirty-seven of the 61 core apps — 61% — launched in 2025 or 2026. The oldest app with real traction shipped in December 2020. Whatever conventions this category appears to have, they were set in the last twenty-four months by a handful of teams, and most of them are still being argued about.

Flicker — the flash of original content before a variant is applied — is the thing this category's marketing argues about. Every serious vendor benchmarks it. It accounts for one review out of 73.

Meanwhile, code left behind after uninstall (7%) plus breaking the store or theme (5%) is 12% of everything merchants complain about: roughly one negative review in eight is about a testing app changing something it should not have, and then not being removable. Add “the app does not work” at 18% and you have a category whose reputational risk is reliability, not milliseconds.

One de-identified example, because the tone matters more than the arithmetic: a two-star review of a Built-for-Shopify app, written twenty minutes after install, reports five blocks of code still on the page after uninstalling. Twenty minutes. That merchant will never install another testing app without checking first.

Seventy-two of 312 positive reviews are some version of “installed in minutes, did not need a developer.” That is the single most repeated sentence in this category's praise, and it means one-click install is table stakes. The leaders solved it years ago. Any product entering now that treats “no developer required” as its differentiator is selling a feature its competitors already gave away.

The other number worth sitting with: theme and template testing — the deepest, most-marketed capability in the category — is named in exactly one positive review out of 312. Merchants are not writing home about test surface area.

The serious entry anchor sits at $49–$99: Shoplift, Intelligems, Elevate, ABsolutely, GemX and ABConvert all start there. Forty-nine percent of the core set has a free plan, which makes free an entry ticket rather than a position. And nearly every plan in the category scales with order or visitor volume, so the price on the listing is a floor, not a bill.

Read the two facts together and the category's economics are legible: the software is priced at $49–$99, and the expensive part — deciding what to test and in what order — is billed as a human being at the top of the ladder.

Zero to seven reviews each. The window is open and it is closing: six apps are currently telling a version of the same story, and the first one to accumulate real reviews will define what merchants expect the story to mean. We are one of the apps in that corridor, which is exactly why we would rather publish the category's actual failure data than another paragraph about AI.

And the one that is not a product at all: Shopify’s own Rollouts, shipped in the Winter ’26 edition. Rollouts are available from the Basic plan and experiments from Grow, the unit is your main theme plus checkout and accounts configuration, headless storefronts are not supported, and there is no public API. For a whole-theme split it is free and it is the best-placed implementation there is. It is the first thing to rule in or out before you buy anything in the table above.