FAQ
Short answers to the questions merchants ask most before and after installing AB Test, with links to the full documentation for each topic.
Do I need to write code?
For most tests, no. URL redirect, template, and theme tests are configured entirely in the admin app with no code — you pick pages, template suffixes, or themes you already have, and AB Test splits the traffic.
The one exception is the JavaScript API test type, where you render the variants yourself and ask AB Test which one is active with window.abtest.isHypothesisActive(). That type requires merchant code by design; the other three do not.
New to the app? Start with a URL redirect or template test — both are no-code and are the quickest way to get a first result.
Will it slow down my store?
AB Test runs as a lightweight theme app embed plus a Web Pixel. The embed assigns a visitor to a variant and stamps an anonymous id; it does not pull in heavy libraries or block your page from rendering. There is no visible flicker for URL, template, and theme tests because the split happens before the storefront paints the assigned experience.
Keep the app embed and the Web Pixel enabled so assignment and measurement work; see Quickstart for where those toggles live.
Does it change my theme?
No. AB Test never creates, edits, or deletes theme files. Template tests serve an alternate template suffix you already built via ?view=<suffix>, and theme tests compare a duplicate or alternate theme you already have via ?preview_theme_id. Publishing any theme change stays entirely in your hands under Online Store > Themes.
Because the app only points traffic at experiences you already control, applying a winner records the decision and stops the split — it does not rewrite your storefront for you.
Does it work with my checkout, Shop Pay, and third-party checkout?
Yes. AB Test attributes revenue using an anonymous visitor id (_ab_vid) that the storefront embed stamps as a Shopify cart attribute. That id rides to the order's note_attributes, and an orders/create webhook books the sale to the assigned variant.
Because attribution is anchored to the order rather than to an on-page pixel, it captures sales even when checkout happens off the storefront — Shop Pay on a separate host, and India third-party checkouts such as Razorpay Magic, GoKwik, Shopflo, and Shiprocket/Fastrr. See Revenue attribution for the full path.
What about privacy and consent?
AB Test reads only an order's id, total, currency, and its own note_attributes._ab_vid. It never reads customer name, email, phone, or address, so it handles no personal information (Protected Customer Data Level 1). Visitors are pseudonymous, identified only by an anonymous id.
Consent is honored through the Shopify Customer Privacy API, which auto-detects consent banners such as OneTrust and Cookiebot. Data is retained for roughly 180 days and then purged. Webhooks are HMAC-verified, traffic is served over TLS, and the mandatory GDPR webhooks are supported. Full details are on the Privacy page.
Is there a free plan?
There is no free tier, but every plan starts with a 15-day free trial so you can run real tests before you pay.
| Plan | Price | Test types | Running tests | Campaigns | Tested visitors / mo |
|---|---|---|---|---|---|
| Base | $19/mo | Any 2 | 2 | 1 | Up to 50,000 |
| Premium | $29/mo | All 4 | 5 | 5 | Up to 200,000 |
| Enterprise | Custom | All 4 | Custom | Custom | Custom |
See Plans for the complete comparison, including how to contact sales for Enterprise.
How long should I run a test?
Let an experiment run at least one to two full weeks so it spans complete weekly cycles — weekends and weekdays convert differently. Wait for enough conversions, not just views, and look for a leading variant whose probability to be best sits around 95% or higher and holds steady over several days.
Avoid stopping the moment a result looks good; early leads often shrink as more data arrives. The Results page explains how to read Bayesian probabilities and when a result is trustworthy.
You do not have to remember to stop a test. Every experiment has a scheduled end date and auto-ends when that date arrives, keeping its data available for you to read and apply.
What happens when I hit my visitor limit?
Each plan includes a monthly cap on tested visitors — 50,000 on Base and 200,000 on Premium. The limit counts visitors assigned to your running tests within a billing month. If you are consistently approaching your cap, upgrade to a higher plan for more headroom, or contact sales about Enterprise. Review the current limits on the Plans page.
How do I get support?
Email yadav.creators@gmail.com with your store URL and a short description of what you are trying to do. If your question is about a specific test type, the pages under Documentation — and the API reference for JavaScript API tests — often have the answer already.