General FAQs
Everything you need to know before running your first experiment with TestSignal — from setup and pricing to capabilities, targeting, and troubleshooting.
Plan, Capabilities & Access FAQs
TestSignal offers easy, transparent monthly pricing — no complicated tiers based on your store’s size or order volume. Just choose the plan that fits your testing needs and get started. See our Pricing Page for full details.
Yes! All three of our plans include a 14-day free trial that gives you full access to run real tests and explore every feature. No commitment required — cancel anytime.
TestSignal is built for Shopify stores. While we’re focused on Shopify today, reach out if you’re on a different platform and interested in support.
Absolutely. TestSignal is fully compatible with every Shopify plan — from Basic to Shopify Plus. All core testing features work out-of-the-box, though certain checkout UI options may depend on your Shopify tier.
To run experiments like a pro, TestSignal needs access to product & variant pricing, shipping settings, your online store theme/editor, and orders & analytics data. We only ask for the essentials—just enough to run your tests safely and effectively.
Product & variant pricing – so price tests hit the mark
Shipping settings – to experiment with different shipping options
Online Store theme/editor – to seamlessly embed test features
Orders & analytics data – to measure results and insights
You can test nearly anything in your Shopify store: product prices, shipping rates, store themes, product images, descriptions, and layouts. The full range of capabilities depends on your plan.
The best test to start with depends on your goals, but most TestSignal users begin with a price test. Testing prices across your store—or starting with your most popular products or collections—usually delivers the fastest and most impactful insights. We recommend trying both a slightly lower and a higher price against your current one to understand the full range of customer response. If shipping is your focus, experimenting with different free-shipping thresholds is a proven way to find the “sweet spot” between AOV and conversion rate. You can also kick off with theme, layout, description, or image tests if you’re looking to improve the shopping experience. For more ideas, explore our test recommendations
Yes — TestSignal allows you to run multiple tests simultaneously. However, if tests overlap (for example, price and layout tests on the same product or page), they can affect each other’s results. To keep your data clean and reliable, you can: run a single combined multivariate test, run tests sequentially for clearer insights, or use audience targeting to make sure your tests don’t overlap.
The ideal test duration depends on a few factors: how big of a change you’re testing, how much traffic or how many orders your store receives, and how confident you want to be in the results. As a general guideline, detecting around a 10% change in conversion rate with roughly 90% confidence usually requires about 300 orders per variant group. TestSignal will help you monitor progress, but giving your test enough time and data is key for reliable insights.
No. Our script is extremely lightweight (under a few KB) and loads asynchronously. It does not block loading or hurt Core Web Vitals.
Overall, yes! TestSignal, a no-code A/B testing app for Shopify stores, will keep running even if you update to Shopify’s One-Page Checkout. How it behaves depends on the type of test:
Price Tests: These will continue to work seamlessly. Shopify Plus users should make sure their Line Item script still includes our Checkout Script. While hiding the applied discount on the One-Page Checkout isn’t possible, we can customize the discount name to your preference.
Shipping Tests: TestSignal will operate without any issues.
Theme Tests: These tests continue to run normally, enabling you to switch between store themes and identify the design that keeps visitors engaged.
Product Image Tests: These tests continue to run normally, giving you the ability to test different visuals and see which images capture customer attention.
Product Description Tests: These tests continue to run normally, letting you experiment with copywriting to discover descriptions that convert best.
Layout Tests: These tests continue to run normally, allowing you to adjust page layouts and optimize how products and content are presented.
No manual copy-paste required. TestSignal integrates seamlessly using Shopify’s App Embeds.
If you change your primary domain but keep the same Shopify store ID, you’ll need to update it in TestSignal settings. Switching domains mid-test may cause visitors to be treated as “new” and may invalidate results — we recommend ending ongoing tests before a domain migration.
Testing FAQs
Editing a live TestSignal experiment is easy — just follow these steps:
Step 1: Pause your test.Just hit “Pause” from the three-dot menu on your experiment in the Experiments page
Step 2: Make your edits.Click the three dots next to your test and select Edit. From there, you can
update products, adjust prices, change shipping rates, tweak layouts, or set
any test group to 0% traffic if you want to effectively remove it. Make sure
to complete the full edit flow and hit Update — your experiment will
automatically continue running with those changes and continue
collecting results.
Keep in mind: after a test has started, you can’t change test group names or add/remove groups entirely — but setting a group to 0% traffic removes it from the active experiment while keeping analytics intact. You can also filter your results by date to compare performance before and after your edits.
Yes — in most cases, the TestSignal support team can help. We can restore ended Price, Theme, Product Image, Product Description, or Layout tests back to a paused state so you can resume them.
Please reach out to our support team
here
for help with this!
Yes, using a multivariate test, you can test multiple elements like layout, images, and product copy together. This helps you understand how different combinations impact conversions. For simpler analysis, consider testing one element at a time.
- Price: Revenue, conversion rate, AOV
- Shipping: Checkout completions, revenue per option
- Theme/Image/Layout/Description: Conversion rate, engagement
A result is generally significant at 90% confidence or higher. Low-traffic stores may need longer tests.
Use the analytics dashboard to compare variants, filter by date or segment, and track conversions, revenue, and AOV to make data-driven decisions.
Preview FAQs
Yes! You can preview both simultaneously. Simply enter preview mode for your Shopify theme first, then enter preview mode for your TestSignal test. The theme will be cached when you open the test preview, letting you see exactly how your variations look and function together.
Traffic FAQs
TestSignal splits visitors evenly across your test variants by default, but you can customize traffic allocation if you want certain variants to receive more or less traffic. This helps focus results on priority variations while still collecting meaningful data from all groups.
Yes! You can change the traffic percentage for each variant at any time. Simply pause the test, make your adjustments in the edit flow, and resume the test. TestSignal will continue collecting results with the new traffic allocation.
The amount of traffic depends on your expected effect size and confidence level. As a guideline: to detect a ~10% conversion change with ~90% confidence, you typically need ~300 orders per variant group. Higher traffic allows tests to reach significance faster.
Yes! TestSignal allows you to target specific audiences or customer segments, such as returning visitors, new visitors, or geographic locations. Segmentation ensures tests run on the right users and provides more actionable insights.
When a test is paused or edited, TestSignal maintains historical data and automatically applies traffic changes once the test is resumed. Any groups set to 0% traffic are effectively removed without losing past results.
No — TestSignal is designed to distribute traffic smoothly and safely. Each variant receives the correct proportion of visitors, so your store performance remains stable while collecting accurate test data.
Yes! In the test setup, you can see and adjust the traffic allocation before launching. This ensures each variant receives the desired share of visitors for accurate results.
TestSignal tracks visitors primarily through browser cookies and session data, so if a customer switches devices (e.g., from mobile to desktop), they may be treated as a new visitor for the purposes of that test. This means their behavior on the new device will be counted independently in the test results.
For most stores, this has minimal impact, especially for high-traffic sites, but it’s something to keep in mind when analyzing results.
Uninstalling FAQs
When you uninstall the app, all active tests will stop running immediately, so variants will no longer be served to visitors. Historical test data will remain accessible in your TestSignal account.
No — uninstalling TestSignal does not alter your store settings, prices, shipping rates, or theme. Any changes made by TestSignal scripts revert to their original state automatically.
Yes! After reinstalling TestSignal, you can resume paused or saved tests. Shipping and price tests may require reconfirmation of settings to ensure they are applied correctly.
No — your historical test data remains safe in your TestSignal account, even after uninstalling. If you want to permanently delete data, contact TestSignal support.
No — your historical test data remains safe in your TestSignal account, even after uninstalling. If you want to permanently delete data, contact TestSignal support.
- Pause all active tests to ensure no experiments are running
- Uninstall TestSignal from your Shopify admin.
TestSignal Team