You turn a user insight into a hypothesis by writing down four things: what you saw, what you think explains it, the change that follows from that explanation, and the one number that should move if you're right. The experiment is then matched to what the page can support. A split test can measure whether the number moved. Usability sessions and interviews can't measure a rate, but they can show whether people understand the page and where they stall, and that can be enough to make a change you record as a judgment.
\n","summary":"A user insight becomes a hypothesis when the observation is separated from the explanation and written as a change, a reason and one primary outcome with guardrails. A split test measures whether the number moved when the page has the traffic for it; usability sessions and interviews show understanding and support a change recorded as a judgment. One assumed pricing-page example runs from observation through ranking, test choice and the recorded result.","publishedAt":"2026-06-07T14:26:11.392","verifiedAt":"2026-09-14T00:00:00","editorialStatus":"APPROVED","lastReviewedAt":"2026-10-10T06:36:12.183","nextReviewDueAt":"2026-12-14T00:00:00","templateVersion":"v2","aliases":["GDD hypothesis format","if then because hypothesis website testing","growth driven design experiments","turning user research into A/B tests","website experiment design GDD","how long to run an A/B test","what does 90 percent statistical significance mean","conversion hypothesis examples","ICE prioritization for A/B testing","how to read an A/B test result","A/B test sample size for low traffic B2B sites"],"confidenceScore":84,"confidenceLabel":"High","canonicalUrl":null},"contributor":{"id":"dc83f9a1-5273-4639-bffb-43d5fa44c2bf","slug":"lean-labs","displayName":"Lean Labs","websiteUrl":null},"contributorOrganizationProfile":null,"contributorPerson":null,"sections":[{"id":"cf675bd4-14e4-4c44-b056-3af933b823bc","sectionKey":"separate_what_you_saw_from_why_you_think_it_happened","sectionType":"markdown_section","heading":"Separate what you saw from why you think it happened","introMarkdown":"An insight usually arrives as an observation and an explanation stuck together, and they need to be pulled apart before anyone designs anything.\n\nSuppose a B2B software company reviews its pricing page. Scroll data shows that most visitors leave before they reach the plan table. In three of the last five recorded sales calls, the prospect asked what the product costs after having visited that page. Those are observations. \"The plan table sits too far down the page\" is an explanation, and other explanations fit the same facts. The visitors might have come from an ad that promised something the page doesn't deliver. The table might be easy to reach and hard to read.\n\n[Wingify's guide to test hypotheses](https://wingify.com/blog/ab-testing-hypothesis/) builds a hypothesis from a problem, a proposed change and a way to measure the result, with the problem drawn from usability sessions, surveys, heatmaps and analytics. Those methods show where people stop, what they do and what they say about it. The reason is still something you infer, and the experiment exists to check that inference.\n\nSo it pays to look more closely before you commit to one explanation. Watch a handful of session recordings on the pricing page, and ask the sales team what prospects said in their own words. Suppose several recordings show people scrolling past the table and still leaving. Those visitors reached the table and left anyway, so there's a second explanation to investigate alongside the first. Position could still matter for them, since a table that arrives late may arrive after interest has gone. A few recordings can't speak for every visitor, and the scroll data still says most people never get that far.","introHtml":"An insight usually arrives as an observation and an explanation stuck together, and they need to be pulled apart before anyone designs anything.
\nSuppose a B2B software company reviews its pricing page. Scroll data shows that most visitors leave before they reach the plan table. In three of the last five recorded sales calls, the prospect asked what the product costs after having visited that page. Those are observations. "The plan table sits too far down the page" is an explanation, and other explanations fit the same facts. The visitors might have come from an ad that promised something the page doesn't deliver. The table might be easy to reach and hard to read.
\nWingify's guide to test hypotheses builds a hypothesis from a problem, a proposed change and a way to measure the result, with the problem drawn from usability sessions, surveys, heatmaps and analytics. Those methods show where people stop, what they do and what they say about it. The reason is still something you infer, and the experiment exists to check that inference.
\nSo it pays to look more closely before you commit to one explanation. Watch a handful of session recordings on the pricing page, and ask the sales team what prospects said in their own words. Suppose several recordings show people scrolling past the table and still leaving. Those visitors reached the table and left anyway, so there's a second explanation to investigate alongside the first. Position could still matter for them, since a table that arrives late may arrive after interest has gone. A few recordings can't speak for every visitor, and the scroll data still says most people never get that far.
\n","outroMarkdown":null,"outroHtml":null,"contentJson":{},"configJson":{},"noteMarkdown":null,"noteHtml":null,"sortOrder":10},{"id":"2c3513a0-f1f5-4fc3-a795-b06bfd0caa38","sectionKey":"write_the_prediction_so_it_can_be_wrong","sectionType":"markdown_section","heading":"Write the prediction so it can be wrong","introMarkdown":"With one explanation chosen, the hypothesis is a sentence with three parts. If we move the plan table above the feature sections, more pricing-page visitors will click through to a plan, because we think most of them leave without ever seeing the plans. The scroll data is the evidence for that belief.\n\nThe \"because\" does more work than it seems to. When a result comes in, it tells you which belief the result bears on. Without it, a losing variation only says that one layout lost.\n\nNext comes the measure. In [the growth-driven design cycle](https://www.growthdrivendesign.com/how-it-works/continuous-improvement), every plan starts with a focus metric, and a hypothesis needs its own version of that: one primary outcome, named before launch. This team chooses the share of pricing-page visitors who click through to a plan. The click is the first thing the new layout could change, and clicks arrive sooner and in larger numbers than closed deals. Closed revenue is the outcome the company cares about more. It arrives months later and in small numbers, and pricing changes and sales activity move it too. So the team judges this test on the click and keeps watching what happens downstream.\n\nThat watching is done with guardrails. A guardrail is an outcome that must not get worse by more than an amount the team agrees on before launch. Moving the table up could bring more clicks from people who haven't read enough to know whether the product fits. So the team also tracks demo requests from that page and the share of them that sales accepts as qualified. If clicks rise while accepted requests fall past the agreed tolerance, the change hasn't done what the business wanted, whatever the primary number says.","introHtml":"With one explanation chosen, the hypothesis is a sentence with three parts. If we move the plan table above the feature sections, more pricing-page visitors will click through to a plan, because we think most of them leave without ever seeing the plans. The scroll data is the evidence for that belief.
\nThe "because" does more work than it seems to. When a result comes in, it tells you which belief the result bears on. Without it, a losing variation only says that one layout lost.
\nNext comes the measure. In the growth-driven design cycle, every plan starts with a focus metric, and a hypothesis needs its own version of that: one primary outcome, named before launch. This team chooses the share of pricing-page visitors who click through to a plan. The click is the first thing the new layout could change, and clicks arrive sooner and in larger numbers than closed deals. Closed revenue is the outcome the company cares about more. It arrives months later and in small numbers, and pricing changes and sales activity move it too. So the team judges this test on the click and keeps watching what happens downstream.
\nThat watching is done with guardrails. A guardrail is an outcome that must not get worse by more than an amount the team agrees on before launch. Moving the table up could bring more clicks from people who haven't read enough to know whether the product fits. So the team also tracks demo requests from that page and the share of them that sales accepts as qualified. If clicks rise while accepted requests fall past the agreed tolerance, the change hasn't done what the business wanted, whatever the primary number says.
\n","outroMarkdown":null,"outroHtml":null,"contentJson":{},"configJson":{},"noteMarkdown":null,"noteHtml":null,"sortOrder":20},{"id":"7ed527ba-165c-40cd-a850-b6bdf2f127ca","sectionKey":"the_pricing_page_hypothesis_part_by_part","sectionType":"table_section","heading":"The pricing-page hypothesis, part by part","introMarkdown":null,"introHtml":null,"outroMarkdown":null,"outroHtml":null,"contentJson":{"rows":[{"cells":["Observation","Most visitors leave before the plan table; prospects on sales calls still ask about cost"]},{"cells":["Explanation being checked","The table sits too far down the page to be seen"]},{"cells":["Change","Move the plan table above the feature sections"]},{"cells":["Primary outcome","Share of pricing-page visitors who click through to a plan"]},{"cells":["Guardrails","Demo requests from the page, and the share sales accepts as qualified, each with an agreed tolerance"]},{"cells":["Decision rule","Set before launch, to suit the testing method in use"]}],"columns":["Part","In this example"]},"configJson":{},"noteMarkdown":null,"noteHtml":null,"sortOrder":30},{"id":"8dd55769-e473-42eb-a988-c88d027f7754","sectionKey":"decide_which_hypothesis_goes_first","sectionType":"markdown_section","heading":"Decide which hypothesis goes first","introMarkdown":"If the team has more hypotheses than it can run, it needs an order.\n\n[Lean Labs' growth-driven design program](https://www.leanlabs.com/solutions/growth-driven-design) shows one way to set it. The agency reviews page behavior, form results, CRM lifecycle data and what the sales team is hearing. It then ranks the candidate changes by expected effect on pipeline, by buyer relevance and confidence, and by effort.\n\nSuppose the software company applies those three to two ideas on its list. The pricing page sits close to a sales conversation, so the team judges the pipeline effect plausible. Scroll data and sales calls point at the same problem, which raises the team's confidence in the idea. On its site, moving a section is a few hours of work. The other idea is a homepage redesign that one executive has asked for. The team judges that the upside could be larger, but it has no visitor evidence behind it and the build would take a full sprint. On this team's scoring, the pricing change goes first.\n\nThose scores are the team's judgments, and another team with different evidence could order the same two ideas the other way. A ranking decides where the next sprint goes. It doesn't forecast a return, and the order should change as results come in. We go further into ordering in [how to organize a growth-driven design roadmap](https://www.answerstack.io/q/gdd-roadmap-prioritization).","introHtml":"If the team has more hypotheses than it can run, it needs an order.
\nLean Labs' growth-driven design program shows one way to set it. The agency reviews page behavior, form results, CRM lifecycle data and what the sales team is hearing. It then ranks the candidate changes by expected effect on pipeline, by buyer relevance and confidence, and by effort.
\nSuppose the software company applies those three to two ideas on its list. The pricing page sits close to a sales conversation, so the team judges the pipeline effect plausible. Scroll data and sales calls point at the same problem, which raises the team's confidence in the idea. On its site, moving a section is a few hours of work. The other idea is a homepage redesign that one executive has asked for. The team judges that the upside could be larger, but it has no visitor evidence behind it and the build would take a full sprint. On this team's scoring, the pricing change goes first.
\nThose scores are the team's judgments, and another team with different evidence could order the same two ideas the other way. A ranking decides where the next sprint goes. It doesn't forecast a return, and the order should change as results come in. We go further into ordering in how to organize a growth-driven design roadmap.
\n","outroMarkdown":null,"outroHtml":null,"contentJson":{},"configJson":{},"noteMarkdown":null,"noteHtml":null,"sortOrder":40},{"id":"7567352f-62c8-4a07-9e0e-4f4f074177e0","sectionKey":"choose_an_experiment_the_page_can_support","sectionType":"markdown_section","heading":"Choose an experiment the page can support","introMarkdown":"A split test fits when two reasonable versions compete and the page gets enough visitors to tell them apart. [HubSpot's page A/B test](https://knowledge.hubspot.com/website-pages/run-an-a-b-test-on-your-page), available on Content Hub Professional and Enterprise, serves two versions at one URL and sends half the visitors to each. HubSpot recommends changing one variable at a time. That advice suits the pricing example, because a variation that moved the table and rewrote the plan names would leave you unable to say which change mattered.\n\nHow long the test needs depends on a few inputs, and [Optimizely's guide to experiment length](https://docs.optimizely.com/experimentation-strategy/docs/how-long-to-run-an-experiment) lists them: the page's current conversion rate, the smallest difference you care to detect, the significance setting, the weekly visitors and the number of variations. The useful consequence is that a small expected difference needs far more data than a large one. A calendar deadline doesn't add evidence, so a test stopped because the month ended has only told you what it had collected by then.\n\nTesting tools also don't all decide the same way. Optimizely's own statistics engine evaluates results continuously and doesn't need a sample size fixed in advance, and the same guide warns that fixed-sample calculators estimate its run time poorly. So write the decision rule before launch, and write it for the tool you're using. In HubSpot, a person picks the winning version and the other becomes a draft. The tool carries out the choice, and the evidence behind it is the team's responsibility.\n\nThis is also where the sprint and the test come apart. Lean Labs runs its improvement work in 30-day sprints, and a sprint is a schedule for the team's work. The test ends when its rule says to stop, which can be after the sprint closes and can be without a clear winner. Let it keep running, and plan the next sprint's work away from the tested page where you can.","introHtml":"A split test fits when two reasonable versions compete and the page gets enough visitors to tell them apart. HubSpot's page A/B test, available on Content Hub Professional and Enterprise, serves two versions at one URL and sends half the visitors to each. HubSpot recommends changing one variable at a time. That advice suits the pricing example, because a variation that moved the table and rewrote the plan names would leave you unable to say which change mattered.
\nHow long the test needs depends on a few inputs, and Optimizely's guide to experiment length lists them: the page's current conversion rate, the smallest difference you care to detect, the significance setting, the weekly visitors and the number of variations. The useful consequence is that a small expected difference needs far more data than a large one. A calendar deadline doesn't add evidence, so a test stopped because the month ended has only told you what it had collected by then.
\nTesting tools also don't all decide the same way. Optimizely's own statistics engine evaluates results continuously and doesn't need a sample size fixed in advance, and the same guide warns that fixed-sample calculators estimate its run time poorly. So write the decision rule before launch, and write it for the tool you're using. In HubSpot, a person picks the winning version and the other becomes a draft. The tool carries out the choice, and the evidence behind it is the team's responsibility.
\nThis is also where the sprint and the test come apart. Lean Labs runs its improvement work in 30-day sprints, and a sprint is a schedule for the team's work. The test ends when its rule says to stop, which can be after the sprint closes and can be without a clear winner. Let it keep running, and plan the next sprint's work away from the tested page where you can.
\n","outroMarkdown":null,"outroHtml":null,"contentJson":{},"configJson":{},"noteMarkdown":null,"noteHtml":null,"sortOrder":50},{"id":"05540241-6e46-43cd-99bb-5fe568d87eee","sectionKey":"when_the_page_is_too_quiet_for_a_split_test","sectionType":"markdown_section","heading":"When the page is too quiet for a split test","introMarkdown":"Some pages don't get the traffic a split test needs in a useful time. Suppose instead that the planning inputs say the pricing test would run for most of a year. A long test can still be sound. But other edits to the page during the run can complicate the comparison, especially if they affect the tested behavior. Prices, campaigns and the product are also likely to change during those months. A team can reasonably decide that the wait isn't worth it.\n\nThe other route answers a different question. Put both layouts in front of a few people who resemble your buyers, and watch them look for the price. Watching matters here because, as [Nielsen Norman Group's guidance on user interviews](https://www.nngroup.com/articles/interviewing-users/) points out, people are poor at recalling exactly what they clicked. Sessions like these show whether people find the table, whether they understand the plans, and where they hesitate. They can't tell you how much the click-through rate will change, because a handful of participants isn't a measure of a rate.\n\nThen ship the version that evidence favors, and record the decision as a judgment. Afterward, compare the click-through rate with the period before, and ask the sales team whether the cost question still comes up on calls. Label both comparisons as uncontrolled, because a campaign, a season or a price change could have moved them in the same weeks. Say in the log that this was a judgment call, so nobody later reads it as a test result.","introHtml":"Some pages don't get the traffic a split test needs in a useful time. Suppose instead that the planning inputs say the pricing test would run for most of a year. A long test can still be sound. But other edits to the page during the run can complicate the comparison, especially if they affect the tested behavior. Prices, campaigns and the product are also likely to change during those months. A team can reasonably decide that the wait isn't worth it.
\nThe other route answers a different question. Put both layouts in front of a few people who resemble your buyers, and watch them look for the price. Watching matters here because, as Nielsen Norman Group's guidance on user interviews points out, people are poor at recalling exactly what they clicked. Sessions like these show whether people find the table, whether they understand the plans, and where they hesitate. They can't tell you how much the click-through rate will change, because a handful of participants isn't a measure of a rate.
\nThen ship the version that evidence favors, and record the decision as a judgment. Afterward, compare the click-through rate with the period before, and ask the sales team whether the cost question still comes up on calls. Label both comparisons as uncontrolled, because a campaign, a season or a price change could have moved them in the same weeks. Say in the log that this was a judgment call, so nobody later reads it as a test result.
\n","outroMarkdown":null,"outroHtml":null,"contentJson":{},"configJson":{},"noteMarkdown":null,"noteHtml":null,"sortOrder":60},{"id":"e86e2f77-9e5f-4f10-8708-69ef1cfda182","sectionKey":"record_the_result_with_the_reasoning_attached","sectionType":"markdown_section","heading":"Record the result with the reasoning attached","introMarkdown":"The cycle's last two steps are to learn from the experiment and to pass what was learned to marketing, sales and service. A result is easy to pass on when the record holds the hypothesis as it was worded before launch, the method used, what happened to the primary outcome and the guardrails, and what the team now believes about the explanation.\n\nThat last item needs its own line in every entry, because a result and an explanation are different things. Suppose the split test ran, and the higher table won on click-through with the guardrails inside their tolerances. Sales still hears the cost question on calls. The record should report those two facts as they are: the measured outcome improved, and the cost question is unresolved. The win is consistent with the position explanation. It doesn't prove it, since the other explanations were never ruled out, and a table higher on the page could draw more clicks for more than one reason. The open question becomes the next hypothesis, which might concern how the plans are described.\n\nA loss needs the same care. It could mean the explanation was wrong, or that the explanation was right and the new layout handled it badly.\n\nA finding like this belongs to more than the website. If prospects can't tell from the page what they'd pay, the sales team can address it early in the first call while the next test runs. There's more on what to do with the numbers in [the metrics to track in growth-driven design](https://www.answerstack.io/q/what-metrics-should-you-track-in-growth-driven-design).","introHtml":"The cycle's last two steps are to learn from the experiment and to pass what was learned to marketing, sales and service. A result is easy to pass on when the record holds the hypothesis as it was worded before launch, the method used, what happened to the primary outcome and the guardrails, and what the team now believes about the explanation.
\nThat last item needs its own line in every entry, because a result and an explanation are different things. Suppose the split test ran, and the higher table won on click-through with the guardrails inside their tolerances. Sales still hears the cost question on calls. The record should report those two facts as they are: the measured outcome improved, and the cost question is unresolved. The win is consistent with the position explanation. It doesn't prove it, since the other explanations were never ruled out, and a table higher on the page could draw more clicks for more than one reason. The open question becomes the next hypothesis, which might concern how the plans are described.
\nA loss needs the same care. It could mean the explanation was wrong, or that the explanation was right and the new layout handled it badly.
\nA finding like this belongs to more than the website. If prospects can't tell from the page what they'd pay, the sales team can address it early in the first call while the next test runs. There's more on what to do with the numbers in the metrics to track in growth-driven design.
\n","outroMarkdown":null,"outroHtml":null,"contentJson":{},"configJson":{},"noteMarkdown":null,"noteHtml":null,"sortOrder":70}],"citations":[{"title":"Continuous Improvement","url":"https://www.growthdrivendesign.com/how-it-works/continuous-improvement","excerpt":"Every plan starts with a focus metric that you want to improve.","quoteText":null,"sourceRole":"PRIMARY","verifiedAt":"2026-09-14T00:00:00","supportsText":"The method owner's specification of the continuous improvement sprint cycle and its four named steps, plan, build, learn and transfer; that every plan starts with a focus metric you want to improve; that the learn step reviews experiments and analyzes data to extract learnings about the audience; an","domain":"growthdrivendesign.com","publisherName":"GrowthDrivenDesign.com"},{"title":"Growth-Driven Design for Websites","url":"https://www.impactplus.com/growth-driven-design","excerpt":"How will you know if an action item is complete? Lay this out explicitly up front.","quoteText":null,"sourceRole":"SUPPORTING","verifiedAt":"2026-09-14T00:00:00","supportsText":"The four fields this guide specifies for a hypothesis statement: expected impact quantified as a single number, effort required as a single number, the metrics measured, and a definition of complete laid out explicitly up front. Also the wishlist brainstorm of 50 to 150 or more ideas produced in a s","domain":"impactplus.com","publisherName":"IMPACT"},{"title":"How to create a strong A/B testing hypothesis","url":"https://vwo.com/blog/ab-testing-hypothesis/","excerpt":"They are derived from at least some evidence. Strong hypotheses originate from usability testing, surveys, heatmaps, or website analytics and not from assumptions.","quoteText":null,"sourceRole":"SUPPORTING","verifiedAt":"2026-09-14T00:00:00","supportsText":"The three traits it gives for a strong hypothesis: derived from at least some evidence gathered through usability testing, surveys, heatmaps or analytics and not from assumption; aimed at altering customer behavior; and focused on deriving customer learning about why visitors behaved as they did and","domain":"vwo.com","publisherName":"VWO"},{"title":"ICE scoring model","url":"https://www.productplan.com/glossary/ice-scoring-model/","excerpt":"Each item receives a ranking from 1-10 for all three parameters, then those three numbers are multiplied, and the product is that item's ICE Score.","quoteText":null,"sourceRole":"INDEPENDENT","verifiedAt":"2026-09-14T00:00:00","supportsText":"Attributes ICE scoring to Sean Ellis and defines its three factors: Impact as how much the project will move the needle on the key metric being targeted, Confidence as the certainty that the project will have the predicted impact, and Ease as the level of effort to complete the project, each scored ","domain":"productplan.com","publisherName":"ProductPlan"},{"title":"A/B testing","url":"https://www.optimizely.com/optimization-glossary/ab-testing/","excerpt":"By testing one element at a time, you can identify which changes increase lead quality and conversion rates.","quoteText":null,"sourceRole":"SUPPORTING","verifiedAt":"2026-09-14T00:00:00","supportsText":"Defines an A/B test as comparing two versions of a page against each other by showing variants randomly to users and judging which achieves better conversion results, and advises testing one element at a time so that changes can be attributed. Used here only for the definition of the instrument and ","domain":"optimizely.com","publisherName":"Optimizely"},{"title":"How long to run an experiment","url":"https://support.optimizely.com/hc/en-us/articles/4410283969165-How-long-to-run-an-experiment","excerpt":"You should run tests for a minimum of one business cycle (seven days) to ensure all kinds of user behavior are accounted for.","quoteText":null,"sourceRole":"PRIMARY","verifiedAt":"2026-09-14T00:00:00","supportsText":"Product documentation for Optimizely's own defaults: the sample size calculator defaults to 90 percent statistical significance, and tests should run for a minimum of one business cycle, which the page defines as seven days, so that all kinds of user behavior are accounted for. The page also states ","domain":"support.optimizely.com","publisherName":"Optimizely"},{"title":"False discovery rate control","url":"https://support.optimizely.com/hc/en-us/articles/4410283967245-False-discovery-rate-control","excerpt":"A false discovery rate of 50% would be alarming. Because Optimizely uses techniques that keep the false discovery rate low (approximately ten percent), your chances of selecting a true winning variation to implement are much higher.","quoteText":null,"sourceRole":"PRIMARY","verifiedAt":"2026-09-14T00:00:00","supportsText":"The vendor's own distinction between the false positive rate, the proportion of false positives out of all negative outcomes, and the false discovery rate, the error rate among declared winners, which is the distinction this answer turns on. Includes the worked case where ten possible conclusive res","domain":"support.optimizely.com","publisherName":"Optimizely (product documentation)"},{"title":"How Not To Run an A/B Test","url":"https://www.evanmiller.org/how-not-to-run-an-ab-test.html","excerpt":"If instead of deciding ahead of time, \"this experiment will collect exactly 1,000 observations,\" you say, \"we'll run it until we see a significant difference,\" all the reported significance levels become meaningless.","quoteText":null,"sourceRole":"INDEPENDENT","verifiedAt":"2026-09-14T00:00:00","supportsText":"The correct statement of what a significance level answers: 'Assuming there is no underlying difference between A and B, how often will we see a difference like we do in the data just by chance?' Also the repeated significance testing error: reported levels assume the sample size was fixed in advanc","domain":"evanmiller.org","publisherName":"Evan Miller"},{"title":"Seven Rules of Thumb for Web Site Experimenters (KDD 2014)","url":"https://exp-platform.com/Documents/2014%20experimentersRulesOfThumb.pdf","excerpt":"If the probability of a true positive effect is low, i.e., most ideas fail to move key metrics in a positive direction, then the probability of a true effect when the p-value is close to 0.05 is still low.","quoteText":null,"sourceRole":"INDEPENDENT","verifiedAt":"2026-09-14T00:00:00","supportsText":"Peer-reviewed paper generalizing from thousands of controlled experiments at Amazon, Booking.com, LinkedIn and Microsoft properties. Supplies the Bayes formula for the probability of a true positive given a statistically significant result, P(TP|SS) = (1-beta)pi / [(1-beta)pi + alpha(1-pi)], with th","domain":"exp-platform.com","publisherName":"Kohavi, Deng, Longbotham and Xu, ACM SIGKDD"},{"title":"All about sample-size calculations for A/B testing: Novel extensions and practical guide","url":"https://arxiv.org/abs/2305.16459","excerpt":"n = 2 p_pool (1 - p_pool) (z_{1-alpha/2} + z_{1-beta})^2 / delta^2, where p_pool = (p_x + p_y)/2.","quoteText":null,"sourceRole":"INDEPENDENT","verifiedAt":"2026-09-14T00:00:00","supportsText":"Equation (2), the two-sample proportion sample size per arm used for every number in the sizing table: n = 2 p_pool (1 - p_pool) (z_{1-alpha/2} + z_{1-beta})^2 / delta^2, with p_pool = (p_x + p_y)/2 and delta the minimum detectable effect. Establishes that required sample scales with the inverse squ","domain":"arxiv.org","publisherName":"Jing Zhou, Jiannan Lu and Anas Shallah, CIKM 2023 (arXiv:2305.16459)"},{"title":"Putting A/B Testing in Its Place","url":"https://www.nngroup.com/articles/putting-ab-testing-in-its-place/","excerpt":"The biggest problem with A/B testing is that you don't know why you get the measured results.","quoteText":null,"sourceRole":"INDEPENDENT","verifiedAt":"2026-09-14T00:00:00","supportsText":"Establishes that A/B testing measures actual behavior under real-world conditions but cannot explain why the measured result happened, because the team is not observing users or listening in on their thoughts, and that it provides data only on the element being tested. Also the recommendation not to","domain":"nngroup.com","publisherName":"Nielsen Norman Group"},{"title":"The three stages of growth-driven design: strategy, launchpad, and continuous improvement","url":"https://www.leanlabs.com/blog/three-stages-of-growth-driven-design","excerpt":"Declaring a winner after 50 visits isn't testing, it's guessing.","quoteText":null,"sourceRole":"CONTRIBUTOR","verifiedAt":"2026-09-14T00:00:00","supportsText":"The contributor's own published positions, cited only inside the contributor perspective: continuous improvement as the ongoing cycle of analyzing site performance, prioritizing changes, testing them and implementing what works; headlines as the highest-yield test, followed by form placement, CTA co","domain":"leanlabs.com","publisherName":"Lean Labs"}],"revisions":[{"revisedAt":"2026-07-17T14:26:11.392+00:00","versionLabel":"Originally published","note":"","reviewedBy":null}],"relatedAnswers":[{"id":"3d083dc8-04bf-4e44-acb9-f13e7060381e","slug":"what-is-design-driven-growth","question":"What is design-driven growth, and how can strategic design drive business growth?","publishedAt":"2026-09-23T00:47:02.738","confidenceScore":78,"confidenceLabel":"Medium","industry":{"id":"70ea3802-1fc6-4fd7-a505-140d38d1c74a","slug":"digital-marketing","label":"Digital Marketing","description":"SEO, content, demand gen, and growth marketing"},"topic":{"slug":"growth-driven-design","label":"Growth-Driven Design","description":null,"schemaKind":null},"contributor":{"id":"dc83f9a1-5273-4639-bffb-43d5fa44c2bf","slug":"lean-labs","displayName":"Lean Labs","websiteUrl":null},"snippet":"Design-driven growth is growth that comes from shaping a product, service or website around user needs and a business goal. Strategic design can drive it when research, the design response and a business measure stay connected. It is a broader idea than growth-driven design, the named three-stage website method.","url":"/q/what-is-design-driven-growth"},{"id":"c18b70fe-5926-46ca-b21b-550c9b03d02c","slug":"gdd-on-hubspot-cms","question":"How do you build a growth-driven design website on HubSpot CMS?","publishedAt":"2026-09-07T14:26:15.656","confidenceScore":86,"confidenceLabel":"High","industry":{"id":"70ea3802-1fc6-4fd7-a505-140d38d1c74a","slug":"digital-marketing","label":"Digital Marketing","description":"SEO, content, demand gen, and growth marketing"},"topic":{"slug":"growth-driven-design","label":"Growth-Driven Design","description":null,"schemaKind":null},"contributor":{"id":"dc83f9a1-5273-4639-bffb-43d5fa44c2bf","slug":"lean-labs","displayName":"Lean Labs","websiteUrl":null},"snippet":"Growth-driven design on HubSpot follows the method's order: strategy, a launchpad site, then improvement cycles. The theme sets which changes editors can make, content staging prepares larger changes while shared assets stay connected to live pages, and page A/B testing on Content Hub Professional and Enterprise compares two versions.","url":"/q/gdd-on-hubspot-cms"},{"id":"6ac7d9be-c9e3-4392-bfce-ccb83fa10923","slug":"evaluate-roi-driven-web-design","question":"How should high-growth companies evaluate ROI-driven website design services?","publishedAt":"2026-09-04T14:26:05.645","confidenceScore":75,"confidenceLabel":"Medium","industry":{"id":"70ea3802-1fc6-4fd7-a505-140d38d1c74a","slug":"digital-marketing","label":"Digital Marketing","description":"SEO, content, demand gen, and growth marketing"},"topic":{"slug":"growth-driven-design","label":"Growth-Driven Design","description":null,"schemaKind":null},"contributor":{"id":"dc83f9a1-5273-4639-bffb-43d5fa44c2bf","slug":"lean-labs","displayName":"Lean Labs","websiteUrl":null},"snippet":"Treat the service as an investment and test it with your own figures: the full cost over a set period, the extra customers needed to cover it, and an agreed way to tell whether the website caused them. Then check the evidence behind the agency's past results, what you will own, and what the work keeps costing.","url":"/q/evaluate-roi-driven-web-design"},{"id":"2f1e68a4-9d9f-4790-aef0-d29edc53761f","slug":"build-custom-gdd-website","question":"How do you build and develop a custom growth-driven design website?","publishedAt":"2026-09-02T14:25:59.899","confidenceScore":75,"confidenceLabel":"Medium","industry":{"id":"70ea3802-1fc6-4fd7-a505-140d38d1c74a","slug":"digital-marketing","label":"Digital Marketing","description":"SEO, content, demand gen, and growth marketing"},"topic":{"slug":"growth-driven-design","label":"Growth-Driven Design","description":null,"schemaKind":null},"contributor":{"id":"dc83f9a1-5273-4639-bffb-43d5fa44c2bf","slug":"lean-labs","displayName":"Lean Labs","websiteUrl":null},"snippet":"You build a custom growth-driven design website by deciding what the site must do for your buyers and business, launching a smaller launchpad site that covers the top of that ranked list, and continuing to develop it in sprints from what real visitors do. Custom refers to the strategy, messaging, design and build choices made for your buyers, not to finishing every page before launch.","url":"/q/build-custom-gdd-website"}],"contributorStats":{"verifiedAnswers":46,"openDisputes":0},"schemaJson":{"@context":"https://schema.org","@type":"Question","name":"How do you turn user insights into growth-driven design hypotheses and experiments?","text":"How do you turn user insights into growth-driven design hypotheses and experiments?","url":"https://www.answerstack.io/q/gdd-hypotheses-experiments","answerCount":1,"datePublished":"2026-06-07T14:26:11.392","about":[{"@type":"Thing","name":"Growth-Driven Design"},{"@type":"Thing","name":"Digital Marketing"}],"acceptedAnswer":{"@type":"Answer","text":"You turn a user insight into a hypothesis by writing down four things: what you saw, what you think explains it, the change that follows from that explanation, and the one number that should move if you're right. The experiment is then matched to what the page can support. A split test can measure whether the number moved. Usability sessions and interviews can't measure a rate, but they can show whether people understand the page and where they stall, and that can be enough to make a change you record as a judgment.","url":"https://www.answerstack.io/q/gdd-hypotheses-experiments","upvoteCount":0,"datePublished":"2026-06-07T14:26:11.392","dateModified":"2026-09-14T00:00:00","citation":[{"@type":"CreativeWork","name":"Continuous Improvement","url":"https://www.growthdrivendesign.com/how-it-works/continuous-improvement"},{"@type":"CreativeWork","name":"Growth-Driven Design for Websites","url":"https://www.impactplus.com/growth-driven-design"},{"@type":"CreativeWork","name":"How to create a strong A/B testing hypothesis","url":"https://vwo.com/blog/ab-testing-hypothesis/"},{"@type":"CreativeWork","name":"ICE scoring model","url":"https://www.productplan.com/glossary/ice-scoring-model/"},{"@type":"CreativeWork","name":"A/B testing","url":"https://www.optimizely.com/optimization-glossary/ab-testing/"},{"@type":"CreativeWork","name":"How long to run an experiment","url":"https://support.optimizely.com/hc/en-us/articles/4410283969165-How-long-to-run-an-experiment"},{"@type":"CreativeWork","name":"False discovery rate control","url":"https://support.optimizely.com/hc/en-us/articles/4410283967245-False-discovery-rate-control"},{"@type":"CreativeWork","name":"How Not To Run an A/B Test","url":"https://www.evanmiller.org/how-not-to-run-an-ab-test.html"},{"@type":"CreativeWork","name":"Seven Rules of Thumb for Web Site Experimenters (KDD 2014)","url":"https://exp-platform.com/Documents/2014%20experimentersRulesOfThumb.pdf"},{"@type":"CreativeWork","name":"All about sample-size calculations for A/B testing: Novel extensions and practical guide","url":"https://arxiv.org/abs/2305.16459"},{"@type":"CreativeWork","name":"Putting A/B Testing in Its Place","url":"https://www.nngroup.com/articles/putting-ab-testing-in-its-place/"},{"@type":"CreativeWork","name":"The three stages of growth-driven design: strategy, launchpad, and continuous improvement","url":"https://www.leanlabs.com/blog/three-stages-of-growth-driven-design"}]}}}