Skip to content
Answer Stack
Open menu

How do you turn user insights into growth-driven design hypotheses and experiments?

✓ Verified Last reviewed by Lean Labs Next review due Dec 14, 2026

Every claim is sourced below

You turn a user insight into a hypothesis by writing down four things: what you saw, what you think explains it, the change that follows from that explanation, and the one number that should move if you're right. The experiment is then matched to what the page can support. A split test can measure whether the number moved. Usability sessions and interviews can't measure a rate, but they can show whether people understand the page and where they stall, and that can be enough to make a change you record as a judgment.

Separate what you saw from why you think it happened

An insight usually arrives as an observation and an explanation stuck together, and they need to be pulled apart before anyone designs anything.

Suppose a B2B software company reviews its pricing page. Scroll data shows that most visitors leave before they reach the plan table. In three of the last five recorded sales calls, the prospect asked what the product costs after having visited that page. Those are observations. "The plan table sits too far down the page" is an explanation, and other explanations fit the same facts. The visitors might have come from an ad that promised something the page doesn't deliver. The table might be easy to reach and hard to read.

Wingify's guide to test hypotheses builds a hypothesis from a problem, a proposed change and a way to measure the result, with the problem drawn from usability sessions, surveys, heatmaps and analytics. Those methods show where people stop, what they do and what they say about it. The reason is still something you infer, and the experiment exists to check that inference.

So it pays to look more closely before you commit to one explanation. Watch a handful of session recordings on the pricing page, and ask the sales team what prospects said in their own words. Suppose several recordings show people scrolling past the table and still leaving. Those visitors reached the table and left anyway, so there's a second explanation to investigate alongside the first. Position could still matter for them, since a table that arrives late may arrive after interest has gone. A few recordings can't speak for every visitor, and the scroll data still says most people never get that far.

Write the prediction so it can be wrong

With one explanation chosen, the hypothesis is a sentence with three parts. If we move the plan table above the feature sections, more pricing-page visitors will click through to a plan, because we think most of them leave without ever seeing the plans. The scroll data is the evidence for that belief.

The "because" does more work than it seems to. When a result comes in, it tells you which belief the result bears on. Without it, a losing variation only says that one layout lost.

Next comes the measure. In the growth-driven design cycle, every plan starts with a focus metric, and a hypothesis needs its own version of that: one primary outcome, named before launch. This team chooses the share of pricing-page visitors who click through to a plan. The click is the first thing the new layout could change, and clicks arrive sooner and in larger numbers than closed deals. Closed revenue is the outcome the company cares about more. It arrives months later and in small numbers, and pricing changes and sales activity move it too. So the team judges this test on the click and keeps watching what happens downstream.

That watching is done with guardrails. A guardrail is an outcome that must not get worse by more than an amount the team agrees on before launch. Moving the table up could bring more clicks from people who haven't read enough to know whether the product fits. So the team also tracks demo requests from that page and the share of them that sales accepts as qualified. If clicks rise while accepted requests fall past the agreed tolerance, the change hasn't done what the business wanted, whatever the primary number says.

Part In this example
Observation Most visitors leave before the plan table; prospects on sales calls still ask about cost
Explanation being checked The table sits too far down the page to be seen
Change Move the plan table above the feature sections
Primary outcome Share of pricing-page visitors who click through to a plan
Guardrails Demo requests from the page, and the share sales accepts as qualified, each with an agreed tolerance
Decision rule Set before launch, to suit the testing method in use

Decide which hypothesis goes first

If the team has more hypotheses than it can run, it needs an order.

Lean Labs' growth-driven design program shows one way to set it. The agency reviews page behavior, form results, CRM lifecycle data and what the sales team is hearing. It then ranks the candidate changes by expected effect on pipeline, by buyer relevance and confidence, and by effort.

Suppose the software company applies those three to two ideas on its list. The pricing page sits close to a sales conversation, so the team judges the pipeline effect plausible. Scroll data and sales calls point at the same problem, which raises the team's confidence in the idea. On its site, moving a section is a few hours of work. The other idea is a homepage redesign that one executive has asked for. The team judges that the upside could be larger, but it has no visitor evidence behind it and the build would take a full sprint. On this team's scoring, the pricing change goes first.

Those scores are the team's judgments, and another team with different evidence could order the same two ideas the other way. A ranking decides where the next sprint goes. It doesn't forecast a return, and the order should change as results come in. We go further into ordering in how to organize a growth-driven design roadmap.

Choose an experiment the page can support

A split test fits when two reasonable versions compete and the page gets enough visitors to tell them apart. HubSpot's page A/B test, available on Content Hub Professional and Enterprise, serves two versions at one URL and sends half the visitors to each. HubSpot recommends changing one variable at a time. That advice suits the pricing example, because a variation that moved the table and rewrote the plan names would leave you unable to say which change mattered.

How long the test needs depends on a few inputs, and Optimizely's guide to experiment length lists them: the page's current conversion rate, the smallest difference you care to detect, the significance setting, the weekly visitors and the number of variations. The useful consequence is that a small expected difference needs far more data than a large one. A calendar deadline doesn't add evidence, so a test stopped because the month ended has only told you what it had collected by then.

Testing tools also don't all decide the same way. Optimizely's own statistics engine evaluates results continuously and doesn't need a sample size fixed in advance, and the same guide warns that fixed-sample calculators estimate its run time poorly. So write the decision rule before launch, and write it for the tool you're using. In HubSpot, a person picks the winning version and the other becomes a draft. The tool carries out the choice, and the evidence behind it is the team's responsibility.

This is also where the sprint and the test come apart. Lean Labs runs its improvement work in 30-day sprints, and a sprint is a schedule for the team's work. The test ends when its rule says to stop, which can be after the sprint closes and can be without a clear winner. Let it keep running, and plan the next sprint's work away from the tested page where you can.

When the page is too quiet for a split test

Some pages don't get the traffic a split test needs in a useful time. Suppose instead that the planning inputs say the pricing test would run for most of a year. A long test can still be sound. But other edits to the page during the run can complicate the comparison, especially if they affect the tested behavior. Prices, campaigns and the product are also likely to change during those months. A team can reasonably decide that the wait isn't worth it.

The other route answers a different question. Put both layouts in front of a few people who resemble your buyers, and watch them look for the price. Watching matters here because, as Nielsen Norman Group's guidance on user interviews points out, people are poor at recalling exactly what they clicked. Sessions like these show whether people find the table, whether they understand the plans, and where they hesitate. They can't tell you how much the click-through rate will change, because a handful of participants isn't a measure of a rate.

Then ship the version that evidence favors, and record the decision as a judgment. Afterward, compare the click-through rate with the period before, and ask the sales team whether the cost question still comes up on calls. Label both comparisons as uncontrolled, because a campaign, a season or a price change could have moved them in the same weeks. Say in the log that this was a judgment call, so nobody later reads it as a test result.

Record the result with the reasoning attached

The cycle's last two steps are to learn from the experiment and to pass what was learned to marketing, sales and service. A result is easy to pass on when the record holds the hypothesis as it was worded before launch, the method used, what happened to the primary outcome and the guardrails, and what the team now believes about the explanation.

That last item needs its own line in every entry, because a result and an explanation are different things. Suppose the split test ran, and the higher table won on click-through with the guardrails inside their tolerances. Sales still hears the cost question on calls. The record should report those two facts as they are: the measured outcome improved, and the cost question is unresolved. The win is consistent with the position explanation. It doesn't prove it, since the other explanations were never ruled out, and a table higher on the page could draw more clicks for more than one reason. The open question becomes the next hypothesis, which might concern how the plans are described.

A loss needs the same care. It could mean the explanation was wrong, or that the explanation was right and the new layout handled it badly.

A finding like this belongs to more than the website. If prospects can't tell from the page what they'd pay, the sales team can address it early in the first call while the next test runs. There's more on what to do with the numbers in the metrics to track in growth-driven design.

Sources

Continuous Improvement

GrowthDrivenDesign.com

Primary source Verified Sep 14, 2026 Supports: The method owner's specification of the continuous improvement sprint cycle and its four named steps, plan, build, learn and transfer; that every plan starts with a focus metric you want to improve; that the learn step reviews experiments and analyzes data to extract learnings about the audience; an

“Every plan starts with a focus metric that you want to improve.”

Growth-Driven Design for Websites

IMPACT

Supporting Verified Sep 14, 2026 Supports: The four fields this guide specifies for a hypothesis statement: expected impact quantified as a single number, effort required as a single number, the metrics measured, and a definition of complete laid out explicitly up front. Also the wishlist brainstorm of 50 to 150 or more ideas produced in a s

“How will you know if an action item is complete? Lay this out explicitly up front.”

How to create a strong A/B testing hypothesis

VWO

Supporting Verified Sep 14, 2026 Supports: The three traits it gives for a strong hypothesis: derived from at least some evidence gathered through usability testing, surveys, heatmaps or analytics and not from assumption; aimed at altering customer behavior; and focused on deriving customer learning about why visitors behaved as they did and

“They are derived from at least some evidence. Strong hypotheses originate from usability testing, surveys, heatmaps, or website analytics and not from assumptions.”

ICE scoring model

ProductPlan

Independent Verified Sep 14, 2026 Supports: Attributes ICE scoring to Sean Ellis and defines its three factors: Impact as how much the project will move the needle on the key metric being targeted, Confidence as the certainty that the project will have the predicted impact, and Ease as the level of effort to complete the project, each scored

“Each item receives a ranking from 1-10 for all three parameters, then those three numbers are multiplied, and the product is that item's ICE Score.”

A/B testing

Optimizely

Supporting Verified Sep 14, 2026 Supports: Defines an A/B test as comparing two versions of a page against each other by showing variants randomly to users and judging which achieves better conversion results, and advises testing one element at a time so that changes can be attributed. Used here only for the definition of the instrument and

“By testing one element at a time, you can identify which changes increase lead quality and conversion rates.”

How long to run an experiment

Optimizely

Primary source Verified Sep 14, 2026 Supports: Product documentation for Optimizely's own defaults: the sample size calculator defaults to 90 percent statistical significance, and tests should run for a minimum of one business cycle, which the page defines as seven days, so that all kinds of user behavior are accounted for. The page also states

“You should run tests for a minimum of one business cycle (seven days) to ensure all kinds of user behavior are accounted for.”

False discovery rate control

Optimizely (product documentation)

Primary source Verified Sep 14, 2026 Supports: The vendor's own distinction between the false positive rate, the proportion of false positives out of all negative outcomes, and the false discovery rate, the error rate among declared winners, which is the distinction this answer turns on. Includes the worked case where ten possible conclusive res

“A false discovery rate of 50% would be alarming. Because Optimizely uses techniques that keep the false discovery rate low (approximately ten percent), your chances of selecting a true winning variation to implement are much higher.”

How Not To Run an A/B Test

Evan Miller

Independent Verified Sep 14, 2026 Supports: The correct statement of what a significance level answers: 'Assuming there is no underlying difference between A and B, how often will we see a difference like we do in the data just by chance?' Also the repeated significance testing error: reported levels assume the sample size was fixed in advanc

“If instead of deciding ahead of time, "this experiment will collect exactly 1,000 observations," you say, "we'll run it until we see a significant difference," all the reported significance levels become meaningless.”

Seven Rules of Thumb for Web Site Experimenters (KDD 2014)

Kohavi, Deng, Longbotham and Xu, ACM SIGKDD

Independent Verified Sep 14, 2026 Supports: Peer-reviewed paper generalizing from thousands of controlled experiments at Amazon, Booking.com, LinkedIn and Microsoft properties. Supplies the Bayes formula for the probability of a true positive given a statistically significant result, P(TP|SS) = (1-beta)pi / [(1-beta)pi + alpha(1-pi)], with th

“If the probability of a true positive effect is low, i.e., most ideas fail to move key metrics in a positive direction, then the probability of a true effect when the p-value is close to 0.05 is still low.”

All about sample-size calculations for A/B testing: Novel extensions and practical guide

Jing Zhou, Jiannan Lu and Anas Shallah, CIKM 2023 (arXiv:2305.16459)

Independent Verified Sep 14, 2026 Supports: Equation (2), the two-sample proportion sample size per arm used for every number in the sizing table: n = 2 p_pool (1 - p_pool) (z_{1-alpha/2} + z_{1-beta})^2 / delta^2, with p_pool = (p_x + p_y)/2 and delta the minimum detectable effect. Establishes that required sample scales with the inverse squ

“n = 2 p_pool (1 - p_pool) (z_{1-alpha/2} + z_{1-beta})^2 / delta^2, where p_pool = (p_x + p_y)/2.”

Putting A/B Testing in Its Place

Nielsen Norman Group

Independent Verified Sep 14, 2026 Supports: Establishes that A/B testing measures actual behavior under real-world conditions but cannot explain why the measured result happened, because the team is not observing users or listening in on their thoughts, and that it provides data only on the element being tested. Also the recommendation not to

“The biggest problem with A/B testing is that you don't know why you get the measured results.”

The three stages of growth-driven design: strategy, launchpad, and continuous improvement

Lean Labs

Contributor · COI Verified Sep 14, 2026 Supports: The contributor's own published positions, cited only inside the contributor perspective: continuous improvement as the ongoing cycle of analyzing site performance, prioritizing changes, testing them and implementing what works; headlines as the highest-yield test, followed by form placement, CTA co

“Declaring a winner after 50 visits isn't testing, it's guessing.”

Revision history

1 revision since publication
Originally published