A/B testing an advisor website means showing two versions of one page element to similar visitors at the same time and keeping the version that produces more booked calls or completed forms. Most independent advisory sites don't get enough traffic to test small tweaks, so the approach that works is to test bold changes on your highest-traffic pages, measure one enquiry metric, and treat every variant as an advertisement under the SEC Marketing Rule. Done that way, testing replaces opinions about your homepage with evidence.
This guide covers what to test, how to size a test for a small-firm site, how to read results without fooling yourself, and where compliance fits.
TL;DR
- Pick one primary metric (a booked call or completed contact form), define it before the test starts, and change only one element per test.
- At a 3% baseline conversion rate, detecting a 20% relative lift takes roughly 14,000 visitors per variant, so low-traffic advisor sites should test bigger changes on busier pages.
- Run tests across full weeks, don't stop when a result first looks good, and log losing and inconclusive tests too.
- Every variant is an advertisement: it must satisfy the SEC Marketing Rule, and your written procedures should cover your website and record retention.
- Roll out the winner, then use what you learned to choose the next test.
Table of Contents
- Why advisors test instead of guessing
- What to test on an advisor website
- Choose the metric before you start
- How much traffic you really need
- Running the test step by step
- Reading results without fooling yourself
- After the test: roll out, then choose the next test
- Compliance: every variant is an advertisement
- Our perspective
- Where we can take this off your plate
- FAQ
- Sources
Why advisors test instead of guessing
Most firms redesign a site on instinct: a partner likes a headline, a designer prefers a color, a competitor's page looks good. Instinct about your own audience is often wrong. Kitces contributor Mikel Bruce makes the point that what a firm assumes prospects will respond to and what they actually respond to are frequently different things, particularly for firms without a marketing or design specialist.
Testing also helps in a regulated business. A controlled experiment gives you a record of why a page says what it says. SiteImprove's Elizabeth Irvine argues that experiments let regulated teams defend decisions with data when compliance or legal reviewers are involved.
The goal is not a prettier site. It is more qualified conversations from the traffic you already earn. If you want a baseline for what "good" looks like first, our piece on closing the advisor website conversion gap walks through the typical range.
What to test on an advisor website
A test needs a single element that can change on its own. MarketingProfs' Eric Hansen warns that even a headline is really several factors (words, font, color, size), so changing all of them at once tells you nothing about which one mattered.
For a prospect-facing advisor site, these elements usually carry the most weight:
| Element | Example of version A vs. B | Why it matters |
|---|---|---|
| Call to action text | "Schedule a Call" vs. "Schedule a Consultation" | Kitces lists CTA wording, size, placement and action type as testable |
| CTA format | Text link vs. button | The CTA stands between you and the prospect |
| CTA action | Call vs. email vs. form | Different prospects prefer different effort levels |
| Homepage headline | Who you serve vs. what you do | First thing every visitor reads |
| Lead magnet title | Checklist vs. guide | Drives opt-ins before a first meeting |
| Form length | Name and email vs. name, email, phone, goal | Friction against lead quality |
Two practical notes. First, rank your ideas. A simple method is impact times confidence divided by effort: a CTA rewrite is cheap and high-impact, while a full redesign is neither quick nor testable as one change. Second, make the variants clearly different. Hansen recommends considerable differences between versions, because subtle changes produce subtle behavior shifts you can't detect. For more ideas on page elements, see this overview of optimizing landing pages.
Write each test as a hypothesis: if we change X, then metric Y will improve, because of reason Z. "If we change the button from "Schedule a Call" to "Schedule a Consultation", booked meetings will rise, because the second feels like a professional service rather than a sales call" is testable. "Make the page better" is not.
Choose the metric before you start
Pick one primary metric and define it in advance. For an advisory firm that is normally a qualified enquiry: a completed contact form, a booked call, or a registration for your webinar or seminar. Page views and time on site are not the goal.
Stable's testing documentation separates three kinds of metric:
- Primary: conversion rate (converting visitors divided by visitors) or lead count from forms and registrations.
- Secondary: bounce rate, time on page, scroll depth, element clicks. They help explain why the primary metric moved.
- Guardrail: measures that must not get worse, such as page load speed or unsubscribes.
Microsoft's experimentation researchers note that changes often improve one aspect while causing regressions in another, so a win on one number should be checked against the others before you ship it. For an advisor, a shorter form that doubles submissions but fills your calendar with unqualified contacts is not a win. Track what happens after the form, not only the submission.
How much traffic you really need
This is where most advisor tests fail before they start. Sample size depends on three things: your baseline conversion rate, the smallest improvement worth detecting, and the risk of a false result you will accept.
The standard settings are 80% power and a 5% significance level. With those, a service-business playbook from XenGrowth shows that a 3% baseline and a 20% relative lift (3.0% to 3.6%) needs roughly 14,000 visitors per variant. A site with a few thousand visits a month would need most of a year for a single page.
The duration math is simple. Divide the required sample by daily traffic per variant. Growthbook's example: with 2,000 visitors needed per variation and 500 visitors a day split across two versions, the test takes 8 days (2,000 ÷ 250). Then round up to whole weeks so weekday and weekend behavior balance out. Omniconvert's guidance is to run at least one to two full weeks, ending on the same weekday you started.
If your numbers say "ten months," you have three options, all from the same playbook:
- Test a bolder change. A dramatically different headline or page layout produces a bigger effect that needs fewer visitors to detect.
- Test your busiest page. Usually the homepage or your main landing page.
- Use a higher-volume secondary metric, such as clicks on the contact button, while keeping qualified enquiries as the primary measure.
Webinar and seminar registration pages are often good candidates because paid and email traffic can be sent to them in volume. Our guide to lead generation landing pages for financial advisors covers how to structure those pages.
Running the test step by step
- Set one goal and write the hypothesis. One conversion event, one change, one reason you expect it to work.
- Build the variant and send it to compliance review first. Both versions must be approved before a single visitor sees either (more on this below).
- Split traffic evenly and randomly. Each visitor should see the same version every time they return.
- Check tracking before launch. SiteImprove recommends QA on tracking, consent and disclosures before a test goes live. Submit a test form on both versions and confirm each one is recorded.
- Run it for the planned length. Cover at least one full business cycle. Avoid testing during major business shifts or holiday traffic spikes, which distort normal behavior.
- Run one test per page at a time. Overlapping tests on the same page make it impossible to attribute a change.
- Document everything, including losers. A landing page testing guide from Knak recommends logging every result so you don't repeat an experiment.
If you test copy often, a tool that tracks copy performance against conversions can help you see which wording patterns keep winning.
Reading results without fooling yourself
Don't stop on the first good day
Watching a dashboard and stopping when the result turns significant inflates false positives. Statsig's documentation explains that sequential testing methods adjust for this, but that early decisions produce underpowered lift estimates. If you need an accurate effect size, wait for the full sample you calculated. Microsoft reports that its typical test lasts 7 days and that repeated peeking must be accounted for.
Check the data before the winner
Atticus Li's analysis guide suggests validating the data first: look for sample ratio mismatch (a 52/48 split on 100,000 users is suspicious), confirm the planned sample was reached, and verify the test ran full weeks. A lopsided split usually means a technical fault, not a real result.
Report a range, not a promise
The same guide advises against promising the headline lift (say, +8%) as a guaranteed outcome. Report the lower bound of the confidence interval as the worst case. When you tell your partners what the new CTA did, say "likely between X and Y," not "it adds 8%."
Treat inconclusive as an answer
If the test ends flat, don't extend it hoping for significance. That is p-value hacking. The result tells you the change has no detectable effect at the size you cared about. Log it, calculate a new sample size if you want to look again, and move to a bolder idea.
After the test: roll out, then choose the next test
Optimizely's guidance on acting on experiment results suggests that with a clear winner you decide whether to implement or test again. Expand messaging wins to other areas, keep iterating after a surprisingly strong result, and move on when returns diminish. Don't ship a variation that fails your business goals. If the test ran during an unusual period, run it again.
Before implementing, check tradeoff metrics too. Reforge's guide says to weigh a significant negative impact on a tradeoff metric against the primary gain.
For an advisor, "apply the winner elsewhere" has a practical meaning: if "Schedule a Consultation" beats "Schedule a Call," use the wording in your webinar emails, your seminar invitations and your contact page. Then the next test might be the form length or the lead magnet title, whichever sits closest to the point where visitors drop off. Our post on a compliance-first lead capture flow maps that path.

Compliance: every variant is an advertisement
Testing does not suspend the rules. Your website copy is subject to the SEC Marketing Rule (Rule 206(4)-1), so each version of a headline or page needs to meet it. In plain terms, an advertisement may not include an untrue material statement, omit a material fact that makes it misleading, state a material fact you can't substantiate, create a misleading implication, discuss benefits without fair treatment of material risks, or present performance unfairly.
Three practical consequences for testing:
- Performance claims need extra care. Gross performance must be shown with net performance at least as prominently, and portfolio or composite results must include one-, five- and ten-year periods. A variant that leads with a return figure is a compliance decision first and a conversion decision second. In a 2024 order, the SEC found an adviser advertised hypothetical performance on its public website without policies ensuring it was relevant to the likely audience.
- Your procedures must cover the website. SEC examiners observed firms whose policies did not address channels such as websites and social media, were informal rather than written, or did not cover preserving advertisements. Keep a dated copy of every variant that ran.
- Testimonials have their own conditions. If a variant adds client quotes, review our summary of the three conditions for testimonials.
The SEC staff also noted that advisers may use tools such as risk-based sample review or preapproved templates. Preapproved variant templates are a natural fit for testing: your chief compliance officer approves the allowed wording options once, and tests only choose among them. For the rule in practical terms, see our four compliance steps for RIAs, and use your compliance officer or securities counsel for decisions about your specific firm. If a message type carries more risk than others, our ranking of fiduciary marketing messages by risk shows where to be most careful.
Our perspective
Most advisors we talk to don't have a testing problem. They have a traffic and time problem. A partner who is already running client meetings is not going to build variants, calculate sample sizes, route each version through compliance and keep a log of every result.
That is why we build advisor websites, webinars, seminars and email follow-up as one connected system rather than separate projects. When the website, the registration pages and the automated emails live together, there is more traffic flowing to the pages that matter, and a lesson from one place carries to the others.
Our team is made up of practicing financial advisors who have grown real practices, so we start from the regulatory and long buying-cycle realities of your business, not from generic agency templates. Testing is most valuable when it sits inside a compliance-friendly process: approved wording, clear records, and a steady rhythm of one change at a time.
The firms that benefit most are the ones that decide to make testing a habit instead of a one-time redesign.
Josh
Where we can take this off your plate
If you would rather spend your hours with clients than in analytics dashboards, we build the pieces a testing program depends on: high-converting advisor websites, lead-generation webinars, fully produced seminars, a compliance-friendly content library, automated email follow-up and custom CRM integration, all tailored to financial services.
We also map a growth strategy around your practice so each page and campaign has a clear job. See the full list of services or start from our growth strategy page.
FAQ
How long should an advisor website A/B test run?
Long enough to reach the sample size you calculated, and never less than one full week. Divide the visitors needed per variant by your daily traffic per variant, then round up to whole weeks. Low-traffic sites should test bolder changes or higher-traffic pages instead of waiting months.
Can a small advisory site with little traffic still test?
Yes, with adjustments. Test one major change instead of small tweaks, focus on your busiest page, and consider clicks on the contact button as a higher-volume secondary metric while keeping booked calls as the primary one. Treat results as directional until you have the full sample.
What should I test first?
Start with the call to action on your highest-traffic page: its wording, placement, or the action it asks for. It sits directly between the visitor and a conversation with you, so improvements there show up in enquiries faster than changes lower on the page.
Do SEC rules apply to test variants?
Yes. Each version is an advertisement under the Marketing Rule, so it should be reviewed before launch and a dated copy retained. Ask your chief compliance officer or securities counsel to approve the variants, especially any that mention performance or client results.
Sources
- Conversion Experiment Playbook for Service-Business Sites
- Optimizing an Advisory Website's Process Page, Kitces
- Conversion Rate Optimization for Financial Services Websites
- What to Test in Multivariate Testing, MarketingProfs
- How to design, size and read a split test
- Patterns of Trustworthy Experimentation, Microsoft Research
- What Is Test Duration?
- Objective A/B test prioritization, Growthbook
- Build an A/B Testing Process for Conversion Rate Optimization, MarketingProfs
- Landing Page A/B Testing: 75 Ideas, Knak
- Frequentist Sequential Testing, Statsig Docs
- How to Analyze A/B Test Results
- Take action based on the results of an experiment, Optimizely
- Decide A/B test next steps, Reforge
- 17 CFR 275.206(4)-1, the Marketing Rule
- SEC Order IA-6646 (2024)
- SEC Division of Examinations Risk Alert on the Marketing Rule (2024)

