Build a conversion funnel and A/B test

This guide measures a three-step signup funnel and compares two experiment variants. The same pattern works for onboarding, checkout, activation, and upgrade flows.

Define the experiment first#

We will measure:

landing_viewed → signup_started → signup_completed

Assign each eligible user once and persist the assignment. Log the same experiment, variant, and durable user_id on every funnel event.

Do not assign on every page view; a user who changes variants makes the comparison invalid.

Log the events#

await logEvent({
  event: "landing_viewed",
  user_id: user.id,
  experiment: "signup_copy_v2",
  variant: user.experiments.signup_copy_v2
});

await logEvent({
  event: "signup_started",
  user_id: user.id,
  experiment: "signup_copy_v2",
  variant: user.experiments.signup_copy_v2
});

await logEvent({
  event: "signup_completed",
  user_id: user.id,
  experiment: "signup_copy_v2",
  variant: user.experiments.signup_copy_v2
});

Use an anonymous ID created by your application before signup if a known user ID is not yet available. Do not use raw IP addresses as the primary identity.

Validate assignment#

Check that a user appears in only one variant:

SELECT
  JSONExtractString(json, 'user_id') AS user_id,
  uniqExact(JSONExtractString(json, 'variant')) AS variants
FROM product_events
WHERE JSONExtractString(json, 'experiment') = 'signup_copy_v2'
GROUP BY user_id
HAVING variants > 1
LIMIT 100

The expected result is empty.

Build a reach funnel#

This query counts distinct users who reached each step in one variant:

SELECT
  multiIf(
    JSONExtractString(json, 'event') = 'landing_viewed', '1. Landing viewed',
    JSONExtractString(json, 'event') = 'signup_started', '2. Signup started',
    JSONExtractString(json, 'event') = 'signup_completed', '3. Signup completed',
    'Other'
  ) AS step,
  uniqExact(JSONExtractString(json, 'user_id')) AS users
FROM product_events
WHERE
  JSONExtractString(json, 'experiment') = 'signup_copy_v2'
  AND JSONExtractString(json, 'variant') = 'treatment'
  AND JSONExtractString(json, 'event') IN (
    'landing_viewed',
    'signup_started',
    'signup_completed'
  )
  AND toDateTime(timestamp) >= subtractDays(now(), 30)
GROUP BY step
ORDER BY step

In Visualization, choose Funnel, use step as the label, and users as the value.

Important: This is a reach funnel: it counts whether a user generated each event in the window. It does not prove strict event order. For flows where order and maximum time-to-convert matter, use ClickHouse’s sequence or windowFunnel functions and validate the query against known users.

Compare variants#

Calculate completion rate by variant:

WITH by_user AS (
  SELECT
    JSONExtractString(json, 'user_id') AS user_id,
    JSONExtractString(json, 'variant') AS variant,
    max(JSONExtractString(json, 'event') = 'landing_viewed') AS entered,
    max(JSONExtractString(json, 'event') = 'signup_completed') AS completed
  FROM product_events
  WHERE
    JSONExtractString(json, 'experiment') = 'signup_copy_v2'
    AND toDateTime(timestamp) >= subtractDays(now(), 30)
  GROUP BY user_id, variant
)
SELECT
  variant,
  sum(entered) AS entrants,
  sum(completed) AS completions,
  round(completions / entrants * 100, 2) AS conversion_rate
FROM by_user
GROUP BY variant
ORDER BY variant

Visualize this as a table or bar chart.

Interpret responsibly#

A higher observed conversion rate does not automatically mean the treatment won.

Before deciding:

confirm sample-ratio balance between variants
run for a full business cycle
predefine the primary metric and stopping rule
check guardrail metrics such as errors, refunds, or downstream retention
verify assignment and instrumentation
use an appropriate statistical test outside GraphJSON when making a consequential decision

GraphJSON gives you the event data and aggregations; experimental validity still depends on the design.