Measure release impact

A deployment is an analytical event. Recording it alongside product and operational events makes it possible to ask whether errors, latency, adoption, or conversion changed after a release.

GraphJSON can show correlation and timing. It does not prove that a deployment caused a change, and it does not replace deployment control, tracing, or rollback automation.

Emit deployment events#

Log a terminal event from the deployment system:

{
  "event": "deployment_completed",
  "deployment_id": "dep_01J...",
  "service": "web",
  "environment": "production",
  "release": "web-2026.07.26.1",
  "commit_sha": "8f2b9a1",
  "branch": "main",
  "provider": "vercel",
  "strategy": "rolling",
  "duration_seconds": 94,
  "schema_version": 1
}

On failure:

{
  "event": "deployment_failed",
  "deployment_id": "dep_01J...",
  "service": "api",
  "environment": "production",
  "release": "api-2026.07.26.3",
  "commit_sha": "b7c10d2",
  "provider": "github_actions",
  "failure_stage": "health_check",
  "duration_seconds": 311,
  "schema_version": 1
}

Use controlled failure stages. Do not send complete build logs, environment variables, command output, or actor email addresses.

Attach the release to application events#

Deployment events identify when a release changed. Application events identify which release handled each request or product action:

{
  "event": "api_request_completed",
  "service": "api",
  "environment": "production",
  "release": "api-2026.07.26.3",
  "route": "/v1/reports",
  "status_code": 200,
  "duration_ms": 184,
  "account_id": "acct_7",
  "schema_version": 1
}

Use a stable release identifier available to every process:

• semantic version
• build number
• immutable image tag
• commit SHA
• platform deployment ID

Do not use latest.

During a rolling or canary deployment, multiple releases can serve traffic simultaneously. Per-event release attribution is more reliable than assuming one timestamp divided all traffic cleanly.

Create a deployment timeline#

SELECT
  toDateTime(timestamp) AS deployed_at,
  JSONExtractString(json, 'service') AS service,
  JSONExtractString(json, 'release') AS release,
  JSONExtractString(json, 'commit_sha') AS commit_sha,
  JSONExtractString(json, 'strategy') AS strategy
FROM deployment_events
WHERE
  JSONExtractString(json, 'event') = 'deployment_completed'
  AND JSONExtractString(json, 'environment') = 'production'
  AND timestamp >= now() - INTERVAL 30 DAY
ORDER BY deployed_at DESC
LIMIT 100

Use a table and link the release identifier to your deployment or source-control system outside GraphJSON where appropriate.

Compare error rate by release#

SELECT
  JSONExtractString(json, 'release') AS release,
  count() AS requests,
  countIf(JSONExtractInt(json, 'status_code') >= 500) AS server_errors,
  round(server_errors / requests * 100, 2) AS error_pct
FROM api_events
WHERE
  JSONExtractString(json, 'event') = 'api_request_completed'
  AND JSONExtractString(json, 'environment') = 'production'
  AND timestamp >= now() - INTERVAL 24 HOUR
GROUP BY release
ORDER BY requests DESC

Compare only releases with enough traffic to interpret. A canary with five requests and one failure has a 20% error rate, but not enough evidence for the same conclusion as a release with a million requests.

Compare latency by release#

SELECT
  JSONExtractString(json, 'release') AS release,
  JSONExtractString(json, 'route') AS route,
  count() AS requests,
  quantile(0.5)(JSONExtractFloat(json, 'duration_ms')) AS p50_ms,
  quantile(0.95)(JSONExtractFloat(json, 'duration_ms')) AS p95_ms,
  quantile(0.99)(JSONExtractFloat(json, 'duration_ms')) AS p99_ms
FROM api_events
WHERE
  JSONExtractString(json, 'event') = 'api_request_completed'
  AND JSONExtractString(json, 'environment') = 'production'
  AND timestamp >= now() - INTERVAL 24 HOUR
GROUP BY release, route
HAVING requests >= 100
ORDER BY p95_ms DESC

Keep route in the comparison. A release serving a different endpoint mix can have a different overall percentile without making any individual route slower.

Measure product impact#

For a product release, compare a defined outcome:

SELECT
  JSONExtractString(json, 'release') AS release,
  uniqExactIf(
    JSONExtractString(json, 'account_id'),
    JSONExtractString(json, 'event') = 'onboarding_started'
  ) AS started_accounts,
  uniqExactIf(
    JSONExtractString(json, 'account_id'),
    JSONExtractString(json, 'event') = 'onboarding_completed'
  ) AS completed_accounts,
  round(completed_accounts / started_accounts * 100, 2) AS completion_pct
FROM product_events
WHERE
  JSONExtractString(json, 'event') IN (
    'onboarding_started',
    'onboarding_completed'
  )
  AND JSONExtractString(json, 'environment') = 'production'
  AND timestamp >= now() - INTERVAL 14 DAY
GROUP BY release
ORDER BY started_accounts DESC

This groups each event by the release that emitted it. It is descriptive, not necessarily a user-level cohort assignment. A user can begin under one release and complete under another.

For a controlled rollout, attach an experiment or rollout assignment and use A/B funnels.

Compare fixed before-and-after windows#

Use the deployment timestamp as a boundary only when traffic allocation changed cleanly.

WITH toDateTime('2026-07-26 18:00:00') AS deployed_at
SELECT
  if(toDateTime(timestamp) < deployed_at, 'before', 'after') AS period,
  count() AS requests,
  countIf(JSONExtractInt(json, 'status_code') >= 500) AS errors,
  round(errors / requests * 100, 2) AS error_pct
FROM api_events
WHERE
  JSONExtractString(json, 'event') = 'api_request_completed'
  AND toDateTime(timestamp) >= deployed_at - INTERVAL 2 HOUR
  AND toDateTime(timestamp) < deployed_at + INTERVAL 2 HOUR
GROUP BY period
ORDER BY period

Control for:

• time of day and day of week
• traffic source
• customer segment
• endpoint mix
• concurrent incidents
• provider changes
• gradual rollout percentage
• instrumentation changes

Build the dashboard#

Recommended tiles:

1. recent production deployments
2. request volume by release
3. 5xx rate by release
4. P95 latency by release and route
5. deployment failures by stage
6. primary product conversion by release
7. guardrail metric such as refunds or support failures
8. current release adoption

Put the active release and reporting window in the dashboard description.

Alert on release guardrails#

Create alerts for:

• deployment failures
• 5xx event count
• P95 latency
• failed background jobs
• checkout or payment failures

GraphJSON should notify an owner; the deployment platform should own rollback.

An alert description should say:

API server-error events exceed the hourly threshold.
Compare by release and route, check the latest deployment health,
then use the deployment platform’s rollback procedure if required.

Handle rollback#

Emit a new event:

{
  "event": "deployment_rolled_back",
  "service": "api",
  "environment": "production",
  "from_release": "api-2026.07.26.3",
  "to_release": "api-2026.07.25.8",
  "deployment_id": "dep_01J...",
  "reason_category": "error_rate",
  "schema_version": 1
}

Use a controlled reason category. Keep the incident narrative in the incident-management system and link it by an opaque incident ID if needed.

Release-review checklist#

• Deployment events use immutable release identifiers.
• Application events include the release that handled them.
• Environment and service are always present.
• Error categories are controlled and redacted.
• Comparisons account for route and traffic mix.
• Canary sample sizes are shown.
• Product impact is not presented as causal without experiment design.
• Alerts point to a documented operational response.
• Rollback events are logged but rollback remains outside GraphJSON.

If Vercel already emits the operational logs you need, connect the Vercel integration and add explicit deployment events for the release boundaries and product facts that logs alone cannot express.