Manage changes to analytics assets

An event field rarely has only one consumer. It can feed a saved SQL query, chart, dashboard, alert, iframe, customer-facing report, spreadsheet export, and an executive definition at the same time.

Treat analytics as a dependency graph:

producer → collection and field → query or visualization
         → dashboard / alert / embed / shared link / export
         → decision, workflow, or customer promise

GraphJSON does not currently present this entire graph as an automatic lineage view. Maintain a small inventory before making breaking changes.

Inventory the assets#

Record at least:

Start with the collections and saved queries most likely to change. Add customer-facing embeds, alerts, and scheduled manual exports next; their consumers often cannot see an editor’s warning.

Use stable internal asset names even when a display title changes.

Download the dependency inventory template to start with the fields above.

Classify the proposed change#

An additive producer change can still be semantically breaking if a query uses SELECT *, groups missing values unexpectedly, or assumes a closed set of categories.

Write a change record#

Before implementation, state:

• what is changing and why
• source and destination contracts
• affected definitions and assets
• data backfill or history behavior
• privacy and retention effect
• validation query and acceptance threshold
• owner and approver
• planned cutover
• rollback trigger and procedure

For a customer-visible metric, also state how old and new values may differ. “More accurate” is not enough; name the population, time, or attribution rule that changed.

Prefer parallel migration#

For a field rename from workspace to account_id:

1. update producers to send both fields
2. verify type, completeness, and cardinality
3. add a v2 query that reads account_id
4. compare old and new results over a settled period
5. move dashboards, alerts, embeds, and API consumers
6. monitor the old field for remaining readers and writers
7. stop writing the old field
8. retire it only after the rollback window

Do not edit the only working query in place when its result columns or meaning change. Duplicate it, use a clear version suffix, and migrate consumers.

Reconcile old and new results#

A useful comparison includes absolute and relative difference:

WITH
  old_definition AS (
    SELECT
      toDate(toDateTime(timestamp)) AS day,
      uniqExact(JSONExtractString(json, 'workspace')) AS value
    FROM product_events
    WHERE JSONExtractString(json, 'event') = 'report_published'
    GROUP BY day
  ),
  new_definition AS (
    SELECT
      toDate(toDateTime(timestamp)) AS day,
      uniqExact(JSONExtractString(json, 'account_id')) AS value
    FROM product_events
    WHERE
      JSONExtractString(json, 'event') = 'report_published'
      AND JSONExtractString(json, 'account_id') != ''
    GROUP BY day
  ),
  days AS (
    SELECT day FROM old_definition
    UNION DISTINCT
    SELECT day FROM new_definition
  )
SELECT
  days.day,
  old_definition.value AS old_value,
  new_definition.value AS new_value,
  new_value - old_value AS absolute_difference,
  if(
    old_value = 0,
    NULL,
    (new_value - old_value) / old_value
  ) AS relative_difference
FROM days
LEFT JOIN old_definition
  ON days.day = old_definition.day
LEFT JOIN new_definition
  ON days.day = new_definition.day
ORDER BY day

Investigate differences by segment and sample events. A matching grand total can hide offsetting errors.

Migrate by asset type#

Collection visualizations#

Duplicate the visualization or retain its exported Data API payload. Verify filters, metric types, time zone, granularity, split cardinality, and empty states.

Saved SQL#

Keep stable result aliases. A column rename can break its chart mapping and every server that normalizes the Data API response. Test source-controlled SQL with metric CI.

Dashboards#

Create a parallel dashboard for a material redesign. Verify every tile’s date range, filter assumptions, and owner before replacing the old link. Coordinate editing: the most recently saved dashboard state can overwrite another editor’s intent.

Alerts#

GraphJSON alerts watch supported Single Line collection visualizations. Check that the replacement uses a compatible graph type, threshold, comparison window, recipients, and evaluation behavior. Exercise the notification path with a safe threshold before relying on it.

Embeds and shared links#

An iframe URL is a bearer capability. Treat URL changes, password changes, and retirement as access-control work. There is not a universal per-viewer revocation mechanism for a link already distributed; rotate or replace the shared asset and remove old access deliberately.

For customer-scoped native reports, version the server-owned registry and authorization behavior along with the query.

Exports#

List people and systems that receive recurring CSV, JSON, PNG, or manual exports. An undocumented spreadsheet formula is still a downstream consumer. Include file column names, time zone, currency unit, and null behavior in the export contract.

Handle collection moves#

Moving from a mixed collection to domain-specific collections changes table names and may change access, retention, and history.

During migration:

1. dual-publish from one durable source, or replay the same event envelope
2. mark the schema and migration version
3. compare counts, distinct IDs, min/max times, and key sums
4. switch readers one by one
5. keep the old path read-only through the rollback window
6. record when the historical boundary changes

Do not make two independent application calls and call that durable dual-write. Use an outbox or queue-backed fan-out as described in durable raw archive and dual-write.

Change metric meaning visibly#

If a calculation changes meaning, give it a new definition version even when the display name stays familiar:

{
  "metric_id": "weekly_active_accounts",
  "definition_version": 4,
  "effective_at": 1785024000,
  "owner": "product-analytics",
  "change": "require one core workflow action"
}

Annotate comparisons that cross the cutover. Recompute history only when the new rules can be applied consistently and the audience understands that prior published numbers may change.

Define rollback before cutover#

A rollback should identify:

• old producer version or event path
• old query and dashboard URLs
• which new data can be ignored safely
• whether data written during the new version is backward-compatible
• who can make the decision
• how customer-visible values will be communicated

Preserve the old asset long enough to test rollback. A deleted query ID or revoked share link may not be reconstructable from memory.

Operate the inventory#

Review critical assets on a regular cadence:

• owner still belongs to the team
• definition still matches the decision
• source fields are populated
• links and embeds still have the intended audience
• alerts still reach an accountable recipient
• exports still have an active consumer
• rollback artifacts are not stale

Retire orphaned assets instead of leaving multiple plausible versions. Record the retirement date and replacement.

Change checklist#

• Source fields, queries, charts, dashboards, alerts, embeds, and exports are inventoried.
• The change is classified as additive, corrective, semantic, breaking, or destructive.
• Definition, owner, acceptance threshold, cutover, and rollback are written.
• Breaking producer and result contracts run in parallel.
• Old and new results are reconciled by period and segment.
• Customer-facing authorization and cache behavior are re-tested.
• Alert delivery and empty/error states are exercised.
• Consumers are moved before the old asset is retired.
• The changelog and metric definition record the effective version.
• Ownership and retirement dates are current.

Continue with metric governance, saved SQL query workflow, and manage and share dashboards.