Operant Studio
OPER-183

fix(ci): drift-check job conflates pending migrations with real drift

Band
SMALL
Suggested
Justin Cooke

Body

feature_id: FEAT-studio-ci-cd

## Context

`.github/workflows/migrate-prod.yml` has two jobs that run `prisma migrate status` against prod:

1. **`migrate` job, `Prisma migrate status (pre-flight, drift check)` step** — has correct logic added in OPER-93 ([REDACTED-DOB]). Greps the combined output for `not found locally in prisma/migrations` (real drift) and treats rc=1 with `have not yet been applied` (pending migrations) as pass, since pending migrations are the whole point of the next `migrate deploy` step.
2. **`drift-check` job, scheduled every 6 hours** (`cron: 17 */6 * * *`) — added in OPER-83 ([REDACTED-DOB]) as a safety net for the silent-skip class of bug. But this job runs `prisma migrate status` and exits 1 on any non-zero return code from the status command. It has no equivalent of the pre-flight step's grep logic.

**Bug:** `prisma migrate status` returns rc=1 for both real drift AND pending migrations. Since [REDACTED-DOB], the `drift-check` job has been failing every 6 hours with `conclusion: failure`, alerting `#dev-alerts` with a "drift" signal when in fact the cause was pending migrations (bot-merged migration PRs that silently skipped `migrate-prod`, tracked separately in OPER-182).

**Signal degradation:** Every failed `drift-check` fires a Slack alert. Justin's team has received these alerts every 6 hours for 2+ days. The signal is now noise — the team can't distinguish "real drift, someone changed prod DB out of band" from "pending migrations, migrate-prod didn't fire." When OPER-182 lands and fixes the silent-skip, this drift-check job will keep firing false positives for any migration PR merged in the window between merge and next scheduled `drift-check` run (up to 6 hours).

**Observed:** Every scheduled run since [REDACTED-DOB] 21:26 UTC has `conclusion: failure`. Latest confirmed at [REDACTED-DOB] 16:21 UTC. Log output shows exactly the pending-migrations path, no `not found locally` line.

## Acceptance Criteria

- [ ] `drift-check` job uses the same grep logic as the `migrate` job's pre-flight step: pass on rc=0, pass on rc=1 with `have not yet been applied` (no drift marker), fail with drift signal only on rc=1 with `not found locally in prisma/migrations`, fail-safe on any other non-zero.
- [ ] The Slack notification body distinguishes "DRIFT DETECTED" (real drift, someone changed prod DB out of band) from "PENDING MIGRATIONS" (migrations queued waiting for migrate-prod to fire). Pending-migrations state should either be silent or use a distinct low-severity signal.
- [ ] Existing pre-flight-step grep logic in the `migrate` job is unchanged.
- [ ] A comment header on the `drift-check` job cross-references OPER-83 (why the job exists) and this ticket (why the grep matters).
- [ ] Regression check: a manual `workflow_dispatch` on a state with only pending migrations (no real drift) does NOT fail the `drift-check` job.

## Risk classification

Sandbox (Risk 3 / Reach 3). The `drift-check` job Slack-alerts only — it does not gate deploys, does not auto-apply, does not modify DB state. A bad fix at worst produces a wrong Slack message; it cannot break prod. writeSets are a single `.yml` file and a Slack message template.

## Depends on

None. Independent of OPER-182 — this ticket fixes the alert signal quality, OPER-182 fixes the underlying silent-skip.

## Blocks

Nothing.

## Related tickets

- OPER-182: fix(ci): migrate-prod silently skips bot-merged migration PRs. This ticket is the companion — OPER-182 fixes the root cause, this ticket fixes the noisy alert that has been firing while the root cause went unfixed.

## Out of scope

- Auto-applying pending migrations from the `drift-check` job (out — that's a policy change, needs PDT input)
- Rewriting the Slack notification format broadly (out — this ticket only differentiates the two states in the body)
- Changing the 6-hour cadence (out — separate concern)
- Alerting on Vercel/DB schema mismatch (out — different signal, would need Vercel deploy hook)


---
<!-- admission-rejected -->
**Admission rejected (unknown_feature_id):** feature_id=FEAT-studio-ci-cd is not declared in the feature-map.md of any admission-enabled product owned by this team.

Valid feature_ids for product studio (products/studio/feature-map.md):

- FEAT-studio-agent-workers
- FEAT-studio-audit
- FEAT-studio-audit-log
- FEAT-studio-branch-protection-rulesets
- FEAT-studio-coverage-policy
- FEAT-studio-dependency-map-policy
- FEAT-studio-dispatcher
- FEAT-studio-env-protection
- FEAT-studio-fabricated-evidence-tripwire
- FEAT-studio-guard-inventory
- FEAT-studio-incident-runbook
- FEAT-studio-mutation-testing
- FEAT-studio-phi-guard
- FEAT-studio-pipeline-observability
- FEAT-studio-planning-validation
- FEAT-studio-platform
- FEAT-studio-product-manifest
- FEAT-studio-qa
- FEAT-studio-qa-projects
- FEAT-studio-review-rounds

... and 2 more; see products/studio/feature-map.md

Attachments

Loading attachments…

Comments

Loading comments…