Migrations that don't take the site down

During a rolling deploy, old and new code run simultaneously against one database. Every migration has to be safe for both, and the failures are locks rather than duration.

Locks, not slowness

A migration that takes thirty seconds and blocks nothing is fine. One that completes in under a second while holding an exclusive lock on a hot table can stall every request that touches it, because they queue behind the lock.

Worse, that queue blocks other things. A quick ALTER TABLE waiting behind one long-running read will itself block every query that arrives after it — the lock queue is ordered, so one slow transaction plus one migration becomes a full outage on that table.

Setting a short lock timeout on migrations converts this from an outage into a failed migration you retry. That is a much better failure.

Adding things is usually safe, with caveats

Adding a nullable column without a default is cheap on modern databases — metadata only.

Adding one *with* a default used to rewrite the whole table; current Postgres does not for most cases, but it is worth confirming for your version rather than assuming.

Adding an index is the classic trap: by default it locks the table against writes for the duration, and on a large table that duration is long. Build it concurrently instead — Rails supports this via algorithm: :concurrently, which requires disabling the transaction wrapper for that migration. Concurrent builds can fail and leave an invalid index behind, so they need a check afterwards.

Removing things needs two deploys

Dropping a column that running code still selects breaks that code immediately. Active Record caches the column list at boot, so old processes will happily SELECT a column that no longer exists.

The safe sequence is: tell Active Record to ignore the column and deploy that. Then drop the column in a second deploy, once no running process expects it.

Renaming is the same problem twice, which is why the pragmatic approach is not to rename at all: add the new column, write to both, backfill, move reads, then remove the old one. Four steps rather than one, and none of them can strand a running process.

If a migration and a deploy have to land in the same instant to be safe, it is not a migration, it is an outage with a schedule.

Backfills belong outside the migration

Updating every row in a large table inside a migration holds a transaction open for the duration and blocks the deploy on it.

Batch it, run it outside the schema migration, and make it resumable. The schema change and the data change are different operations with different risk profiles, and coupling them means a failure in the slow half rolls back the fast half.

Making this automatic

Reviewing every migration by hand for these properties does not scale and depends on whoever is reviewing having read this list.

There are gems that check migrations against exactly these rules and fail the build with an explanation and a safe alternative. That converts institutional knowledge into a linter, which is where knowledge like this belongs — the failure mode of the manual approach is a quiet Friday deploy by someone who joined last month.