The 55% page-load win was four queries and a cache key
Cutting clinical-study page loads by more than half under real traffic. Not a rewrite — profiling, eager loading, and one honest look at what was cacheable.
The pages were slow under load. The team's instinct was that the front end had grown too heavy, and there was a plan to start pulling components apart.
The profile said otherwise. Almost all of it was server time, and almost all of that was the database — not one catastrophic query, but a lot of small ones that only appeared together under real traffic.
Measure the thing users experience
The first correction was what we were timing. Local page loads were acceptable, which is why the problem kept getting waved away. Production under concurrency was a different system: warm caches behaved differently, connection pools were contended, and the slowest requests were several times the median.
So the target became the p95 of server response time on the real endpoint, measured in production. Not the median, which hid the problem, and not local timings, which lied.
Where it went
Four categories, in the order they contributed.
**N+1 queries in a nested render.** A collection rendered a partial per row, and that partial touched an association. One query became several hundred. Individually each was sub-millisecond, which is exactly why nobody had noticed — nothing looked slow in the logs, there was just a great deal of it.
**A count that walked the whole table** to render a number in a header, on every request, for a value that changed a few times a day.
**Serialisation of records we then discarded**, loading full objects to read two columns.
**Cache keys that never hit**, because they included a timestamp that changed on every request. The cache was working perfectly and was completely useless.
Eager loading, and picking the right verb
The N+1 fix is includes, and most people stop at knowing that. The interesting part is
that Rails gives you three tools with genuinely different behaviour.
preload always runs separate queries — one for the parent records, one per
association. It cannot filter on the association, because as far as SQL is concerned
the tables never meet. It is usually the gentlest on memory.
eager_load builds a single LEFT OUTER JOIN. You can filter and sort on associated
columns, at the cost of a wider result set, with parent rows repeated once per child.
On a one-to-many with many children that duplication is not trivial.
includes decides for you, picking separate queries normally and switching to a join
when it detects that you are referencing the association in a condition. Convenient,
and occasionally surprising — a where on an associated table silently changes your
query strategy and your memory profile.
I default to preload when I am only displaying data, and reach for eager_load
deliberately when I need to filter. Being explicit means the query plan does not change
because someone added a where clause three months later.
Making N+1 impossible instead of fixing it again
Fixing the N+1s took an afternoon. Stopping them coming back needed something
structural, and that is strict_loading.
With it enabled, touching an association that was not explicitly loaded raises rather than quietly issuing a query. The N+1 stops being a performance characteristic you might notice in a dashboard and becomes a failing test.
The overhead is negligible — enabling it across ten thousand records adds under a millisecond — while a single unnoticed N+1 over a hundred records can add hundreds of milliseconds to a request. That trade is not close.
A performance regression that raises in CI is a bug. One that only shows up in a latency graph is a discussion, and discussions lose to feature work.
The caching part
With the queries fixed, caching was worth doing rather than a way to hide the problem — an important ordering. Caching a slow query does not make it fast, it makes it intermittently fast and much harder to reason about.
The cache keys were the actual bug. Including anything per-request — a timestamp, a
request id, a Time.current — guarantees a miss every time while looking entirely
correct in code review. Rebuilt around the record and its updated_at, the same code
began hitting, and expensive fragments were computed on write instead of on every read.
For the count in the header: a counter cache. The number is maintained when rows change rather than derived when anyone looks.
What it added up to
Better than 55% off page load under high traffic, with no rewrite, no new infrastructure, and no framework swapped out. The most expensive change was a counter cache column and a migration.
The lesson I keep relearning is that "the app is slow" is almost never one thing. It is four or five ordinary things stacked up, each individually defensible, and the profile tells you which order to remove them in. Guessing does not.