Ruby & Rails

Elasticsearch wasn't the problem. My queries were.

Halving search response time while raising result accuracy 35% — mostly by rewriting what I was asking for, not what I was asking it on.

16 September 2025 4 min read Mohammad Aaquib Jawed

Property search on the platform was slow and, more damagingly, wrong often enough that people had stopped trusting it. Users had learned to scroll past the first few results, which is the clearest possible signal that ranking is broken.

The instinct in the room was to scale the cluster. That would have made the wrong results arrive faster.

Two problems wearing one costume

Users reported a single complaint — "search is bad" — that turned out to be two unrelated failures.

Latency came from cluster configuration and unbounded queries: shard layout that did not match access patterns, and queries requesting far more data than the page displayed.

Relevance came from the queries not reflecting how people actually searched. This was the bigger problem and the less obvious one, because it does not appear in any performance dashboard.

The relevance fix was rewriting the question

The original implementation was a generic multi-match across every indexed field with default weighting. It is the obvious first implementation and it treats a match on a property description as equivalent to a match on its locality.

What people actually did was search by location, then filter by concrete attributes. Locality terms should dominate scoring. A description mentioning a neighbourhood in passing should not outrank a property in it.

Rewriting with explicit field boosts, bespoke aggregations for the filters people used, and a scoring function that reflected the real hierarchy of intent lifted result accuracy 35% — without touching a single node.

If your search returns the wrong thing, more shards return the wrong thing in parallel.

A related lesson: analysers matter more than query syntax. Location names have abbreviations, spellings and colloquial forms. Getting synonym handling and tokenisation right at index time fixed a category of "no results for something that obviously exists" that no amount of query tuning would have reached, because the term was never in the index in a findable form.

Then the infrastructure

With ranking fixed, the latency work was worth doing. Cluster configuration for the actual access pattern, bounded result sets, and requesting only the fields the page renders rather than whole documents — together, a 50% cut in response time.

Worth doing, and much less interesting than it looked from the outside. It was second in priority and would have been a poor place to start, because a faster wrong answer is still a wrong answer.

Measuring relevance at all

The awkward part of a relevance project is that "better" is not self-evident. Latency you can time. Relevance needs a definition before it needs a fix.

What worked was cheap rather than clever. A set of real queries taken from logs, weighted by how often they actually occurred. For each, a small hand-labelled set of results that ought to appear near the top. Then a simple score: for these queries, how often is a good result in the first handful?

It is not a research-grade methodology. It was enough to tell whether a change helped, which is the entire requirement — and it converted an argument about taste into a number that moved. Without it, every ranking change is somebody's opinion against somebody else's.

The other habit worth keeping is watching what users do after searching. A search followed by an immediate second, more specific search is a failure, whatever the relevance score says.

Deep pagination is a trap

One specific thing worth flagging because it bites people at exactly the wrong moment.

Asking for results far into a result set is disproportionately expensive in a distributed search engine: each shard must produce everything up to that offset so the coordinating node can merge and discard nearly all of it. Page one is cheap. Page five hundred is not, and the cost is borne by the whole cluster.

Most users never go deep. Crawlers and scripts do, constantly, and that is how a rare expensive query becomes a steady background load. Capping how deep pagination can go, and using a cursor-based approach where deep traversal is genuinely needed, removed a class of slow query we had been treating as unexplained variance.

The other half of that codebase

The same system had race conditions in its concurrent booking flow — two users could progress against the same slot under load.

The fix was mutex-based synchronisation and atomic operations on the critical paths, cutting race conditions 60%. Different problem, same underlying lesson: find the actual mechanism before reaching for capacity. Nobody would have solved a race condition by adding servers, but plenty of people would have tried to solve slow search that way.

What I take from it

When users say something is slow, find out whether they mean slow or wrong. They use the same word, the causes are unrelated, and fixing the one they did not mean produces a confident report that changes nothing.

And when the fix is "configure more infrastructure", check that the thing you are scaling is the thing that is broken. Scaling is satisfying, measurable, and frequently addresses the symptom you can see instead of the cause you have not looked for yet.

All writing Reply by email