Field notes — essays & research notes

Things I've written down.

Ruby, Rails, performance, security and applied machine learning. Updated when something feels finished.

55 posts · page 4 of 6
BIG DATA 05 / 22 / 26

One task running for an hour while 199 sit idle

Skew is the most common reason a Spark job is slow, and it looks exactly like a job that is nearly finished.

3 min
RUBY & RAILS 05 / 19 / 26

YJIT is free performance, until you read the memory graph

15–30% faster for a flag you probably already have on. What it costs per worker, and the one case where you should turn it down rather than off.

4 min
RUBY & RAILS 05 / 13 / 26

Processing a million rows without loading a million rows

`map` builds an array. On a large collection that array is the problem, and the fix is usually one method name.

3 min
MLOPS 05 / 08 / 26

The model was fine. The features were different.

A model that scores well offline and poorly in production usually isn't seeing the same inputs it was trained on.

2 min
SECURITY 05 / 06 / 26

Secrets that stay secret

The dangerous part is not storage. It's that a secret in git history is still there after you delete the file — and .gitignore won't save you.

3 min
RUBY & RAILS 05 / 01 / 26

Stop guessing: profiling Ruby properly

Benchmarking a suspicion confirms the suspicion. A sampling profiler tells you where the time actually went.

3 min
RUBY & RAILS 04 / 28 / 26

Retries fell 30% when I stopped retrying the wrong things

A retry is a bet that the failure was temporary. Most retried jobs were losing that bet — and the ones that won were doing their work twice.

5 min
RUBY & RAILS 04 / 22 / 26

Reading a query plan without fear

EXPLAIN output looks impenetrable and answers three questions. Those three cover most of what makes a query slow.

3 min
SECURITY 04 / 15 / 26

Your logs are a database nobody secured

Logs are copied to aggregators, retained for months and read by people who would never be granted access to the production database.

2 min