Things I've written down.
Ruby, Rails, performance, security and applied machine learning. Updated when something feels finished.
2 posts in Big Data
BIG DATA
05 / 22 / 26
One task running for an hour while 199 sit idle
Skew is the most common reason a Spark job is slow, and it looks exactly like a job that is nearly finished.
3 min
BIG DATA
11 / 03 / 25
One line of PySpark, 34% off the runtime
Broadcast joins, two million records, and why Spark's default query plan is worth arguing with when you know something the optimiser doesn't.
4 min