Every few years the machine learning world falls hard for something new. Transformers, diffusion models, agents, whatever is currently setting the group chat on fire. Meanwhile, in the background, the same algorithm from 2014 keeps winning Kaggle competitions and shipping to production, doing the unglamorous work of guessing which customers will churn and which loans will default. Nobody writes poems about the workhorse. So today, because it’s Friday, I did.
[The poem]
O gradient booster, sculptor of splits,
you take my weak learners and sharpen their wits.
Where one shallow tree would stumble and guess,
a thousand in sequence converge on success.
Each stump learns the sins of the stump that came prior,
descending the loss like a monk through the mire.
Regularized gently (both L1 and L2),
you punish complexity, humble and true.
The neural nets preen with their billions of weights,
demanding GPUs and warehouse-sized rates,
while you, on a laptop, with tabular grace,
still quietly win every Kaggle-bound race.
With column subsampling and shrinkage applied,
you prune away hubris, keep variance tied.
Early stopping, your mercy; your histogram, art,
second-order Taylor expansions of heart.
So here’s to you, XGBoost, workhorse supreme,
the sensible answer to every team’s dream.
When someone says “transformer,” I smile and say,
“Sure… but have you tried boosting? It still wins today.”
Have a good weekend.
