At some point, every data engineer has to confront a slightly uncomfortable truth about how they work: the tools they use are not just tools but habits, and those habits quietly shape how they think, how they build, and ultimately what kind of systems they produce. That realization tends to hit hardest when someone points directly at a pattern and gives it a name, which is exactly what happened when I started talking about notebook engineering and received a very direct question in response, which was essentially this: if notebooks are the problem, then what does it actually look like to break free from them?
That question is much harder to answer than simply criticizing the behavior, because it forces you to move beyond pointing out flaws and into explaining what better looks like in practice.
I finally hit that point that every engineer eventually reaches with a tool they once loved, that moment where frustration quietly builds over time and then suddenly flips into a decision, not because of one catastrophic failure but because of the accumulation of too many small ones. That was me with Polars. After years of using it, promoting it, and even putting it into production workloads early on, I reached the point where I removed it entirely from a set of critical pipelines and replaced it with DuckDB without much hesitation.
Before anyone jumps in to defend Polars or call this an overreaction, it is worth understanding that this was not a spur-of-the-moment decision. This was the result of years of real-world usage, repeated friction, and a growing realization that what matters most in a production data platform is not theoretical performance or benchmark wins, but consistency, predictability, and trust that the tool behaves the same way every time it runs.
