Ok, Spark isn’t dead. Before you leave, I’m sorry for lying to you. Sorta. Kinda. Not really.

Undoubtedly Apache Spark has reached its zenith, shot like a rocket out of the Databricks barrel into the sky. The world is shifting though, even if ever so imperceptible.

Read more

I recently spent some time poking around Agent Bricks from Databricks, and it’s a pretty good representation of where we are in the AI cycle right now. Whether you’re skeptical or all-in, it’s hard to ignore the fact that agent-based systems are no longer theoretical. They’re here, and they’re being used to automate real workflows.

I’m not particularly interested in the hype, though. What matters is whether something useful can actually be built. We’ve reached a point where building AI systems is no longer technically difficult. Tools like Claude can generate large portions of the code, frameworks are maturing quickly, and platforms like Databricks are packaging everything into approachable interfaces. The barrier to entry has dropped significantly. The real question now is not whether you can build an agent, but whether you can build one that delivers value.

Read more