Introduction to Databricks Asset Bundles.

Need a gentle introduction to Databricks Asset Bundles, what they are, how to use them, why to use them? Look no further you hobbit.

Read more

Fivetran buys DBT. People get mad.

Well, we all knew that open source wasn’t a real thing anymore. This just confirms it. I don’t use DBT much, I think it’s for whimps and script kiddies. Anywho, I love watching Linkedin and Reddit explode with anger at Fivetran buying DBT. Everyone things dbt core is done. Who cares. Babies.

Read more
,

Running Llama 3.1 8B Locally (LangChain and SQLite)

Things have changed a lot in the last year related to LLMs and AI; on the one hand, it seems the AI skeptics for coding are increasingly confined to the corners of the internet. Everyone is dancing around in the middle, not sure of where everything should fall. Clearly, if we don’t use AI at […]

Read more

DuckDB on Mother Duck. Lake House Glory?

Read more

The Semantic Layer – Real or Not?

Read more

The Era of the YAML Engineer

You know, I did fight it for a long time, and I’m still fighting it. Look, no one wants to become a Terraform engineer; that is pain and suffering. But, we all understand the benefits of IAC (infrastructure as code), and SHOULD be using it in our daily tech lives, or pushing towards it. But […]

Read more
, , ,

DuckDB vs Polars. Wait. DuckDB and Polars.

So, the classic newbie question. DuckDB vs Polars, which one should you pick? This is an interesting question, and actually drives a lot of search traffic to this website on which you find yourself wasting time. I thank you for that. This is probably the most classic type of question that all developers eventually ask […]

Read more

Full vs Incremental Data Loads Explained

Read more
, , ,

Apache Iceberg Writes with DuckDB (or not)

Well, all the bottom feeders (Iceberg and DuckDB users) are howling at the moon and dancing around a bonfire at midnight trying to cast their evil spells on the rest of us. Apache Iceberg writes with DuckDB? Better late than never I suppose. Your witchy ways won’t work on me. Not going to lie, Iceberg […]

Read more
, ,

How to tune Spark Shuffle Partitions.

So, you’re just a regular old Data Engineer crawling along through the data muck, barley keeping your head above the bits and bytes threatening to drown you. At point in time you were full of spit and vinegar and enjoyed understanding and playing with every nuance known to man. But, not you are old and […]

Read more