Rust. AWS S3. Scripts. Python. Datetimes. Files. Etc.

I was recently sucked into doing something like a caveman … writing Rust by hand with the wonderful Zed IDE to solve a Python problem. Ahh … what a wonderful world we live in. It felt strangely good to do such things; it brought me back to the days pre-LLM when I would fight tooth and nail with that borrow checker and compiler. Like it or not, those days are mostly done as far as the future of Software Development goes … I’m sure there will be little hobbit holes of bearded Rust developers hacking away by hand long into the future, but that will be the exception, not the rule.

Anywho, the entire process did remind me of why such hand-writing of Rust is important as a mental exercise, at least.

Read more

, ,

Delta Kernel (Rust) and the future of the Lakehouse.

I recently spent some time with my old love, that is, Rust; fawning over the old days when I learned to fight the borrow checker without my own two worn-out hands, no LLMs to do the work for me. I still think it’s probably one of the most fun languages to learn, for many reasons. Yes, it can be a bit verbose, but it also teaches you to think about your programs and problems in a different way. It’s fast, way fast, and simply brings joy. It’s also a great tool to write building blocks with, CLIs and other crates that can be consumed by others.

I’ve also been an avid user of the Lakehouse Architecture since before it was cool.

Read more

, ,

Rust + DuckDB – Fast meets Faster. Yowza.

There are many things I love in this life. God. Family. The Great Outdoors. Writing code. Reading good books. Writing on Substack. Friends. Pizza. Fishing. Ah well, you get the point. What a time to be alive, the world in tatters, AI upending everything we have taken for gospel for the last … well forever. Because I’m an old curmudgeon, I learned to write Perl as my introduction to code, used to run LAMP stacks on macs underneath my desk and watching everyone to try hack it. That being said, while everyone else is ringing their hands in the error and retching like Gollum at how AI has stolen our precious, our code; I just ignore it all, and do both, use AI for coding when required or expected, and still writing Rust by hand when I get chance for fun.

Read more

,

Writing Rust … by Hand. No LLMs.

I recently had a bit of a self-identity crisis; of course, it was tied to spending way too much time on social media. I do try to avoid the worst of it, got rid of Twitter (X) this year, and mostly stick to Substack, LinkedIn, YouTube, occasional Reddit, and Hacker News. Probably still too much.

I’ve always seen it as a form of self-preservation, probably due to growing up the youngest of five in the middle of the prairie next to the river, in the only county without a working stoplight.

I try to be pragmatic and never dogmatic, except when someone says “Production” and “Notebook” in the same sentence.

Read more

, , ,

Introduction to Polars in 2 Minutes

Polars is the hot new Rust based Python Dataframe tool that is taking over the world and destryoing Pandas even as we speak. You want the quick and dirty introduction to Polars? Look no farther.

, , ,

Reading and Processing JSON with Rust vs Python.

Have you ever wondered about being explicit in your code vs being vague? I think about this a lot as I’m writing code on a daily basis. I’ve found I like being explicit and verbose when writing code, rather than being vague in what I’m doing most of the time.

Read more

, , , , ,

Why did Golang lose to Rust for Data Engineering?

A few years ago I wasn’t sure, who was going to win, Golang seemed to be popular, and still is for that matter. When I first wrote a little Golang (~2+ years ago) I was just trying to see what the hype was all about. The funny thing is, at the time, and today, it seems like the Golang syntax is much simpler than Rust, easier to learn and pick up by far.

Read more

, , ,

Writing Apache Spark with Rust! Spark Connect Introduced.

I never thought I would live to see the day, it’s crazy. I’m not sure who’s idea it was to make it possible to write Apache Spark with Rust, Golang, or Python … but they are all genius.

As of Apache Spark 3.4 it is now possible to use Spark Connect … a thin API client on a Spark Cluster ontop of the DataFrame API.

You can now connect backend systems and code, using Rust or Golang etc, to a Spark Server and run commands and get results remotely. Simply amazing.  A new era of tools and products is going to be unleashed on us.  We are no longer chained to the JVM. The walls have been broken down. The future is bright.

, ,

Why I Love Rust, but Deploy Python

I’m not sure if others have this same problem, maybe they are lucky, they get to build in their favorite language 24/7, it’s their tool of choice. I feel like I have a great burden to bear, a heavy one. I love to write Rust … but I deploy Python. Even when I know I could write Rust … Python gets deployed.

Read more