Thoughts on Software Engineering with AI

I’ve been using AI coding tools seriously for about six months, first on side projects and now as my default engineering workflow. I wanted to write down some of the practices, impressions, and mental models that have helped me make sense of them so far.

If you’re coming from a skeptical point of view, I’d recommend reading Mitchell Hashimoto’s My AI Adoption Journey first.

Once men turned their thinking over to machines…

Frank Herbert, Dune

Brief Background

I spent four years as a backend/infrastructure engineer at Palantir, then three years managing an engineering team. I was not a day one AI adopter. I found early chatbots wrote bad code, and I saw many early AI-assisted PRs that required heavy cleanup. My attitude changed when I started playing around with Claude Code last summer on a side project. The combination of no external pressure, a totally unfamiliar domain (TS web app), and an easy terminal-based UI created the type of addictive, counting-down-the-seconds-until-rate-limits-reset experience that a lot of people have shared in recent months.

That experience was part of why I switched from managing back to being an individual contributor. Part of what made me effective as a manager was that I had been a strong engineer first. I could teach others because I had done the work myself. If AI tools were becoming central to how engineers work, and I hadn’t internalized them, I needed to go do that first. It was also part of my motivation for joining Normal Computing. An agile startup close to the cutting edge is a better place to figure this out than a larger company.

What I learned using these tools

It’s actually a skill, and one you need to work on.

There was a time when I would roll my eyes at “vibe coders,” and I still do in certain contexts, but one thing that becomes obvious quickly is that there are good ways to use these tools and bad ways. The bad ways stick out the most in the form of sloppy PRs and breaks to production that the author can’t explain, but for folks using LLMs well it’s practically invisible. A good bar: others shouldn’t be able to tell you used an agent just by looking at your work. Like any skill, it takes practice. The way I use these tools has changed tremendously over time, and no doubt will continue to evolve.

The tools today are good, and they are only going to get better

Not everyone is sold on this yet, but the tools as they exist today are quite good and capable of solving many engineering challenges. More importantly, they’re only going to get better. It’s anyone’s guess how high the ceiling is, but it’s already higher than most engineers believed. Because these tools will keep improving, there’s a lot of return on investment from switching earlier. The skills you build now will compound as the tools mature. Even if you believe the quality of the models has plateaued, it’s clear that the state of the art in LLM harnesses like Claude Code or AI-first IDEs like Google’s Antigravity is progressing rapidly. Improvements in these systems alone will continue producing gains for some time.

Fundamentals are as important as ever

A common stage of AI adoption is wondering if you have anything valuable left to contribute. For me this period was short. I had already experienced the mindset shift from authoring code myself to delegating and reviewing the work of other engineers. In either case the good news is that strong engineering fundamentals are still extremely valuable. I’d argue more than ever. Thinking about issues like good API design, complexity management, maintainability, and verifiability has never been more crucial. Perhaps the most classic failure mode of both junior engineers and LLMs is thinking at only one level of complexity, asking only what change needs to be made in this file or module, and missing how local choices impact the larger system both today and down the road as it continues to scale and evolve. I could opine on fundamentals for a long time, but for now I’m just going to plug my favorite software engineering book, A Philosophy of Software Design.

Use the right tool for the job

It is very easy to become overly dependent on LLMs and get stuck in a fruitless back and forth. Sometimes you need to crack open your IDE to find the information you need. Other times you’re better off authoring the code yourself. The common thread is avoiding autopilot and staying aware of whether your current approach is actually working. This is not a new issue introduced by LLMs. I’ve seen many engineers, including myself, waste time trying to debug an issue by carefully studying a PR diff when what they really need to do is exec into a running pod and figure out what’s actually happening. There are many different ways to be lazy, and LLMs are just another tool that can be both used and misused.

Iterate, iterate, iterate

When I’m programming, my first implementation is to get things working, and then I iterate for factors like code quality, execution speed, and testability. It seems unfair to expect LLMs to operate differently, and yet a lot of attention is placed on “one shotting” an implementation. My advice is to forget about one shotting and work until you have something you’re happy with. Don’t forget to review your PR in GitHub one last time before sending it out for review. To reiterate an earlier point, the end product of LLM-based work in production should be indistinguishable from authoring it yourself.

We need to rethink programming as theory building

One of the more important ideas I’ve encountered is Programming as Theory Building, which argues that a program’s true essence resides in the mental “theory” built by its developers rather than the source code itself. When those developers leave, the program effectively “dies” because new maintainers lack the underlying intuition, leading to systemic decay that documentation alone cannot prevent. If LLMs are increasingly used to author changes to software, then it seems like the process of theory building is undermined.

But I’m not sure I buy the premise. Mental models don’t scale. People switch teams and leave companies all the time. If you’re relying on folks’ mental models for your codebase, you have a system that fundamentally can’t grow. Even worse, a common failure mode I’ve seen is engineers writing software that makes sense to them and their mental model, but makes sense to no one else. Maybe mental models are more bug than feature. They create implicit knowledge that should be explicit, and they let engineers get away with code that only makes sense to them.

This is part of the reason why engineers have developed tools like API design, standardization, and documentation: to help other engineers build a mental model quickly. Working with AI models is like working with someone who has literally zero mental model of the code. Every session is brand new. They only have the available context to help them. If LLMs are struggling with your codebase, it might be because you’re over-relying on your own mental model. In fact, one possible upside of LLM engineering is that you’re constantly operating without that mental model, and thus have a reason to invest in codifying the model in, well, code. I’m still thinking through this, but I find it useful to reframe what’s a clear downside of LLMs, less deep experience with the codebase, into an opportunity for better engineering.

You have no reason not to explore

The best software engineers I’ve encountered have one thing in common: a supernatural ability to go beyond their domain, build context on other systems/codebases, and contribute the necessary changes. I’ve tried to emulate this, but it doesn’t come naturally. I’ve historically felt happiest working in a space that I was very familiar with, where I had a strong mental model and could make changes with a high degree of confidence. My favorite unlock from using LLMs is that they’ve made me much more confident working across domains. The experience of first cloning a repo and trying to figure out where to start has been replaced with a back and forth conversation where I ask an LLM to explore the repository and answer my questions. I can make assertions and have LLMs quickly verify or refute them, and I can author changes with a reasonable degree of confidence far more quickly. These tools make it easier than ever to explore unfamiliar code yourself and form your own hypotheses before asking someone else to explain it to you. It’s a practice I strongly recommend.

LLMs let me prioritize things that I wouldn’t have done otherwise

Building off the last item, LLMs have made the hardest part of a project, starting it, so much simpler. For me this has meant a lot more dev environment customization and tool setup. Previously, I would have felt like I was wasting valuable work time to try custom NVIM plugins or fiddle with the ZMK firmware on my keyboard. Now I can point an agent at it and get something pretty good very quickly. Better yet, I can ask the LLM silly questions like what the shortcut is for iterating between chunks in my git diff plugin, which makes me feel more confident that I won’t get blocked experimenting with new things.

Workflow patterns that help me

The setup details change constantly, but a few patterns have stuck.

The first is disposable worktrees. I heavily use this clwt shortcut, which automatically opens up a new worktree based off of main for a given project and opens up Claude. Usually I work on one thing, but I can have up to three projects going in parallel without mixing branches, local state, or half-finished agent work.

The second is giving agents somewhere to put and recover context. I’ve added the .local folder to my global git ignore, and I symlink that folder between worktrees. During Claude sessions I use it to dump thoughts, plans, agent documentation, etc. for reference by other agents without needing to worry about polluting git. I also try to use generic AGENTS.md type standards when possible to avoid vendor lock-in. If you write good AGENTS.md files, your instructions work across Claude Code, Cursor, Codex, and whatever comes next.

I also find LLMs especially useful on dense inspection tasks. For example, I’m experimenting with MCP servers like Grafana because reading through logs is often exactly the kind of tedious, high-context work where an agent can help me form a hypothesis faster.

My actual Claude sessions are nothing revolutionary. I’m pretty sensitive to the garbage in, garbage out problem, so I like to make sure to start with the right context and have it plan first. I find myself managing context a lot and handing off to new sessions when the context fills up and quality degrades. Some of the over-selling and hype of AI products makes people believe that they can do anything with zero effort from the user, which leads to disappointing results. You’ll learn more by doing than by reading the nth “how to use Claude Code” post. Spend more time leveling up your actual software engineering knowledge than jumping from one hyped AI product to the next.

A few setup details

My current setup is pretty simple. I use Ghostty as my terminal, iterate mostly in Claude Code, and open up NVIM when I want an editor. I picked up NVIM specifically because I was using Claude in the terminal more, and I like being able to open and close a lightweight editor quickly versus waiting for something more featureful but bloated like IntelliJ to open up.

I’m conservative about tool permissions, so I maintain a list of allowed actions for these repos, though I’m increasingly tempted to just let it go and wait until a PR is open to iterate on, much like if I was reviewing another dev’s code. I’ll sometimes spin up Codex if I find that Opus is stuck on something and I need a second opinion, and I could see myself moving to an OSS wrapper like OpenCode in the near future if needed, for example if costs get too high, other models get better, or a different harness proves superior.

For details on my specific configuration, I manage public-facing development environment setup in dotfiles, a public repo with personal configuration that applies across machines and contexts:

• Editor config (nvim, vim)
• Shell config (zsh)
• Claude Code settings and global AGENTS.md
• GitHub CLI config
• Personal git identity

References

A Philosophy of Software Design by John Ousterhout - The first coding book I read that finally made all of the feedback from my mentors click; the most important pieces are an understanding of what complexity is and how to manage it.

Verifiability is the Limit by Alperen Keles - Argues that verifiability, not capability, is the fundamental limit of LLM-assisted coding; success depends on how easily code can be verified in a given domain.

Software Engineering: Past, Present, and Future with Grady Booch - Podcast episode where an industry veteran explains how to think about LLM trends as a natural evolution of programming, akin to the adoption of object-oriented programming.

Neural Networks by 3Blue1Brown - I find it helpful to get some intuition for what these tools are actually doing, so that when they make mistakes you can better understand why; this is just one video in an excellent series I recommend.

My AI Adoption Journey by Mitchell Hashimoto - A recent blog post about personal AI adoption from the cofounder of HashiCorp.

Irrational Exuberance blog by Will Larson - I love Will Larson’s books on engineering management, and I try to stay up to date on his blog. Some particularly good/relevant recent entries include What Can Agents Actually Do and Facilitating AI Adoption at Imprint, though it’s the non-AI content about engineering management where he really shines.

Designing Machine Learning Systems by Chip Huyen - Good background on ML infrastructure and system design.

AI Engineering by Chip Huyen - Good background on building applications with foundation models. Surprisingly, I found Designing Machine Learning Systems more informative.