Mea culpa: I was wrong. The artificial intelligence (AI) singularity is, in fact, here. Whether we like it or not, AI isn’t something that will possibly, maybe impact software development in the distant future. It’s happening right now. Today. No, not every developer is taking advantage of large language models (LLMs) to build or test code. In fact, most aren’t. But for those who are, AI is dramatically changing the way they build software. It’s worth tuning into how they’re employing LLMs like ChatGPT to get some sense of how you can use such tools to make yourself or your development teams much more productive.
AI-driven ambition
One of the most outspoken advocates for LLM-enhanced development is Simon Willison, founder of the Datasette open source project. As Willison puts it, AI “allows me to be more ambitious with my projects.” How so? “ChatGPT (and GitHub Copilot) save me an enormous amount of ‘figuring things out’ time. For everything from writing a for loop in Bash to remembering how to make a cross-domain CORS request in JavaScript—I don’t need to even look things up anymore, I can just prompt it and get the right answer 80% of the time.”
For Willison and other developers, dramatically shortening the “figuring out” process means they can focus more attention on higher-value development rather than low-grade trial and error.
For those concerned about the imperfect code LLMs can generate (or outright falsehoods), Willison says in a podcast not to worry. At least, not to let that worry overwhelm all the productivity gains developers can achieve, anyway. Despite these non-trivial problems, he says, “You can get enormous leaps ahead in productivity and in the ambition of the kinds of projects that you take on if you can accept both things are true at once: It can be flawed and lying and have all of these problems … and it can also be a massive productivity boost.”
The trick is to invest time learning how to manipulate LLMs to make them what you need. Willison stresses, “To get the most value out of them—and to avoid the many traps that they set for the unwary user—you need to spend time with them and work to build an accurate mental model of how they work, what they are capable of, and where they are most likely to go wrong.”
For example, LLMs such as ChatGPT can be useful for generating code, but they can perhaps be even more useful for testing code (including code created by LLMs). This is the point that GitHub developer Jaana Dogan has been making. Again, the trick is to put LLMs to use, rather than just asking the AI to do your job for you and waiting on the beach while it completes the task. LLMs can help a developer with her job, not replace the developer in that job.
“The biggest thing since the World Wide Web”
Sourcegraph developer Steve Yegge is willing to declare, “LLMs aren’t just the biggest change since social, mobile, or cloud—they’re the biggest thing since the World Wide Web. And on the coding front, they’re the biggest thing since IDEs and Stack Overflow, and may well eclipse them both.” Yegge is an exceptional developer, so when he says, “If you’re not pants-peeingly excited and worried about this yet, well … you should be,” it’s time to take LLMs seriously and figure out how to make them useful for ourselves and our companies.
For Yegge, one of the biggest concerns with LLMs and software is also the least persuasive. I, for one, have wrung my hands that developers relying on LLMs still have to take responsibility for the code, which seems problematic given how imperfect the code is that emerges from LLMs.
Except, Yegge says, this is a ridiculous concern, and he’s right:
All you crazy m——s are completely overlooking the fact that software engineering exists as a discipline because you cannot EVER under any circumstances TRUST CODE. That’s why we have reviewers. And linters. And debuggers. And unit tests. And integration tests. And staging environments. And runbooks. And all of … Operational Excellence. And security checkers, and compliance scanners, and on, and on and on! [emphasis in original]
The point, to follow Willison’s argument, isn’t to create pristine code. It’s to save a developer time so that she can spend more time trying to build that pristine code. As Dogan might say, the point is to use LLMs to generate tests and reviews that discover all the flaws in our not-so-pristine code.
Yegge summarizes, “You get the LLM to draft some code for you that’s 80% complete/correct [and] you tweak the last 20% by hand.” That’s a five-times productivity boost. Who doesn’t want that?
The race is on for developers to learn how to query LLMs to build and test code but also to learn how to train LLMs with context (like code samples) to get the best possible outputs. When you get it right, you’ll sound like Higher Ground’s Matt Bateman, gushing “I feel like I got a small army of competent hackers to both do my bidding and to teach me as I go. It’s just pure delight and magic.” This is why AWS and other companies are scrambling to devise ways to enable developers to be more productive with their platforms (feeding training material into the LLMs).
Stop imagining a future without LLM-enabled software development and instead get started today.