Personalized Software

AI will definitely revolutionize the way we build and use software, but in which ways? I will share my thoughts based on some recent observations and my own experiences.

Here are two facts about the current state of AI coding agents:

They have the ability to finish a project end to end without human intervention (only a minimal initial prompt is needed with the help of Ralph Loop), but we need to grant them the permission to do arbitrary operations to maximize their capabilities. Claude’s C Compiler (CCC) is a good example of how capable these agents are — they can build a complete C compiler from scratch (not really because they use existing test suits as a very good source of oracle but still impressive) and it can compile some real world C projects like sqlite.
That said, projects fully automated by AI agents can have bad design choices and they are hard to reason about and maintain. Basically, they are not readable and no one really wants to read them. CCC also shows evidence on this point: CCC-compiled sqlite can be 158,000x slower in some cases and the author also mentioned that it’s really hard to add any new features to it as every new change will break the existing code like a whack-a-mole.

Based on these two points, it’s fair to say that fully automated coding agents are well suited for cases where functionality is more important than maintainability like personal use or single-purpose projects.

Safety is a big concern even in those personal use cases because it will interfere with other things live on that computer, and the implication is twofold here:

The agent itself should not be trusted — we do not know which operations it will perform.
The code generated by agents should not be trusted — we do not know what the code is doing under the hood because no one reviews it.

Both the agent and the code generated by it should be sandboxed to let use harness the power of it without worrying.

As tokens become cheaper and sandboxing techniques mature, I envision the following pictures:

Everyone can quite easily create, share, customize and iterate their own personalized application software with only a few words. The applications do not need to be bug-free (it never has been though), it only needs to be functional enough to solve problems at hand. Bugs can be fixed on the fly whenever users encounter them.
Infrastructure code that is meant for wide production usage will still be maintained and reviewed by humans with the help of AI. Specifically, AI can play an important role in verifying the correctness of the code.
We need a secure API layer for the sandboxed personalized software to interact with the environment (e.g., persistence, networking, etc.).