This is an extract from On Program Synthesis and Large Language Models - Why it is unlikely new developments in machine intelligence will eventually make programming obsolete by Hans Hüttel (published in Communications of the ACM, January 2025 - Vol. 68 No. 1).

The article contains further details and is well worth reading.

Premise

Recent advancements in AI, particularly large language models (LLMs), have led some experts to claim that traditional programming will become obsolete. Matt Welsh argues that AI will replace conventional programming with trained models and, for simpler cases, AI will automatically generate code.

However, this claim is controversial. Complex software like operating systems, game engines and SaaS platforms are unlikely to be entirely replaced by AI. Moreover, generating correct code from a specification is a well-known challenge in computer science. Theoretical results show that program synthesis is computationally difficult and requires significant resources.

Programming in English?

The idea that English could become the language of programming is not new. In the 1950s, people believed FORTRAN would eliminate the need to write code, but this never happened. Today, some suggest that AI tools like ChatGPT enable programming in natural language, but human languages are inherently ambiguous and require precise syntax for effective communication with AI.

Challenges in generating correct code

The core issue of generating code from a specification, known as program synthesis, involves finding a program that meets specific constraints. Theoretical results classify this problem as PSPACE-complete, meaning it requires an exponential number of steps relative to the input size. As a result, LLM-based tools cannot reliably produce correct code.

AI as a Tool, Not a Replacement

While LLMs can assist programmers by suggesting code and improving productivity, they do not signal the end of programming. AI-based approaches have well-documented limitations and should be seen as complementary rather than as a replacement for traditional programming.

The true value of natural language in software development lies in facilitating discussions between developers and users to refine software requirements. AI can assist in this process, but the challenge of ensuring correct code remains unresolved.