What the Claude Code Source Leak Tells Us About Software Engineering

During this long holiday I finally had a chance to sit down and read about the Anthropic code that recently leaked. I’d like to summarize and share what I found, and connect it to the future of software engineering and to studying it, as a lecturer who teaches this subject, and to why it is still necessary.

Quick version. On March 30, Anthropic accidentally leaked the source code of Claude Code, the program you use to talk to code, but through a black developer-style terminal screen. Normally when you write a program in TypeScript or JavaScript, you can distribute it through a site called npm (think of it as an app store for developers). Before uploading, we typically transform and minify the code to reduce file size, for example function calculateContextWindow(tokens) might become just f(t).

This is a problem for developers who need to debug minified code, so a source map is usually created, acting as a map between the minified code and the original source. This file contains the full source code. Normally it isn’t sent to npm; developers add it to a file called .npmignore (which says what files not to upload).

But Anthropic forgot to list the source map file in .npmignore. The code leaked out, more than 500,000 lines of it.

So what did we find? A disclaimer first: I didn’t sit down and read 500,000 lines. When the code leaked, people saved it and built it into an MCP (an added capability for AI) so people can explore it. If you want to try it, add claude mcp add code-explorer -- npx -y claude-code-explorer-mcp, then type in Claude things like “summarize the architecture” or “show me insights.”

Let me give a little context on the AI industry right now. Most of the industry sees a tool like Claude Code as just a middleman that takes a command from a person, passes it to the model, waits for the result, and shows it. As if the model is smart enough, and you just route commands to it and everything’s done, nothing complicated needed. But what turned up in the leaked code has a much more complex structure than expected. There is a memory system, feature flags, multi-agent coordination, and security validators hidden in there. It is not just a simple prompt wrapper. This is sometimes called a harness, or orchestration layer, the layer that controls how the LLM works. (The Thai translation would be something like “สายรัด”, which sounds silly, so I’ll just keep the English word harness.)

Here is what I found in the orchestration layer, in order of complexity. To repeat, what leaked is only the shell of Claude Code, not the AI model they use. (I personally like items 3 and 5 especially.)

(1) Self-healing query loop. Claude Code doesn’t work as a plain request-response (ask and answer with the AI). It works continuously. If the model runs out of output budget in the middle of a task, the system doesn’t show an error to people. It injects an invisible meta-message so the model can continue, or, in worse cases, switches to another model altogether.

(2) Context compaction. The attention mechanism of an LLM has to compute the weight of the relationship between every word and every other word in the context. This scales quadratically: if the context gets 2 times longer, the work goes up 4 times, getting slower and more expensive. So Claude Code has a compaction system that automatically removes low-value text from the context window.

(3) KV cache optimization. This one is subtle and rather elegant: Anthropic sorts the tool list alphabetically every time before sending it to the API. A KV cache (Key-Value cache) is a system that saves what the model has already processed. Imagine the model has to read and understand every available tool before it can answer a question. This step is called prefill, and it takes time. If the tool list is the same on every request, the model has already stored this and can skip the step entirely. But if the tool list is in a random order, even with exactly the same tools, just swapped positions, the stored cache can’t be matched and everything has to be read again. Just sorting the list alphabetically saves a huge amount of time and money in the long run. This way of thinking takes real knowledge of system internals to come up with.

(4) Bash security validators. Before Claude gets to run commands in the terminal, they pass through 23 validators written in more than 2,000 lines. They detect everything from Unicode zero-width space injection (hiding invisible characters to change the meaning of a command) to IFS null-byte injection, which are attack vectors that most people barely know, but without protection, you could be hit with prompt injection.

(5) Anti-distillation flag. This one is very interesting. Anthropic hid a flag called ANTI_DISTILLATION_CC that deliberately puts fake tools into the system prompt, in order to contaminate training data that competitors might capture for model distillation. (Distillation is when you collect the inputs and outputs of a big model and use them as training data to train a smaller model to behave similarly, a method competitor companies commonly use.)

There are also details like model codenames and model development plans, but I’ll leave those out of this post.

What I’d like to invite you to think about next is this. Lately there’s a belief that AI will replace software engineers because the models are smart enough: just vibe code, tell the model what you want, and hit accept. No need to study CS, understand data structures, or know algorithms; the model handles it all. I’m fairly confident this may be true in the future, but not yet.

Let me quickly explain how an LLM thinks and how that differs from people. If you’ve heard of Daniel Kahneman’s “Thinking, Fast and Slow” (2011), there are two systems of thinking. System 1 is fast and automatic, catches patterns, and responds instantly, for example seeing 2+2= and getting 4 without thinking. System 2 is slow and deliberate, reasoning step by step, for example solving a complex logic problem or designing a system you’ve never seen before. An LLM (the AI everyone uses now) is a System 1 that has simply been trained on huge amounts of data.

Last year a research paper described LLMs as working through a process called context-directed extrapolation: using the context they receive to activate patterns learned from training data and combine them, rather than reasoning from first principles (Madabushi, 2025). This explains why LLMs look very smart on tasks similar to their training data but do poorly on truly novel tasks.

There’s also an Apple paper titled “The Illusion of Thinking” (Shojaee, 2025), which tested reasoning models such as o3 and Claude 3.7 on problems of varying complexity and found what’s called a reasoning cliff. (If you took the CP Scholarship AI Specialized Track with me, you’ve probably heard me tell this three times, haha.) Once complexity passes a threshold, performance doesn’t decline gradually but collapses at once. And more surprisingly, as complexity rises, the models actually try less, measured by the number of tokens used for thinking. It’s like a person who meets a hard exam and gives up sooner than usual.

A clear example is the Tower of Hanoi, a disk-moving game that requires planning many steps. With few disks, the models do very well. Once you go up to 8 or more, even when you give the correct algorithm clearly, the models still fail. Not for lack of compute, but because they have no real mechanism for tracking the state of a complex system, unlike humans, who can build a mental model of the system and simulate it in their heads.

Now back to the world of software engineering. One study analyzed how AI tools affect productivity on the job (Dell’Acqua, 2025) and concluded that using AI tools is a jagged frontier: there is no clear line for which kinds of work will improve. If the work is inside AI’s frontier, such as writing, analysis, or summarization with clear patterns, people who use AI work 25.1% faster and get better results. But if the work is outside the frontier, such as novel problems, ones that need specific domain judgment, or decisions of a kind never seen in training data, people who use AI are 19% less likely to solve the problem correctly than people who don’t.

AI doesn’t just fail to help. It makes things worse, because people trust output that looks credible but is wrong. The problem is that this frontier line is invisible to the naked eye. Without the fundamentals to verify results yourself, you can’t tell which tasks are inside the frontier and which are outside. People who have studied CS/CE have more tools to assess this than people with no foundation.

Back to Anthropic’s code. The questions you need to be able to answer before building a production AI system are all outside the LLM’s frontier.

• Why can read tools run in parallel but write tools must run serially? If you don’t know about concurrency and race conditions, you may not be able to answer.
• Why does the context need to be compacted, and how do you compact it without deleting important memory? If you don’t understand the attention mechanism, you may not be able to decide.
• Why does the KV cache fail to match when the tool list is reordered, and how do you stabilize it? If you don’t know system internals, there’s no way you’d think of sorting alphabetically.

No model can answer these questions on its own. Not because the models aren’t smart enough, but because they are outside the LLM’s frontier. They require System 2 thinking: reasoning from first principles about systems that aren’t in the training data.

So the economic moat in the AI era isn’t the model, because models are commoditizing and the differences between frontier models will keep shrinking. The moat is in the people who can build systems, and build the harness for these models.

540 million years ago there was an event called the Cambrian Explosion, in which a great many new life forms appeared at once in a short period, after the environment became favorable. This code leak may be the same kind of trigger for the AI engineering field, because now everyone has a blueprint of the harness Anthropic has been using.

The people who benefit most are those with enough CS/CE fundamentals to read that code, or to understand why Anthropic made those decisions, understand the hidden trade-offs, and then build on it or adapt it to their own use cases. People who just vibe code, wait for AI to generate, and hit accept will only be at the end of the pipe of the people who built the harness for them to use.

AI doesn’t replace software engineers. AI gives software engineers who understand fundamentals the chance to build things far better than before. But they have to build a good enough harness first, and that harness needs real computer science, not just the ability to prompt. As a closing note, though, software engineers who can do only middling system architecture may also be left behind, if they haven’t learned what AI can do and how it can help write code.

What do you get from studying CS or computer engineering? You get the foundation that tells you what you’re building, what to ask, and when AI is taking you out of the jagged frontier without your knowing. No model can replace those three things.

Pai

References

• Shojaee et al. (2025). “The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity.” Apple Machine Learning Research. arXiv:2506.06941
• Madabushi et al. (2025). “Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors.” arXiv:2505.23323
• Kahneman, D. (2011). “Thinking, Fast and Slow”.
• Dell’Acqua et al. (2025). “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality.” Organization Science, 36(1)