Prompt engineering has been marketed as a new kind of technical skill, somewhere between programming and psychology. Job postings offer six figures for it. Courses sell it as a profession. The framing is that you’re crafting inputs to coax optimal outputs from a language model, like a skilled artisan working with a new medium.
That’s flattering but wrong. What you’re actually doing is debugging a system whose internals are invisible to you, whose behavior isn’t deterministic, and whose failure modes have no stack trace. It’s debugging with the logs turned off.
The System Has State You Cannot Inspect
When a traditional program misbehaves, you have recourse. You can add logging, inspect variables, step through execution, and read the code. The bug exists somewhere in a finite, readable space. You can find it because you can, in principle, see everything the system sees.
With a large language model, none of that applies. The “state” of the system during inference is billions of floating-point weights shaped by training data you didn’t choose, plus whatever the model has encoded about how those weights interact. When GPT-4 gives you a confident but wrong answer, there is no line of code to blame. The failure is distributed across a structure so large that even the people who built it can’t tell you exactly why a specific output occurred.
Prompt engineering is the practice of poking this system from the outside and inferring its internal logic from behavioral observations. That’s debugging. Specifically, it’s the most frustrating kind: debugging without source access, the kind where you’re inferring why a bug existed from its effects alone.
Techniques Are Empirical, Not Principled
The canonical prompt engineering techniques, chain-of-thought prompting, few-shot examples, role assignment, structured output formatting, were not derived from a theory of how language models work. They were discovered by noticing that certain inputs produced better outputs and then reverse-engineering a story about why.
Chain-of-thought prompting, described in a 2022 paper from Google Brain, improved model performance on reasoning tasks. The explanation offered was intuitive: asking the model to reason step-by-step before answering gives it more “space” to work through the problem. That’s plausible but it’s not a mechanistic explanation. It’s a behavioral observation dressed up as a theory.
This is how debugging without source access always works. You find a workaround, you develop a mental model for why it helps, and you use that model to generate more hypotheses. The model might be completely wrong about the underlying mechanism and still be useful for generating workarounds. That’s fine for debugging. It’s a thin foundation for a discipline.
Prompt Sensitivity Is the Equivalent of a Heisenbug
One of the stranger properties of language models is that semantically identical prompts can produce meaningfully different outputs. Changing “think carefully” to “think step by step” shifts performance on some benchmarks. Adding a trailing space changes outputs. Telling the model it’s an expert in something nudges it toward different responses than asking the same question without that framing.
This behavior has a name in traditional debugging: a Heisenbug, a bug that changes behavior when you try to observe or modify it. The bug that disappears when you add logging is the classic form. Prompt sensitivity is the LLM equivalent. The act of adding a clarifying phrase to your prompt is itself an intervention that changes the system’s behavior in ways you didn’t fully intend and can’t fully predict.
Engineers who’ve spent time doing serious prompt work describe a familiar frustration: a prompt that works reliably in testing starts drifting in production, or works well on one model version and breaks on the next. There’s no diff to look at. The system changed underneath you.
The Output Has No Error Code
Traditional debugging gives you structured failure signals. An exception has a type and a message. A failed HTTP request returns a status code. Even a crashed program leaves a core dump. These signals are imperfect, but they constrain the problem space.
Language models fail silently and confidently. The model doesn’t know when it’s wrong, which means it can’t tell you, which means you’re reading the output and making your own judgment call. As we’ve covered, your LLM has no idea when it’s wrong, and that uncertainty propagates to everyone evaluating its outputs. Debugging a system that never throws an exception is much harder than debugging one that does.
This shapes what prompt engineering actually involves in practice: writing evaluations, running outputs through rubrics, comparing generations across prompt variants, and slowly building a test suite that approximates ground truth. That’s not a creative writing process. It’s QA work on an opaque system.
The Counterargument
The pushback here is that calling prompt engineering “just debugging” undersells the genuine skill involved. And there’s something to that. Developing accurate mental models of a system’s behavior, even from the outside, is hard and valuable. The people who are good at it aren’t just guessing randomly. They’ve built real intuitions about how models respond to framing, specificity, persona assignment, and decomposition.
But this objection doesn’t challenge the core claim. Good debuggers also build accurate mental models from behavioral observation. Skilled reverse engineers reconstruct internal logic without source access. These are legitimate and difficult skills. Calling them what they are doesn’t diminish them.
The problem with the “prompt engineering as new discipline” framing isn’t that it overstates the difficulty. It’s that it obscures the nature of the work, which makes it harder to improve systematically. If you think you’re crafting prompts, you iterate on craft intuitions. If you understand you’re debugging an opaque system, you reach for testing frameworks, behavioral characterization, and systematic evaluation. The second approach scales. The first mostly doesn’t.
The Position, Restated
Prompt engineering is real work that requires real skill. But dressing it up as something categorically new obscures what’s actually happening: you are debugging a complex system from the outside, without source access, with non-deterministic behavior and silent failure modes. The sooner practitioners adopt that framing, the sooner the field develops tools and methods that match the actual problem. The aesthetic of prompt craftsmanship is fine. Just don’t let it crowd out the engineering.
