Prompt

A language model receives everything a caller wants it to know — its role, the conversation, documents, tool results — in a single context window. That input is the prompt. Some vendor docs use the word more narrowly: OpenAI’s text generation guide calls the input parameter the prompt, as distinct from instructions.

How a prompt works

OpenAI’s 2023 ChatML preview showed the structure plainly. A conversation is one token sequence; each message opens with a special delimiter token, followed by the role name (“system”, “user”, “assistant”) as ordinary text. In the structured form, user text cannot forge those delimiters. The document warns that a raw-string form would allow injections “similar to SQL injections”. Anthropic’s Messages API marks the line at the API instead: the system prompt goes in a top-level system parameter, not in the message list.

The marks tell the model where each piece of text came from. Whether it acts on that is down to training, and the same ChatML preview admits its models had seen few system messages and paid “much more attention to user examples”. The delimiters are protected; the authority is not.

The researchers who introduced the instruction hierarchy argue that models often treat a system prompt as the same priority as text from untrusted users. Their remedy is more training, not a runtime check.

Why it matters

Because the marks in a prompt record where text came from but cannot make the model obey it, any piece of text — a retrieved document, a tool result, a user’s message — can act as an instruction. That is a consequence of the design, and it is the condition that makes prompt injection possible.

Model settings shape the output, but the text you write is the main lever. Writing prompts that consistently produce the desired output is a craft skill, and the field studying it is called prompt engineering.

Questions and answers

What is a prompt in AI?

A prompt is the complete input sent to a language model in one turn. It can contain instructions, conversation history, documents, and tool results, all arriving as one sequence of tokens the model reads from start to finish.

What is the difference between a prompt and a system prompt?

A system prompt is one section within a prompt — the part written by the application developer and placed before the conversation begins. The full prompt also includes the user's message, any prior conversation, and tool results. The system section is marked as such, but the model decides how much weight to give it.

Why does prompting matter as a skill?

Because a language model has no way to deduce intent from an ambiguous input, the words chosen and the order they appear in significantly affect what comes back. Prompting is the practice of writing inputs that consistently produce useful, accurate outputs.