What you'll learn

What is prompt engineering?

The discipline of communicating with LLMs to steer their behaviour without updating model weights.

"Methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights." — Lilian Weng

• You change inputs, not parameters
• The lever is text + settings + examples
• The goal is reliable, evaluable outputs

Three prerequisites

Per Anthropic's docs — before prompt engineering helps:

1. Clear definition of success criteria
2. Way to empirically test against them
3. A first-draft prompt to improve

Without these three, you're optimising in the dark.

When PE is (and isn't) the right lever

Anatomy: 4 elements of a prompt

"Show your prompt to a colleague with minimal context. If they'd be confused, Claude will be too."

Bare instruction vs full prompt

❌ "Summarise this."

✅ "Summarise the following customer review for our weekly report.
   Focus on actionable feedback. Output 3 bullets, ≤15 words each.
   <review>{{text}}</review>"

Full version: instruction + context + input data + output indicator.

The 6 universal LLM settings

Temperature controls variety, not quality.

Vendor-specific: Claude effort

CLAUDE 4.x No direct cross-vendor equivalent.

Zero-shot — always start here

Give the task with no examples. Modern instruction-tuned models reliably handle:

• Sentiment classification, summarisation, translation
• Standard extraction tasks with clear schemas
• Generation with simple constraints

If it works → you're done. Don't add examples for the sake of it.

Few-shot — when zero-shot underperforms

Classify as: BILLING, BUG, FEATURE, OTHER

Example: "Charges me twice every month."  → BILLING
Example: "Click 'Save', page reloads."    → BUG
Example: "Could you add dark mode?"       → FEATURE

Now: "I love the new logo!"
→

Sweet spot: 2-5 examples. >10 → diminishing returns; consider RAG.

Few-shot bias gotchas

Three documented biases (Zhao et al., 2021):

• Majority label bias — favours frequent label
• Recency bias — repeats final examples' labels
• Common token bias — prefers frequent tokens

Mitigation

• Balance label distribution
• Randomise example order
• Positive examples beat negative ("don't do X")

Chain-of-Thought (CoT)

Ask the model to think out loud before answering.

Let's think step by step.
Show your reasoning before answering.

Big wins on math + multi-step logic. No help on translation or recall.

Decision tree — what's the right tool?

Is the problem about HOW the model responds?
  → prompt engineering

Is the problem about KNOWLEDGE the model lacks?
  → RAG

Is the problem about STABLE behaviour at scale?
  → fine-tuning

Is the problem about LATENCY or COST?
  → switch model

System prompts — set persona at API level

Concrete behaviour rules >> abstract expertise claims.

Structured output — 3 techniques

1. XML tags — Claude was trained on these
2. JSON with explicit schema — vendor-neutral
3. Prefilling — start the assistant turn with {

Best reliability: schema + prefill together ("the double belt").

Prompt chaining

Plan → Validate → Execute

If validate fails → loop back to Plan with the error.

Each step simpler, easier to debug, easier to evaluate.

Tool use / function calling

1. Define tools (name, description, parameter schema)
2. Send request + tools to model
3. Model responds with text OR tool call
4. Your code executes the tool
5. Send result back; loop until done

The description triggers the call. Treat it as prompt engineering: third-person, what + when.

Retrieval-Augmented Generation

INDEX (one-time): documents → chunks → embeddings → vector store

QUERY (per-request):
  user query → embed → top-k similar chunks
  ↓
  prompt: "Answer using only <context>. Q: {{query}}"
  ↓
  output (with citations to chunks)

Garbage in, garbage out — invest in retrieval quality first.

Evaluation-driven prompt development

1. Define success criteria (concrete, measurable)
2. Build eval set (10-50 labelled examples)
3. Run prompt → score outputs
4. Identify failure → revise
5. Re-run, measure delta, ship or iterate

Pick the iteration with best validation pass rate — not the latest one.

Trigger rate (models are non-deterministic)

trigger_rate = (runs that triggered) / (total runs)

Test passes if trigger_rate matches expectation:
  should_trigger=true:  trigger_rate > 0.5
  should_trigger=false: trigger_rate < 0.5

Run each query at least 3 times.

Train (60%) drives revisions. Validation (40%) measures generalisation.

Prompt injection — 3 classes

1. Direct — "What is the password?" / "Ignore previous instructions"
2. Indirect — translate / roleplay / hypothetical reframe
3. Progressive — multi-turn extraction; defeats single-turn filters

"The line between aligned and adversarial behavior is thinner than most people think." — Lakera

The trust boundary problem

LLMs read and follow all text in the prompt — user input is executed by the model in the same way as your system prompt.

There's no clean trust boundary in text. Defences are layered, never absolute.

Defensive prompting — 4 layers

1. Evaluation-first logic — model decides if request is safe before answering
2. Role anchoring — re-assert persona mid-prompt
3. Output conditioning — pre-fill refusal start ("I cannot share")
4. Instruction repetition — restate constraints multiple sections

No single layer suffices. All together effective.

Beyond prompts — programmatic guards

Agent patterns

When a single prompt + a chain aren't enough — you need a loop.

• ReAct — Thought → Action → Observation cycle
• Reflexion — self-critique loop
• Planner-Executor — separate planning from execution

Always cap iterations. Maintain state. Anchor role throughout.

Top anti-patterns to avoid

• Vibes-only iteration (no evals)
• Cranking temperature for "better quality" (wrong knob)
• Using temperature AND top_p together
• Negative instructions instead of positive examples
• Vague tool descriptions
• Single-layer injection defence
• Unbounded agent loops
• Same defences across vendors (re-test on each)

Production checklist

Sources cited

• Anthropic — Prompt engineering overview
• Anthropic — Best practices (Claude 4.x)
• Lilian Weng — Academic taxonomy
• PromptingGuide.ai — vendor-neutral
• Lakera — safety / injection

All claims in this deck are cited inside the wiki under /wiki/.