Context Engineering explained in the simplest way possible 👇

Context engineering is basically giving the LLM only the right information it needs.

When LLMs get too much context, they start to hallucinate and forget important details (even system prompts).

Our job as AI engineers is to fit as much useful information as possible into the smallest number of tokens.

Here are some ways to do this (just examples, there are more! The last one is especially creative):

1. Trimming noise from the context

When we get returns from API calls, they often look like this:

{
  "tags": [
    {
      "name": "v1.2.3",
      "commit": "abc123",
      "date": "2024-03-15T10:00:00Z"
    },
    {
      "name": "v1.2.2",
      "commit": "def456",
      "date": "2024-03-14T15:30:00Z"
    }
  ]
}

This response contains a lot of extra characters (noise). An easy fix is to remove characters like \, {}, etc., to save tokens.

2. Using Sub-LLMs

Imagine we have a Coding Agent, and a user asks to create a new feature. First, we need a detailed plan. Instead of making the main agent think through everything, we delegate planning to a separate agent. The main coding agent doesn’t need to know how the plan was made—it just needs the finished plan.

3. Modifying memory

Tool calls use a lot of tokens. Each call involves four steps:

1. User prompt
2. AI initiates the tool call
3. The tool returns its response
4. AI sends the final reply

After this is done, all the details aren’t usually important anymore.

Instead of passing back the full four-step process each time, we simplify it into XML tags like this:

Here's everything that happened so far:

<slack_message>
  From: @alex
  Channel: #deployments
  Text: Can you deploy the backend?
</slack_message>

<list_git_tags>
  intent: "list_git_tags"
</list_git_tags>

<list_git_tags_result>
  tags:
  - name: "v1.2.3"
    commit: "abc123"
    date: "2024-03-15T10:00:00Z"
  - name: "v1.2.2"
    commit: "def456"
    date: "2024-03-14T15:30:00Z"
  - name: "v1.2.1"
    commit: "ghi789"
    date: "2024-03-13T09:15:00Z"
</list_git_tags_result>

The difference here is we’re sending just one simplified snapshot instead of all four steps.

We also avoid passing unnecessary information like tool call IDs and parameters that don’t matter anymore, keeping the context shorter.

Conclusion

Context engineering isn’t only about saving tokens, it’s about making sure your LLM stays clear and accurate in the least amount of tokens as possible.