An LLM Tooling Guide for Skeptical Developers

Programming has changed a lot in the last couple of years. Jobs feel tighter, everything is moving fast, and now apparently even my grandma can ship a pretty complex iOS to-do app.

Still, beyond all the panic this can trigger in “real” developers, these tools are also genuinely useful. They can help us build faster, explore ideas faster, and sometimes understand code faster too. And if you are anything like me, and spent a while resisting this whole wave because software felt like a craft, you might feel a bit behind. Or maybe you are already using these tools, but not really sure how they work.

This post is not a rant about machines doing our thinking for us. I want to understand both how LLMs work, including their limitations, and how these new tools are built and used.

My goal here is simple: build a practical mental map of the LLM tooling stack, from the models themselves all the way to prompts, tools, agents, MCP, skills, and retrieval.

Part 1: The Model

This first part is more theoretical, but it matters because the rest of the tooling stack is mostly built around the strengths and limitations of the model itself. If you are already familiar with how LLMs work, you can skim this section and jump to Part 2.

LLM Basics

Natural Language Processing, or NLP, has been with us for a long time, and it was mostly about combining text and machine learning. It has not always been tied to deep learning or neural networks until recently. Its original goal was not to generate text. It was to automate manual tasks that required understanding text, for example translation or text classification.

Research goes on and we arrive at Large Language Models, or LLMs, today’s subject. Before modern GPT-style LLMs, one important architecture was the encoder-decoder model, whose goal was to translate from a source language to a target language.

The encoder learns the source sequence and sends that information to the decoder. The decoder then predicts the most likely matching sequence in the target language one token at a time.

But it was found that using only the decoder can also work for predicting the next token in a sequence. That lets us remove the source language and start generating grammatically correct sentences. This is what GPT-style LLMs are based on. What do we do next? Increase the size of the model and add more parameters. It went from GPT-1 with 117 million to GPT-3 with 175 billion. With a greater number of parameters, the model can start storing not only information about the grammar of language but also facts and opinions.

This is known as the transformer architecture, which was proposed in the paper “Attention is All You Need”, published by Google in 2017 and implemented in every major LLM today. Here is the diagram used in the paper:

Another concept worth mentioning here is embeddings. Embeddings are vector representations of text or other data. Attention is different: it is the operation the transformer performs on those representations to understand how tokens relate to one another. So embeddings are the representation, and attention is what the model does with that representation. Embeddings tell the model what each token is. Attention helps the model decide how tokens influence one another.

If you did not understand any of what I attempted to explain, stick with this: LLMs are a black box where we send words and it gives us the most probable word that comes next. That matters because we created an algorithm that learned the structure of a huge amount of human knowledge.

https://www.0xkato.xyz/how-llms-actually-work/

Training

Since models are trained on mostly crawled data from the internet, and probably other copyrighted sources, the parameters of the model can be thought of as a lossy compression of all that data. Once we finish training, what we get are document generators, which are useful but not fit for every task. What we want is something closer to an assistant that can answer our questions.

In order to get our assistant we divide the training into two stages:

• Stage 1: Train it with a lot of text, this gives us our base model which understands how text works. This can cost millions since we need several days and thousands of GPUs. We have been discussing this type of training.
• Stage 2: Fine tune it, also known as alignment, this gives us our “assistant” model. After we finish stage 1, we swap the training dataset for a new one that was labeled by humans, preferably experts. This labeled data can be formed as questions and answers so the model better understands what new structure should be followed. This is usually much cheaper than stage 1 and can be done frequently. At this stage we prefer quality over quantity.

Now you can see why chat was the first interface we implemented to interact with LLMs. The model was explicitly trained to respond to prompts in that format.

During training, the model learns the embedding matrix and the attention layer parameters. During inference, it uses those learned parameters to look up embeddings and compute attention for the current input. This is why embeddings matter not only for models themselves, but also later on in search, retrieval, recommendations, and RAG systems.

Another common approach is to compare two answers and ask a human to choose the better one, using that preference for fine tuning. This is known as Reinforcement Learning from Human Feedback or RLHF. But nowadays labeling has been done collaboratively between humans and assistant models.

Once training is complete, we have a file containing all parameters that can be loaded into the model so we can start asking questions.

And lastly, once training stops, the model cannot directly absorb new knowledge into its parameters. That is why models have cutoff dates, which roughly mark how recent their training data is. They can go on the web and search for new information, but that data will not be in the parameters themselves. Rather, it will be an interpretation of what they found.

Tokens

Now that we understand what LLMs are and how they are made, let’s dive into some of the limitations.

If you understand what the model can and cannot do on its own, the rest of the stack starts to make much more sense.

When LLMs are predicting the next word in a sequence, we do not actually spill out the word, we return a token. The token is the fundamental building block that LLMs understand for writing and reading text. A token can be a word, part of a word, a punctuation mark, or even a single character. You can use this page to see how an LLM could create tokens based on the text it is fed.

Here is an example of the tokenizer on GPT-5.

And here is another on GPT-3.

I used Lorem Ipsum for the tokenizer, a placeholder text in Latin. You can see that it detects more whole words with GPT-5. This is because the vocabulary increased and we can find more meaning in the relationships within those models.

The reason why we do this is that AI models, in general, do not deal with raw text, but rather numbers, and breaking text into smaller chunks creates a sort of standardized vocabulary of mathematical IDs. This is how we charge money for the models by the usage of tokens.

The attention technique defined in “Attention is All You Need” is a way for LLMs to understand the meaning and relationship between tokens. In order to do this, we need to compare each word with every other one, having a complexity of O(N^2). This means that those comparisons require storing data proportional to N^2 in GPU memory. Because attention gets expensive as sequences grow, models can only process a limited number of tokens at once. That token limit is what we call the context window.

You can picture it like this:

Consequently, whenever we use an application like ChatGPT, the model is not only receiving our new message but rather our whole conversation to process our latest request.

Conceptually, context windows are the memory of an LLM, they represent the “knowledge” an LLM has at a given moment. The main problem is that context windows have a fixed size, so what should we do when we are about to reach maximum capacity without losing what the LLM already knows? Here is where prompt and context engineering come into play, which we will be talking about later on.

Model Types

One key distinction is between training and inference. Training goes through a lot of data and generates the parameters of a neural network that can be used to predict text, while inference is the act of using those frozen parameters to predict text. Training shapes what the model can do while inference is the model doing it.

This concept is shared by almost every machine learning model, but we need to discuss it here because LLMs blur the line. Giving context to an LLM can feel a bit like training a model.

So we have already talked about base models, which are the document generators, and assistant models, which can answer questions, but nowadays we can find reasoning models, which have a lot of things happening in the background.

Reasoning models use internal tokens known as reasoning tokens before answering requests. This helps the model check a lot of options and paths for creating better answers or solutions. These were created to tackle more complex tasks like coding or research. You need to use these models in a reasonable way since we are also being charged for the reasoning tokens, which we do not have access to. We can see how many we used but not what they actually did. Most models already let you choose how high you want the reasoning to be, and the higher it is, the more tokens it consumes.

At a high level, the flow looks like this:

The “thinking” reasoning models create is just more inference produced by the same mechanism. This kind of behavior comes from how these models are trained and tuned, often with techniques that reward better final answers on complex tasks. In practice, the models learn that generating reasoning steps before answering can lead to better outcomes.

There are also multimodal models, which are not limited to text for input or output. Depending on the model, they can also work with images, audio, or video. This is done by converting those formats into tokens too so the model can also process them.

If you want to learn more about LLMs, I heavily recommend checking this playlist by Andrej Karpathy.

Prompting

Prompts are what we tell an LLM. They usually contain instructions and context so a task can be achieved. We call prompt engineering the practice of optimizing prompts to efficiently use LLMs in applications.

The elements of a prompt are the following:

• Instructions: What we want the prompt to achieve
• Context: What extra information it needs to achieve
• Input data: The way the user is going to interact with the LLM
• Output indicator: A mark to tell the LLM where to write the output

In chat-based APIs, prompts are usually structured as messages with roles instead of one raw block of text. These roles exist because not all text in the prompt should be treated the same way.

• System: Sets high-level behavior and constraints for the model. This is where persistent instructions usually live.
• User: Contains the actual request or question from the user.
• Assistant: Contains previous model responses, which helps preserve continuity across the conversation.
• Tool: Contains outputs from tools or functions, so the model can treat them as external results instead of normal dialogue.

With these elements in mind, we can create a simplified assistant-style prompt, like the kind used in tools such as ChatGPT (without a lot of capabilities):

[Instructions]
You are ChatGPT, a helpful assistant.
Answer clearly and briefly.
If the user asks for code, provide a minimal working example.
If you are unsure about a fact, say so.

[Context]
The user is working on a Python web app and prefers short explanations.

[Input data]
User: How do I read a JSON file in Python?

[Output indicator]
Assistant:

In order to make it behave like it does in production, we would likely need to create a prompt that follows its spec on how an LLM should behave.

You can configure how deterministic the LLM can be with two parameters:

• Temperature: Controls how sharp or flat the token probabilities are, making the output more deterministic or more random.
• Top P: Limits sampling to the smallest set of likely tokens whose total probability mass reaches the chosen value.

Now that we understand what prompting is, let’s briefly explain two simple and popular techniques.

• Few-shot prompts: As simple as just providing examples of how it should behave

Classify the sentiment of the review as positive, neutral, or negative.

Review: "The battery lasts all day and the screen looks great."
Sentiment: positive

Review: "It works fine, but nothing really stands out."
Sentiment: neutral

Review: "It stopped working after two days."
Sentiment: negative

Review: "The camera is decent, but the app crashes a lot."
Sentiment:

• Chain-of-thought (CoT): Also provide examples, but you must provide some reasoning before showing the answer.

Solve the problem and show the reasoning before the final answer.

Question: A shirt costs 20 dollars and has a 25 percent discount. What is the final price?
Reasoning: 25 percent of 20 is 5. Subtract 5 from 20.
Answer: 15 dollars

Question: A laptop costs 1000 dollars and has a 10 percent discount. What is the final price?
Reasoning:

It has its own variant called Zero-shot CoT, where no examples are provided and you just add “Let’s think step by step” at the end. This can encourage more step-by-step reasoning behavior in some models.

This also helps explain why prompting matters so much for reasoning models. Many of the behaviors we associate with reasoning can be encouraged through prompting, even though reasoning models are also specifically trained and tuned to do this more reliably. That said, modern reasoning models often do not need explicit chain-of-thought prompting and can work better with simpler instructions.

At the end of the day, prompting is about giving the model the right structure to solve a task. Sometimes that means clearer instructions, sometimes it means examples, and sometimes it means better context. That naturally leads us to context management, which is the problem of deciding what information the model should actually see.

I highly recommend checking the Prompting Guide for more techniques or watch Elvis Saravia’s Overview.

Context Management

When we use an LLM for a long time, we fill the context window. At that point, no more “knowledge” can be added to the current session, so we need to find a way to reduce the context in order to keep using the session and lose the least amount of information.

The strategy can vary a lot depending on what kind of application is using the LLM, usually a chat interface or an agent.

Some of the most common techniques for context management are:

• Truncation: Drop the oldest messages or least relevant context
• Summarization: Compress earlier conversation into a shorter memory
• Retrieval: Only bring back the pieces of information needed for the current turn

In practice, most systems combine these techniques instead of relying on only one of them.

I found some talks about how companies have been solving context management, but these are more related to agents, like The Engineering of AI Agents, How we solved Context Management in Agents, and Agentic Search. You can also check LangChain’s docs.

Part 2: Giving Models Capabilities

This is the practical core of the post. Here we move from the model itself to the systems built around it, with code examples for tools, agents, MCP, skills, and plugins. You can find the fully working code discussed here in this repo.

Tools and Function Calling

Tool calling is what lets an LLM stop being just a text generator and start interacting with real systems.

Tools are capabilities that the model can use to do things outside of the model itself. This allows the LLM to search the web, read a file, query a database, or communicate with a REST API.

Function calling is the most common way to expose those capabilities to a model. Your application defines the tool, the model asks to use it, your application executes it, and then the model answers with the result.

If you want to develop a custom tool, for example one that gets the current time, you need to do the following:

1. Write a script that gets the current time
2. Define its inputs and output clearly
3. Expose it to the model
4. Handle the model’s request to run it

A tool usually starts as a normal function in your app. The extra work is describing it to the model and handling the call safely. The model does not execute the function itself. It only asks your application to run it.

At a high level, this is the interaction we are implementing:

Here is a minimal example in Python using the OpenAI SDK to add the capability of looking up the time in a specific timezone:

def get_current_time(timezone: str = "UTC") -> dict:
    now = datetime.now(ZoneInfo(timezone))
    return {
        "timezone": timezone,
        "current_time": now.isoformat(),
    }

That is just a normal function in our app. The next step is exposing it to the model as a tool:

tools = [
    {
        "type": "function",
        "name": "get_current_time",
        "description": "Get the current time for a given timezone.",
        "parameters": {
            "type": "object",
            "properties": {
                "timezone": {
                    "type": "string",
                    "description": "An IANA timezone like America/Mexico_City or UTC",
                }
            },
            "required": ["timezone"],
            "additionalProperties": False,
        },
    }
]

This part is only metadata. It tells the model the tool name, what it does, and which arguments it expects.

Now we send the user request and the tool definitions to the model:

first_response = client.responses.create(
    model="gpt-5",
    input="What time is it in Mexico City?",
    tools=tools,
)

tool_call = None
for item in first_response.output:
    if item.type == "function_call":
        tool_call = item
        break

The first response is usually not the final answer. If the model wants to use a tool, it returns a function_call telling our app which tool to run and with which arguments.

Then our application executes the real function and sends the result back:

if tool_call and tool_call.name == "get_current_time":
    args = json.loads(tool_call.arguments)
    result = get_current_time(args["timezone"])

    second_response = client.responses.create(
        model="gpt-5",
        previous_response_id=first_response.id,
        input=[
            {
                "type": "function_call_output",
                "call_id": tool_call.call_id,
                "output": json.dumps(result),
            }
        ],
    )

    print(second_response.output_text)

So the loop is:

1. our app defines the function
2. our app describes it as a tool
3. the model asks to use it
4. our app runs it
5. the model answers with the result

The reason there is a first and second response is that the model does not execute the tool by itself. The first response is the model asking for the tool, and the second response is the model answering after our application gives it the tool output.

The diagram above is a simplified version of the flow, so it does not match the code exactly. In practice, your application sends both the user prompt and the tool definitions to the model, and later sends back a function_call_output item rather than just a generic tool result.

If you are a bit more curious, you can check out how the older local shell tool works. You can also read more about tools here.

The example I provided is one way to define a tool. It is known as function calling, and it is the most common way we integrate other services with a model. OpenAI also provides other kinds of tools that do not follow the same schema, such as web search, shell, and file search.

Be aware that https://www.quandri.io/engineering-blog/mcp-is-dead TBD

We cannot create native built-in tool types of our own, but we can create custom function tools, a custom free text tool, or a MCP server.

Agents and Workflows

Workflows were already a thing before LLMs came along. In essence, a workflow is just a sequence of steps to complete a task, usually consisting of:

1. Get a request
2. Fetch data
3. Process it
4. Return it

We can find platforms like Zapier, IFTTT, and AWS Step Functions that were already pretty famous. We used them for marketing automation, data pipelines, integrations, and customer support. But after the AI explosion, we started integrating LLMs, so the workflow process looks something like this now:

1. Get a request
2. Choose a tool
3. Run the tool
4. Process the result
5. Reply

Now with these new capabilities, we can build AI for customer support agents, assistants, document processing pipelines, and meeting summaries.

The new buzzword is agents. An agent is a system built around a model that can reason through a task, use tools, keep state, and decide what to do next. This is much more autonomous than a workflow, since it can keep going until it thinks the task is finished or needs input from the user.

A workflow is the process. An agent is the model-driven system moving through that process.

The main difference from workflows is that this is not only a sequence of steps. It moves through those steps and makes decisions along the way. So let us implement one to understand it better.

We will be implementing an agent that investigates why the latest deployment failed and tells us what likely caused it. The possible behavior would be:

1. Read the request
2. Check deployment logs
3. Inspect recent commits
4. Look at test failures
5. Maybe inspect environment variables
6. Form a hypothesis
7. Summarize likely cause

Here is a minimal example in Python:

def get_deployment_logs() -> str:
    return """Deploy failed at step: migrate_database
Error: relation "users" does not exist
Service: api
Environment: production"""


def get_recent_commits() -> str:
    return """a1b2c3 Add startup migration for billing tables
d4e5f6 Rename users table access in API layer
g7h8i9 Update deployment config"""


def get_test_failures() -> str:
    return """No failing tests in the latest CI run.
Unit tests: passed
Integration tests: passed"""


def get_environment_summary() -> str:
    return """DATABASE_URL is set
Migration step runs before API boot
Latest release uses a fresh production image"""

Those are the “tools” our agent can use. In a real system they would call logs, git, CI, or deployment APIs. Here they just return text so we can focus on the agent loop itself.

tools = [
    {
        "type": "function",
        "name": "get_deployment_logs",
        "description": "Get the latest deployment logs as plain text.",
        "parameters": {
            "type": "object",
            "properties": {},
            "required": [],
            "additionalProperties": False,
        },
    },
    {
        "type": "function",
        "name": "get_recent_commits",
        "description": "Get recent commits related to the latest deployment as plain text.",
        "parameters": {
            "type": "object",
            "properties": {},
            "required": [],
            "additionalProperties": False,
        },
    },
]

Just like before, these are only the tool definitions. Now we send the task to the model:

response = client.responses.create(
    model="gpt-5",
    input=(
        "Investigate why the latest deployment failed. "
        "Use the available tools, form a hypothesis, and summarize the likely cause."
    ),
    tools=tools,
)

At this point the model may ask for one tool, several tools, or none at all. The agent loop is what keeps the process going until there are no more tool calls.

while True:
    tool_calls = [item for item in response.output if item.type == "function_call"]

    if not tool_calls:
        print(response.output_text)
        break

    tool_outputs = []
    for tool_call in tool_calls:
        output = run_tool(tool_call.name, {})
        tool_outputs.append(
            {
                "type": "function_call_output",
                "call_id": tool_call.call_id,
                "output": output,
            }
        )

    response = client.responses.create(
        model="gpt-5",
        previous_response_id=response.id,
        input=tool_outputs,
        tools=tools,
    )

And this tiny helper is what maps a tool name to the real implementation:

def run_tool(name: str, _arguments: dict) -> str:
    if name == "get_deployment_logs":
        return get_deployment_logs()
    if name == "get_recent_commits":
        return get_recent_commits()
    if name == "get_test_failures":
        return get_test_failures()
    if name == "get_environment_summary":
        return get_environment_summary()
    raise ValueError(f"Unknown tool: {name}")

This is where agents start to differ from a single tool call example. Instead of asking for one tool and then answering, the model can keep going through several rounds of tool use until it has enough information to produce a final answer.

We have covered the basics of how to write an agent, but you should also read this very good blog post by fly.io.

Model Context Protocol

Protocols are a set of rules for how two systems communicate. Without protocols, the internet would never work. A protocol is a specification for how you should send and receive messages.

For the same reason, MCP was invented to standardize the way we connect models to external capabilities. It differs from function calling by providing a standard way to expose capabilities between systems, like an editor, a CLI, chat interfaces, and coding assistants.

The participants of MCP are:

• MCP Host: The AI application that coordinates and manages MCP connections.
• MCP Client: Maintains a connection to an MCP server and obtains context from that server for the MCP host to use
• MCP Server: The program that serves context data, regardless of where it runs

The MCP primitives define what clients and servers can offer each other:

• Tools: Executable functions that AI applications can invoke to perform actions.
• Resources: Data sources that provide contextual information to AI applications.
• Prompts: Reusable templates that help structure interactions with language models.

The communication happens through JSON RPC, which is mostly JSON with a few extra fields. An MCP server can run locally or remotely. There is no need to go too deep into that here.

To learn more about the spec, check this.

For now, let us implement an MCP server and client that allow an LLM to consult internal documentation stored in markdown files.

I will use the official Python SDK and its FastMCP helper. We will expose local markdown files as resources and also add a small search tool.

First, we load the markdown files from a local docs/ directory:

from pathlib import Path

from mcp.server.fastmcp import FastMCP


BASE_DIR = Path(__file__).parent
DOCS_DIR = BASE_DIR / "docs"
mcp = FastMCP("internal-docs")


def load_docs() -> dict[str, str]:
    docs: dict[str, str] = {}
    for path in DOCS_DIR.glob("*.md"):
        docs[path.stem] = path.read_text(encoding="utf-8")
    return docs


DOCS = load_docs()

Now we expose those files as resources. This lets an MCP client read them through a standard resource URI:

@mcp.resource("docs://{name}")
def get_doc(name: str) -> str:
    return DOCS.get(name, "Document not found.")

Then we add a tool that searches those same docs and returns matching text:

@mcp.tool()
def search_docs(query: str) -> str:
    query = query.lower()
    matches: list[str] = []

    for name, content in DOCS.items():
        if query in content.lower() or query in name.lower():
            matches.append(f"# {name}\n\n{content}")

    if not matches:
        return "No matching documents found."

    return "\n\n---\n\n".join(matches)

That is the server side. The official Python SDK is async-first, so the client example is wrapped in a small main() function:

import anyio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client


server_params = StdioServerParameters(
    command="python3",
    args=["server.py"],
)


async def main() -> None:
    async with stdio_client(server_params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()

            resources = await session.list_resources()
            tools = await session.list_tools()
            refund_doc = await session.read_resource("docs://refunds")
            search_result = await session.call_tool(
                "search_docs",
                arguments={"query": "refund"},
            )

So the client can discover resources, list tools, read a document, and call search_docs through the same protocol. This example shows the two most important MCP ideas. Resources expose information the model can read, and tools expose actions the model can call. In this case both happen to be backed by the same local markdown files.

Skills

A skill is a reusable set of instructions that tells an agent how to approach a task. Basically, it is guidance for using capabilities well.

In the Agent Skills format, a skill is a directory with a file named SKILL.md that has three elements:

• name: Short identifier for the skill
• description: Tells the agent when to use this skill
• body: Instructions the agent follows when the skill activates

Alongside SKILL.md, we can find other directories named:

• scripts: Executable code
• references: Documentation
• assets: Templates or resources

You can check the Agent Skills open standard here.

For now, let’s implement a skill that guides our agent on how to use the MCP server we just built. It should teach the agent when to search the docs, how to choose the right resource, and how to summarize the result.

At its core, the skill is just a markdown file with reusable instructions:

---
name: use-internal-docs
description: Use this skill when you need to answer questions from internal documentation exposed by the MCP docs server.
---

# Use Internal Docs

Use the internal docs MCP server whenever the task depends on product policy, onboarding steps, or any other internal documentation.

## Workflow

1. Start with `search_docs` using a short query built from the user's request.
2. If a result looks relevant, read the matching resource directly.
3. Prefer exact policy language over guesses.
4. If multiple docs match, compare them before answering.
5. If nothing relevant is found, say so clearly instead of inventing an answer.

This is why skills are guidance instead of capability. The MCP server already exposes the docs and the search tool. The skill simply teaches the agent when to use them and how to behave once it has the results.

If you want to go deeper on agent skills, check out this paper. Also check out this skill for UI.

Plugins

A plugin is a packaged extension that bundles things like tools, MCP servers, skills, configuration, and metadata into something installable.

This example is Codex specific. I am following the Codex plugin manifest spec, which uses a required .codex-plugin/plugin.json file and optional paths like skills and mcpServers. Inside the plugin root, Codex expects that manifest at .codex-plugin/plugin.json.

The plugin directory looks like this:

plugin/
├── .codex-plugin/
│   └── plugin.json
├── .mcp.json
├── mcp/
│   ├── server.py
│   └── docs/
│       ├── onboarding.md
│       └── refunds.md
└── skills/
    └── use-internal-docs/
        ├── SKILL.md
        └── references/
            └── example-questions.md

The main manifest is plugin.json:

{
  "name": "internal-docs-plugin",
  "version": "0.1.0",
  "description": "Package the internal docs MCP server and its companion skill.",
  "skills": "./skills/",
  "mcpServers": "./.mcp.json",
  "interface": {
    "displayName": "Internal Docs Plugin",
    "shortDescription": "Search internal docs through MCP with a packaged skill."
  }
}

This is how the plugin connects the packaged parts:

• skills: "./skills/" tells Codex where to load the bundled skills from
• mcpServers: "./.mcp.json" tells Codex where to load the bundled MCP server config from

Then .mcp.json points to the bundled MCP server:

{
  "mcpServers": {
    "internal-docs": {
      "command": "python3",
      "args": ["./mcp/server.py"],
      "cwd": "."
    }
  }
}

To distribute and install the plugin in Codex, the documented local flow uses a marketplace.json file. This file tells Codex which plugins exist and where to load them from.

.agents/
└── plugins/
    └── marketplace.json

A minimal marketplace.json looks like this:

{
  "name": "local-example-plugins",
  "interface": {
    "displayName": "Local Example Plugins"
  },
  "plugins": [
    {
      "name": "internal-docs-plugin",
      "source": {
        "source": "local",
        "path": "./plugin"
      },
      "policy": {
        "installation": "AVAILABLE",
        "authentication": "ON_INSTALL"
      },
      "category": "Productivity"
    }
  ]
}

With that file in place, you can register the current directory as a plugin marketplace:

codex plugin marketplace add .

That command registers the current directory as a plugin marketplace source. From there, Codex can discover the plugin defined in marketplace.json through its plugin management UI and install it from there.

Conclusion

I wrote this post because I did not want to keep using these tools without understanding what they actually were. Once I started looking closely, a lot of the magic became easier to place. Some of it is model capability, but a lot of it is engineering around the model: prompting, context management, tools, agents, MCP, skills, and packaging.

I still have mixed feelings about what all of this means for programming. These tools are useful, and the research around them is genuinely creative, but they also change what parts of software work feel valuable and enjoyable. I do not think the old world of fully hand-crafted code is coming back, so for now I would rather understand this stack clearly than pretend it is not reshaping the work.