The Skills Map for Building and Deploying AI Applications

The single most useful thing someone handed me this year is a one-page map of the skills that make up building and deploying AI applications. It was built by analyzing a large number of job postings, structured expert interviews, and survey responses — so it reflects what the field actually rewards, not what a single company's blog thinks is fashionable.
The map is worth sitting with. AI Engineering sits at the top, and one of its four branches — Building and deploying AI applications — expands into six competencies:
- LLM foundations
- Grounding models with data
- Building agentic systems
- Evaluation-driven development
- Operating in production
- Machine learning foundations
That's the map. What follows is my field guide to each node — what the skill means, where to go to actually build it, and how the pieces fit together. I've linked the primary sources and tools I trust so you can go deeper on any branch.
The one idea underneath all six
Before the branches: the difference between AI applications and ordinary software is that an AI application's output is less predictable. You don't know in advance what an LLM will say, or what a supervised model will predict. That uncertainty changes the shape of the work.
Traditional software is planned. AI software is iterative. You build a piece, examine it, and decide what to try next — a sequence of steps driven by the intermediate results. Skilled AI engineers are good at that loop:

The six competencies are the toolkit you reach into at every step of that loop. Being able to skillfully decide what to do next is what lets you build reliable systems on top of unreliable components. That's the whole game.
1. LLM foundations
Understanding how large language models tokenize input and generate output is what lets you predict when to count on them and when they'll fail. This is not trivia — it's the difference between guessing and reasoning about:
- tokenization and context windows — what fits, what gets truncated, and the cost of a cache miss vs. a cache hit (OpenAI's tokenizer guide and Anthropic's prompt engineering overview are the canonical practical references; Lilian Weng's survey is the deep one);
- knowledge cutoff and reasoning effort — knowing a model can't know what it wasn't trained on, and how to spend inference budget;
- sampling parameters — temperature, top-p, and why "more creative" is rarely what a production system wants;
- special features — tool calling, multimodal input, and when a smaller specialized model beats a giant general one.
The Transformer architecture is the substrate for all of it (Vaswani et al., 2017 — Attention Is All You Need). When you understand the foundations you can make the real calls: which model, or which mix of models, and whether a specialized technique like fine-tuning or self-hosting is worth it (Hu et al., 2021 — LoRA; Dettmers et al., 2023 — QLoRA).
I've written about this from the sovereignty angle — when the model is local and you control the weights, "LLM foundations" becomes an operational discipline, not a vendor dependency. See Context Engineering: the real full-stack paradigm and The Model Is Not the Product.
2. Grounding models with data
LLMs need good input context to produce useful output. Vector-search RAG was the early answer, but the toolkit has grown well past it. The actual decision is what to include in the prompt versus what to let the model retrieve on demand, and which representation fits the data and the query:

- vector index (RAG) — embed and similarity-search over documents (Chroma, Weaviate, LlamaIndex);
- knowledge graph — entities and relations for multi-hop queries (Microsoft GraphRAG);
- semantic layer over structured data — customer records and OLAP reached through tools rather than stuffed into a prompt (a private knowledge-graph build with local agents);
- in-prompt context — curated records packed into the window (retrieval architecture: memory systems that compound).
Whatever you pick, you still have to turn documents — text, PDFs, HTML, images — into LLM-ready inputs, and engineer pipelines that keep that data clean and fresh. Stale grounding is worse than no grounding. For the framing of "RAG is one branch, not the tree," Huyen Chip's GenAI Platform design is the clearest industry write-up, and her AI Engineering field guide is worth the read alongside the skills map.
3. Building agentic systems
Agentic systems run from workflows that chain a fixed sequence of LLM calls to a harness that lets the model pick its own next step. The engineering decisions:
- architecture — what to chain, what to parallelize, when to use code vs. an LLM, and where the fallbacks live (Anthropic — Building Effective Agents; Kaggle — Agents whitepaper);
- the tool surface — what the model can call, including MCP, CLI, and sandbox execution (OpenAI Agents guide, LangGraph);
- memory and long sessions — how context is managed across a long run (Sovereign Memory Bank: autonomous cognitive memory for agents);
- multi-agent orchestration — when one agent stops being enough (autonomous architectures and self-improving inference);
- going to production safely — guardrails, adversarial inputs, data exfiltration, and governance (OWASP Top 10 for LLM Applications, NVIDIA NeMo Guardrails, LLM-Guard).
The field is moving fast — voice agents, computer-use agents, generative UI. Each is a specialization of the same harness. I've been building these fully local and giving local systems hands via a real execution environment.
4. Evaluation-driven development
This is the one I'd pick if I could only keep a single skill. The engineers who are genuinely good at building AI systems are the ones who can drive a disciplined evals + error-analysis loop. It focuses effort on directions that actually pay off, instead of random thrashing.
Building good evals is its own deep skill:
- exploratory analysis of traces and outputs, combined with product and business judgment, to decide what to measure;
- the menu of eval types — deterministic (code-based) checks, LLM-as-a-judge, and human-in-the-loop — and when each applies (DeepEval, DSPy for programmatic optimization);
- evaluating your evals — so the measurement itself keeps improving.
The right approach varies by project and by stage of the project, which is why it's hard to master. But once you have it, development becomes systematic rather than random — exactly the loop in the first diagram.
5. Operating in production
Operating AI software differs from traditional software because of its unpredictability, cost, and latency. The production skill set:
- observability — understand real-world performance, detect drift, and respond fast to model failures and adversarial prompt injection (Arize Phoenix, Evidently AI for drift and evaluation);
- statistical CI/CD — regression testing calibrated to the risk of a mistake, not a fixed suite;
- cost and latency optimization — model-choice optimization, distillation, fine-tuning, and workflow simplification when you have many users.
This is where "the loop is the product" becomes literal: if the system runs continuously, observability is the operating system (If the loop is the product, observability becomes the OS — when that post is live; the theme runs through the whole observatory series).
6. Machine learning foundations
Modern LLMs are built with machine learning — supervised and reinforcement learning. Everyone I know who is good at building with LLMs also understands ML and deep learning at some depth, and many applications still need a model someone else trained or one you train yourself.
Foundations here mean knowing the popular models and their tradeoffs (accuracy, training speed, inference speed), and how to engineer the data to train and evaluate them. The mental frameworks — bias/variance, error analysis, engineering your data — are the same ones that let you navigate uncertain output everywhere else on this map:
- Attention Is All You Need (Transformers);
- LoRA and QLoRA (efficient fine-tuning);
- the RL behind modern post-training, including RLHF and RLAIF / constitutional methods;
- distillation for cost/latency.
Labeling and data-curation tooling like Label Studio and serving layers like MLServer round out the practical stack.
How to use the map
The map isn't a checklist to finish — it's a diagnostic. When a build is stuck, the six nodes tell you which missing skill is the bottleneck:
- output is unpredictable in a way you can't reason about → LLM foundations.
- the model keeps missing facts → grounding with data.
- the workflow is brittle or can't scale to open-ended tasks → agentic systems.
- you're changing things and hoping → evaluation-driven development.
- it works in the demo and dies at scale → operating in production.
- you can't train or tune the part that matters → ML foundations.
Every bit you learn makes the next system better. And the complement that holds the whole map together is plain software engineering — the discipline that turns a promising prototype into something safe to run. That's the next letter.
Sources & further reading
- The skills map itself (the header image) — synthesized from job-postings, expert interviews, and surveys; reproduced with attribution to its author.
- Attention Is All You Need — Vaswani et al., 2017
- LoRA — Hu et al., 2021 · QLoRA — Dettmers et al., 2023
- RLHF — Ouyang et al., 2022 · Constitutional AI — Bai et al., 2022
- Distilling the Knowledge in a Neural Network — Hinton et al., 2015
- Anthropic — Building Effective Agents
- Model Context Protocol · OpenAI Agents · LangGraph
- Microsoft GraphRAG · Chroma · Weaviate · LlamaIndex
- OWASP Top 10 for LLM Applications · NVIDIA NeMo Guardrails · LLM-Guard
- DeepEval · DSPy · Arize Phoenix · Evidently AI
- Huyen Chip — GenAI Platform · AI Engineering field guide
- Kaggle — Agents whitepaper · Prompt Engineering Guide
- Related writing on this site: Context Engineering, The Model Is Not the Product, Private Knowledge Graph with Local Agents, Sovereign Memory Bank, Autonomous Architectures, Breaking Free from ChatGPT, objective05-exec, Retrieval Architecture.
Related research
Jul 14, 2026
Synthesizing Memory with Agent: A Local-First Architecture for Persistent AI State
The prevailing paradigm in AI agent development treats memory as an external retrieval service, a failure mode that fragments intelligence across the model, the context window, and the vector store.
Oct 23, 2026
LLMjacking: Building the 'Skynet Mode' Recursive Attacker
A deep dive into 'LLMjacking'—where autonomous agents hijack exposed LLMs to drive their own attacks. We deconstruct the architecture of recursive agents, 'Skynet mode' autonomy, and how a stolen brain becomes the primary attacker.
Oct 23, 2026
Operation Ouroboros: The Recursive Intelligence Parasite
What happens when an attacker stops treating language models as individual services and starts treating them as a distributed pool of reasoning capacity? What happens when exposed inference endpoints, autonomous agents, poisoned retrieval systems, and compromised cloud identities become components of a single operational architecture?
Jul 15, 2026
The Sovereign Knowledge Compiler: Compile-Time Memory for Local-First AI Agents
A revised architecture for agent memory that treats cognition as something compiled once into inspectable artifacts rather than retrieved fresh on every query — grounded in how OpenAI's Agents SDK, Mem0, and Hindsight are already converging on distillation, and extended with decay, conflict resolution, and CRDT-based device sync.
Jul 14, 2026
The Recursive Research Compiler: Turning Compile-Time AI Inward with the Knowledge Compiler SDK
A case study in Compile-Time AI: how the Knowledge Compiler SDK turns Markdown into inspectable intermediate representations, and how pairing it with an agent like Hermes lets a codebase compile its own next generation of passes.
Jul 5, 2026
Retrieval Architecture: Memory Systems That Compound
Memory systems and retrieval architecture for sovereign AI. Sovereign Memory Bank, Dynamic Persona MoE RAG, Objective05, and GraphRAG — the subsystems that make retrieval compound over time.