Core AI Agent Concepts

Summary:

AI Agent concepts and terminology used by Outerlimit and the wider community.

Agent

An AI system that perceives context, decides on a course of action, and executes it through tools or APIs, often over multiple steps, with limited or no human intervention between steps. Distinct from a chatbot, which simply responds to each prompt in isolation.

Agentic AI  

AI systems built around one or more agents that autonomously pursue goals by decomposing them into subtasks, reasoning, invoking external tools, and adapting based on results. The defining feature — and the source of most novel security risk — is that the system takes actions in the world, not just generates text.

Agency / Autonomy

The degree of freedom an agent has to act on its access without checking back with a human. High autonomy increases capability but also blast radius: a single manipulated instruction can trigger a long chain of consequential actions.

A2A (Agent-to-Agent protocol)

An open protocol for communication and task delegation between independent agents. Introduces trust and authentication challenges, since one agent must decide whether to trust instructions or data coming from another.

Context window

The amount of text (prompt, retrieved data, conversation history, tool output) a model can consider at once. Everything in the window is treated as potentially actionable input, which is why untrusted content placed in it is dangerous.

Human-in-the-loop (HITL)  

A control pattern where a human must approve specific agent actions before they execute (e.g. sending money, deleting data). Human-on-the-loop is the lighter variant where a human monitors and can intervene but does not pre-approve each step.

MCP (Model Context Protocol)

An open standard for connecting models and agents to external tools, data sources, and services through a uniform interface. Widely adopted, but its dynamic "discover and connect at runtime" nature creates supply-chain and tool-poisoning exposure (see ASI04).

Memory

Persisted state an agent carries across steps or sessions. Short-term memory is the working context of a task; long-term (or episodic) memory is stored and retrieved later. Memory that can be written by untrusted input is a poisoning target (see ASI06).

Multi-agent system (MAS)

An architecture where several specialised agents collaborate, delegate, and communicate to accomplish a goal. Adds attack surface around inter-agent trust, message integrity, and cascading failure.

Orchestration

The coordination layer that routes tasks between an agent's reasoning, its tools, its memory, and (in a MAS) other agents. The orchestrator is a high-value control point for enforcing guardrails.

Planning / Task decomposition

The process by which an agent breaks a high-level goal into an ordered set of steps. Manipulating the plan (rather than a single output) is the essence of goal hijacking.

RAG (Retrieval-Augmented Generation)

A pattern where a model retrieves relevant documents from a knowledge store and uses them to ground its response. Because retrieved content enters the context window, a poisoned knowledge base becomes an indirect prompt-injection vector.

ReAct (Reason + Act)

A common agent loop in which the model alternates between reasoning about what to do and acting (calling a tool), then observing the result and reasoning again.

System prompt

The instruction set that defines an agent's role, rules, and constraints, supplied separately from user input. Treated as privileged configuration; its leakage (see LLM07) can reveal guardrails to an attacker.

Tool use / Function calling

The mechanism by which an agent invokes external capabilities (APIs, databases, shell commands, code execution). Tools are where an agent's actions become real, so tool access is the primary thing to scope and monitor.

Author
Neil Larkins
COO
·
Outerlimit