What Are AI Agents, Actually?
An agent is a system where the model decides the next step. That's it. Most things called 'agents' aren't - they're workflows with LLM-powered steps, which is usually the right architecture anyway. Understanding the actual distinction helps you buy smarter and build better.
TL;DR: An agent is a system where the model decides the next step. That’s it. Most things called “agents” aren’t - they’re workflows with LLM-powered steps, which is usually the right architecture anyway. The term has become marketing noise. Understanding the actual distinction helps you buy smarter and build better.
Open any AI vendor’s website. Count how many times you see “agent.”
Chatbots are agents. Workflows are agents. Anything with an LLM is an agent. The term has been stretched so thin it means nothing. Which is a problem when you’re trying to figure out what you’re actually buying - or building.
Let me cut through it.
The Actual Definition
An agent is a system where the model decides what happens next. Not you. Not your code.
An agent is a system where the model decides what happens next.
Not you. Not your code. The model observes the current state, reasons about what to do, takes an action, observes the result, and decides the next move. The loop is model-driven.
This is different from a workflow, where you define the path in code. Document arrives → classify it → extract fields → validate → route. The model might power individual steps (classification, extraction), but the orchestration is yours. You wrote the logic. The model executes within it.
The distinction isn’t about sophistication or capability. It’s about control. Who decides what happens next - your code or the model?
What “Agents” Usually Aren’t
Most things marketed as agents are something else:
Not a chatbot. A chatbot responds to prompts. You ask, it answers. There’s no autonomous action, no multi-step execution, no tool use beyond generating text. Calling a chatbot an “agent” is like calling a calculator a robot.
Not a workflow with LLM steps. This is the most common confusion. You build a pipeline: ingest document, run it through GPT for classification, apply business rules, output result. That’s a workflow. The LLM is a component, not the orchestrator. Your code decides the path.
Not RAG. Retrieval-augmented generation fetches relevant context and feeds it to a model. That’s a capability, not agency. The model doesn’t decide to retrieve - your system does.
Not an API with a nice UI. Wrapping an LLM API in a chat interface doesn’t make it an agent. It makes it a chat interface.
What Agents Actually Look Like
A true agent has an observe → think → act → observe loop where the model controls the loop.
You give it a goal: “Research competitor pricing strategies and summarize findings.”
The agent decides: search for competitors, read results, decide which links are relevant, fetch those pages, extract pricing information, realize it needs more context, search again with refined terms, synthesize across sources, decide when it has enough, produce summary.
No predetermined sequence could replicate this because the steps depend on what the agent finds along the way. That’s the value proposition: navigating uncertainty that can’t be specified in advance.
Tools give agents capabilities - search, code execution, API calls, file operations. The model decides when and how to use them. That decision-making is what makes it an agent.
Why Most “Agents” Are Actually Workflows (And That’s Fine)
Here’s an honest admission: I’ve built what companies call “multi-agent systems.” Most were workflows with LLM-powered steps.
One project - regulatory document review for pharma - ran six “agents” in parallel: grammar checking, format validation, consistency analysis, reference verification, abbreviation tracking, section gap detection. Sounds agentic. But the architecture was deterministic: file triggers processing, document gets chunked, all six analyzers run in parallel on each chunk, results get deduplicated, Excel report generated.
The model didn’t decide the path. The code did. Each “agent” was really a specialized prompt running within an orchestrated workflow.
Was this wrong? No. It was the right architecture for the problem. The process was predictable. We knew exactly what analysis steps were needed. Encoding that in code made the system debuggable, auditable, and reliable. Giving models the freedom to decide their own path would have added complexity without benefit.
Most business problems are like this. The process is knowable. You can draw it on a whiteboard. When that’s true, a workflow beats an agent every time.
When Agents Actually Make Sense
Agents genuinely help when two conditions are met:
The path can’t be specified in advance. If you can draw the decision tree, you don’t need an agent - you need a workflow that implements that tree. Agents belong where the tree would be infinite because each branch depends on what you discover.
Failures can be caught before they matter. Agents make mistakes. If those mistakes get caught through automated verification - tests that pass or fail, outputs validated against criteria - the system can iterate toward correctness. If mistakes propagate before detection, you have a problem.
Research tasks fit both criteria. Code generation with test feedback fits. Open-ended analysis with human review before decisions fits.
High-stakes processes with slow feedback loops rarely fit. Regulatory submissions, financial transactions, anything where errors are expensive and detection is slow - these need the predictability of workflows, not the flexibility of agents.
The “AI Agent” Marketing Problem
Everyone slaps “agent” on everything because it sounds sophisticated. “AI-powered workflow” doesn’t have the same ring as “autonomous AI agent.”
I’ve seen it everywhere. Every vendor is now “AI-powered.” No-code automation sellers call their Zapier alternatives “AI agents.” Prompt “cheat-sheets” promising autonomous workflows you’ll never actually use. Tools genuinely useful for prototyping get treated like production-ready autonomous systems because the marketing implies capability that isn’t there.
This isn’t necessarily the vendors’ fault. The hype cycle creates pressure to use the hottest terms. But it makes buying decisions harder when you can’t tell what you’re actually getting.
Questions to Ask Vendors
When someone pitches you an “AI agent,” ask:
“What decisions does your agent make autonomously?” If they can’t answer clearly, it’s probably not an agent - or it’s an agent making decisions you haven’t thought through.
“What happens when it makes a mistake?” Real agents need real guardrails. How are errors detected? How are they corrected? What’s the blast radius of a bad decision?
“Can you show me the decision logic?” If the system is actually a workflow, they should be able to show you the flow. If it’s truly agentic, they should be able to explain what the model decides and what constraints bound those decisions.
“Why does this need agency instead of automation?” This is the killer question. If they can’t articulate why the path must be model-determined rather than code-determined, they probably built a workflow and called it an agent.
Scoping Before Solutioning
When clients tell me “we want an AI agent,” I ask them to describe the problem.
What are the steps you take today? What’s the workflow? Where do you need judgment versus where do you need labor? Labor steps are automation candidates - workflows with LLM-powered components. Judgment steps might need agency, or they might need humans with better tools.
This decomposition usually reveals that the judgment component is smaller than assumed. What looks like “we need an AI agent to handle customer support” often breaks down to: 80% routine cases (workflow), 15% information lookup with standard patterns (workflow with retrieval), 5% genuine judgment calls (humans, or agents with tight guardrails and human review).
Automating the 80% with a reliable workflow beats deploying an agent that tries to handle 100% and fails unpredictably on the hard cases.
The Containment Principle
When you deploy genuine agency, containment matters. What can the agent access? What’s the worst thing it could do if it misinterprets a goal or gets manipulated?
You don’t want your internal chatbot executing credit card purchases based on user requests. Every capability you give an agent is a capability that can be misused - through bugs, misaligned goals, or external manipulation. Scope permissions tightly. Log everything. Build kill switches. Assume “unexpected” will happen; design so it doesn’t mean “catastrophic.”
The Honest Take on Agentic Experience
I’ll be direct: I don’t have extensive experience deploying fully autonomous agents in production. Few people do - the technology is genuinely new, and the conditions where true agency makes sense are narrower than marketing suggests.
What I have built: workflows that incorporate agent-like flexibility at specific points, bounded by deterministic orchestration. Systems where the model reasons within constraints rather than controlling the entire flow. This hybrid approach - workflows with agent-powered steps - handles most real-world problems better than either pure approach. The FDA document review system is a concrete example: six specialized analyzers running within a deterministic pipeline.
The skill isn’t building agents. It’s knowing when you need one and when a workflow will serve you better. Usually, it’s the workflow.
The Right Mental Model
Think of the agent question as: who decides the next step?
If your code decides, it’s a workflow. Use workflows when the path is predictable, when you need auditability, when errors must be traceable, when reliability matters more than flexibility.
If the model decides, it’s an agent. Use agents when the path genuinely can’t be specified in advance, when exploration is the point, when failures can be caught quickly, when flexibility matters more than predictability.
Most production systems are workflows with agent-like components at specific points. That’s not a compromise - it’s good architecture.
The marketing will keep calling everything an agent. Now you know what the word actually means.
If you’re trying to figure out whether you need an agent, a workflow, or something in between - let’s talk. I’ve built enough of both to know when each makes sense.