Back

What AI Agents Can and Can't Do Reliably Right Now

"AI agent" gets used to describe two very different things, and mixing them up is where most of the disappointment with this technology comes from. One is a narrow, well-scoped system that reads an input, applies a defined set of rules or classifications, and takes a specific action. The other is a general-purpose assistant expected to handle open-ended, multi-step tasks with judgment close to a human's. The first works well today. The second is much less reliable than the marketing around it suggests, and it's worth being specific about why.

What reliably works today

Reading unstructured text and extracting structured information out of it is a genuine strength. Feed a model an inbound customer email and ask it to identify the sender's intent, urgency, and what service they're asking about, and it will do that consistently well, especially when the categories it's choosing between are clearly defined. Classification tasks like this, sorting a message into one of a known set of buckets, are one of the most dependable uses of current AI.

Drafting text within constraints is another. Generating a first-pass reply to a common inquiry, summarizing a document, or reshaping one piece of content into a different format all tend to produce genuinely usable output, provided a person reviews it before it goes out rather than trusting it blindly.

Short, well-defined action chains hold up reasonably well too: read a message, look up a record, update a field, send a notification. Each step is simple enough that the model rarely gets confused about what it's supposed to do.

Where reliability drops off, and why

The core problem is that errors compound across steps. If a model is right 85% of the time at each individual step of a task, which sounds fairly good in isolation, a ten-step workflow built entirely on that model succeeds end to end only around 20% of the time, because the errors stack. This is one of the most consistently documented failure patterns in current research on agent reliability, and it explains why long, autonomous, multi-step workflows are so much harder to trust than a single well-scoped classification task.

Edge cases are the other consistent weak point. A system that hasn't been explicitly tested against unusual or malformed inputs tends to fail on a meaningful share of real-world interactions, not because the model is bad in general but because production traffic always includes cases nobody anticipated during testing. A message in a different language, a form field left blank, two unrelated questions in one inquiry. These trip up systems that perform beautifully on the tidy examples used to build and demo them.

Integration brittleness matters too, and it's the least discussed of the three. APIs change their response format without warning. Authentication tokens expire. A spreadsheet column gets renamed. None of these are AI failures at all, technically, but they show up as the automation "breaking," and a system with no way to notice and flag that kind of failure will just keep running on bad assumptions until someone happens to notice the output looks wrong.

What this means in practice

The workflows that hold up well in production tend to be narrow and specific rather than broad and autonomous: read this type of message, decide this specific classification, take this specific action, log it, and flag anything the system isn't confident about for a person to check. The workflows that disappoint tend to be the ones asked to handle an entire open-ended process end to end with no human checkpoint anywhere in it.

The right conclusion here is about design, not avoidance: build agents around where they're actually strong, with a human still in the loop at the points where judgment or an unusual case is likely to show up. A well-scoped agent handling lead qualification, with uncertain cases routed to a person, is a genuinely reliable tool. The same underlying model asked to run an entire customer relationship autonomously, with no oversight, is asking for a level of consistency the technology doesn't reliably deliver yet.

A simple way to test reliability before trusting a workflow

Before relying on an AI-driven step for anything customer-facing, it's worth deliberately feeding it a batch of real, messy historical examples, not the tidy ones used to build it, and checking the output by hand. Include the awkward cases on purpose: the inquiry that doesn't fit any category cleanly, the message with a typo that changes its meaning, the one written in frustration rather than politely. How the system handles that batch is a far better predictor of real-world reliability than how it performs on the five clean examples used during development, and it's a cheap check to run before anything goes live.

The honest version

Current AI agents are excellent at narrow, well-defined, checkable tasks, and they get meaningfully less reliable as a task gets longer, more open-ended, and further from a human's ability to spot-check the output. Building automation around that reality, instead of around what a demo video makes it look capable of, is the difference between a system that quietly works for a year and one that quietly produces wrong answers nobody noticed for a month.

Related reading

Common Mistakes When Automating Customer InquiriesThe failure modes this reliability gap causes in practice. AI Lead Qualification & Auto-Response SystemA narrow, well-scoped agent of exactly the kind described here.

If you're weighing up where an AI agent would genuinely hold up in your business, describe the process and I'll give you an honest assessment.

Start a conversation What GENZAX does