Building Scalable AI Agents: Chirag Agrawal Explains How

Host: 2025 is the year of the agents. We talk a lot about AI agents, but how do they actually work? Why do we need agents? Why don't we just have one big supercomputer? What are the risks of programming an agent, and what do we need to care about with memory and all those things?

Today I talked to Chirag Agrawal. He comes from one of the big tech companies and he's really good at explaining these concepts to us non-technical folks. So keep listening and you'll learn a lot and take away a lot from it.

Host: So Chirag, it's great to have you on the podcast. First of all, welcome.

Chirag: Thank you for having me. Really appreciate it.

Host: What got you interested in AI? Why AI?

Chirag: Great question. I've always been fascinated by the problem of search. During my undergraduate years, that's one of the problems I worked on for a long time. Since I was learning computer science, I decided to build my own little search engine. It could take a PDF book, and I built a Google-like interface for it. You could search the book and it would tell you what page the answer is on.

That got me really interested in natural language processing, because it felt powerful: just looking for information, with the computer assisting you. Over the years, I've come a long way. I've spent a lot of time working for a big AI consumer product, and now the problems we tackle are a lot more complicated than searching a PDF book. But that's where it started.

Host: That's interesting, because most people would say "data is important," but you're more like "it's about finding the right thing." I remember Google once saying they wanted to become a "find engine," not a search engine. Are we on that path now?

Chirag: I think Google really solved information retrieval. But AI, and you can see this with where ChatGPT is going, is solving a different problem: what actions to take.

For example, you could always search for hotels. But now AI can help you find the exact hotel you'd like, narrow down a hundred options into one action, and then you can go and book it. That narrowing down is where I think AI is taking us.

Host: And that's probably not just one thing happening. You have one interface, but in the background there are lots of agents running around.

Chirag: Exactly. Thinking of agents as humans, or how humans work together, is a helpful analogy. It doesn't perfectly match how you design agents, because sometimes LLMs behave in very inhuman ways. But for conceptualizing a multi-agent system, it helps.

Once you go into the details, you realize there's a lot of engineering required. They don't just work out of the box.

Host: You said they behave "inhuman" in certain ways. Do you have an example?

Chirag: Yes. Here's a simple one we tackled at work.

Say there are two agents. One can tell you the weather. Another can turn on your sprinkler. A human would naturally understand that the sprinkler agent controls sprinklers and the weather agent provides weather.

But if you put them together without the right context and ask: "If it's not raining, turn on my sprinkler," the assistant might message the sprinkler agent and say something like: "If you hear from the weather agent that it's not raining, then turn yourself on."

That's silly, because those agents don't "know" each other. It's like telling one person: "When my other friend tells you something, do this," even though they've never met. AI makes mistakes like this unless you engineer the context and the boundaries.

And that's with just two agents. At scale, the complexity grows fast.

Host: If you have thousands of agents, you first have to find which agents can even do the task.

Chirag: Exactly. Then you need routing, task planning, task breakdown. You need a search solution to identify the right tools or agents.

There's a well-known concept in AI called the ReAct loop: the agent reasons, then acts, then reasons again based on what happened, and repeats until the goal is achieved.

And you can build hierarchies: sometimes the "act" is delegating part of the work to another agent, like how you'd delegate at work.

Host: If we have a complex business process, how many agents are we talking about?

Chirag: Anywhere from one to thousands. There's no real limit.

Take a high-level goal like "write technology articles." If you try to automate that, you might want specialized workers: one finds topics, one checks novelty, one gathers data, one writes code, one runs experiments. Then you might have higher-level coordinators managing those, and another layer that interfaces with the user, because talking to a human is different than talking to another machine.

Then you get into memory. Do all agents share memory like a hive mind? Sometimes that helps. Sometimes it's disastrous because too much shared context throws agents off track.

Then there's synchronization: if agent A knows something and agent B knows something else, how do they coordinate without assuming the other has the same context? Humans constantly manage that implicitly in conversation. Agents need explicit mechanisms.

Host: That's interesting, because you and I didn't start from zero, we already have some shared context. With agents, you have to create that deliberately.

Chirag: Exactly. In our architecture, agents do share memory, and it's good for the user because the user talks to one assistant and continuity doesn't break.

But it also has downsides: agents can get confused, try to use APIs they don't have access to, or attempt actions outside their authority. You need checks and guardrails, and even then things can slip through.

If you isolate context, you reduce confusion, but then you have the synchronization problem.

Host: Why do we have agents at all? Why not one supercomputer to rule them all, like in science fiction?

Chirag: If we had a magical LLM with an infinite context window and perfect recall, you could dump everything into one prompt: all tools, all data, all instructions. Maybe then you wouldn't need separate agents.

But that's not reality. LLMs get confused, especially in multi-turn conversations. And action-taking is categorically harder than informational Q&A.

A lot of people use ChatGPT mostly for informational queries. That often works because the model can answer from what it already learned. But if you want an assistant to take real-world actions, like booking a cab, turning on sprinklers, making payments, calling third-party APIs, the system needs reliable planning and execution. With more context and more tools, mistakes become more likely, and the failure modes get worse.

Also, LLMs aren't perfect at instruction following. You can't dump pages of instructions and expect perfect compliance. To ship something reliable, you narrow responsibilities. A well-defined agent does one thing well.

It's like why we don't have one superhuman who can do every job: you can't fit everything in one head, and you don't have every tool.

Host: Like a Swiss Army knife: it can do a lot, but nothing perfectly.

Chirag: Exactly.

Host: People say "2025 is the year of the agent," but when did people actually start building functioning agents?

Chirag: If you define an agent broadly as software that accomplishes goals on your behalf by taking actions, agents have existed for a long time, like early assistants such as Siri. They weren't great, but the idea existed.

The modern "AI agent" wave really accelerated after ChatGPT became mainstream. The definition got sharper, and capability took a leap.

A good example is RAG, retrieval-augmented generation. It's basically: search your own data, retrieve relevant pieces, then summarize them into an answer. That's an agent-like pattern and it became very popular as these models took off.

Host: For people outside the field, RAG is like giving the AI access to your documents, so answers are more grounded and current?

Chirag: Yes. Another way to put it: models are trained up to a point in time. If you want your AI to use fresh information, like your company's internal docs or something you wrote last night, you index that data, search it when a question comes in, and then the model uses what it finds. That gives assistants real-time-ish access to your information.

Host: So "AI agent" also means it's not just deterministic software. It can interpret natural language, plan steps, and act.

Chirag: Right. Traditional software follows fixed flows. AI agents can plan the steps dynamically because they have an LLM "brain."

Natural language matters not only for user-to-agent interaction, but also for agent-to-agent interaction. Before, systems talked via rigid APIs. If you can instead communicate in plain language and both sides can understand it, coordination becomes much more general. You can layer AI over existing systems and suddenly you get much richer interactions.

Host: Do you see some industries adopting agents faster than others?

Chirag: Engineers were early adopters. Coding agents are already excellent. I like Claude Code and Cursor a lot.

Outside software, I've seen fast adoption in compliance, because compliance work is tedious and document-heavy. Also medicine, where RAG-style retrieval is useful. And law firms, because they need to search massive document corpora.

In general, any profession where a lot of the job is interacting with a computer can benefit, because computers are basically APIs and processes underneath, and AI can work with those.

Host: Finance seems like a natural fit too.

Chirag: People are experimenting, like trading with AI, but I wouldn't trust it with serious money yet. A classic danger is a fat-finger event: one mistaken action causes a cascade because automated systems interpret it as a market signal. That's a very inhuman failure mode unless you constrain the system carefully.

Host: Constraints and guardrails matter.

Chirag: Exactly. That's another reason agents are useful: you can scope them narrowly, monitor them for specific failure modes, and reduce risk.

Host: What's your favorite way to use AI?

Chirag: At work, I use it a lot for coding. Personally, I use it heavily for research. I don't use Google as much anymore because AI tools reduce effort a lot.

I also use AI to interact with my own documents, like summarizing something I wrote years ago. And meeting summarization is huge at work: good summaries, action items, tracking what we decided.

One experimental thing I built was an agent that helps with marketing on Reddit. It reads posts, finds ones relevant to what I'm building, and leaves a helpful comment that also mentions the product. I don't abuse it because it can annoy people and contribute to "dead internet" if it's misused. The key is constraints so it stays within boundaries and doesn't mislead.

Host: People are getting better at spotting AI-written content too.

Chirag: They are. There are telltale patterns and repeated phrases. Even things like overusing em dashes can be a giveaway. You need thoughtfulness, and ideally transparency and guardrails.

Host: Last question: Terminator or Matrix. Do you think machines will take over the world?

Chirag: I think agents will do a big part of our jobs in the next 5 to 10 years. But I don't think it's going to be a sudden "overnight takeover" like Terminator.

If something like AGI happens, it'll likely be built slowly, with a lot of engineering and oversight. That gives us time to add guardrails and limit harm. So a cinematic takeover seems unlikely, but life will change significantly.

Host: I have good days and bad days. Mostly I'm not afraid, but sometimes I worry about things like warfare and drones. Still, I agree with your view: it's a process, not a Deus ex machina. Lots of people build these systems, and that lowers the chance that one person creates something catastrophic in isolation.

Also, AI is expensive to build, which may be a good thing. It reduces the odds of a lone actor creating something truly dangerous.

Chirag: That's a good point. The cost barrier can reduce certain risks.

Host: Where can people find you?

Chirag: People can reach out on LinkedIn. I also have a website, chirag.io. If LinkedIn doesn't work, go to the website, my socials are linked there and you can contact me.

Host: Thank you for the interview. It was great. You explained things really well for non-technical people. Loved it.

Chirag: Thank you for inviting me. It was a pleasure talking to you.