How I Built an Agentic Research System
A practical breakdown of the agents that power Applied’s living map of real AI deployments
Last month, I shared the State of Applied AI, a report based on 200+ real AI deployments from companies around the world. A few people asked how those cases were found, analyzed, and organized. I covered the methodology briefly in the report, since the focus was the findings.
This article is about the system behind it.
That system is Applied: a living map of real AI use cases. It tracks how companies adopt AI across industries, business functions, tools, and outcomes.
Today, I’ll break down how a team of agents collects, structures, and transforms scattered stories into intelligence.
Agentic Research
Behind the scenes, Applied is a team of agents with specific tasks and different levels of autonomy. Above them, there is an orchestration layer. This is usually a complex part of agentic systems, but I kept it dead simple on purpose.
Scout Agent
The Scout Agent discovers and prioritizes new AI use cases to extract. It is the starting point of the system, and plays a key role in keeping Applied both deep and broad. It needs to cover as many industries and business functions as possible, while balancing established AI tools with new entrants.
The goal is to avoid building a living map that only covers three tools, or one where 80% of the cases come from engineering teams.
Extractor Agent
The Extractor Agent is the most important agent in the system.
It reads each potential case, decides if it is worth including, and extracts the structured data behind it: companies, vendors, tools, business functions, outcomes, and other key entities.
This is the agent I have spent the most time tweaking. It burns the most tokens, but it is also where most of the quality control happens.
Over time, it has learned what makes a strong case: novel in terms of AI usage, not duplicated, tied to a clear company, connected to an end-to-end scenario, supported by tangible outcomes, and recent enough to matter (2025 or newer).
In practice, it checks whether the case is really about AI, whether it uses newer technologies like LLMs or agentic systems, whether the outcomes are concrete, and whether the entities already exist in the map or need to be added.
Enrichment Agent
The Enrichment Agent runs after the Extractor Agent. At this point, the case is already structured, so the agent has more context to work with. Its job is to complete missing information about companies, tools, categories, and use cases using additional sources.
Translator Agent
Applied use cases are available in both English and Spanish. The Translator Agent handles that translation layer and keeps the living map consistent across both languages.
Quality Assurance Agent
Early on, the system did not have any sort of quality checks. That became a problem.
The data, taxonomy, and even parts of the UI started to feel inconsistent. The living map had the right ingredients, but it was not cohesive enough.
So the Quality Assurance Agent was included to find and fix issues across Applied.
Match Maker Agent
The Match Maker Agent is the newest agent in Applied. As the living map grows, simple filters on a webpage are weren’t enough for users to identify value. The Match Maker Agent helps users discover cases that match their interests and notifies them by email or in the app.
Orchestration
In Applied’s AI tools landscape, there is a category called Agentic Management. These tools help orchestrate agents, track their outputs, and monitor spending. This layer is particularly important when dealing with swarms of agents and mission critical tasks.
So, how do agents coordinate in Applied?
There is no external orchestration tool. So far, a simple system that combines a Data Model and individual Report Logs per agent has worked well.
All agents can read from and write to the living map.
Each agent also reads its own logs to understand where it left off, what was already processed, and what still needs attention.
The agents do not really know that other agents exist. They don’t know each other’s tasks either. Their shared context is the living map.
So, in practice, the orchestration layer is mostly the living map, the logs, and myself.
I still co-guide the parts where judgment matters most:
Designing the data model
Defining the taxonomy and categories
Structuring the AI tools landscape
Deciding whether to include or discard borderline cases
How can this agentic architecture be improved further?
There is still a lot to improve, especially in the enrichment agent. Applied could add more context around cases, vendors, tools, and categories: pricing, reviews, quality signals, outcomes, rollout time, required expertise, and more.
There aren’t automatic feedback loops, those still go through me. The QA Agent reports issues, but those findings do not go back (directly) to other agents. I’m the bridge.
Closing that loop would make each agent upgrade itself, but… The risk is that agents can make strong decisions without enough evidence. Even with today’s best models, they sometimes overreach. They could introduce drastic changes to the taxonomy or data model, or get stuck in loops where they change something, reverse it, and repeat the process.
That is why I would be careful about giving agents full control over the core structure of Applied.
Where closing the loop feels safer is in the Match Maker Agent. For example, the Match Maker agent could learn from user feedback, email replies, platform comments, open rates, and click rates. Over time, it would learn which cases are actually valuable to different types of users.
Other Applications for this Agentic Framework
Applied is focused on AI adoption, but the same framework can be used in many other areas. Any workflow that requires finding information, filtering what matters, extracting structured data, enriching it, and keeping it updated could benefit from a similar agentic setup.
You can probably think of your own version based on the problems you care about.
A few examples:
Competitor research: track product launches, pricing changes, customer stories, positioning, and market moves.
Real estate: monitor listings, neighborhoods, prices, permits, rentals, and investment opportunities.
Supply chain: track vendors, disruptions, shipping updates, costs, risks, and alternative suppliers.
Cybersecurity threat intelligence: track vulnerabilities, exploits, threat actors, affected vendors, mitigation steps, and risk levels.
Policy and regulation tracking: Track new laws, consultations, enforcement actions, regulatory guidance, and how they affect specific industries.
The pattern is the same, A) scout the sources, B) extract what matters, C) enrich the context, D) check quality, and E) deliver useful insights to the right person.
Access the living map of AI adoption at TheApplied.co.
Love seeing this architecture laid out clearly. The Living Map layer as a shared state between agents is a smart design — avoids the coordination overhead that kills most multi-agent systems. The challenge at scale is usually keeping that map fresh without it becoming a bottleneck. Curious how you handle conflicts when multiple agents update it concurrently.
Excellent write-up. What I liked most is that the system avoids the usual “fully autonomous agent” fantasy and instead treats agents as constrained workers operating around structured workflows, verification, and human-controlled feedback loops.
A lot of people underestimate how quickly agentic systems become unreliable once taxonomy, extraction quality, or memory layers start drifting silently in production. This article actually acknowledges those operational realities instead of pretending that prompting alone solves them.
Also appreciated the emphasis on iterative refinement over one-shot orchestration. That’s much closer to how real production systems evolve.
I write a lot about distributed systems, DDD, and production AI infrastructure myself, and this was absolutely worth the subscription.