March 17th, 2025, by Jakub Zavrel, Founder and CEO, Zeta Alpha.
As companies move beyond the initial excitement of prototyping chat-based generative AI, business leaders are increasingly seeking tangible returns on their AI investments in terms of productivity, cost savings, and growth. One of the most exciting current paths forward is AI Agents. For knowledge management (KM) professionals, the key question now is how to develop and deploy AI agents, and go beyond chat interactions and question answering to perform sophisticated knowledge-intensive work as true intelligent assistants.
AI Agents work by chaining together building blocks composed of Large Language Model (LLMs) prompts combined with larger quantities of data than humans can typically digest. AI Agents break down goals into simpler tasks to address sophisticated domain-specific tasks that might take people days of work to complete, a key characteristic of work that generates real business value (Rosenbush, 2024), and interact with existing IT systems through APIs.
Imagine AI agents that not only answer questions and create reports, but contribute to better tactical and strategic decision-making from existing knowledge in real time. The knowledge management function would finally be able to create a competitive advantage.
The Evolution from Chatbots, via RAG to AI Agents
Most enterprises have started their AI journey with basic chat-based assistants, which offer a convenient interface for users to interact with LLMs. As LLMs are trained on public information from the internet, they cannot deal very well with proprietary company knowledge. However, they can quite easily be extended to pull information from internal sources through Retrieval-Augmented Generation (RAG). The rapid evolution of LLMs and Generative AI (GenAI) has led to broad adoption of chatbots, allowing employees to quickly produce content, and get help with simple day-to-day questions for which existing Enterprise Search systems have been failing for decades.
Although such RAG chatbots can be prototyped quickly, with open source frameworks like LangChain or LlamaIndex and off-the-shelf AI models, in the end they often fall short of the expectations of domain experts when it comes to tasks that require a precise understanding of domain knowledge. This is one of the key reasons why only around 30% of the AI prototypes end up going in production (Deloitte, 2024).
RAG, while powerful, is not a silver bullet; it relies heavily on the quality of the underlying retrieval system. For enterprises to trust their AI Agents, it’s essential to have a well-tuned, domain-specific retrieval system foundation that understands the unique context and nuances of the organization’s data, terminology and knowledge domains.
Imagine an analyst investigating emerging market trends. Unlike traditional RAG systems that generate brief summaries, an AI Research Agent divides the task into sub-goals, performs multiple searches, assesses the relevance of each source, synthesizes insights, and organizes key trends into a cohesive report. Rather than overwhelming the analyst with raw data, the agent distills actionable intelligence from internal and public sources, achieving in minutes what would normally take days.
Domain-Specific AI Use Cases
Research and Development (R&D) departments often review literature to identify scientific trends and validate new innovations. AI Agents can streamline this process by summarizing findings from various studies, identifying gaps, and even proposing further research. But the applications of AI Agents in KM extends far beyond R&D. Here are some domain-specific use cases where AI agents are making a measurable impact:
Knowledge Extraction from Technical Manuals:
AI Agents can automate the extraction of essential information from complex technical manuals, e.g. creating customized maintenance schedules and troubleshooting guides based on actual maintenance logs for each machine. This reduces human error and minimizes downtime in production facilities.
Onboarding and Offboarding:
For onboarding, AI Agents organize and tailor training materials, ensuring new hires quickly access relevant information suited to their skill levels. During offboarding, these agents capture departing employees’ knowledge, preserving expertise and maintaining continuity.
Compliance Analysis:
AI agents can review contracts, policies, and procedures with regulatory insight, flagging compliance issues and saving time for legal teams while ensuring adherence to evolving standards.
Intellectual Property (IP) Research:
AI agents streamline patent research by identifying relevant patents, comparing claims, and suggesting new opportunities, freeing IP professionals to focus on strategic analysis rather than exhaustive document searches.
The Rise of AI Agent Ecosystems: Interoperability and Recent Innovations
For AI agents to achieve their full potential, they need the ability to interact not only with humans but also with other agents, tools, and systems. Big Tech is setting the stage for agent ecosystems (Ghaffary, 2024), where multiple AI agents can work in tandem. Microsoft’s “Copilot” vision, for example, provides for a marketplace of AI Agents, allowing seamless interaction with ubiquitous tools like MS Teams, Excel, and Word. Salesforce, with their new Agentforce product line, offers agents that pull data from across the Salesforce ecosystem to provide direct action on customer support and sales.
In these ecosystems, agents communicate with each other and understand and access APIs of existing business applications, which opens up completely new possibilities for KM workflows. With the introduction of agents that can understand and operate a computer screen such as the recent model launched by Anthropic, the IT integration puzzle also seems largely solved. We are already seeing significant growth in the AI agent startup ecosystem, with hundreds of new companies emerging in 2024, and this number is projected to expand to thousands by 2025 (Owyang, 2024).
While critics of AI Agents are quick to point out the analogy to early AI assistants like Microsoft's Clippy, which only offered very basic, and often intrusive help, today's AI agents have evolved significantly and the underlying LLM technology plays in a different league altogether. In 2024, these agents can operate seamlessly in the background, autonomously managing complex tasks, scheduling, information retrieval and API use. Eighty-two percent of business leaders surveyed said they expect to integrate AI Agents into their businesses in the next few years (Boulton, 2024).
Embracing the Future of AI for Knowledge Management
AI agents represent a new frontier in knowledge management, transitioning from simple interactions to sophisticated, autonomous problem-solving for complex tasks. For enterprise leaders, the opportunity is clear: deploying domain-specific AI agents that leverage reliable retrieval systems and operate within dynamic ecosystems can significantly enhance team capabilities. Gartner advises IT leaders to "look for agentic AI in your technology stack" (Coshow, 2024). At the same time, the goal of AI implementation should not be to replace human judgement, but to augment the workforce with superpowers that boost productivity, and increase quality and business growth, in a world where the supply of experts and skilled labor is increasingly limited.
Zeta Alpha’s work in this field demonstrates the transformative potential of these agents. With the Zeta Alpha RAG Agents SDK, the recently introduced AI Research Assistant, and many domain-specific applications across knowledge-intensive industries, we showcase how AI agents can be developed and deployed in production at scale. In this way, AI isn’t just augmenting human knowledge management—it is redefining the possible in enterprise KM.
As enterprises embrace this shift, leaders will be faced with questions about implementation, integration, and reliability. However, those who take the leap will find that the rewards go beyond incremental improvements—they’re unlocking true KM superpowers that will shape the future of their organizations. The evolution from chatbots to AI agents is here, and for forward-thinking leaders, the time to act is now.
References
Clint Boulton, August 19, 2024. AI agents loom large as organizations pursue generative AI value, CIO Magazine. https://www.cio.com/article/3488553/ai-agents-loom-large-as-organizations-pursue-generative-ai-value.html
Tom Coshow, October 1, 2024. Intelligent Agents in AI Really Can Work Alone. Here’s How. Gartner. https://www.gartner.com/en/articles/intelligent-agent-in-ai
Deloitte. August 2024, The State of Generative AI in the Enterprise - Moving from potential to performance. https://www2.deloitte.com/us/en/pages/consulting/articles/state-of-generative-ai-in-enterprise.html
Shirin Ghaffary, February 15, 2024. Tech Companies Bet the World is Ready for 'AI Agents'. Bloomberg. https://www.bloomberg.com/news/newsletters/2024-02-15/tech-companies-bet-the-world-is-ready-for-ai-agents
Jeremiah Owyang, September 12, 2024. Plot Your Path: The 2024 AI Agent Ecosystem Map. Web Strategist. https://web-strategist.com/blog/2024/09/12/plot-your-path-the-2024-ai-agent-ecosystem-map/
Kylie Robison, October 10, 2024. Agents are the future AI companies promise - and desperately need. The Verge. https://www.theverge.com/2024/10/10/24266333/ai-agents-assistants-openai-google-deepmind-bots
Steven Rosenbush, October 16, 2024. AI Agents Can Do More Than Answer Queries. That Raises a Few Questions. Wall Street Journal. https://www.wsj.com/articles/ai-agents-can-do-more-than-answer-queries-that-raises-a-few-questions-15009853 A slightly modified version of this blog post appeared in KMWorld 2024 Sound Off: Conference Highlights and 2025 Forecast.