Why LLMs Aren't Enough: The Need for Specialized AI Agents

Riding the LLM Wave: Their Superpowers and Shortcomings

In the last year, we've witnessed a surge in the usage of Large Language Models (LLMs), advanced AI systems capable of understanding and generating human-readable text. It's no surprise, given their versatility in what they can do, from explaining complex concepts to aiding developers in debugging code or even concocting yummy recipes for your next meal. These models can comprehend prompts written in human-readable language, akin to speaking with a knowledgeable mentor and receiving intelligible responses.

Despite their diverse applications, LLMs currently have limitations in handling complex tasks that require accessing current or dynamic information. This is due to their knowledge cutoffs, which are based on the time of their pretraining.

For instance, ChatGPT 3.5 has a knowledge cutoff of January 2022. So, inquiries about 2022’s highest-grossing film or tomorrow's weather would likely draw a gentle reminder of said knowledge cutoff and suggest you seek other sources.

Another limitation is their vulnerability to hallucinations, generating content that appears factual with utmost confidence, though lacks accuracy, or worse- is entirely false.

Rather than using them as a primary information source, it might be more helpful to see them as a reasoning aid, or “reasoning engines”. This makes LLMs incredibly valuable for tasks that demand logical reasoning. So, instead of utilizing their shaky knowledge retrieval skills, what if there was a way we could leverage their reasoning skills? Let’s put a pin in this.

Running in Circles: Task Navigation

You may want help on an overall objective, say a primary goal with numerous subtasks. What would you do? You might start by listing tasks, directing ChatGPT, and manually handling those outside its scope, such as researching current events. Maybe, as these tasks are crossed off the list, you realize you need to reprioritize these tasks. Or wait! Turns out you need to add more tasks to this list. The list grows, and you’re trapped in task-evolving, priority-shifting hell.

A tedious ordeal, indeed. What if managing all this chaos was someone else’s job? Enter AI agents, programs that can take an objective and perform and manage all the tasks we discussed above.

How exactly can AI agents do all this? Let’s revisit LLMs’ reasoning prowess and how AI agents can leverage it.

The Power of Synergy: LLMs and AI Agents

We feed our agent a prompt containing our desired objective. Paired with an LLM, our agent deciphers the prompt, constructs a task list, and dynamically adapts as it completes tasks, constantly reshaping until the objective is reached. This automated process not only simplifies complexity into seamless efficiency but also empowers individuals without expertise to translate their ideas into reality, fostering innovation.

These AI agents can be configured to access and browse the internet, compensating for LLMs’ limitations regarding real-time data. They might even run your computer autonomously, automating tasks requiring human-input. Perhaps we might pair an agent with domain-specific LLMs, unlocking a vast realm for specialized AI agents in industry-specific tasks.

It’s also worth noting that agents predate LLMs, showcasing a longstanding history of autonomy-driven frameworks such as JADE (Java Agent Development Framework) and JAICS (Java Intelligent Agent Componentware). Agents have existed as an approach to automating tasks and processes, laying the foundation for the sophisticated AI agents we employ today and serving as the backbone of many automated systems long before the rise of LLMs.

Pairing LLMs with agents makes for a natural conversation workflow that allows users to take part in the process when needed while still understanding the behind-the-scenes operations. This seamless integration facilitates collaboration between humans and AI, bridging the gap between technical expertise and user-friendly interaction. Users can now harness the power of AI without extensive training, fostering a more inclusive and accessible environment for innovation.

Agents Today

Various open-source projects allow you to interact with agents, create custom agents, or even build production ready multiagent-based applications. The following are a few notable projects.

1. BabyAGI: BabyAGI is a python script for an AI-powered task management system. This project leverages several agents with respective jobs to create, prioritize, and execute tasks based on a given objective. These 3 agents are powered by LLMs. Another non-LLM-based agent is used for storing results in a database to maintain the context.
2. AutoGPT: AutoGPT is another task-driven project with a vision to make AI accessible to everyone, providing users the tools to build and customize their own agents, as well as providing out-of-the-box agents.
3. Superagent: Superagent is a framework that allows you to build, manage, and deploy AI assistants powered by one or many agents. You can use these assistants to automate workflows, generate content, or use them in support chatbots that can reference documents to help answer customer questions.
4. AutoGen: Microsoft’s Autogen is another open-source framework for building applications powered by agents with a focus on conversational interactions between multiple agents. AutoGen allows you to define roles and capabilities to agents, how they interact, and the extent of human participation if needed.

Other notable agents and agent frameworks to checkout include CrewAI, MetaGPT, and AgentGPT.

Real World Applications

Now that we've explored the task-performing capabilities of LLM-based agents, let's delve into real-world scenarios across different sectors. These scenarios showcase how specialized agents cater to specific industry needs with their unique roles.

1. Healthcare: Specialized agents can assist in medical research by analyzing vast volumes of clinical data to identify patterns in diseases or treatment effectiveness. They could help in creating personalized treatment plans based on patient records and medical literature, aiding doctors in making informed decisions. Additionally, they might streamline administrative tasks by processing patient queries and scheduling appointments. These agents could make use of Med-PaLM, Google’s LLM designed for healthcare.
2. Finance: In finance, these agents, possibly paired with BloombergGPT or FinGPT, could predict market trends by analyzing historical data and news articles. They could automate customer service by addressing routine inquiries about account information or transaction details. Additionally, they might assist in risk assessment and compliance by analyzing legal documents and regulations to ensure adherence to financial laws.
3. Legal Services: Specialized agents could perform legal research by scanning through immense legal databases to find precedents and relevant case laws. They could assist in drafting contracts and legal documents by providing suggestions based on various clauses and legal standards. Moreover, they might automate document review processes, increasing efficiency in law firms.
4. Customer Support: For customer support, these agents could handle frequently asked questions, troubleshoot basic technical issues, and provide product information to customers. They could also assist in sentiment analysis of customer feedback to identify areas for improvement in products or services. Furthermore, they might aid in routing more complex queries to human representatives while providing relevant context.
5. Manufacturing: In manufacturing, specialized agents might optimize supply chain management by analyzing market trends and forecasting demand for raw materials. They could assist in quality control by analyzing product specifications and identifying potential defects or areas for improvement. Additionally, these agents might facilitate maintenance processes by predicting equipment failures based on historical data, minimizing downtime.

What's Next?

While LLMs might not be the go-to for complex and meticulous tasks alone, tapping into their reasoning abilities and teaming them up with AI agents unlocks a realm of opportunities across industries. This collaboration empowers the handling of routine tasks and industry-specific challenges. Whether it's tailoring healthcare treatments, predicting market trends in finance, or conducting thorough legal research, blending LLMs with specialized AI agents sparks a revolution in efficiency, decision-making, and customer interactions. This synergy between LLMs and AI agents points to an era where tasks once deemed daunting now invite innovation, transforming possibilities into realities. Harnessing the power of multiagent systems adds a layer of sophistication to this collaboration, paving the way for seamless orchestration and heightened problem-solving capabilities.

At Dragonscale Industries, we are working on Rustic AI, an open-source framework set to redefine how you build AI systems using agents. By enabling users to build a "Guild" of specialized agents, Rustic AI goes beyond task execution, emphasizing understanding and evolution. Imagine the collective power of specialists, making their expertise accessible to anyone with a specific goal in mind. Rustic AI represents a step forward in AI evolution, promising a future where the collaboration of intelligent systems brings knowledge and capabilities to the fingertips of those seeking to achieve their objectives.