In my earlier blog post, we introduced the Microsoft Agent Framework.
We saw how the new open-source SDK from Microsoft blends the best features from Semantic kernel, Auto Gen and the Process Framework.
In this blog post we take an initial first look at the Microsoft agent framework in terms of the components that form this new open-source SDK.
Specifically, we cover the following:
- What is an AI agent
- Available agent types
- When to use an AI agent
- When not to use an AI agent
- Conversations and threads
- Agent Function tools
- Agents as function tools
- Memory and memory types
- Middleware
- Background processing
- Observability
- Workflows
This blog post is about the main concepts, components and patterns that are core to shipping AI agents using the Agent Framework.
Based on prior project experience, I find some of these components and tooling to be similar to others I’ve used in the past. For example, ChatHistory in Bot Framework and Semantic Kernel.
These are also the topics I will be digging into deeper over the next few weeks and months.
~
What Is an AI Agent
The industry has grappled with definitions for an AI agent the dust is straight over and most agree that AI agent is a software application that consists of:
- Mechanisms to accept human prompts
- A language model to generate responses from human prompts
- features to identify and select relevant tooling to taken actions to satisfy a human prompt
Access to vector databases and more recently, MCP servers are also common capabilities that we’re seeing being part of the agentic AI landscape.
~
Available Agent Types
The Microsoft Agent Framework provides support for multiple types of agents. All agents are derived from a common base class, AIAgent. The available agent types can be seen below.
Throughout my Semantic Kernel projects, I typically use the OpenAI ChatCompleteion types.
Learn more about available agent types here.
~
When to Use an AI Agent
AI agents are useful in situations where you need autonomy, iterative decision making and conversational experiences.
Common use cases include:
- customer support
- education and self-development
- code generation or analysis
- brainstorming and ideation
Due to their non-deterministic nature, I tend to advise that AI agents are best deployed in low to medium risk use cases.
~
When Not to Use an AI Agent
AI agents can go rogue and hallucinate. Due to this, I often don’t use or advise using AI agents in high-risk use cases.
Hallucinations and the non-deterministic nature of AI agents mean you need some form of guard rails to verify an AI agents output or action.
Typically, the ways you can enforce guard rails:
- introduce human in the loop
- write traditional code, methods
- create a specific function tool for the agent to call (a form of traditional code)
All this said, an AI agent can struggle with complex tasks that involve multiple steps or decision points.
Complex tasks can require the invocation of multiple tools, API calls, and custom code.
It can be hard for a single agent to orchestrate, maintain state and manage.
The older Process Framework was an initial attempt to remedy this. A new technology is available to within the Agent Framework to help with this -Workflows.
We’ll cover Workflows in detail in a future blog post.
I dive into hallucinations in my Pluralsight course “Vector Databases and Embeddings for Developers”.
You can find this course here.
~
The Anatomy of an AI Agent
Multiple Components come together to form an AI agent. Lights on park where some of these are you’ll notice that many of them are like what was available in Semantic Kernel. Some are new.
Conversations and Threads
Agents are stateless and don’t maintain any between calls. To have a multi-turn conversation with an agent, you need to create an object to hold the conversational state and pass this object to the agent when running it.
This is where conversations and threads come into play.
To create the conversation state object, you call the GetNewThread method on an agent instance.
If you don’t call this method, an disposable in-memory thread is created. This is only available for the duration of the run.
Agent Function Tools
Most agents require access to external capabilities and functionalities. This is where function tools come in handy. These are like Native Functions and Plugins that are available in Semantic Kernel.
Human in The Loop
A new feature that I don’t recall in Semantic Kernel -but is available in the Agent Framework, is being able to implement a function tool that require human approval before execution.
You can implement this manually of course but having an out of the box option is helpful.
To implement human-in-loop, you simply wrap your function tools with the ApprovalRequiredAIFunction command. A nice touch.
Agents as Function Tools
When building function tools, they will likely handle a discrete task. A handful of inputs and single output or action.
The Agent Framework had a feature that lets you use an instance of the AIAgent class as a function tool.
This is done by calling the AsAIFunction() method and providing it as a tool to another agent.
This makes it possible for you to daisy-chain agents and build more complex workflows.
One agent passing the baton to the other.
Memory and Memory Types
You need a way to maintain context between conversations. This is where memory comes into play.
Memory can also be used to store user or application settings that you want to load when a human or machine interacts with your AI agent.
There are 2 flavours of memory in the Agent Framework:
- short term memory
- long term memory
Several memory mechanisms are available. You can use in-memory, databases or a specialised memory service. This offering reminds me of the Bot Framework days.
Short-Term Memory – Chat History
Multiple memory offerings are available when working with short term memory.
- In-memory chat history
- Service / model inferred chat history storage
- 3rd party chat history storage
In-memory Chat History Storage
This a good option to use in several use cases. Mainly during development or if you are using a service that doesn’t support in-service stored of chat history.
When using this option, the AgentFramework stores chat history and agent memory in the AgentThread object by default. Both agent and human data is stored.
One nice feature is out of the box chat reduction.
When interacting with a model, the chat history can grow. You don’t want it continually growing throughout the conversation.
I published a blog post about this when working with Semantic Kernel. In that post, I detail how you can optimise chat history. With the Microsoft Agent Framework, you get this out of the box.
It’s done by implementing the Microsoft.Extensions.AI.IChatReducer interface.
Service / Model Inferred chat history storage
Certain models require or have options to store chat history in-service. One example of this is the Open AI Responses API.
When using a service/model inferred chat history, the Agent Framework will store the id of the remote chat history in the AgentThread object.
Learn more about Open AI Responses API here.
3rd Party Chat History Storage
If you want a more robust chat history storage mechanism and none are available in the service/model inference option you are using, you can use a 3rd party chat storage mechanism.
Learn how to create your own custom message store here.
Middleware
Sometimes you need to intercept agent interactions at different points. This is where middleware comes into play.
Often I’ll use middleware to inject logging perform security checks, trap errors or to sanitise content being received or sent by an agent.
Logic in middleware often sets outside your core agent or application code.
The agent framework offers three different types of middleware:
- agent run middleware
- function calling middleware
- a chat client middleware
All the types of middleware are implemented via a function callback.
Learn more about middleware here.
Background Responses
In my opinion, this is one of the most exciting features and the Agent Framework. Most of the agents we have seen in the last few years involved a traditional chatbot experienced -human prompt and agent response. A one-to-one interaction.
These are helpful but in the real world, an approach is often needed to handle operations that can take hours to complete.
This might be tasking an agent with processing thousands of database records, crunching financial figures or generating new metrics.
Consider implementing background responses as part of your RAG pipeline during content ingestion, chunking, pre-processing or post-processing operations.
Background responses are enabled by setting AllowBackgroundResponses = true in the AgentRunOptions parameter.
At the time of writing, only agents that use the Open AI Responses API support background processing.
These are available via directly via Open AI and Azure Open AI Responses Agent.
Find a working example here.
Observability
Observability helps you track and monitor how your agent is performing. Use this to help you diagnose any issues that may arise during the execution of your agent.
Like ASP.NET, logs can be captured via logging framework of your choice. The Agent framework also integrates with Open Telemetry. You can also use the Open Telemetry SDK to export metrics to an Azure Monitor Resource.
Learn more about observability here.
~
Workflows
In my mind, Workflows are the replacement for the Process Framework. Agent Framework Workflows help you enforce guardrails within your agentic AI system.
Whereas an AI agent is dynamic and non-deterministic in terms of how it reaches a defined goal, a workflow is explicitly defined.
In my opinion, this has been a missing piece in the agentic AI ecosystem. It helps provide more comfort around using agentic ai in medium to high-risk use cases.
Of course, you still need a human-in-the-loop process for verification purposes. I will cover Workflows in future blog posts.
Learn more about workflows here.
~
Summary
In this blog post, we’ve taken an initial first look at the Microsoft agent framework. We’ve look at some of the key components that form this new open-source SDK.
In future blog posts, we’ll explore each of these in more detail.
~
Enjoy what you’ve read, have questions about this content, or would like to see another topic?
Drop me a note below.
You can schedule a call using my Calendly link to discuss consulting and development services.
~
Further Reading and Resources
Some more resources to help you:
Code and Engaging
Videos
Learn
~
Log in
