At Microsoft Ignite 2024, Microsoft released Azure AI Agent Service. In Azure AI Foundry, you can directly build AI Agents based on intelligent business through Azure AI Foundry SDK and Azure AI Agent Service UI. Compared with Azure AI Assistant API, Azure AI Agent Service allows you to have more flexible models, such as directly calling open source LLM such as Llama 3, Mistral and Cohere, better enterprise function access (Microsoft Fabric, Microsoft SharePoint, Azure AI Search, Microsoft Bing, etc.), stronger enterprise security mechanisms and data storage methods.

Azure AI Agent Service is in the Public Preview stage. Using Azure AI Foundry SDK, developers can quickly build agents based on Azure AI Agent Service using Python or C#. Enterprises will have different AI Agents based on their business, so how should these AI Agents be combined in the workflow? We need to use AutoGen or Semantic Kernel to orchestrate these three AI Agents.

Tips for creating AI Agents in Azure AI Foundry SDK

When creating a project in Azure AI Foundry, it is recommended to use this template 

Note

1. Azure AI Agent Service is currently in Public Preview, so we need to use it in specific regions and models.

You can also use the Azure AI Foundry model catalog

• • Llama 3.1-70B-instruct
  • Mistral-large-2407
  • Cohere command R+

     2. Azure AI Foundry SDK now supports Python / C# versions

Python for Azure AI Foundry SDK

pip install azure-ai-projects pip install azure-identity

.NET for Azure AI Foundry SDK

dotnet add package Azure.AI.Projects --version 1.0.0-beta.1

 

     3. If you want to learn how to create an AI Agent with Azure AI Foundry SDK, please refer to https://learn.microsoft.com/en-us/azure/ai-services/agents/quickstart

The following is an introduction to a scenario, such as a blog scenario. We can define three AI Agent Services around this creation scenario: an agent for content collection, an agent for writing skills, and an agent for saving content functions. Then, we can orchestrate the AI ​​Agent through AutoGen / Semantic Kernel.

Azure AI Agent Service defines a single agent

Azure AI Agent Service defines more single agents for specific intelligent tasks. For example, you can define an agent for data mining, an agent that links to Azure Function, or an agent that uses Function Calling to access third-party email services. For the agent mentioned in the above scenario, we can define it like this

🔍Content collection agent

The content collection agent mainly searches for the content of the blog outline through search engines to add more detailed content to the blog. Azure AI Agent Service provides Grounding with Bing service for extension, and you can easily access Bing search.

Samples

• Python https://github.com/kinfey/MultiAIAgent/blob/main/03.AzureAIAgentWithAutoGen01.ipynb
• C# https://github.com/kinfey/MultiAIAgent/blob/main/08.AzureAIAgentWithSK01.ipynb

📖Writer Agent

Agents who write skills can directly complete the written content through LLM.

🛠️Save Agent

The agent that saves content completes the work of saving content through the code-interpreter of Azure AI Agent Service.

Samples：

• Python https://github.com/kinfey/MultiAIAgent/blob/main/01.AzureAIAgentCode.ipynb
• C# https://github.com/kinfey/MultiAIAgent/blob/main/05.AzureAIAgentCodedotNET.ipynb

Use AutoGen / Semantic Kernel to orchestrate multi-agents

In enterprise scenarios, we have different single agents to help us complete different tasks. Based on different tasks, we can combine different agents to complete related tasks. If we need to send a blog, we can combine the above three agents to complete the task. AutoGen / Semantic Kernel both support the orchestration of agents. We can use AutoGen's Function Calling / Semantic Kernel's Plugin to complete the binding of Azure AI Agent Service to complete the orchestration of the above three agents.

The following is the specific architecture

AutoGen's orchestration

Sample - click

Semantic Kernel's orchestration

Sample - click

By orchestration multi-agents, we can easily completely different workflows. The combination of multiple agents allows us to quickly completely different tasks. With the enhancement of LLM/SLM capabilities, there will be better integration in enterprise business functions. AI Agent as Service will come, and better integration of multiple agents can improve our work efficiency and move into the real AI era.

Resources

1. Learning about Azure AI Agent Service https://learn.microsoft.com/en-us/azure/ai-services/agents/
2. Learning about Microsoft AutoGen https://microsoft.github.io/autogen/dev/
3. Learning about Microsoft Semantic Kernel https://github.com/microsoft/semantic-kernel

 

Our most popular RAG solution for Azure has now been deployed thousands of times by developers using it across myriad domains, like meeting transcripts, research papers, HR documents, and industry manuals. Based on feedback from the community (and often, thanks to pull requests from the community!), we've added the most hotly requested features: support for multiple document types, chat history with Cosmos DB, user account and login, data access control, multimodal media ingestion, private deployment, and more.

This open-source RAG solution is powerful, but it can be intimidating to dive into the code yourself, especially now that it has so many optional features. That's why we're putting on a 10-part live series in January/February, diving deep into the solution and showing you all the ways you can use it. 

Register for the whole series on Reactor or scroll down to learn about each session and register for individual sessions. We look forward to seeing you in the live chat and hearing how you're using the RAG solution for your own domain. See you in the streams! 👋🏻

The RAG solution for Azure

13 January, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we'll dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat Join us for the kick-off session, where we'll do a live demo of the RAG solution and explain how it all works. We'll step through the RAG flow from Azure AI Search to Azure OpenAI, deploy the app to Azure, and discuss the Azure architecture.

Customizing our RAG solution

15 January, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we'll dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat In our second session, we'll show you how to customize the RAG solution for your own domain - adding your own data, modifying the prompts, and personalizing the UI. Plus, we'll give you tips for local development for faster feature iteration.

Optimal retrieval with Azure AI Search

20 January, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat Our RAG solution uses Azure AI Search to find matching documents, using state-of-the-art retrieval mechanisms. We'll dive into the mechanics of vector embeddings, hybrid search with RRF, and semantic ranking. We'll also discuss the data ingestion process, highlighting the differences between manual ingestion and integrated vectorization

Multimedia data ingestion

22 January, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat Do your documents contain images or charts? Our RAG solution has two different approaches to handling multimedia documents, and we'll dive into both approaches in this session. The first approach is purely during ingestion time, where it replaces media in the documents with LLM-generated descriptions. The second approach stores images of the media alongside vector embeddings of the images, and sends both text and images to a multimodal LLM for question answering. Learn about both approaches in this session so that you can decide what to use for your app.

User login and data access control

27 January, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat In our RAG flow, the app first searches a knowledge base for relevant matches to a user's query, then sends the results to the LLM along with the original question. What if you have documents that should only be accessed by a subset of your users, like a group or a single user? Then you need data access controls to ensure that document visibility is respected during the RAG flow. In this session, we'll show an approach using Azure AI Search with data access controls to only search the documents that can be seen by the logged in user. We'll also demonstrate a feature for user-uploaded documents that uses data access controls along with Azure Data Lake Storage Gen2.

Storing chat history

29 January, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat Learn how we store chat history using either IndexedDB for client-side storage or Azure Cosmos DB for persistent storage. We'll discuss the API architecture and data schema choices, doing both a live demo of the app and a walkthrough of the code.

Adding speech input and output

3 February, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat Our RAG solution includes optional features for speech input and output, powered either by the free browser SDKs or by the powerful Azure Speech API. We also offer a tight integration with the VoiceRAG solution, for those of you who want a real-time voice interface. Learn about all the ways you can add speech to your RAG chat in this session!

Private deployment

5 February, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat To ensure that the RAG app can only be accessed within your enterprise network, you can deploy it to an Azure virtual network with private endpoints for each Azure service used. In this session, we'll show how to deploy the app to a virtual network that includes AI Search, OpenAI, Document Intelligence, and Blob storage. Then we'll log in to the virtual network using Azure Bastion with a virtual machine to demonstrate that we can access the RAG app from inside the network, and only inside the network.

Evaluating RAG answer quality

10 February, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat How can you be sure that the RAG chat app answers are accurate, clear, and well formatted? Evaluation! In this session, we'll show you how to generate synthetic data and run bulk evaluations on your RAG app, using the azure-ai-evaluation SDK. Learn about GPT metrics like groundedness and fluency, and custom metrics like citation matching. Plus, discover how you can run evaluations on CI/CD, to easily verify that new changes don't introduce quality regressions.

Monitoring and tracing LLM calls

12 February, 2025 | 11:30 PM UTC | 3:30 PM PT

Register for the stream on Reactor

In this series, we dive deep into our most popular, fully-featured, and open-source RAG solution: https://aka.ms/ragchat When your RAG app is in production, observability is crucial. You need to know about performance issues, runtime errors, and LLM-specific issues like Content Safety filter violations. In this session, learn how to use Azure Monitor along with OpenTelemetry SDKs to monitor the RAG application.

Welcome in! You’ve entered, Only Malware in the Building. Join us each month to sip tea and solve mysteries about today’s most interesting threats. Your host is Selena Larson, Proofpoint intelligence analyst and host of their podcast DISCARDED. Inspired by the residents of a building in New York’s exclusive upper west side, Selena is joined by N2K Networks Dave Bittner and Rick Howard to uncover the stories behind notable cyberattacks. 

Being a security researcher is a bit like being a detective: you gather clues, analyze the evidence, and consult the experts to solve the cyber puzzle. On this episode, we talk about the year's most impactful cyber trends and incidents—from the Snowflake hack and Operation Endgame to the rise of multi-channel scams and explosive growth in web inject attacks. Ransomware continued to wreak havoc, especially in healthcare, while callback phishing and MFA-focused credential attacks kept defenders on high alert. Join us as we reflect on these challenges and look ahead to what’s next in 2025.

Download audio: https://traffic.megaphone.fm/CYBW3926030913.mp3?updated=1736189894

This article covers setting up Elastic Job agents, creating jobs, and automating tasks across multiple databases. Streamline database management, schedule tasks efficiently, and enhance performance with Elastic Database Jobs.