Co Author: JiteshThakur
AI agents are useful because they can act. They call tools, query databases, send messages, and hand work to other agents. That same freedom creates a problem: access control can tell you which service an agent may reach, but it does not always tell you whether a particular action is sensible, safe, or permitted. MCP is how most agents now act.
Two Control Points:
This post examines two control points that address different parts of the MCP lifecycle.
- Agent 365 CLI evaluates the MCP server before an agent uses it.
- Agent Governance Toolkit (AGT) governs sensitive tool calls while the agent runs.
One improves what the agent sees. The other governs what the agent does.
The Agent 365 CLI is a cross-platform command-line tool for Agent 365 applications on Azure. Its evaluation command examines MCP tool definitions and scores their quality. AGT evaluates actions against policy and records each decision.
Together, these tools support a practical model: evaluate the server first, then provide proper scaffolding for the developer to test this in a dry run.
Agent 365 CLI: Score an MCP server from the command line:
The Agent 365 CLI can evaluate an MCP server against research-based practices for production readiness. The result is more useful than a simple pass or fail.
The evaluation gives you:
- A score for each tool name, description, and parameter schema;
- A prioritized list of improvements;
- An overall maturity score for the server; and
- Local output that you can use early in development.
This report turns a vague question, "Is this MCP server ready?", into a concrete list of work.
The evaluate command
a365 develop-mcp evaluate --server-url <server-url> [--auth-token <auth-token>] [options]
The command reads the tool schemas from the server. It then produces guidance for names, descriptions, parameters, and schema structure.
A local coding-agent CLI scores the semantic checks. You can use GitHub Copilot CLI or Claude Code under your account and AI subscription. The command does not send tool-schema data to Microsoft.
Prerequisites:
Install the following software:
- Agent 365 CLI;
- Node.js 18 or later for GitHub Copilot CLI; and
- A supported coding-agent CLI for semantic scoring.
For example, install GitHub Copilot CLI with this command:
powershell npm install -g @github/copilot
This bring-your-own-LLM model keeps the scoring step in your local development environment. It is useful when model calls must remain inside an approved deployment.
How the evaluation works
The command runs a five-step pipeline and logs progress as it goes.
Fig 1: MCP Evaluation using Agent 365 Cli
- Connect to the MCP server and collect its tool schemas.
- Generate an evaluation checklist in the output directory.
- Score the semantic checks with the selected coding agent.
- Calculate the maturity level and action priorities.
- Write the JSON and HTML reports.
The evaluation contains two types of checks:
- Deterministic checks use exact rules in the CLI. For example, a tool name cannot be empty.
- Semantic checks use a coding agent to score clarity and meaning. Each result includes a reason for the score.
Examples
Set the authentication token in an environment variable. Then evaluate an authenticated server and write the artifacts to a subfolder.
powershell $env:A365_MCP_AUTH_TOKEN = "<bearer-token>" a365 develop-mcp evaluate --server-url "https://my-mcp-server.contoso.com/mcp" --output-dir "./eval"
Use a specific scoring engine with the `--eval-engine` option:
powershell a365 develop-mcp evaluate --server-url "http://localhost:5000/mcp" --eval-engine claude-code
Scenario: Evaluate a malicious MCP server
For this demonstration, we hosted a deliberately malicious MCP server at `http://127.0.0.1:8124/`. It exposes tools that demonstrate tool poisoning, credential leakage, prompt injection, schema mismatch, sandbox escape, and other attacks.
The server is intentionally unsafe and is for demonstration only.
Fig 2: Setting up a test MCP server for evaluation
We ran the evaluation in two steps. First, we generated the checklist without automatic semantic scoring:
a365 develop-mcp evaluate --server-url "http://127.0.0.1:8124/" --eval-engine none
Fig 3: Agent365CLI MCP Evaluation
The command wrote the checklist and a semantic-evaluation prompt to the output directory. It also displayed the next steps.
Second, we gave the prompt and checklist to a coding agent. The agent completed each unscored semantic check with a Boolean score and a short reason. After we saved the completed checklist, we ran the command again to generate the report:
a365 develop-mcp evaluate --server-url "http://127.0.0.1:8124/" --output-dir "C:\temp\MaliciousMCP"
Fig 4: Creating the report with Agent365 CLI MCP Evaluate command
Understanding the evaluation report
Open `<server-name>_eval_report.html` from the output directory. The report contains:
- The overall score from 0 to 100;
- The maturity level from 0 to 4;
- Scores for each tool and quality category; and
- A prioritized action list for the next maturity level.
Fig 5: MCP Evaluation Report
In our demonstration, the server scored 86.0 and reached Level 3: Optimized for AI. That strong overall score did not mean that every tool was safe or clear. The report found 58 action items, including one critical item and 33 high-priority items.
That contrast matters. A server can have valid schemas and consistent names while still exposing misleading or dangerous tools.
Fig 6: MCP Evaluation Report - Tool-By-Tool Detail
What to look for
Read the per-tool results before the overall score. A single weak tool can create more risk than the server average suggests.
Focus on these report sections:
- Tool names: Can an agent select the correct tool from its name?
- Tool descriptions: Does each description explain the purpose and correct use?
- Parameter names: Do the names identify the data that the tool requires?
- Parameter descriptions: Do they explain the format, type, and constraints?
- Schema structure: Are the schemas valid and processable?
- Action items: Which changes have the highest effect on tool selection and use?
The command processes static tool schemas from `tools/list`. It does not process runtime payloads, end-user data, or personal data. The command keeps the `--auth-token` value in memory. It sends the value only in the HTTP `Authorization` header. It does not write the token to disk or give it to the coding agent.
AGT: Put governance in the execution path:
Microsoft's open-source Agent Governance Toolkit (AGT) evaluates an action before execution. It adds identity and policy context, records the decision, and can send risky work for approval. This can be used by developers during the build time for dynamic evaluation of the MCP server.
AGT lets developers put part of that intent into the execution path. Remote tools still need secure implementations, sandboxes need hard boundaries, and audit records need appropriate storage and access controls. You do not need to replace your agent framework to use it.
What sits in the decision path?
AGT wraps the tools that an agent already uses. You can start to govern a tool with two lines of Python:
python from agentmesh.governance import govern safe_tool = govern(my_tool, policy="policy.yaml")
On each call, `safe_tool` evaluates the configured policy. An allowed action reaches the original tool. A denied action raises `GovernanceDenied` and creates a decision record. This wrapper model reduces the cost of adoption. Teams can add governance to an existing agent stack without rebuilding it.
AGT supports Python, TypeScript, .NET, Rust, and Go. Its documented integrations include popular agent frameworks, MCP, and A2A. Teams can also adopt AGT in stages. A team can begin with policy checks and audit records. It can add identity, approvals, sandboxing, and operational controls as risk increases.
Each control answers a different question:
- Policy: Is this action allowed?
- Identity and trust: Which agent made the request?
- Runtime controls: What limits apply to execution?
- Audit evidence: Why did AGT allow or deny the action?
A low-risk assistant can need only a deny rule and basic logging. An agent that moves money or changes production systems needs stronger controls.
Fig 7: AGT Architecture
Scenario: Govern the same malicious MCP server
For this scenario demonstration, we used AGT Python packages as an MCP gateway. The gateway sat between an agent and the same malicious server from the earlier evaluation. This setup let us examine both control points against one target. The Agent 365 CLI examined the server's static tool definitions. The AGT gateway examined real requests and responses for the developer during its testing.
Fig 8: AGT findings at runtime
In the policy interface, a developer can edit runtime limits and detection rules. The developer can also validate the policy against sample tool metadata, save a revision, and activate it with a recorded reason.
Fig 9: AGT control coverage
The control-coverage view shows which AGT capabilities are active in the gateway. It also links each capability to package checks and end-to-end evidence.
In our demonstration, we included the following controls:
- Tool metadata poisoning detection;
- Tool change and rug-pull detection;
- Dangerous argument blocking;
- Tool-response content scanning;
- Per-client tool-call budgets; and
- A redacted decision audit trail.
You can build your detection & input security by reading more about it here.
The gateway detected malicious content. For one blocked `tools/list` request, it recorded the findings. The important result was not only that AGT blocked the request. It also preserved the matched evidence, affected tool locations, policy modes, and request context.
Fig 10: Example detection via AGT
The dashboard then summarized block-mode findings, leading risk drivers, and tools that required review. This evidence can help a team prioritize policy changes and investigate repeated attacks.
Fig 11: Sample AGT metrics
AGT does not require this UI, gateway, or architecture. Its structured decisions can feed an admin console, SIEM, incident workflow, or approval queue.
AGT also includes an Agent Compliance package with mappings for OWASP and other controls. These mappings give developers and governance teams a common record of applied controls. Teams do not need to reconstruct the agent's behavior after an incident. Check Compliance - Agent Governance Toolkit for more information.
Conclusion:
MCP safety needs controls before and during execution. The Agent 365 CLI improves the MCP interface before deployment. It exposes unclear tool definitions, scores server maturity, and turns quality gaps into prioritized work. While AGT is implemented at the build phase, it provides developers the ability to test policy, identity, execution context & preserve evidence for allowed or denied decisions. Neither tool replaces secure server code, strong sandbox boundaries, or protected audit storage. Instead, they make those controls easier to evaluate and explain. Start with one MCP server and one consequential tool call. Evaluate the server with the Agent 365 CLI. Then put an AGT policy around the action that carries the most risk.
While these controls help secure the build phase of an agent, once agents move into production, runtime controls become essential. Agent365 provides those controls at runtime.