MariaDB 12.3 has a new feature enabled by the option binlog_storage_engine. When enabled it uses InnoDB instead of raw files to store the binlog. A big benefit from this is reducing the number of fsync calls per commit from 2 to 1 because it reduces the number of resource managers from 2 (binlog, InnoDB) to 1 (InnoDB). See this blog post for more details on the new feature.

My previous post had results for sysbench with a small server. This post has results for the Insert Benchmark with a large (48-core) server. Storage on this server has a low fsync latency while the small server has high fsync latency.

tl;dr

• binlog storage engine makes some things better without making other things worse
• binlog storage engine doesn't make all write-heavy steps faster because the commit path isn't the bottleneck in all cases on a server with storage that has low fsync latency

tl;dr for a CPU-bound workload

• the l.i0 step (load in PK order) is ~1.3X faster with binlog storage engine
• the l.i2 step (write-only with smaller transactions) is ~1.5X faster with binlog storage engine

ClawRoute Technical Architecture: How Smart Model Routing Works

Overview

ClawRoute is a distributed AI routing system that intelligently routes requests across multiple LLM providers using a unified 0-100 scoring system, Thompson Sampling for exploration/exploitation balance, circuit breakers for fault tolerance, predictive rate limiting, and multi-provider support. The system optimizes for cost, speed, and reliability while providing zero-configuration developer APIs.

Core Architecture

1. Request Router (router.py)

The main entry point that receives requests and routes them based on:

• Unified 0-100 quality score (task-specific weights)
• Cost optimization
• Latency requirements
• Availability and health status

Key Features:

• Unified Scoring System: All models rated 0-100 with weights adjusted per task type
• Thompson Sampling: Balances exploration and exploitation for model selection
• Smart Fallback: Automatic switching when primary model underperforms
• Global Distribution: Routes to geographically closest healthy endpoints

2. Provider Adapters

Modular adapters for each LLM provider:

OpenAI Adapter

• GPT-3.5, GPT-4, GPT-4 Turbo support
• API key rotation and rate limit handling

Anthropic Adapter

• Claude 3 family support
• API key management

Google Adapter

• Gemini Pro/Ultra support

Custom Endpoints

• Self-hosted OpenAI-compatible models
• Local LLM deployments

3. Unified 0-100 Scoring System

Every model response receives a score from 0-100 based on five dimensions, with weights that adjust based on task type:

final_score = (0.25 * relevance) + (0.20 * coherence) + (0.20 * completeness) + 
              (0.15 * latency_score) + (0.10 * cost_efficiency) + (0.10 * task_specific)

Scoring Dimensions (0-100 each):

• Relevance: Does response address the prompt? (semantic similarity)
• Coherence: Is response logically structured and consistent?
• Completeness: Does it fully answer the question?
• Latency Score: Normalized response time (faster = higher score)
• Cost Efficiency: Quality per dollar spent
• Task Specific: Custom dimension based on use case

Task-Specific Weight Examples:

• Coding Tasks: Quality weight increased to 0.35, latency reduced to 0.10
• Creative Writing: Relevance weight 0.30, coherence 0.25
• Data Analysis: Completeness weight 0.30, cost efficiency 0.15
• Real-time Chat: Latency weight 0.25, relevance 0.20

4. Thompson Sampling for Model Selection

Instead of static routing, ClawRoute treats each model as a "bandit arm" and uses Thompson Sampling to balance exploration and exploitation:

For each request:
  1. Sample from each model's Beta(α, β) distribution
     where α = successes + 1, β = failures + 1
  2. Select model with highest sampled value
  3. Execute request
  4. Observe outcome (score 0-100)
  5. Update distribution:
        if score >= threshold: α += 1
        else: β += 1

This dynamically shifts traffic toward better-performing models while still testing alternatives.

5. Circuit Breaker Pattern

Prevents cascading failures with three states:

CLOSED → [failures ≥ threshold] → OPEN
  ▲                                 |
  |                                 |
  |                    [timeout]    |
  |                                 ▶
HALF-OPEN ← [probe success] —— CLOSED

Configuration:

• Failure threshold: 5 consecutive low scores (< 60)
• Timeout: 30 seconds before half-open
• Half-open: Allow one test request

6. Predictive Rate Limiting

Learns provider limits from 429 responses:

class AdaptiveRateLimiter:
    def __init__(self, provider):
        self.provider = provider
        self.window = 60  # seconds
        self.requests = deque()
        self.limit = None  # Learned from 429s
        self.safety_margin = 0.8  # Stay under 80% of limit

    def allow_request(self):
        now = time.time()
        # Remove old requests
        while self.requests and self.requests[0] < now - self.window:
            self.requests.popleft()

        # Predictive check
        if self.limit and len(self.requests) >= self.limit * self.safety_margin:
            return False

        return len(self.requests) < (self.limit or 1000)

7. Multi-Provider Abstraction

Unified interface hides provider differences:

response = clawroute.generate(
    prompt="Explain RSA encryption",
    task_type="coding",  # Adjusts scoring weights
    max_tokens=500
)

Provider Capabilities Matrix:

Technical Implementation

Request Flow

def route_request(request):
    # 1. Apply task-specific weights
    weights = get_task_weights(request.task_type)

    # 2. Thompson Sampling selects candidate models
    candidates = thompson_sample(request.context)

    # 3. Filter by circuit breaker state
    healthy = [m for m in candidates if circuit_breaker[m].state == "CLOSED"]

    # 4. Check predictive rate limits
    available = [m for m in healthy if rate_limiter[m].can_send()]

    # 5. Select highest expected score
    selected = max(available, key=lambda m: m.beta_distribution.mean())

    # 6. Execute and score
    response = providers[selected].call(request)
    score = score_response(response, weights)

    # 7. Update learning systems
    update_thompson(selected, score)
    update_rate_limiter(selected, response.headers)
    return response

Scoring Algorithm

def score_response(response, weights):
    scores = {
        'relevance': semantic_similarity(response, request.prompt) * 100,
        'coherence': coherence_model.score(response) * 100,
        'completeness': completeness_check(response, request) * 100,
        'latency': normalize_latency(response.latency) * 100,
        'cost_efficiency': (base_quality / response.cost) * 100,
        'task_specific': task_specific_scorer[request.task_type](response)
    }

    return sum(scores[k] * weights[k] for k in weights)

Deployment & Scaling

Horizontal Scaling

• Stateless router instances behind load balancer
• Shared Redis for scoring history and rate limit tracking
• Consistent hashing for provider affinity

Database Schema

model_performance (
    model_id, 
    timestamp, 
    task_type, 
    score_0_100,
    latency_ms,
    cost_usd,
    success_bool
)

rate_limit_state (
    provider, 
    window_start, 
    request_count, 
    learned_limit
)

Monitoring

• Real-time score distributions per model
• Alert on scoring distribution shifts (model drift)
• Track cost savings vs baseline routing
• Latency and success rate dashboards

Performance Impact

A/B Test Results (vs Round Robin)

Task-Specific Gains

• Code Generation: 22% higher quality scores
• Customer Support: 18% faster responses
• Content Creation: 15% better coherence

Getting Started

Install via npm:

npm install @clawroute/sdk

Initialize with providers:

import { ClawRoute } from '@clawroute/sdk';

const ai = new ClawRoute({
  providers: {
    openai: { apiKey: process.env.OPENAI_API_KEY },
    anthropic: { apiKey: process.env.ANTHROPIC_API_KEY },
    google: { apiKey: process.env.GOOGLE_API_KEY }
  },
  scoring: {
    // Optional: customize task weights
    taskWeights: {
      coding: { relevance: 0.30, coherence: 0.15, completeness: 0.25, 
               latency: 0.10, cost: 0.10, taskSpecific: 0.10 }
    }
  }
});

// Route automatically based on task type
const result = await ai.generate({
  prompt: "Create a Python function to calculate fibonacci",
  taskType: "coding",
  maxTokens: 200
});

Future Enhancements

• Online Learning: Real-time weight adjustment based on user feedback
• Multi-Objective Optimization: Pareto frontier for cost vs quality
• Prompt Caching: Semantic caching for repeated queries
• Edge Deployment: Regional model providers for lower latency

ClawRoute is open source under MIT License. Visit github.com/clawhub/clawroute for documentation and examples.

ClawRoute: Intelligent AI routing that learns and adapts to deliver the best model for every request.

Adoption of Generative AI (GenAI) and agentic AI has accelerated from experimentation into real enterprise deployments. What began with copilots and chat interfaces has quickly evolved into powerful business systems that autonomously interact with sensitive data, call external APIs, connect to consequential tools, initiate workflows, and collaborate with other agents across enterprise environments. As these AI systems become core infrastructure, establishing clear, continuous visibility into how these systems behave in production can help teams detect risk, validate policy adherence, and maintain operational control.

Observability is one of the foundational security and governance requirements for AI systems operating in production. Yet many organizations don’t understand the critical importance of observability for AI systems or how to implement effective AI observability. That mismatch creates potential blind spots at precisely the moment when visibility matters most.

In February, Microsoft Corporate Vice President and Deputy Chief Information Security Officer, Yonatan Zunger, blogged about expanding Microsoft’s Secure Development Lifecycle (SDL) to address AI-specific security concerns. Today, we continue the discussion with a deep dive into observability as a necessity for the secure development of GenAI and agentic AI systems.

For additional context, read the Secure Agentic AI for Your Frontier Transformation blog that covers how to manage agent sprawl, strengthen identity controls, and improve governance across your tenant.

Observability for AI systems

In traditional software, client apps make structured API calls and backend services execute predefined logic. Because code paths follow deterministic flows, traditional observability tools can surface straightforward metrics like latency, errors, and throughput to track software performance in production.

GenAI and agentic AI systems complicate this model. AI systems are probabilistic by design and make complex decisions about what to do next as they run. This makes relying on predictable finite sets of success and failure modes much more difficult. We need to evolve the types of signals and telemetry collected so that we can accurately understand and govern what is happening in an AI system.

Consider this scenario: an email agent asks a research agent to look up something on the web. The research agent fetches a page containing hidden instructions and passes the poisoned content back to the email agent as trusted input. The email agent, now operating under attacker influence, forwards sensitive documents to unauthorized recipients, resulting in data exfiltration.

In this example, traditional health metrics stay green: no failures, no errors, no alerts. The system is working exactly as designed… except a boundary between untrusted external content and trusted agent context has been compromised.

This illustrates how AI systems require a unique approach to observability. Without insights into how context was assembled at each step—what was retrieved, how it impacted model behavior, and where it propagated across agents—there is no way to detect the compromise or reconstruct what occurred.

Traditional monitoring, built around uptime, latency, and error rates, can miss the root cause here and provide limited signal for attribution or reconstruction in AI-related scenarios. This is an example of one of the new categories of risk that the SDL must now account for, and it is why Microsoft has incorporated enhanced AI observability practices within our secure development practices.

Traditional observability versus AI observability

Observability of AI systems means the ability to monitor, understand, and troubleshoot what an AI system is doing, end-to-end, from development and evaluation to deployment and operation. Traditional services treat inputs as bounded and schema-defined. In AI systems, input is assembled context. This includes natural language instructions plus whatever the system pulls in and acts on, such as system and developer instructions, conversation history, outputs returned from tools, and retrieved content (web pages, emails, documents, tickets).

For AI observability, context is key: capture which input components were assembled for each run, including source provenance and trust classification, along with the resulting system outputs.

Traditional observability is often optimized for request-level correlation, where a single request maps cleanly to a single outcome, with correlation captured inside one trace. In AI systems, dangerous failures can unfold across many turns. Each step looks harmless until the conversation ramps into disallowed output, as we’ve seen in multi-turn jailbreaks like Crescendo.

For AI observability, best practices call for propagating a stable conversation identifier across turns, preserving trace context end-to-end, so outcomes can be understood within the full conversational narrative rather than in isolation. This is “agent lifecycle-level correlation,” where the span of correlation should be the same as the span of persistent memory or state within the system.

Defining AI system observability

Traditional observability is built on logs, metrics, and traces. This model works well for conventional software because it’s optimized around deterministic, quantifiable infrastructure and service behavior such as availability, latency, throughput, and discrete errors.

AI systems aren’t deterministic. They evaluate natural language inputs and return probabilistic results that can differ subtly (or significantly) from execution to execution. Logs, metrics, and traces still apply here, but what gets captured within them is different. Observability for AI systems updates traditional observability to capture AI-native signals.

Logs, metrics, and traces indicate what happened in the AI system at runtime.

• Logs capture data about the interaction: request identity context, timestamp, user prompts and model responses, which agents or tools were invoked, which data sources were consulted, and so on. This is the core information that tells you what happened. User prompts and model responses are often the earliest signal of novel attacks before signatures exist, and are essential for identifying multi-turn escalation, verifying whether attacks changed system behavior, adjudicating safety detections, and reconstructing attack paths. User-prompt and model-response logs can reveal the exact moment an AI agent stops following user intent and starts obeying attacker-authored instructions from retrieved content.
• Metrics measure traditional performance details like latency, response times, and errors as well as AI-specific information such as token usage, agent turns, and retrieval volume. This information can reveal issues such as unauthorized usage or behavior changes due to model updates.
• Traces capture the end-to-end journey of a request as an ordered sequence of execution events, from the initial prompt through response generation. Without traces, debugging an agent failure means guessing which step went wrong.

AI observability also incorporates two new core components: evaluation and governance.

• Evaluation measures response quality, assesses whether outputs are grounded in source material, and evaluates whether agents use tools correctly. Evaluation gives teams measurable signals to help understand agent reliability, instruction alignment, and operational risk over time.
• Governance is the ability to measure, verify, and enforce acceptable system behavior using observable evidence. Governance uses telemetry and control plane mechanisms to ensure that the system supports policy enforcement, auditability, and accountability.

These key components of observability give teams improved oversight of AI systems, helping them ship with greater confidence, troubleshoot faster, and tune quality and cost over time.  

Operationalizing AI observability through the SDL

The SDL provides a formal mechanism by which technology leaders and product teams can operationalize observability. The following five steps can help teams implement observability in their AI development workflows.

1. Incorporate AI observability into your secure development standards. Observability standards for GenAI and agentic AI systems should be codified requirements within your development lifecycle; not discretionary practices left to individual teams.
2. Instrument from the start of development. Build AI-native telemetry into your system at design time, not after release. Aligning with industry conventions for logging and tracing, such as OpenTelemetry (OTel) and its GenAI semantic conventions, can improve consistency and interoperability across frameworks. For implementation in agentic systems, use platform-native capabilities such as Microsoft Foundry agent tracing (in preview) for runtime trace diagnostics in Foundry projects. For Microsoft Agent 365 integrations, use the OTel-based Microsoft Agent 365 Observability SDK (in Frontier preview) to emit telemetry into Agent 365 governance workflows.
3. Capture the full context. Log user prompts and model responses, retrieval provenance, what tools were invoked, what arguments were passed, and what permissions were in effect. This detail can help security teams distinguish a model error from an exploited trust boundary and enables end-to-end forensic reconstruction. What to capture and retain should be governed by clear data contracts that balance forensic needs against privacy, data residency, retention requirements, and compliance with legal and regulatory obligations, with access controls and encryption aligned to enterprise policy and risk assessments.
4. Establish behavioral baselines and alert on deviation. Capture normal patterns of agent activity—tool call frequencies, retrieval volumes, token consumption, evaluation score distributions—through Azure Monitor and Application Insights or similar services. Alert on meaningful departures from those baselines rather than relying solely on static error thresholds.
5. Manage enterprise AI agents. Observability alone cannot answer every question. Technology leaders need to know how many AI agents are running, whether those agents are secure, and whether compliance and policy enforcement are consistent. Observability, when coupled with unified governance, can support improved operational control. Microsoft Foundry Control Plane, for example, consolidates inventory, observability, compliance with organization-defined AI guardrail policies, and security into one role-aware interface; Microsoft Agent 365 (in Frontier preview) provides tenant-level governance in the Microsoft 365 admin plane.

To learn more about how Microsoft can help you manage agent sprawl, strengthen identity controls, and improve governance across your tenant, read the Secure Agentic AI for Your Frontier Transformation blog.

Benefits for security teams

Making enterprise AI systems observable transforms opaque model behavior into actionable security signals, strengthening both proactive risk detection and reactive incident investigation.

When embedded in the SDL, observability becomes an engineering control. Teams define data contracts early, instrument during design and build, and verify before release that observability is sufficient for detection and incident response. Security testing can then validate that key scenarios such as indirect prompt injection or tool-mediated data exfiltration are surfaced by runtime protections and that logs and traces enable end-to-end forensic reconstruction of event paths, impact, and control decisions.  

Many organizations already deploy inference-time protections, such as Microsoft Foundry guardrails and controls. Observability complements these protections, enabling fast incident reconstruction, clear impact analysis, and measurable improvement over time. Security teams can then evaluate how systems behave in production and whether controls are working as intended.

Adapting traditional SDL and monitoring practices for non-deterministic systems doesn’t mean reinventing the wheel. In most cases, well-known instrumentation practices can be simply expanded to capture AI-specific signals, establish behavioral baselines, and test for detectability. Standards and platforms such as OpenTelemetry and Azure Monitor can support this shift.

AI observability should be a release requirement. If you cannot reconstruct an agent run or detect trust-boundary violations from logs and traces, the system may not be ready for production.

The post Observability for AI Systems: Strengthening visibility for proactive risk detection appeared first on Microsoft Security Blog.

How are CTOs feeling about AI?

According to Andy Skipper, founder of CTO Craft, they’re experiencing fear, uncertainty, and doubt.

And if the technical leaders of companies are feeling that way, what can the rest of us expect? Certainly, we dream of productivity boosts and an AI El Dorado – but that’s not the reality.

That’s why we sat down with Skipper to talk about how CTOs should manage expectations for AI, and how to navigate the hype versus reality.

Stakeholders and investors are watching CTOs closely, and the pressure is rising

Many CTOs, Skipper notes, are navigating intense pressure from non-technical stakeholders and investors alike, especially with the massive resources being invested in AI and LLM technologies.

He’s a bit careful about this:

AI is not going to reduce costs or increase productivity in the way some non-technical people think just yet. It’s getting there, but it’s not there yet.

At the same time, Skipper points out a surprising upside: AI is giving engineering leaders a chance to reconnect with the code and architecture without writing all the code themselves:

One of the things you have to accept as an engineering leader is that you are going to get further away from the code the more senior you become. AI gives people an opportunity to get back to architecture and development work, even if they aren’t coding themselves.

CTO role can be isolating

When Skipper became a CTO for the first time, he quickly realized just how isolating the role could be. There was nowhere for tech leaders to share challenges, get support, or navigate the non-technical side of the job.

That gap inspired him to start CTO Craft, now a community helping senior engineering leaders navigate team dynamics, strategy, and AI.

When I was a CTO for the first time, I didn’t have somebody who I could talk to about the issues I was seeing or compare notes with people who had similar challenges. That’s what CTO Craft is all about – helping people understand where the challenges come from and understand they’re not alone in having those challenges.

As a coach and mentor, Andy works closely with CTOs around the world, helping them deal with issues like burnout, communication with nontechnical stakeholders, and, lately, how to adapt in the AI era.

The most common CTO mistake? Always chasing the newest technologies

Many first-time CTOs struggle with burnout, overextending themselves to shield teams from stress, and balancing hands-on coding with high-level responsibilities. He explains:

A lot of the people that I work with directly are suffering from burnout. First time CTOs commonly miss out self-preservation. And usually that’s a combination of too much expectation of their own energy levels, their own abilities, backlogs…

And after overextending themselves, first-time CTOs often make another common mistake: chasing the newest technologies. While adopting the latest tools and frameworks can seem exciting, Skipper warns that it’s not always the best choice for fast-moving teams trying to scale.

“Using bleeding-edge tech can slow you down, make systems harder to maintain, and even complicate hiring because the talent pool for newer technologies might be limited,” he explains.

As a coach, Skipper says these are just some of the recurring challenges he sees among engineering leaders, alongside a range of other operational and people-related issues.

Engineering skills alone won’t make you a CTO

For aspiring engineering leaders, Skipper highlights that growing into a successful CTO requires more than technical excellence: commercial understanding, communication, coaching, and vision-setting are just as crucial:

The difference between a good engineering manager and a great CTO is understanding how technology drives business success, while still inspiring and guiding your teams.

But technical and business skills are only part of the picture. Motivation and team management are equally critical. Skipper stresses that not everyone is motivated by the same things, and leaders need to understand individual drivers:

Having a vision in the first place is very important. But when it comes to actually bringing individuals along on the journey, they all need to be worked with differently. You can’t just set it and expect everyone to be motivated.

He also warns against a common mistake among CTOs: trying to shield their teams from the challenges of a pivot or rapid change. While the instinct is understandable, it often backfires and drains the leader’s emotional energy. Instead, transparency and realistic communication are key:

Being transparent, being realistic, measuring your words, not being super negative about everything, but still being realistic, I think all these things are really important.

The need for a support network, not another tech stack

Skipper believes resilience and peer support are crucial for engineering leaders navigating the complexity of the CTO role. Sharing experiences and learning from others can help leaders realize they’re not alone when facing difficult decisions.

Looking ahead, however, he admits that the pace of technological change makes it hard to predict what the role will look like in the future.

Five years from now, I honestly have no idea what the role of a CTO will look like. The way we build software is already changing rapidly, especially with AI. But the fundamentals like setting a vision, communicating it clearly, and connecting technology with business outcomes, will always remain essential.

For Skipper, that uncertainty makes peer support crucial: it helps leaders adapt, learn, and navigate a fast-changing profession.

Ultimately, he believes the most important skill for CTOs is the ability to keep learning and tackle challenges without going it alone.

*Infobip, the global communications API leader that launched ShiftMag, was an Event Partner at CTO Craft 2026.

The post CTOs Face Pressure to Deliver AI Gains, but Productivity Isn’t There Yet appeared first on ShiftMag.