Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
165086 stories
·
33 followers

Science at the Frontier: How scientists use ChatGPT

1 Share
Explore how more than 1,800 scientists use ChatGPT in research. See findings on daily workflows, reported productivity, and expectations for discovery.
Read the whole story
alvinashcraft
just a second ago
reply
Pennsylvania, USA
Share this story
Delete

Hugging Face and OpenRouter join forces to make open-weight models easy to run

1 Share
Read the whole story
alvinashcraft
15 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

An MCP Tool Call Can Resolve and Still Fail

1 Share

An await finishes. The application returns a result. The trace contains a completed tool span. The test is green.

The tool also returned isError: true.

There is no contradiction. JavaScript promise completion and MCP tool success are different signals. Treating them as the same signal creates a surprisingly quiet failure: an agent can continue after a rejected operation while a completion-only test reports success.

I maintain AgentInspect. This article uses a synthetic test against agent-inspect@6.31.35 and @agent-inspect/mcp@6.31.35 to separate those signals. It makes no model calls and does not describe a production incident.

Three questions hiding inside “success”

The MCP tools specification distinguishes protocol-level errors from errors reported inside a tool result. A valid result can carry isError: true.

For an application, there is another boundary beyond either of those: whether the requested effect actually happened.

Question Evidence What it does not establish
Did the call return? Promise resolution or protocol response That the tool accepted the operation
Did the tool report failure? MCP result's isError The final state of an external system
Did the intended outcome occur? Independently observed state or receipt That every earlier execution step was correct

A booking tool might respond normally to explain that inventory is unavailable. An email tool might accept a message without proving delivery. A database timeout might leave the write outcome uncertain.

Those cases deserve different application behavior. They should not collapse into a single green check.

MCP response handling separates rejected promises, returned tool errors, and independently verified outcomes.

A returned error is evidence to evaluate, not an instruction to retry automatically.

Start with a deliberately small reproduction

Install matching package versions:

npm install --save-exact agent-inspect@6.31.35 @agent-inspect/mcp@6.31.35

This JavaScript fixture is intentionally not a real MCP server. It isolates the returned-result branch of the wrapper:

import assert from "node:assert/strict";
import { mkdtemp, readdir } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { inspectRun } from "agent-inspect";
import { wrapMcpClient } from "@agent-inspect/mcp";

const traceDir = await mkdtemp(join(tmpdir(), "mcp-result-"));
const response = {
  isError: true,
  content: [{ type: "text", text: "Synthetic request rejected" }],
};

const client = wrapMcpClient(
  { async callTool() { return response; } },
  { serverName: "fixture" },
);

const returned = await inspectRun(
  "returned-tool-error",
  () => client.callTool({ name: "lookup_policy", arguments: {} }),
  { traceDir, silent: true },
);

assert.strictEqual(returned, response);
const [filename] = (await readdir(traceDir))
  .filter(name => name.endsWith(".jsonl"));

The identity assertion matters. Instrumentation should not silently replace the caller's result with a different object or invent an exception. In this release, the wrapper preserves the result and records a failed outcome observation for the MCP tool error.

The application still owns the response: explain the rejection, choose an authorized fallback, request clarification, or stop. The tracing adapter records evidence; it does not choose the recovery policy.

Test the observation, not just completion

Append this to the same fixture:

import { openTraceFile } from "agent-inspect/readers";
import {
  defineTraceContract,
  evaluateTraceContractRead,
} from "agent-inspect/checks";

const trace = await openTraceFile(join(traceDir, filename));

const completionOnly = evaluateTraceContractRead(
  trace,
  defineTraceContract({ run: { requireCompleted: true } }),
);

const withOutcomes = evaluateTraceContractRead(
  trace,
  defineTraceContract({
    run: { requireCompleted: true },
    observations: { failOn: ["failed"] },
  }),
);

assert.equal(completionOnly.status, "pass");
assert.equal(withOutcomes.status, "fail");

These are measured results from the included test harness, not a hand-drawn CLI transcript. TraceContract is a beta surface; pin the release and keep the fixture beside the contract.

Notice what the second check says: a captured observation reported failure. It does not claim that AgentInspect independently queried a booking system or verified a database transaction. The observation comes from the MCP response. Its provenance determines its evidentiary strength.

Don’t turn every tool error into a retry

isError is not a retryability classification.

A malformed argument, denied permission, unavailable resource, and transient dependency failure can all produce unsuccessful tool results. Retrying the first two without changing anything adds work rather than recovery. Retrying a side-effecting action after an ambiguous response can duplicate an effect.

Prefer an application-owned, bounded policy. Separate read-only recovery from writes. Where a remote service supports idempotency, follow that service's actual contract. Neither the wrapper nor a trace field makes an operation exactly-once.

Also avoid parsing arbitrary error prose into policy decisions. If your server exposes reviewed structured error categories, test those categories. If it does not, preserve uncertainty instead of guessing from the word “timeout.”

Expand the fixture at the integration boundary

The fake client proves a narrow behavior. Before adopting a wrapper, add real SDK tests for normal results, thrown errors, isError, cancellation, method receivers, and lifecycle methods. Verify captured calls against an independently counted server handler.

The companion examples include an installed MCP SDK client/server test using an in-memory transport. That covers SDK method compatibility without implying that network reconnects, authentication, or every transport have been tested.

This distinction is the habit worth keeping: every green result should name the boundary it actually exercised.

If your current agent test only asserts that a promise resolved, add one fixture in which the tool explicitly refuses the request. Does the run still look successful? If so, the missing assertion is probably not about the final sentence.

Implementation references

Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Robert Pattinson is Amazon’s first James Bond

1 Share

Robert Pattinson will take on the role of James Bond in Amazon MGM's first film in the 007 franchise. The next James Bond film will be directed by Denis Villeneuve using a script written by Steven Knight, while Amy Pascal and David Heyman will produce.

The upcoming James Bond film will be the first from Amazon, which snapped up the rights to the franchise through its acquisition of MGM Studios in 2021. Amazon later gained creative control over the spy series from Barbara Broccoli and Michael G. Wilson in 2025 after a years-long stalemate. There hasn't been a new James Bond film since Daniel Craig's last installment as the spy in 2021's No T …

Read the full story at The Verge.

Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Designing a Reliable AI Workflow on Azure: Production Lessons from Translation Automation

1 Share

 

Many teams can build an AI prototype that works. A trigger arrives, the model produces a plausible response, and the demo ends with the expected file or message. But what happens when that prototype becomes a workflow that other people depend on?

Imagine an AI-assisted process that prepares a recurring report. The notification arrives on time, but the report is blank. On another run, the research is useful but lands in the wrong template. A retry then replaces a version that someone had already reviewed. In each case, the model may have responded normally while the application still delivered the wrong operational result.

I encountered versions of these failures while operating Localizeflow, an independent translation orchestration project built around Co-op Translator. A translation could appear to finish without producing a complete, reviewable document. A credential failure could look like a missing result. Overlapping requests could place too much work and state inside one long-running worker.

Those experiences changed the question I ask after an AI prototype succeeds. The next question is not simply, “Can the model do the task?” It is, “Can the workflow record the request, survive failure, validate the final artifact, and return something another person can trust?”

This article uses translation as one worked example, but the lessons apply to any long-running AI workflow that produces a report, document, batch, or repository change. We will follow three failure classes from prototype to production, then add one operating requirement that makes the workflow's progress visible.

A working prototype proves only one thing

The translation prototype was deliberately simple. A trigger started the process, the process downloaded the source, called the model, wrote the translated files, and uploaded the result. It proved that the model could translate a document.

It did not prove that the workflow had used the intended input revision, preserved a partial result, rejected stale output, or left enough evidence to explain a failure. It also did not prove that the resulting document was ready for someone else to use.

That distinction became visible in public repository work. MCP for Beginners pull request #926 shows an incomplete translation batch that stopped with two of 19 translation tasks complete and was closed without merge. The screen records the observable state, but it does not establish the root cause or prove that every generated file was blank.

Figure 1. Public failure evidence from MCP for Beginners pull request #926. The Localizeflow progress record showed two of 19 translation tasks complete, and the pull request was closed without merge. This screen establishes the observable outcome, not the root cause or the contents of every generated file.

By contrast, AI Agents for Beginners pull request #790 connects a completed translation update to a visible diff, review findings, follow-up fixes, and a repository decision. The Localizeflow bot opened the update, and the repository merged it on October 30, 2026. The merge proves that the repository accepted the change. It does not prove translation accuracy, time saved, or a separate deployment. We will return to this accepted result at the end.

The two records illustrate the central distinction in this article:

A model response, a completed process, a delivered artifact, and an accepted result are different outcomes.

Translation exposed that distinction, but it is not unique to translation. A report generator can return text while omitting a required section. A post-processor can damage a valid response. An uploader can publish the wrong revision. A retry can replace work that a person had already approved.

What the workflow had to add

The production-oriented design starts by treating each request as durable work rather than a long-running model call. Figure 2 applies that pattern to an independent translation example. The sections that follow explain the responsibility at each handoff.

Figure 2. Independent reference architecture for hosting Co-op Translator on Azure. A GitHub documentation change enters an orchestrator hosted in Azure Container Apps, which records the request and starts an Azure Container Apps Job. The job runs Co-op Translator and calls Azure OpenAI to produce a candidate translation. Deterministic checks gate the pull request, and a person decides whether to merge it. This is an independent reference design, not a description of an existing project's internal implementation.

Note: Azure Container Apps is suited to service-style workloads such as HTTP APIs, while Container Apps Jobs runs tasks that start, perform a finite unit of work, and stop. In this design, the Container App hosts the request-facing API and orchestrator, while each accepted translation attempt runs as a separate Job.

The container app receives the GitHub change, validates the scope, records the logical job, and decides when an attempt may start. The job runs Co-op Translator, calls the deployed Azure OpenAI model, writes an attempt-scoped artifact, reports its outcome, and exits. Azure OpenAI performs model inference, but it does not own workflow state, validation policy, or merge authority.

First, accept the work. The application checks that the caller, input, scope, and policy are allowed. In this worked example, the accepted request connects a GitHub documentation change to the repository, source revision, file scope, and target language. In a report workflow, it may mean the data source, reporting period, template, and recipient.

Next, record and orchestrate it. The orchestrator hosted in Azure Container Apps saves the logical job before starting execution. The record connects the request to the input revision, policy version, attempt history, and eventual decision. If the process stops, the work still has an identity and a state. In this design, the application layer detects work, records the logical job, and coordinates bounded executions.

Then, execute a bounded attempt. The orchestrator remains the request-facing service boundary in Azure Container Apps while an Azure Container Apps Job runs Co-op Translator for the finite unit of work. The worker receives the recorded job ID, writes to an attempt-specific workspace, and reports its outcome.

Finally, validate and decide. The job's output remains a candidate. Application checks establish whether the artifact is complete, current, and within the allowed scope. Only then does the application prepare a reviewable change. In this worked example, that change is a pull request, and a person decides whether to merge it.

The model call remains important, but it is no longer asked to carry workflow state, validation policy, and delivery authority by itself.

What changes after the prototype works?

Starting with the prototype was the right decision. A first version should answer the smallest useful question: can the model perform the task on one known input? Building job records, isolated workers, recovery logic, validation rules, and monitoring before that question is answered can become overengineering.

The mistake is not starting simple. It is treating the simple version as production-ready. Reliability features should be added when the workflow crosses a real boundary: more inputs, overlapping requests, external dependencies, consequential delivery, or users who need to understand and recover from failure.

The original prototype connected a trigger directly to a model call and treated the returned output as success. That was enough to prove the basic capability. Most state lived inside the process, however, so a stopped run or incomplete artifact left little durable evidence.

The production-oriented workflow keeps the model call but changes the boundaries around it. The trigger records a job, each execution becomes an attempt, generated files remain separate from accepted state, and a retry cannot silently erase the failed attempt.

Start with the simplest workflow that can prove the task. Add complexity only when a real failure boundary justifies it.

Figure 3. The model call remains. Production adds durable state, bounded execution, artifact checks, and human review.

This change matters because real operation adds failure surfaces that a successful demo does not exercise. A trigger can arrive twice. A credential can be invalid. The source can change while work is running. A model can return only part of a document. Application code can damage a valid response. A validator can reject acceptable variation. An external system can complete an action while the workflow loses the response.

While writing this article, I went back through every Co-op Translator release, from v0.2.1 to v0.23.3. Reading them in order felt a little like replaying the project from the first prototype to the workflow it became. I had forgotten how many of the harder problems appeared only after the model was already returning translations. Credentials failed, chunks were reassembled incorrectly, retries needed boundaries, and valid output could still be rejected.

Looking back helped me choose the cases below. I wanted to share the problems that became visible only after the prototype worked, because they are the kinds of problems other teams may meet when a proof of concept becomes an automation people depend on.

Three failures changed the design most.

Lesson 1: A successful model call can still produce a failed artifact

In early runs, a translation could appear to finish while omitting source lines or changing protected Markdown structure. The provider request had completed, but the document was not complete enough to review.

Co-op Translator pull request #568 records a reproducible version of this behavior. Azure OpenAI returned HTTP 200 with finish_reason='stop', but a response could still omit source lines or lose, duplicate, or reorder protected Markdown placeholders. The affected responses were below the provider token limit and did not report a length or content-filter stop.

The recovery logic therefore stopped treating a normal provider completion as proof of a valid artifact. It validated each translated chunk, preserved the verified prefix, and retried only the missing or damaged portion when possible.

I described how that review path evolved, including marker checks, bounded retries, preserved human edits, and recorded failures, in What production taught us about reviewing AI translations.

The model is not the only place where an artifact can break. Issue #505 showed a translated Markdown table that rendered correctly for the first four rows and then collapsed into unformatted text.

Figure 4. Cropped evidence from Co-op Translator issue #505. Application reassembly inserted an extra newline at the chunk boundary, so the final Markdown table broke after row 03 even though translated chunks were available. Pull request #508 documents the fix. This is not a model-quality failure.

The one-line repair was conceptually simple: join the translated chunks without injecting a new newline. The lesson was larger. Valid model outputs can become an invalid document after the response, so validation must inspect the final assembled artifact.

The earlier article Fixing Broken Markdown in AI Translation: Hardening a Production Pipeline examines the wider class of structural failures, including damaged code fences, links, lists, images, anchors, and CJK formatting.

Guardrails can also be wrong. Pull request #581 documents a valid Italian translation that preserved the protected structure but moved ordinary prose naturally. The validator retried the valid result and eventually failed, so the fix narrowed the invariant to the structure that actually needed protection.

These cases led to one operating rule: treat every generated response as a candidate. Keep the logical job, attempt, candidate, and accepted result separate. Pull request #493 applied the same principle by preserving the last usable translation when a replacement failed.

Where policy permits, quarantine a rejected candidate with its validation report. Where the content cannot be retained, preserve safe structural diagnostics such as the file, attempt ID, failed check, and expected and actual counts. Do not copy sensitive source content into routine logs.

The same rule applies to a report workflow. A report with a missing required section is not a completed report, even if the model returned a normal response. A template populated with damaged fields is not safe to deliver, even if the original content was correct.

Try this before production: remove one required section or protected marker from an otherwise normal candidate. The attempt should fail without replacing the previous accepted result, and the failure record should explain which check rejected it.

Lesson 2: A missing result may be a dependency failure

Another failure occurred before translation quality could matter. A credential problem prevented the provider from producing a usable response. In the early workflow, the expected artifact simply did not appear, while the underlying authentication problem surfaced only as a generic process failure.

A blind retry would repeat a request that could not succeed. The workflow needed to distinguish “the model produced a bad candidate” from “a required dependency was unavailable.”

Co-op Translator pull request #211 added a lightweight connectivity check that exercises the real provider data path before translation begins. It checks the endpoint, API version, credential, and deployment. Authentication failures such as 401 or 403 stop the run with actionable feedback instead of appearing after a long batch.

Figure 5. Authentication failure becomes failed with a stable reason and does not publish an empty artifact. After repair, the workflow starts a new attempt under the same logical job.

A startup check is valuable, but it cannot guarantee that a token or credential will remain valid throughout a long execution. A 401 or 403 received during execution should enter the same stable failure category. The job can become failed with a reason such as provider_authentication, while an operator repairs the endpoint, permission, or credential and starts a new attempt without losing the earlier history.

The same pattern applies to retrieval indexes, databases, document stores, and delivery APIs. Report the dependency failure instead of allowing it to masquerade as missing or low-quality AI output.

For Azure resources that support Microsoft Entra authentication, managed identities reduce the secrets that an application must store and rotate. Any remaining credentials belong in the execution environment or a secret store, never in prompts, repository content, or job payloads.

Try this before production: use an invalid endpoint or unavailable credential. The workflow should stop before expensive work, record a stable dependency reason, and publish no empty artifact. After repair, the new attempt should remain connected to the original request.

Lesson 3: Long-running work needs a bounded execution

The first hosting path ran translation work through the same long-lived Container App process that served the application. When requests overlapped, the process had to coordinate more than one workspace, more than one translation state, and more memory-intensive work at the same time. Cleanup and failure isolation became harder to reason about.

This was not evidence that Azure Container Apps inherently leaks memory. Container Apps can scale continuously available services horizontally. The problem was the execution model: a translation batch has a beginning, an end, and an artifact, while the worker behaved like a continuously running service.

Figure 6. Keep the continuously available API in Azure Container Apps and give each finite attempt its own Azure Container Apps Job execution. Retries, cleanup, and admission control still require application policy.

Microsoft describes a container app as a continuously running service and a Container Apps Job as a task that starts, performs a finite unit of work, and stops. That made the split clearer. The API and dispatcher could remain in a container app; Figure 2 groups those continuously available application responsibilities as the orchestrator. Each accepted unit of work could start a separate job execution linked to the logical job and attempt IDs.

The job definition supplies the container image, resource allocation, timeout, and retry policy. Each execution has its own finite process lifetime, execution identity, status, logs, and an attempt-scoped workspace when the application creates one. A finite process lifetime does not remove the need to control memory or clean temporary files, but it limits how far one failed attempt can spread. Data written to persistent storage still needs an explicit retention and cleanup policy. Bound cost and permissions the same way: cap model usage per attempt, and give the dispatcher, worker, validator, and publisher only the access each one needs.

The dispatcher can admit independent requests up to an application-defined limit and deliberately serialize work that might conflict, such as simultaneous updates to the same repository. Container Apps Jobs parallelism controls replicas within one execution. It does not replace request-level admission control.

The application should not rely on external side effects happening exactly once. A retry, restart, or lost response can still occur. Attempt ownership, idempotent state transitions, and reconciliation of an existing pull request or downstream record remain application responsibilities.

Try this before production: submit two independent requests together. Each should retain its own execution ID, workspace, result, logs, and retry path. Then submit two requests that would update the same downstream resource and confirm that the dispatcher applies the intended serialization policy.

Then test the handoffs that fail quietly: submit the same request twice, change the source while an attempt is running, and lose the response after the downstream record is created. The recorded job, input revision, and current attempt owner should lead the workflow to one logical job, no stale output, and no duplicated downstream record.

Lesson 4: Make progress visible before the final artifact appears

A long-running AI workflow should not force a developer to infer its health from whether a file eventually appears. Without searching raw console logs, a person should be able to answer four questions: Was my request accepted? Is the work still advancing? Where did it stop? What result is waiting for review? The workflow should expose the current attempt, how many units are running, complete, queued, or failed, when progress last changed, and whether a reviewable artifact is ready.

Those answers come from a small set of job states. Status should describe evidence, not optimism, so each state names what must be on record before a job can enter it. The following is a reference state model for this design, not a description of Localizeflow's current internal state machine. Here, accepted means the orchestrator has durably recorded the request and accepted it for processing. It does not mean that execution has completed.

StateWhat it meansEvidence on record
acceptedThe request is stored for processing.Request ID, input revision, scope, policy
runningA current attempt owns the job.Attempt ID, ownership expiry
validatingAn artifact exists and checks are running.Candidate artifact, checks run so far
ready_for_reviewValidation passed and a reviewable change is available.Validation result, artifact diff
failedThe latest attempt ended without a valid artifact.Failing stage, stable reason, safe diagnostics
supersededA newer request or input revision replaced this job.Link to the replacement
approvedA person accepted the reviewable change.Reviewer identity, downstream decision record
rejectedA person declined the reviewable change.Reviewer identity, recorded reason

approved and rejected come from the downstream decision, not from the workflow finishing. For failed, name the stage, such as dependency, generation, assembly, validation, or delivery, so the workflow never retries blindly.

The Localizeflow progress records in the public pull requests make that distinction concrete. The failed record in Figure 1 exposes partial completion and failed jobs instead of presenting an empty destination as the only symptom. The completed record shown at the end exposes a different state using the same vocabulary. A developer can see that the automation is advancing, stopped, or ready for the next decision without treating a model response as the only signal.

Progress is not quality. 19 / 19 completed means that the execution units reported completion; it does not mean every translation is accurate. Operational visibility must connect to artifact validation, a reviewable diff, and the final human decision. The dashboard explains what the workflow is doing. The checks and reviewer decide whether its result is acceptable. Whether the workflow saves time or improves quality needs its own measurements: review time, retries, dependency failures, superseded results, human corrections, and the model, container, and storage resources each run consumes. Collect them before claiming improvement.

Try this before production: start a multi-unit request, then fail one worker or make a dependency unavailable. The progress view should stop advancing, identify the failed or blocked unit, preserve the earlier attempt, and lead an operator to the relevant evidence. After a retry, the new attempt should be visible without erasing the old one.

Build for the moment after the demo

The prototype answered whether a model could perform one task. Production asked a harder question: could the surrounding application explain failure, preserve usable work, reject unsafe output, and return something a person could trust?

Translation revealed three failure classes, but the design lesson is broader. A normal model response can still produce a failed artifact. A missing result can be a dependency problem. A finite unit of work becomes easier to isolate when its process lifetime is finite too.

Most reliability concerns in an AI workflow are familiar automation concerns: state, retries, dependencies, concurrency, and delivery. What AI adds is a probabilistic boundary. Even when the model call succeeds, its response may be incomplete, structurally invalid, or simply unsuitable for the intended use. A reliable design must therefore validate the artifact, preserve rejected candidates safely, and keep consequential acceptance outside the model.

An AI workflow must handle ordinary automation failures and also assume that a successful model call can produce an unacceptable result.

A reliable AI workflow connects the accepted request, input version, execution attempt, candidate artifact, validation result, and human decision. When a run fails or the input changes, that chain should explain what happened and what should happen next.

Figure 7. The public Localizeflow run record linked from AI Agents for Beginners pull request #790 shows the overall state, all 19 work units complete, no active work, no recovery attempts, and no terminal failures. These fields make execution observable. Review findings still led to follow-up changes before the repository merged the pull request on October 30, 2026. Neither completed progress nor the merge alone proves translation accuracy, time saved, or deployment.

That is the purpose of the progress view: make the AI and automation workflow inspectable while it is running, then connect completion to validation and a visible repository decision. A green progress bar is useful evidence, but it becomes trustworthy only as part of that larger chain.

Start with one input, one expected artifact, and one downstream decision: in the translation example, one document, one language, and one test repository. Prove that one valid run stops at ready_for_review and waits for a person, then widen the scope. Track the work, preserve the evidence, validate the result, and return something a person can inspect.

Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Debugging SEO for a bilingual nonprofit site with Bing Webmaster Tools and Microsoft Clarity

1 Share

I run the website for a small foundation in Malaysia. We do food aid, school supplies for kids from low-income families, and practical help when a family hits a rough patch. The site is in English and Bahasa Melayu, and volunteers write our stories in a small admin area.

A few weeks ago I set myself a simple goal: when someone in Malaysia searches for help, or for a way to give, they should find us. I expected a weekend of tweaking titles. Instead I spent a lot longer finding problems that were invisible in a browser and very visible to a crawler.

This post walks through the whole thing as a set of test cases. Each one has what I expected, what I actually found, what I changed, and how I proved the fix worked. Most of the evidence comes from Microsoft tools: Bing Webmaster Tools, Microsoft Clarity with Copilot, Azure AI Translator and IndexNow. I've kept the parts that are still broken too, because I think those are the most useful.

The Microsoft side of the setup

I won't go into the rest of our stack. It's a modern web framework with a CDN in front, and that's all that matters for this post. The interesting part is what sits around it.

Bing Webmaster Tools is where I check indexing, run SEO scans, submit the sitemap and look at keyword data. Microsoft Clarity records how real visitors use the site, and its built-in Copilot answers questions about that data in plain English. Azure AI Translator keeps the English and Malay versions of every story in step, so we never publish in one language only. And IndexNow, the open protocol Bing supports, tells search engines the moment a page changes. All four are free or close to it, which matters a lot when you're a charity.

Our home page. It looked fine to me, which was the whole problem.

Test case 1: does Bing actually like our home page?

My first stop was URL Inspection in Bing Webmaster Tools. It has two tabs, and the difference matters. Bing Index shows the page as Bing last stored it, while Live URL fetches it right now. Together they make a perfect before and after tool.

The stored copy of our English home page, from 31 October, had two problems:

Bing Index view: "Title too long" (error) and "Alt attribute for images is missing" (notice).

The title problem was everywhere, not just on the home page. Every title had the full foundation name appended, and plenty of them ran past 70 characters, so Bing and Google cut them off halfway through a sentence. I wrote a small function that fits each title into 70 characters. It tries the full name first, then our short initials, and drops the brand completely when the page title is long enough on its own. Unit tests keep it honest.

The alt text problem was funnier. I expected the logo and team photos to be missing descriptions. I didn't expect a pile of story images whose alt text was literally "BlockNote image", the default label from our rich text editor. Those now fall back to the story title, and every other image has a real description.

 

Then I ran the Live URL test on the same page:

Live URL view, same page, same day: "No SEO/GEO issues found".

 

Test case 1: home page title and alt text
ToolBing Webmaster Tools, URL Inspection
ExpectedNo SEO issues on the English home page
FoundTitle too long (error) and missing alt text (notice)
Fix70 character title budget with a short brand fallback; real alt text on every image
ResultLive URL test shows no SEO/GEO issues

 

The unit tests that guard the title rules and the story checklist.

Test case 2: is our structured data telling the truth?

This one was a bit embarrassing. Every page carries a block of JSON-LD that tells search engines "this is an NGO, here's the address, here's how to reach us." Ours had an address in a city we don't operate from, a placeholder phone number, a founding year nobody could confirm and social links someone had guessed. All of it came with a template, and none of it came from us.

For a charity, trust is the product, so I deleted every invented value. The structured data now reads from a settings page we actually maintain, and an empty field is left out instead of guessed. Stories carry NewsArticle and breadcrumb markup, and the Contact page has a short FAQ with matching FAQPage markup.

Our live NGO markup. Every value comes from settings (contact details hidden for this post).

URL Inspection reports "1 Markup type found" on the stored copy, so Bing is reading it. If you do one thing after this post, open that section for your own home page. It's the quickest way to see who Bing thinks you are.

 

Test case 2: structured data
ToolBing Webmaster Tools, URL Inspection (Markup)
ExpectedOrganisation details that match reality
FoundInvented address, placeholder phone number and guessed social links from a template
FixMarkup now reads from our own settings; empty fields are left out
ResultBing reads the markup and every value in it is real

 

Test case 3: are the Malay pages really Malay?

Our Malay pages had English headings, English buttons and sometimes whole English paragraphs, and some links on them dropped visitors onto the English version. To a crawler that looks like one confused page rather than two languages, and a lot of Malaysians search in Malay.

Bing leans on the language a page declares more than on hreflang, so I made sure every Malay page declares itself as Malay and every English page as English. I added hreflang with an x-default for Google, and fixed the links so you stay in the language you started in.

Translation is where Azure came in. Nobody on our team has time to write every story twice, so the story editor has a "Translate whole story" button that calls Azure AI Translator. Out of the box it mangled some of our words, like asnaf, zakat, B40 and our programme names. I added a small glossary that protects those terms before the text goes to the API and puts them back afterwards, and the glossary has its own tests.

 

The Malay home page, Malay all the way through, buttons included.

 

The safety net is an audit script I run against the live site. It visits every URL in the sitemap and fails if a page's language doesn't match its address, if the main text reads like the other language, if a link loses its language prefix, or if an image is broken.

Today's run against production: 18 pages and 21 images, PASS.

Test case 3: language consistency
ToolProduction audit script and Bing URL Inspection
ExpectedMalay pages fully in Malay, English pages fully in English
FoundMixed languages on Malay pages and links that switched language
FixFinished translations, declared language per page, hreflang, Azure AI Translator with a protected glossary
ResultAudit passes on 18 pages and 21 images, and it has already caught one regression

 

Test case 4: can crawlers even render the page?

Our robots.txt blocked the folder that serves the site's styles and scripts. Crawlers could read the text but couldn't render the page the way a visitor sees it, and because everything looked perfect in a browser, nobody had noticed.

Removing that rule was the main fix. While I was there, I made every page declare the www address as its canonical home, marked donation receipts and the 404 page as noindex, and rebuilt the sitemap so it only lists www addresses. Then I resubmitted it in Bing Webmaster Tools.

Bing Webmaster Tools > Sitemaps: 1 sitemap, 0 errors, 0 warnings, 18 URLs discovered.

 

I did hit a setback here. My first Site Scan was pointed at the bare domain rather than www. It scanned exactly one page, hit an HTTP 4xx error and stopped. Scan the address you actually serve.

 

My first Site Scan, on the bare domain: 1 page scanned, 1 error (Http 400-499).

 

While writing this post I also noticed that the bare domain redirects to www with a temporary redirect. For a permanent move you want a permanent redirect, so search engines hand everything over to the www address with confidence. That one is on my list.

 

Test case 4: crawlability and sitemap
ToolBing Webmaster Tools, Sitemaps and Site Scan
ExpectedCrawlers can render every page and find every URL
Foundrobots.txt blocked our styles and scripts; the first Site Scan stopped after one page
FixUnblocked the folder, one canonical www address, a clean sitemap resubmitted
Result1 sitemap, 0 errors, 18 URLs discovered

 

Test case 5: what are real visitors doing?

Our admin had fields for analytics IDs that someone had filled in long ago. They never saved. There was no error and no warning, the values just quietly disappeared, so we thought we were measuring visitors and we weren't. I rebuilt that settings page, then opened the live page source to check every value really rendered. After a few surprises I wasn't trusting anything I hadn't seen with my own eyes.

Then I added Microsoft Clarity, and it changed how I think about this whole project. Setup is one script tag, and I linked it to Bing Webmaster Tools so the two share data. Here's the dashboard for the last three days:

 

Clarity dashboard, last 3 days: 61 sessions (2 bot sessions excluded), 1 page per session, 77.25% scroll depth, 21 seconds active time, 6.56% dead clicks, 0% rage clicks.

 

Three numbers jumped out. People view one page per visit and leave, which tells me nothing is pulling them to a second page, and that's a content and linking problem rather than a technical one. Scroll depth is 77%, so when people arrive, they actually read. And 6.56% of sessions had a dead click, where someone clicked something that did nothing. On a site this small, that's worth chasing.

Asking Clarity Copilot

I don't have time to watch every recording, so I opened Copilot in the Clarity dashboard and asked it straight out: "Where are visitors getting stuck or frustrated on the site, and what should I fix first?"

Clarity Copilot's actual answer. It lists dead clicks by page and recommends starting with the Contact pages.

 

Copilot worked through the sessions and came back with dead clicks on our Contact pages in both languages and on a couple of blog pages, no rage clicks and no significant JavaScript errors. Its advice was to start with the Contact pages. I liked that it didn't stop at a number. It told me where to look.

Checking Copilot's answer with a heatmap

I wanted to see it for myself, so I opened the click heatmap for the Malay Contact page:

Click heatmap for /ms/contact. The most clicked element is our office address, which wasn't a link.

 

There it is. The most clicked thing on the page was our office address, with 2 of the 6 clicks, and it was plain text. People expect to tap an address and get a map, especially on a phone. We had a separate "Open in Google Maps" link underneath, but nobody went for it, so I made the whole address block the map link and shipped it the same day. Clarity also masks text by default (you can see the numbers and email hidden in the heatmap), which made me much more comfortable using it on a charity site.

 

Test case 5: dead clicks on the Contact page
ToolMicrosoft Clarity dashboard, Copilot and click heatmap
ExpectedFew or no dead clicks
Found6.56% of sessions had dead clicks, mostly on the Contact pages, where visitors clicked the plain-text address
FixThe whole address block is now the map link
ResultLive; I'll recheck the same heatmap after a week

Clarity also caught my own mess

This one I didn't see coming. The Heatmaps page lists your most visited pages, and two of the top entries were addresses on localhost. That was me, testing on my own laptop, because the analytics script loaded everywhere and my development sessions were mixed in with real visitors.

Popular pages in Clarity. Rows 2 and 9 are my local machine, not real visitors.

 

Our IndexNow code already refused to run anywhere except the live site, so I gave analytics the same rule. It was a one-line change. The same list also showed our 404 page with the foundation name twice in its title, so that got its own proper title too. Two bugs found for free.

Test case 6: my fix broke the contact form

This is the one that taught me to test outside the SEO bubble. When I made www the canonical address, our contact form started rejecting submissions. Its security check only accepted requests from the address it already knew about, and after my change everyone was arriving on www.

The fix was small, and I added tests so it can't quietly break again. Funnily enough, Clarity Copilot's first recommendation was also to test the contact forms and buttons. Two different tools pointed at the same page. If you change your URLs, test your forms.

 

Test case 6: contact form after the canonical change
ToolManual form tests and Clarity Copilot
ExpectedThe contact form works on the www address
FoundSubmissions were rejected after the canonical change
FixSecurity check updated and covered by automated tests
ResultThe form works and the tests guard against a repeat

 

Test case 7: can new pages reach Bing quickly?

Our site is small and Bing doesn't crawl it often, so waiting days for a new story to show up felt silly. I wired up IndexNow. Whenever a story or programme is published, edited, unpublished or deleted, the site sends both language versions to the IndexNow endpoint. Scheduled posts send theirs when they go live, and there's a "Send all pages now" button for big batches. The request itself is tiny:

 

POST https://api.indexnow.org/indexnow
{
  "host": "www.example.org",
  "key": "<your key>",
  "keyLocation": "https://www.example.org/<your key>.txt",
  "urlList": ["https://www.example.org/en/blog/...", "https://www.example.org/ms/blog/..."]
}

I hit two gotchas. The key file has to be served as plain text, not wrapped in an HTML page. And only the live site should send pings, otherwise test builds tell Bing about addresses that don't exist.

 

Test case 7: getting new pages to Bing
ToolIndexNow and Bing Webmaster Tools
ExpectedNew and changed stories reach Bing within hours
FoundA small site that gets crawled rarely, so new stories took days to appear
FixAutomatic IndexNow pings on publish, edit, unpublish and delete, in both languages
ResultEvery change is submitted the moment it happens

 

Test case 8: what's happening in AI search?

People ask Copilot and ChatGPT about charities now, and I wanted those answers to be right about us. I added llms.txt and llms-full.txt, plain summaries built from our live content, and a robots.txt that explicitly welcomes the main AI crawlers.

Bing Webmaster Tools has a new AI Performance report (still in beta) that shows how often your pages are cited as sources in Microsoft Copilot answers. Ours is honest and humbling:

 

Bing Webmaster Tools > AI Performance (beta): 0 citations, 0 cited pages over 3 months.

 

Zero. Then, while grabbing screenshots for this post, I found part of the reason other AI tools can't see us. Our CDN had quietly added its own managed block to the top of our robots.txt that disallows GPTBot, ClaudeBot, Google-Extended and others. Further down, my own rules allow the same bots, so the file argues with itself.

Our live robots.txt. The managed block at the top disallows AI crawlers (red); my rules further down allow them (green).

 

It isn't only the file, either. When I sent requests pretending to be those crawlers, some of them were turned away before they ever reached the site:

403 means blocked at the edge. GPTBot and ClaudeBot are refused; PerplexityBot and Bingbot get through.

 

Bingbot gets through, so Copilot can read us, and our zero citations are more likely down to having only a handful of stories so far. ChatGPT and Claude can't read us at all right now, though, which makes the llms.txt work pointless for them. This is a real decision rather than a bug. I'm leaning towards letting AI search and answer bots in while still blocking crawlers that only collect content for training.

 

Clarity has an AI Visibility section as well, which tracks AI bot activity and citations. Mine says "almost ready" because the domain needs verifying through Bing Webmaster Tools first, which is my next five-minute job.

 

Clarity > AI Visibility: waiting for domain verification via Bing Webmaster Tools or Google Search Console.

 

Test case 8: AI crawler access
ToolBing AI Performance, Clarity AI Visibility and crawler user-agent tests
ExpectedAI assistants can read the site and llms.txt
Found0 Copilot citations so far; some AI crawlers are blocked at the CDN and robots.txt contradicts itself
FixWaiting on a decision about which AI crawlers to allow
ResultOpen

 

Helping our writers do the right thing

Everything above is plumbing. People find you through content, and our stories are written by volunteers who aren't SEO people. Handing them a style guide was never going to work, so the guidance lives inside the story editor. As they write, a 13-point checklist scores the story: is the focus phrase in the title, is the title the right length, is there a real search description, does every photo have a description, is it long enough, and does it link to another page. A preview shows the result as it will appear in search, using the same title rules as the live site. One click adds Malaysian search phrases in English or Malay, and "Translate whole story" uses Azure AI Translator so every story exists in both languages.

 

The story editor's "Search and sharing" panel. This story sits at 70% and tells you exactly what's missing.

 

Bing's Keyword Research taught me something too: Malay keyword data is thin. "Bantuan asnaf" (help for asnaf families) is a phrase people really use, but Bing doesn't have enough data to show a trend, and the top results are government and zakat bodies. We're not going to win that head term. We can win the long tail, with specific stories about specific programmes in specific places.

 

Bing Keyword Research for "bantuan asnaf": not enough data for a trend, and the top 10 are government and zakat sites.

 

Finally, I built a small Search insights page in our admin that uses the Bing Webmaster API and Google Search Console to show both engines side by side. It highlights phrases where we rank between 8 and 20, close to page one, and page-one results that nobody clicks. Bing is still collecting data for our www address, so that page will get more useful over the next few weeks.

Conclusion

When I started, I thought SEO was about keywords. For a small site like ours it turned out to be mostly about trust and hygiene: telling search engines the truth, making sure they can actually read the page, and making sure both languages are really there. Every fix in this post came from looking at the site the way a crawler does, not the way I do.

 

The Microsoft tools did most of the work. Bing Webmaster Tools gave me the clearest to-do list I had, and the Bing Index and Live URL tabs in URL Inspection gave me a clean before and after for every fix. Clarity showed me what real visitors do, and Copilot turned a pile of sessions into one clear instruction: start with the Contact page. Azure AI Translator keeps both languages in step, and IndexNow gets new stories in front of Bing within hours. None of it needed a budget.

 

Writing this post paid off too. Collecting the screenshots turned up three more bugs: my own laptop counting as a visitor, a doubled title on the 404 page, and an address people kept tapping because they expected a map. All three are fixed and live. Two things are still on my list, deciding how much access AI crawlers get and making the bare domain redirect permanent. After that comes the slow part, writing more stories and linking them together so visitors read a second page.

 

If you're doing this for a small nonprofit, look before you add anything, write your checks down as tests, and find out what sits in front of your site. And if you've already worked through the AI crawler question, I'd really like to hear how you handled it in the comments.

Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories