Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
140434 stories
·
32 followers

Why AI Demands a New Approach to Observability

1 Share

AI is transforming industries in ways we couldn’t have imagined just a few short years ago, from automating customer service to streamlining supply chain management. But, with these exciting advancements come new challenges. As AI systems become more integral to everyday business operations, the need to monitor their performance, behavior, and decision-making processes has never been greater.

According to the National Institute of Standards and Technology (NIST), AI applications require rigorous oversight, as inadequate management can lead to unforeseen or inequitable outcomes. Unfortunately, existing observability solutions fall short by focusing solely on performance, rather than other attributes unique to AI.

This article sheds light on observability, defining what it is, why it’s essential for managing AI systems, and why traditional approaches are inadequate.

So, What Exactly Is Observability?

Observability platforms help companies monitor, analyze, and understand the performance and health of their systems, including logs, metrics, and traces. Traditional monitoring systems track basic metrics like server status or network latency. They’re an evolution of traditional network monitoring solutions, but with a broader scope and more advanced capabilities. These platforms take monitoring further by empowering teams to answer questions like, “What’s causing these performance issues?” or “Why is this behavior happening?” It provides deeper, actionable insights into system health and performance, helping teams solve issues before they impact users.

Observability offers several key benefits for businesses, like accelerating issue resolution and reducing downtime. It can also help businesses optimize resources, predict failures before they happen, and align system health with key business metrics. Ultimately, observability is about making smarter, faster decisions to keep things running smoothly.

Why Traditional Observability Tools Fall Short for AI

AI systems introduce a new set of variables that require careful monitoring, particularly when interacting with AI models that engage with users in real-time.

Traditional observability systems were designed to track predefined metrics like CPU usage, vs. capturing GenAI response inaccuracies or mitigating malicious user inputs. This is where advanced AI observability steps in, offering a way to manage these systems effectively and ensure they perform as expected and, importantly, that they do so in an ethical and secure way.

Let’s take a deeper dive into the unique needs of AI systems:

  • AI “Grey Areas”: AI presents unique grey areas for performance monitoring. An AI system may appear to perform correctly (e.g., a user enters a prompt and the AI answers), but the response is suspect. For example, if a company’s chatbot promotes a competitor over its own product, the performance is fine, but the content of the response is a problem.
  • Data Quality Monitoring and Security: It is crucial to make sure that AI does not leak sensitive information such as customer PII, confidential business data, or proprietary information about the AI system itself — especially as nefarious actors may try to trick it into doing so.
  • User Interaction:  True measurement of AI effectiveness requires insights not only into the system’s performance but also into user interaction. While metrics like response times and accuracy are important, they don’t provide the full picture. Feedback about the AI’s responses to user queries is essential, so that businesses know if the AI is responding as expected and meeting customers’ needs.
  • AI Output Issues: AI outputs must be continuously monitored for toxicity (i.e., harmful or offensive content such as hate speech or biased language) and hallucinations, where the model generates misleading or false information. Without proper oversight, these issues can undermine user trust, spread misinformation, and lead to unintended consequences
  • Compliance: AI systems must adhere to strict and ever-evolving regulations and compliance standards that are difficult to keep up with. These include the EU Artificial Intelligence Act, California Consumer Privacy Act, and several others.

Why Advanced AI Observability Systems Are Essential

Observability tools tailored explicitly for AI systems address these unique risks with the following features:

  1. Comprehensive User Interaction Monitoring: Provides full visibility into user interactions, including conversation histories, logins, and use of tokens (i.e. units of data consumed by AI models). This granular tracking helps teams identify suspicious resource consumption, detect cost harvesting attempts (i.e. query spamming to drive up operational costs for an AI system), and optimize budgets without compromising performance.
  2. Real-Time Issue Detection and Resolution: Identifies problems such as poor response accuracy, latency spikes, and malicious user inputs. This allows teams to address underperforming AI agents before they negatively impact the user experience.
  3. AI Evaluation Engine: Includes both quality and security evaluators, as well as the ability to customize to meet specific needs. The quality evaluators analyze both user and AI interactions to detect potential issues, such as toxicity and AI hallucinations. Meanwhile, the security evaluators continuously monitor for potential risks, enabling early identification and helping secure AI systems against malicious actors.
  4. Compliance and Regulation Adherence: Helps businesses stay on top of evolving regulatory requirements regarding AI accuracy, reliability, and ethical use, while also mitigating legal and reputational risks.

The Future of AI Observability

As AI systems grow more complex, the limitations of traditional observability tools are becoming more apparent. Standard observability solutions only focus on performance and are not designed with AI’s unique challenges in mind, so they struggle to effectively track and manage the nuances of AI behavior.

Fortunately, emerging observability platforms are being designed to address the unique challenges posed by AI solutions, providing deeper insights into user interactions and risks, such as toxicity, hallucinations, and security vulnerabilities.

One exciting development we anticipate seeing soon is the integration of agentic AI into the observability process. This would be a game-changer, with agents able to diagnose and solve problems with minimal human assistance.

For organizations aiming to stay competitive, AI observability must be a core component of their strategy. By focusing on AI-specific metrics, businesses can ensure a seamless and high-quality AI environment, harness the full potential of the technology, and achieve positive, tangible business outcomes.

The post Why AI Demands a New Approach to Observability appeared first on The New Stack.

Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Assigning and completing issues with coding agent in GitHub Copilot

1 Share

You’ve used GitHub Copilot to help you write code in your IDE. Now, imagine assigning Copilot an issue, just like you would a teammate—and getting a fully tested pull request in return. 

That’s the power of the new coding agent in GitHub Copilot. Built directly into GitHub, this agent starts working as soon as you assign it a GitHub Issue or prompt it in VS Code. Keeping you firmly in the pilot’s seat, the coding agent builds pull requests based on the issues you assign it.

This isn’t just autocomplete. It’s a new class of software engineering agents that work asynchronously to help you move faster, clean up tech debt, and focus on the work that really matters. Let’s explore how this coding agent works and how it can help you find new ways of working faster. ✨

Oh, and if you’re a visual learner we have you covered. 👇

Coding agent in GitHub Copilot 101

This new coding agent, which is our first asynchronous software engineering agent, is built on GitHub Actions and works like a teammate. You assign it an issue, let it do the work, and then review its outputs before changing or accepting them. It also incorporates context from related issues or PR discussions and can follow custom repository instructions that your team has already set.

You assign Copilot an issue and it plans the work, opens a pull request, writes the code, runs the tests, and then asks for your review. If you leave feedback, it’ll revise the PR and keep going until you approve. 

The process isn’t instant—it takes a little time to compute and run. But it’s already helping developers work faster and more efficiently. 

According to Brittany Ellich, Senior Software Engineer at GitHub, traditional advice for devs has been to do one thing at a time, and do it well. But with the new coding agent, GitHub can now help you do more things well, like:

  • Offloading repetitive, boilerplate tasks like adding and extending unit tests
  • Maintaining better issue hygiene and documentation with quick typo fixes and small refactors
  • Improving user experience by fixing bugs, updating user interface features, and bolstering accessibility

By assigning these low- to medium-complexity tasks to the coding agent, you may finally have the bandwidth to focus on higher-level problem solving and design, tackle that tech debt that’s been piling up, learn new skills, and more.

Even though Copilot is doing the work, you’re in control the entire time: You decide what to assign, what to approve, and what should be changed.

How to get the coding agent to complete an issue

Step one: Write and assign the issue to Copilot

This is where you’ll be most involved—and this step is crucial for success. Think of writing the issue like briefing a team member: The more context you give, the better the results (like any other prompt). 

Make sure to include:

  • Relevant background info: Why this task matters, what it touches, and any important history or context. 
  • Expected outcome: What “done” looks like.
  • Technical details: File names, functions, or components involved.
  • Formatting or linting rules: These are especially important if you use custom scripts or auto-generated files. You can add these instructions for Copilot so it’s automatically reflected in every issue. 

Once you’ve written the issue, it’s time to assign it to Copilot—just like you would a teammate. You can do this via github.com, the GitHub Mobile app, or through the GitHub CLI. 

Copilot works best with well-scoped tasks, but it can handle larger ones. It just might take a little bit longer. You don’t have to assign only one issue; you can batch-assign multiple issues, which is great for tasks like increasing test coverage or updating documentation.

Here are a few tips and tricks that we’ve found helpful:

  • You can use issue templates with fields like “description” and “acceptance criteria” to make writing issues easier and more consistent across your team. 
  • If your repo includes custom instructions (such as which files are auto-generated or how to run formatters), Copilot will use these to improve its output.
  • The agent can actually see images included in its assigned issues on GitHub, so you can easily share images of what you want your new feature to look like, and the agent can run with it. 

Step two: Copilot plans the code 

Once you assign Copilot an issue, it will add an 👀 emoji reaction. Then it will kick off an agent session using GitHub Actions, which powers the integrated, secure, and fully customizable environment the coding agent is built on. 

This environment is where Copilot can explore and analyze your codebase, run tests, and make changes. The coding agent will simultaneously open both a branch and a pull request, which will evolve as Copilot works. 

Copilot will read your issue and break it down into a checklist of tasks, then update the pull request with this checklist. As it completes each task, Copilot checks it off and pushes commits to the branch. You can watch the session live, view the session logs later, or refresh the PR to see how Copilot is reasoning through the task. These are updated regularly for increased visibility, so you can easily spot problems if they arise.

Step three: Copilot writes the code

This is where the magic happens. Once you see the “Copilot started work” event in the pull request timeline, you’ll know the wheels are turning. Here’s what happens next:

  • Copilot modifies your codebase based on the issue.
  • It runs automated tests and linters if they’re present in your repo and updates or generates tests as needed.
  • Copilot will also push commits iteratively as it completes tasks.

You can see the work happening in real time, and if you notice that something looks off, you can step in at any point to make sure things are going in the right direction before Copilot passes it back to you.

Step four: Review and merge the pull request

This is another stage where you’ll need to be involved. Once Copilot finishes the work, it will tag you for review. You can either:

  • Approve the pull request
  • Leave comments
  • Ask for changes

Copilot will automatically request reviewers based on the rules you’ve set in your repo. And if needed, you can go through multiple review cycles until you get your desired outcome—just like with a human teammate. 

Once the pull request is approved:

  • The change can now follow your repo’s merge and deploy process.
  • The agent session will end.
  • If needed, a human can take over from the branch at any time. 

🚨One important thing to note: The person who created the issue can’t be the final approver. You’ll need a peer, manager, or designated reviewer to give the green light. This promotes collaboration and ensures unreviewed or unsafe code doesn’t get merged.

And you’re done! ✅ 

Like any other tool (or teammate), Copilot’s coding agent might need a little prodding to deliver exactly the output you want. Remember, the biggest factor to success starts with how you write the issue (Copilot can also help you write those faster). 

Here are a few tips on how to get the most out of Copilot: 

  • Write comprehensive issues: Clear, scoped, and well-documented issues lead to better results.
  • Start small: Try using the agent for tests, docs, or simple refactors.
  • Troubleshooting: If Copilot gets stuck, tag it in a comment and add more context. Iterating and refining the issue requirements can also help.

Take this with you 

AI and LLMs are improving at a rapid pace. “The models we’re using today are the worst ones we’ll ever use—because they’re only getting better,” says Ellich. And coding agents are already proving useful in real workflows. 

Try using the coding agent on a sample repo. See what it can do. And start building your own agentic workflows. Happy coding!

Visit the Docs to get started with the coding agent in GitHub Copilot.

The post Assigning and completing issues with coding agent in GitHub Copilot appeared first on The GitHub Blog.

Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Better CSS Shapes Using shape() — Part 3: Curves

1 Share

If you’re following along, this is the third post in a series about the new CSS shape() function. We’ve learned how to draw lines and arcs and, in this third part, I will introduce the curve command — the missing command you need to know to have full control over the shape() function. In reality, there are more commands, but you will rarely need them and you can easily learn about them later by checking the documentation.

Better CSS Shapes Using shape()

  1. Lines and Arcs
  2. More on Arcs
  3. Curves (you are here!)

The curve command

This command adds a Bézier curve between two points by specifying control points. We can either have one control point and create a Quadratic curve or two control points and create a Cubic curve.

Bézier, Quadratic, Cubic, control points? What?!

For many of you, that definition is simply unclear, or even useless! You can spend a few minutes reading about Bézier curves but is it really worth it? Probably not, unless your job is to create shapes all the day and you have a solid background in geometry.

We already have cubic-bezier() as an easing function for animations but, honestly, who really understands how it works? We either rely on a generator to get the code or we read a “boring” explanation that we forget in two minutes. (I have one right here by the way!)

Don’t worry, this article will not be boring as I will mostly focus on practical examples and more precisely the use case of rounding the corners of irregular shapes. Here is a figure to illustrate a few examples of Bézier curves.

Comparing two curved lines, one with one control point and one with two control points.

The blue dots are the starting and ending points (let’s call them A and B) and the black dots are the control points. And notice how the curve is tangent to the dashed lines illustrated in red.

In this article, I will consider only one control point. The syntax will follow this pattern:

clip-path: shape(
  from Xa Ya, 
  curve to Xb Yb with Xc Yc
);

arc command vs. curve command

We already saw in Part 1 and Part 2 that the arc command is useful establishing rounded edges and corners, but it will not cover all the cases. That’s why you will need the curve command. The tricky part is to know when to use each one and the answer is “it depends.” There is no generic rule but my advice is to first see if it’s possible (and easy) using arc. If not, then you have to use curve.

For some shapes, we can have the same result using both commands and this is a good starting point for us to understand the curve command and compare it with arc.

Take the following example:

This is the code for the first shape:

.shape {
  clip-path: shape(from 0 0,
    arc to 100% 100% of 100% cw,
    line to 0 100%)
}

And for the second one, we have this:

.shape {
  clip-path: shape(from 0 0,
    curve to 100% 100% with 100% 0,
    line to 0 100%)
}

The arc command needs a radius (100% in this case), but the curve command needs a control point (which is 100% 0 in this example).

Two rounded shapes that appear to have similar curves, one using an arc and another using a curve.

Now, if you look closely, you will notice that both results aren’t exactly the same. The first shape using the arc command is creating a quarter of a circle, whereas the shape using the curve command is slightly different. If you place both of them above each other, you can clearly see the difference.

This is interesting because it means we can round some corners using either an arc or a curve, but with slightly different results. Which one is better, you ask? I would say it depends on your visual preference and the shape you are creating.

In Part 1, we created rounded tabs using the arc command, but we can also create them with curve.

Can you spot the difference? It’s barely visible but it’s there.

Notice how I am using the by directive the same way I am doing with arc, but this time we have the control point, which is also relative. This part can be confusing, so pay close attention to this next bit.

Consider the following:

shape(from Xa Ya, curve by Xb Yb with Xc Yc)

It means that both (Xb,Yb) and (Xc,Yc) are relative coordinates calculated from the coordinate of the starting point. The equivalent of the above using a to directive is this:

shape(from Xa Ya, curve to (Xa + Xb) (Ya + Yb) with (Xa + Xc) (Yb + Yc))

We can change the reference of the control point by adding a from directive. We can either use start (the default value), end, or origin.

shape(from Xa Ya, curve by Xb Yb with Xc Yc from end)

The above means that the control point will now consider the ending point instead of the starting point. The result is similar to:

shape(from Xa Ya, curve to (Xa + Xb) (Ya + Yb) with (Xa + Xb + Xc) (Ya + Yb + Yc))

If you use origin, the reference will be the origin, hence the coordinate of the control point becomes absolute instead of relative.

The from directive may add some complexity to the code and the calculation, so don’t bother yourself with it. Simply know it exists in case you face it, but keep using the default value.

I think it’s time for your first homework! Similar to the rounded tab exercise, try to create the inverted radius shape we covered in the Part 1 using curve instead of arc. Here are both versions for you to reference, but try to do it without peeking first, if you can.

Let’s draw more shapes!

Now that we have a good overview of the curve command, let’s consider more complex shapes where arc won’t help us round the corners and the only solution is to draw curves instead. Considering that each shape is unique, so I will focus on the technique rather than the code itself.

Slanted edge

Let’s start with a rectangular shape with a slanted edge.

A slanted rectangle shape in two stages, first with sharp edges, then with curved edges.

Getting the shape on the left is quite simple, but the shape on the right is a bit tricky. We can round two corners with a simple border-radius, but for the slanted edge, we will use shape() and two curve commands.

The first step is to write the code of the shape without rounded corners (the left one) which is pretty straightforward since we’re only working with the line command:

.shape {
  --s: 90px;  /* slant size */

  clip-path: 
    shape(from 0 0,
    line to calc(100% - var(--s)) 0,
    line to 100% 100%,
    line to 0 100%
    );
}

Then we take each corner and try to round it by modifying the code. Here is a figure to illustrate the technique I am going to use for each corner.

Diagrammiong a rounded rectangular shape in three stages, first with sharp edges, then with points indicating where the curve control points are, then the completed shape.

We define a distance, R, that controls the radius. From each side of the corner point, I move by that distance to create two new points, which are illustrated above in red. Then, I draw my curve using the new points as starting and ending points. The corner point will be the control point.

The code becomes:

.shape {
  --s: 90px;  /* slant size */

  clip-path: 
    shape(from 0 0,
    Line  to Xa Ya,
    curve to Xb Yb with calc(100% - var(--s)) 0,
    line to 100% 100%,
    line to 0 100%
    );
}

Notice how the curve is using the coordinates of the corner point in the with directive, and we have two new points, A and B.

Until now, the technique is not that complex. For each corner point, you replace the line command with line + curve commands where the curve command reuses the old point in its with directive.

If we apply the same logic to the other corner, we get the following:

.shape {
  --s: 90px;  /* slant size */

  clip-path: 
    shape(from 0 0,
    line  to Xa Ya, 
    curve to Xb Yb with calc(100% - var(--s)) 0,
    line  to Xc Yc,
    curve to Xd Yd with 100% 100%,
    line to 0 100%
    );
}

Now we need to calculate the coordinates of the new points. And here comes the tricky part because it’s not always simple and it may require some complex calculation. Even if I detail this case, the logic won’t be the same for the other shapes we’re making, so I will skip the math part and give you the final code:

.box {
  --h: 200px; /* element height */
  --s: 90px;  /* slant size */
  --r: 20px;  /* radius */
  
  height: var(--h);
  border-radius: var(--r) 0 0 var(--r);
  --_a: atan2(var(--s), var(--h));
  clip-path: 
    shape(from 0 0,
    line  to calc(100% - var(--s) - var(--r)) 0,
    curve by calc(var(--r) * (1 + sin(var(--_a)))) 
              calc(var(--r) * cos(var(--_a)))
    with var(--r) 0,
    line  to calc(100% - var(--r) * sin(var(--_a))) 
              calc(100% - var(--r) * cos(var(--_a))),
    curve to calc(100% - var(--r)) 100%  with 100% 100%,
    line to 0 100%
    );
}

I know the code looks a bit scary, but the good news is that the code is also really easy to control using CSS variables. So, even if the math is not easy to grasp, you don’t have to deal with it. It should be noted that I need to know the height to be able to calculate the coordinates which means the solution isn’t perfect because the height is a fixed value.

Arrow-shaped box

Here’s a similar shape, but this time we have three corners to round using the curve command.

The final code is still complex but I followed the same steps. I started with this:

.shape {
  --s: 90px; 

  clip-path: 
    shape(from 0 0,
    /* corner #1 */
    line to calc(100% - var(--s)) 0,
    /* corner #2 */
    line to 100% 50%,
    /* corner #3 */
    line to calc(100% - var(--s)) 100%,

    line to 0 100%
    );
}

Then, I modified it into this:

.shape {
  --s: 90px; 

  clip-path: 
    shape(from 0 0,
    /* corner #1 */
    line  to Xa Ya
    curve to Xb Yb with calc(100% - var(--s)) 0,
    /* corner #2 */
    line  to Xa Ya
    curve to Xb Yb with 100% 50%,
    /* corner #3 */
    line  to Xa Yb
    curve to Xb Yb with calc(100% - var(--s)) 100%,

    line to 0 100%
    );
}

Lastly, I use a pen and paper to do all the calculations.

You might think this technique is useless if you are not good with math and geometry, right? Not really, because you can still grab the code and use it easily since it’s optimized using CSS variables. Plus, you aren’t obligated to be super accurate and precise. You can rely on the above technique and use trial and error to approximate the coordinates. It will probably take you less time than doing all the math.

Rounded polygons

I know you are waiting for this, right? Thanks to the new shape() and the curve command, we can now have rounded polygon shapes!

Three rounded polygon shapes, first a pentagon, second a triangle, and third an octagon.

Here is my implementation using Sass where you can control the radius, number of sides and the rotation of the shape:

If we omit the complex geometry part, the loop is quite simple as it relies on the same technique with a line + curve per corner.

$n: 9; /* number of sides*/
$r: .2; /* control the radius [0 1] */
$a: 15deg; /* control the rotation */

.poly {
  aspect-ratio: 1;
  $m: ();
  @for $i from 0 through ($n - 1) {
    $m: append($m, line  to Xai Yai, comma);
    $m: append($m, curve to Xbi Ybi with Xci Yci, comma);
  } 
  clip-path: shape(#{$m});
}

Here is another implementation where I define the variables in CSS instead of Sass:

Having the variables in CSS is pretty handy especially if you want to have some animations. Here is an example of a cool hover effect applied to hexagon shapes:

I have also updated my online generator to add the radius parameter. If you are not familiar with Sass, you can easily copy the CSS code from there. You will also find the border-only and cut-out versions!

A rounded shape with six sides next to a cutout of a rounded shape with six edges.

Conclusion

Are we done with the curve command? Probably not, but we have a good overview of its potential and all the complex shapes we can build with it. As for the code, I know that we have reached a level that is not easy for everyone. I could have extended the explanation by explicitly breaking down the math, but then this article would be overly complex and make it seem like using shape() is harder than it is.

This said, most of the shapes I code are available within my online collection that I constantly update and optimize so you can easily grab the code of any shape!

If you want a good follow-up to this article, I wrote an article for Frontend Masters where you can create blob shapes using the curve command.

Three blob shapes in a single row, each colored with a gradient that goes left to right from dark orange to light orange.

Better CSS Shapes Using shape()

  1. Lines and Arcs
  2. More on Arcs
  3. Curves (you are here!)

Better CSS Shapes Using shape() — Part 3: Curves originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.

Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Building a Modern Python API with Azure Cosmos DB: A 5-Part Video Series

1 Share
I’m excited to share our new video series where I walk through building a production-ready inventory management API using Python, FastAPI, and Azure Cosmos DB NoSQL. This project demonstrates modern async patterns, clean architecture, and enterprise-grade features like batch operations and optimistic concurrency control.
This builds on our Getting Started Series, check out those videos to grasp the fundamentals first.

Video 1: Data Modeling with Pydantic

In the first video, I dive deep into data modeling using Pydantic v2, showing how to create robust, type-safe models with built-in validation. The key insight is using model inheritance to separate concerns between what clients send and what the database returns.

class Product(BaseModel):
    """Fields that are intrinsic to a product"""
    name: Annotated[str, Field(min_length=1, max_length=255)]
    description: Annotated[Optional[str], Field(None, max_length=1000)]
    category: Annotated[str, Field(min_length=1, max_length=100)] 
    price: Annotated[Decimal, Field(gt=0, decimal_places=2)]
    sku: Annotated[str, Field(min_length=1, max_length=50, pattern=r"^[A-Z0-9-]+$")]
    quantity: Annotated[int, Field(ge=0)] = 0
    status: ProductStatus = ProductStatus.ACTIVE

    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)

class ProductResponse(Product):
    """All product fields plus system-generated fields."""
    id: str
    etag: str = Field(alias="_etag")  # Cosmos DB concurrency control token
    last_updated: datetime
The model hierarchy ensures that input validation happens automatically, while response models include system-generated fields like ETags for optimistic concurrency control.

Video 2: Client Configuration and Connection Management

The second video covers setting up a production-ready Azure Cosmos DB client with proper authentication, connection pooling, and singleton pattern implementation. I show how to use Azure’s managed identity for secure, credential-free authentication.

async def _ensure_client() -> CosmosClient:
    """
    Ensures a single CosmosClient instance exists for the lifetime of the application.
    """
    global _client, _credential
    if _client is None:
        try:
            logger.info("Initializing Cosmos DB client with DefaultAzureCredential")
            _credential = DefaultAzureCredential()

            client_options = {
                "connection_timeout": 60, 
            }

            _client = CosmosClient(COSMOSDB_ENDPOINT, _credential, **client_options)

            logger.info(
                "Cosmos DB client initialized successfully",
                extra={
                    "endpoint": COSMOSDB_ENDPOINT,
                    "auth_method": "DefaultAzureCredential",
                },
            )
        except Exception as e:
            logger.error(f"Failed to initialize Cosmos DB client: {e}")
            raise
    return _client
This pattern ensures efficient resource usage while maintaining thread safety in async environments.

Video 3: Async Operations Done Right

In the third video, I demonstrate how to leverage FastAPI’s async capabilities with Azure Cosmos DB’s async SDK. The combination of async/await patterns with dependency injection creates clean, testable code.

async def update_product(
    container: ContainerProxy,
    identifier: VersionedProductIdentifier,
    updates: ProductUpdate
) -> ProductResponse:
    """Update an existing product with optimistic concurrency control."""
    update_dict = updates.model_dump(exclude_unset=True)
    normalized_category = normalize_category(identifier.category)
    update_dict["last_updated"] = datetime.now(timezone.utc).isoformat()
    
    # Create patch operations for partial updates
    patch_operations = []
    for key, value in update_dict.items():
        if key not in ["id", "category", "_etag"]: 
            patch_operations.append(
                {"op": "set", "path": f"/{key}", "value": value}
            )
    
    try:
        result = await container.patch_item(
            item=identifier.id,
            partition_key=normalized_category,
            patch_operations=patch_operations,
            headers={"if-match": identifier.etag},  # ETag for concurrency control
        )
        return ProductResponse.model_validate(result)
    except Exception as e:
        handle_cosmos_error(e, operation="update", product_id=identifier.id)
The use of patch operations enables efficient partial updates while ETags prevent concurrent modification conflicts.

Video 4: Batch Operations for Performance

The fourth video explores batch operations, showing how to process multiple items efficiently using Azure Cosmos DB’s batch API and Python’s asyncio for concurrent execution across partitions.

async def _execute_batch_by_category(
    items_by_category: dict[str, list[T]],
    process_func: Callable[[str, list[T]], Any],
    result_extractor: Callable[[Any], list[R]] | None = None
) -> list[R] | list[Any]:
    """
    Generic function to execute batch operations grouped by category 
    with concurrent processing.
    """
    # Schedule tasks to run concurrently for each category
    tasks = [
        asyncio.create_task(process_func(category, items))
        for category, items in items_by_category.items()
    ]
    
    # Wait for all tasks to complete and gather results
    results = await asyncio.gather(*tasks, return_exceptions=True)
    
    # Check for exceptions and collect successful results
    all_results = []
    exceptions = []
    
    for result in results:
        if isinstance(result, Exception):
            exceptions.append(result)
        else:
            if result_extractor and result is not None:
                all_results.extend(result_extractor(result))
    
    if exceptions:
        raise exceptions[0]
    
    return all_results
This generic approach allows efficient batch processing while respecting Azure Cosmos DB’s partition key requirements.

Video 5: Centralized Error Handling and Logging

The final video demonstrates how to implement centralized error handling that transforms low-level database errors into meaningful application exceptions, complete with structured logging for observability.

def handle_cosmos_error(e: Exception, operation: str, **context: Any) -> None:
    """Convert Cosmos DB exceptions to application-specific exceptions."""
    
    if isinstance(e, CosmosBatchOperationError):
        status_code = getattr(e, 'status_code', None)
        if status_code == 409:
            error_message = str(e).lower()
            if "unique index constraint" in error_message and "/sku" in error_message:
                sku = context.get("sku", "unknown")
                raise ProductDuplicateSKUError(
                    f"Product with SKU '{sku}' already exists"
                ) from e
            else:
                # Handle ID conflicts - prefer showing SKU if available
                sku = context.get("sku")
                if sku:
                    raise ProductAlreadyExistsError(
                        f"Product with SKU '{sku}' already exists"
                    ) from e

@app.exception_handler(ApplicationError)
async def application_error_handler(request: Request, exc: ApplicationError) -> JSONResponse:
    """Handle application-specific errors with structured logging."""
    logger = logging.getLogger(request.scope.get("route").endpoint.__module__)
    
    # Log at appropriate level based on status code
    log_level = "warning" if exc.status_code < 500 else "error"
    extra = {"error_type": type(exc).__name__}
    
    getattr(logger, log_level)(str(exc), extra=extra, exc_info=exc)
This approach ensures that errors are handled consistently across the application while providing meaningful feedback to API consumers.

Conclusion

This project demonstrates how modern Python features combined with Azure Cosmos DB can create a robust, scalable API. Key takeaways include:
  • Leveraging Pydantic for strong typing and validation
  • Implementing proper async patterns for optimal performance
  • Using batch operations to minimize round trips to the database
  • Building comprehensive error handling for production reliability
  • Maintaining clean architecture with clear separation of concerns
The complete source code is available on GitHub, and I encourage you to watch the video series for detailed explanations. Whether you’re building a new API or modernizing an existing one, these patterns will help you create more maintainable and performant applications.  Happy coding!

Leave a review

Tell us about your Azure Cosmos DB experience! Leave a review on PeerSpot and we’ll gift you $50. Get started here.

About Azure Cosmos DB

Azure Cosmos DB is a fully managed and serverless NoSQL and vector database for modern app development, including AI applications. With its SLA-backed speed and availability as well as instant dynamic scalability, it is ideal for real-time NoSQL and MongoDB applications that require high performance and distributed computing over massive volumes of NoSQL and vector data.

To stay in the loop on Azure Cosmos DB updates, follow us on XYouTube, and LinkedIn.

To quickly build your first database, watch our Get Started videos on YouTube and explore ways to dev/test free.

The post Building a Modern Python API with Azure Cosmos DB: A 5-Part Video Series appeared first on Azure Cosmos DB Blog.

Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete

What Comes After the LLM: Human-Centered AI, Spatial Intelligence, and the Future of Practice

1 Share

In a recent episode of High Signal, we spoke with Dr. Fei-Fei Li about what it really means to build human-centered AI, and where the field might be heading next.

Fei-Fei doesn’t describe AI as a feature or even an industry. She calls it a “civilizational technology”—a force as foundational as electricity or computing itself. This has serious implications for how we design, deploy, and govern AI systems across institutions, economies, and everyday life.

Our conversation was about more than short-term tactics. It was about how foundational assumptions are shifting, around interface, intelligence, and responsibility, and what that means for technical practitioners building real-world systems today.

The Concentric Circles of Human-Centered AI

Fei-Fei’s framework for human-centered AI centers on three concentric rings: the individual, the community, and society.

Image created by Adobe Firefly

At the individual level, it’s about building systems that preserve dignity, agency, and privacy. To give one example, at Stanford, Fei-Fei’s worked on sensor-based technologies for elder care aimed at identifying clinically relevant moments that could lead to worse outcomes if left unaddressed. Even with well-intentioned design, these systems can easily cross into overreach if they’re not built with human experience in mind.

At the community level, our conversation focused on workers, creators, and collaborative groups. What does it mean to support creativity when generative models can produce text, images, and video at scale? How do we augment rather than replace? How do we align incentives so that the benefits flow to creators and not just platforms?

At the societal level, her attention turns to jobs, governance, and the social fabric itself. AI alters workflows and decision-making across sectors: education, healthcare, transportation, even democratic institutions. We can’t treat that impact as incidental.

In an earlier High Signal episode, Michael I. Jordan argued that too much of today’s AI mimics individual cognition rather than modeling systems like markets, biology, or collective intelligence. Fei-Fei’s emphasis on the concentric circles complements that view—pushing us to design systems that account for people, coordination, and context, not just prediction accuracy.

Spatial Intelligence: A Different Language for Computation

Another core theme of our conversation was Fei-Fei’s work on spatial intelligence and why the next frontier in AI won’t be about language alone.

At her startup, World Labs, Fei-Fei is developing foundation models that operate in 3D space. These models are not only for robotics; they also underpin applications in education, simulation, creative tools, and real-time interaction. When AI systems understand geometry, orientation, and physical context, new forms of reasoning and control become possible.

“We are seeing a lot of pixels being generated, and they’re beautiful,” she explained, “but if you just generate pixels on a flat screen, they actually lack information.” Without 3D structure, it’s difficult to simulate light, perspective, or interaction, making it hard to compute with or control.

For technical practitioners, this raises big questions:

  • What are the right abstractions for 3D model reasoning?
  • How do we debug or test agents when output isn’t just text but spatial behavior?
  • What kind of observability and interfaces do these systems need?

Spatial modeling is about more than realism; it’s about controllability. Whether you’re a designer placing objects in a scene or a robot navigating a room, spatial reasoning gives you consistent primitives to build on.

Institutions, Ecosystems, and the Long View

Fei-Fei also emphasized that technology doesn’t evolve in a vacuum. It emerges from ecosystems: funding systems, research labs, open source communities, and public education.

She’s concerned that AI progress has accelerated far beyond public understanding—and that most national conversations are either alarmist or extractive. Her call: Don’t just focus on models. Focus on building robust public infrastructure around AI that includes universities, startups, civil society, and transparent regulation.

This mirrors something Tim O’Reilly told us in another episode: that fears about “AI taking jobs” often miss the point. The Industrial Revolution didn’t eliminate work—it redefined tasks, shifted skills, and massively increased the demand for builders. With AI, the challenge isn’t disappearance. It’s transition. We need new metaphors for productivity, new educational models, and new ways of organizing technical labor.

Fei-Fei shares that long view. She’s not trying to chase benchmarks; she’s trying to shape institutions that can adapt over time.

For Builders: What to Pay Attention To

What should AI practitioners take from all this?

First, don’t assume language is the final interface. The next frontier involves space, sensors, and embodied context.

Second, don’t dismiss human-centeredness as soft. Designing for dignity, context, and coordination is a hard technical problem, one that lives in the architecture, the data, and the feedback loops.

Third, zoom out. What you build today will live inside ecosystems—organizational, social, regulatory. Fei-Fei’s framing is a reminder that it’s our job not just to optimize outputs but to shape systems that hold up over time.

Further Viewing/Listening



Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Enhance AI security with Azure Prompt Shields and Azure AI Content Safety

1 Share

A strong defense against prompt injection attacks

The AI security landscape is constantly changing, with prompt injection attacks emerging as one of the most significant threats to generative AI app builders today. This occurs when an adversary manipulates an LLM’s input to change its behavior or access unauthorized information. According to the Open Worldwide Application Security Project (OWASP), prompt injection is the top threat facing LLMs today1. Help defend your AI systems against this emerging threat with Azure AI Content Safety, featuring Prompt Shields—a unified API that analyzes inputs to your LLM-based solution to guard against direct and indirect threats. These exploits can include circumventing existing safety measures, exfiltrating sensitive data, or getting AI systems to take unintended actions within your environment.

Prompt injection attacks

In a prompt injection attack, malicious actors input deceptive prompts to provoke unintended or harmful responses from AI models. These attacks can be classified into two main categories—direct and indirect prompt injection attacks.

  • Direct prompt injection attacks, including jailbreak attempts, occur when an end user inputs a malicious prompt designed to bypass security layers and extract sensitive information. For instance, an attacker might prompt an AI model to divulge confidential data, such as social security numbers or private emails.
  • Indirect, or cross-prompt injection attacks (XPIA), involve embedding malicious prompts within seemingly innocuous external content, such as documents or emails. When an AI model processes this content, it inadvertently executes the embedded instructions, potentially compromising the system.

Prompt Shields seamlessly integrates with Azure OpenAI content filters and is available in Azure AI Content Safety. It defends against many kinds of prompt injection attacks, and new defenses are regularly added as new attack types are uncovered. By leveraging advanced machine learning algorithms and natural language processing, Prompt Shields effectively identifies and mitigates potential threats in user prompts and third-party data. This cutting-edge capability will support the security and integrity of your AI applications, helping to safeguard your systems against malicious attempts at manipulation or exploitation. 

Prompt Shields capabilities include:

  • Contextual awareness: Prompt Shields can discern the context in which prompts are issued, providing an additional layer of security by understanding the intent behind user inputs. Contextual awareness also leads to fewer false positives because it’s capable of distinguishing actual attacks from genuine user prompts.
  • Spotlighting: At Microsoft Build 2025, we announced Spotlighting, a powerful new capability that enhances Prompt Shields’ ability to detect and block indirect prompt injection attacks. By distinguishing between trusted and untrusted inputs, this innovation empowers developers to better secure generative AI applications against adversarial prompts embedded in documents, emails, and web content.
  • Real-time response: Prompt Shields operates in real time and is one of the first real-time capabilities to be made generally available. It can swiftly identify and mitigate threats before they can compromise the AI model. This proactive approach minimizes the risk of data breaches and maintains system integrity.

End-to-end approach

  • Risk and safety evaluations: Azure AI Foundry offers risk and safety evaluations to let users evaluate the output of their generative AI application for content risks: hateful and unfair content, sexual content, violent content, self-harm-related content, direct and indirect jailbreak vulnerability, and protected material.
  • Red-teaming agent: Enable automated scans and adversarial probing to identify known risks at scale. Help teams shift left by moving from reactive incident response to proactive safety testing earlier in development. Safety evaluations also support red teaming by generating adversarial datasets that strengthen testing and accelerate issue detection.
  • Robust controls and guardrails: Prompt Shields is just one of Azure AI Foundry’s robust content filters. Azure AI Foundry offers a number of content filters to detect and mitigate risk and harms, prompt injection attacks, ungrounded output, protected material, and more.
  • Defender for Cloud integration: Microsoft Defender now integrates directly into Azure AI Foundry, surfacing AI security posture recommendations and runtime threat protection alerts within the development environment. This integration helps close the gap between security and engineering teams, allowing developers to proactively identify and mitigate AI risks, such as prompt injection attacks detected by Prompt Shields. Alerts are viewable in the Risks and Alerts tab, empowering teams to reduce surface area risk and build more secure AI applications from the start.

Customer use cases

AI Content Safety Prompt Shields offers numerous benefits. In addition to defending against jailbreaks, prompt injections, and document attacks, it can help to ensure that LLMs behave as designed, by blocking prompts that explicitly try to circumvent rules and policies defined by the developer. The following use cases and customer testimonials highlight the impact of these capabilities.

AXA: Ensuring reliability and security

AXA, a global leader in insurance, uses Azure OpenAI to power its Secure GPT solution. By integrating Azure’s content filtering technology and adding its own security layer, AXA prevents prompt injection attacks and helps ensure the reliability of its AI models. Secure GPT is based on Azure OpenAI in Foundry Models, taking advantage of models that have already been fine-tuned using human feedback reinforcement learning. In addition, AXA can also rely on Azure content filtering technology, to which the company added its own security layer to prevent any jailbreaking of the model using Prompt Shields, ensuring an optimal level of reliability. These layers are regularly updated to maintain advanced safeguarding.

Wrtn: Scaling securely with Azure AI Content Safety

Wrtn Technologies, a leading enterprise in Korea, relies on Azure AI Content Safety to maintain compliance and security across its products. At its core, Wrtn’s flagship technology compiles an array of AI use cases and services localized for Korean users to integrate AI into their everyday lives. The platform fuses elements of AI-powered search, chat functionality, and customizable templates, empowering users to interact seamlessly with an “Emotional Companion” AI-infused agent. These AI agents have engaging, lifelike personalities, interacting in conversation with their creators. The vision is a highly interactive personal agent that’s unique and specific to you, your data, and your memories.

Because the product is highly customizable to specific users, the built-in ability to toggle content filters and Prompt Shields is highly advantageous, allowing Wrtn to efficiently customize its security measures for different end users. This lets developers scale products while staying compliant, customizable, and responsive to users across Korea.

“It’s not just about the security and privacy, but also safety. Through Azure, we can easily activate or deactivate content filters. It just has so many features that add to our product performance,” says Dongjae “DJ” Lee, Chief Product Officer.

Integrate Prompt Shields into your AI strategy

For IT decision makers looking to enhance the security of their AI deployments, integrating Azure’s Prompt Shields is a strategic imperative. Fortunately, enabling Prompt Shields is easy.

Azure’s Prompt Shields and built-in AI security features offer an unparalleled level of protection for AI models, helping to ensure that organizations can harness the power of AI without compromising on security. Microsoft is a leader in identifying and mitigating prompt injection attacks, and uses best practices developed with decades of research, policy, product engineering, and learnings from building AI products at scale, so you can achieve your AI transformation with confidence. By integrating these capabilities into your AI strategy, you can help safeguard your systems from prompt injection attacks and help maintain the trust and confidence of your users.

Our commitment to Trustworthy AI

Organizations across industries are using Azure AI Foundry and Microsoft 365 Copilot capabilities to drive growth, increase productivity, and create value-added experiences.

We’re committed to helping organizations use and build AI that is trustworthy, meaning it is secure, private, and safe. Trustworthy AI is only possible when you combine our commitments, such as our Secure Future Initiative and Responsible AI principles, with our product capabilities to unlock AI transformation with confidence. 

Get started with Azure AI Content Safety


1OWASP Top 10 for Large Language Model Applications

The post Enhance AI security with Azure Prompt Shields and Azure AI Content Safety appeared first on Microsoft Azure Blog.

Read the whole story
alvinashcraft
3 hours ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories