Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
162973 stories
·
33 followers

From copy-paste to platform thinking

1 Share

At Octopus Deploy, we maintain a collection of static websites we call microsites. They host public-facing content like our product documentation, our external handbook, and the blog you’re reading this article on. Contributors come from across the organization, including support, engineering, product, marketing, and our people team.

Each microsite is built on the Astro framework and lives in its own GitHub repository. For years, they were built, tested, and deployed using GitHub Actions and Octopus Deploy. The setup worked, but over time, the gap between “it works” and “it’s working well” kept growing, especially when we stepped back to consider what great software delivery looks like today.

We continuously release functionality in Octopus to help teams improve software delivery, and we wanted to bring those improvements to our microsites. Eating your own dog food and all that! We also knew the build had friction, and there were parts of GitHub Actions we wanted to improve. What started as “let’s enable ephemeral environments for the microsites” became a series of adjustments to our deployment pipeline.

They were all for the better, but along the way, we hit paper cuts, had fascinating discussions, and faced decisions that seemed simple on the surface yet quickly made us think differently on topics like where the platform team’s responsibility ends and the application team’s begins, how opinionated a shared process should be, and how much flexibility is too much.

This article anchors a series of posts that go deeper into specific pipeline improvements. Here, I want to set the scene, while the pieces that follow get into the technical detail, the decisions we made, and the rough edges we hit along the way.

Crucially, while I’ll be using our microsites to tell the story, the patterns and considerations apply well beyond static sites, and they surfaced when I worked with customers building and delivering software. If you’ve ever tried to standardize processes across multiple teams, manage shared delivery infrastructure, or define the right lines of ownership, this will likely feel familiar.

The way things were

None of what I’m about to describe was a crisis. The microsites were online, content was being published, builds and releases were functioning, and things were fine on the surface. But fine has a way of masking friction, and friction has a way of compounding in the background until someone decides to act. What started with “let’s enable ephemeral environments on the microsites” led us to look more broadly at how we build and deploy changes, and that surfaced several areas we felt could be improved.

I want to give you a peek behind the curtain at what things looked like before we started. I’m not going to cover every paper cut and rough edge here, as those get their own dedicated articles. But I want you to see that, on their own, none of these things were really that big of a deal. It was only when we stepped back and looked at the delivery cycle holistically that the case for change became clear.

The copy-paste loop

Each microsite lived in its own GitHub repository, and each repository had its own set of GitHub Actions workflows to build, test, and validate changes. We had four core GitHub Actions workflows, which at a high level achieved the following tasks:

  1. Spellcheck (branch)
  2. Markdown linting (branch)
  3. Build, test, package, push the package to Octopus, and trigger the release (branch)
  4. Build, test, package, push the package to Octopus, and trigger the release (main)

Four workflows per repo across five repos meant 20 workflow files in total. Any future microsites would only make it worse, as another repo meant another set of workflow copies to maintain. And this was just for the CI side. The idea behind this was that if a change was made to one of these workflows, it would be abstract enough to be easily copied and pasted into the same workflow file in the other microsite repositories. This pattern will be familiar to many software teams. A build process works well for a particular technology stack, so when the next application comes along using the same stack, an engineer copies the working example as a starting point. It’s pragmatic, it’s fast, and on day one, it makes complete sense.

The problem with copy-paste, even if you’re copying from a template file, is that it’s a snapshot from a point in time, not an ongoing linked relationship. The moment you paste something into a second place, you have two things to maintain instead of one. Over time, changes are made in one repo but not in the others, sometimes intentionally, sometimes not. A node version gets bumped in one workflow and forgotten in another. Someone tweaks a step while testing something and doesn’t propagate it across. Another engineer just isn’t aware of all the different applications using the same workflow, and before long, five repos that were supposed to be identical have drifted apart, and nobody has a clear picture of which one is the source of truth.

The same pattern played out for deployments. Each microsite had its own project and deployment process, defined in Octopus Configuration Language (OCL) and stored in the repository. This is great, as the deployment process is stored in code alongside the application, and benefits from all the great things we’ve defined as an industry by putting configuration in version control. For the microsites, the deployment process was identical across all five sites. Or at least it was supposed to be. Making a change meant updating one project, verifying it worked, and then manually propagating the change to the other four projects. In practice, this meant five pull requests, five reviews, and five merges.

Even though we designed variables and the deployment process to be project-agnostic, the overhead of something as small as changing the format of a Slack notification message is hard to justify and only gets worse at scale. When the overhead of making a change is too high, people stop making them. Over time, that means fewer improvements, and fewer improvements mean everything starts to feel slow and heavy. The pipeline becomes something people work around rather than something they invest in.

The deeper problem is the drift. Copy-paste gives you the illusion of consistency without the guarantee of it. It is very easy for a human to forget to update one of the files or test a change against one of the sites, and then completely forget to update the other sites. From a Platform Engineering and compliance perspective, saying “this is how we build and deploy” with confidence when operating this way is impossible.

Deployment times that test our patience

Deploying a microsite should be straightforward. At the end of the day, it is a static website; it’s not as complex as much of today’s software development and architecture. Build the site, push the files to the destination, clear the cache if needed, and send a notification. In practice, a full deployment of one of our larger microsites, like the blog or docs, would take more than 40 minutes.

The culprit was the way files were transferred to S3. The step responsible for the upload compares each file in the source against the existing files in the destination before deciding whether to copy it. For a site with thousands of images, each generating multiple derivative files during the build, the comparison loop took too long.

Alternatives were explored briefly. SFTP was considered, and questions were raised about whether the upload step could be improved or replaced with a more efficient alternative. While the answer to that was almost certainly “yes,” the deployment was working; it wasn’t a priority to dig into further, and the impression at the time was “that’s how S3 works.” When something is functioning well enough, and the path to a better solution isn’t obvious, a targeted workaround tends to win.

That workaround was a new function in the build - allow me to introduce pnpm thin. Rather than deploying the full site on every run, pnpm thin stripped out any image assets that hadn’t been modified in the last 14 days. Running pnpm thin was added to the build process (well, all 5 copies of the build processes!), and fewer files meant fewer comparisons, which brought deployment times down to around 10 minutes.

An issue with pnpm thin was the assumption it came with. It assumed the destination already contained all the images stripped during the build. For an incremental deployment to a long-running production environment, that assumption was fine. For a fresh deployment to a brand new environment, like an ephemeral environment spun up for a pull request, it wasn’t. Most of the site’s images would simply be missing.

It’s a good example of how workarounds can quietly constrain future decisions. The pnpm thin function solved the immediate problem, but it made a full, clean deployment effectively impossible without first understanding why half the images weren’t showing up.

The fading static staging environment

Each microsite had two static environments that Octopus deployed to - staging and production. Early on, staging was a genuine part of the process, proposed changes were deployed to the staging environment, and people relied on it. But over time, use faded, and the reasons why are worth understanding because they’re not unique to our microsites.

Two things eroded it. The first was contention. When someone needed to do more thorough testing, they’d ask others not to overwrite the staging environment while they worked. I’m sure you’ve seen similar messages bounce around your Slack or Teams channels:

Please don’t use the staging environment today, I’m testing out some changes I’ve been working on. Thanks heaps!

It seems like a reasonable request, but realistically, it meant everyone else had to find another way to check their changes. Local previews filled that gap, and once people found a faster workaround that didn’t require coordinating with anyone, they stuck with it. Interestingly, relying on running the sites locally introduces other pain points, which I’ll touch on below when discussing how to preview changes.

This also meant that, as someone who wanted to deploy your change to the staging environment, you felt like you had to ask around first to see if you were about to interrupt someone else. Messages like “Anyone in the blog staging environment at the moment?” weren’t unusual to see flying around Slack, though admittedly, those were more pleasant than seeing the “Who just overwrote my deployment in the blog staging environment!?” messages! Add in a globally distributed team and the friction that is time zones to the equation, and you’re in for a world of hurt.

The second was the feedback loop. Build times already stretched to around 10 minutes. A deployment to staging added more to that. When you can verify something locally in a few seconds, waiting 20 or more minutes for a staging deployment is hard to justify, especially for a content change where the diff is right there in the GitHub UI.

Delayed feedback loops are the conditions in which workarounds grow. The longer you wait for a build, a deployment, or a shared environment to free up, the more attractive any shortcut becomes. It’s a rational response to friction, and once the shortcut becomes the default, the original agreed and documented process falls out of the team’s workflow.

By the time we looked at this properly, most contributors were testing locally if they checked at all, and reviewers were reading diffs in GitHub’s portal rather than seeing changes live. The first time many changes were seen in a real hosted environment was in production.

The effort to preview changes

The staging situation pointed to a broader gap. Unless the static staging environment was used, there was no easy way for a contributor to share a live, hosted version of their change with someone else. No way for a reviewer to click a link and see the proposed change running in a real environment. To do it in staging meant:

  1. Knowing you were all clear to use the staging environment without annoying someone else who may be testing/reviewing their change
  2. Knowing which Octopus Deploy instance and project to go to
  3. Having access to the target project in Octopus Deploy
  4. Knowing how to perform a deployment to the staging environment
  5. Knowing the URL for the staging environment to share with others

Technical contributors would often just run the site locally on their machine. Clone the repo, install dependencies, run pnpm dev, and open localhost. It’s not onerous if you know what you’re doing. But many people contributing to the microsites aren’t developers. For them, the local preview instructions in the readme present a barrier, not a solution.

For reviewers, the situation isn’t much better. To see a proposed change live, you’d need to check out the branch, follow the same local setup instructions, and run the site yourself. In practice, most reviewers looked at the diff in GitHub and called it done. That works for typo fixes or, honestly, even new documentation pages or blog posts. However, it’s less reassuring for layout changes, new components, or anything where seeing it rendered matters.

What we wanted was simple. Open a pull request, get a URL, and see your change in a real hosted environment. Ideally, one that mirrored the production infrastructure, allowing us to test changes in a production-like environment. While static website hosting is straightforward, there are nuances with different hosting options. We wanted to provide this to contributors without them needing to know anything about the underlying infrastructure, without needing access to Octopus, without needing to run anything locally. That experience didn’t exist yet, and building it was what set this whole project in motion, which admittedly was a bit of eat your own dog food when we introduced ephemeral environments in Octopus.

Organically growing infrastructure

The microsites were provisioned and were running in AWS, using S3 as the endpoint and CloudFront distributions in front of them. It worked, but they were initially provisioned and configured manually and had been left largely untouched since.

Without Infrastructure as Code (IaC), there was no reliable way to know whether the configuration of each bucket and distribution was consistent across all sites. If a change was required, it had to be made manually, one resource at a time. If something ever needed to be recreated from scratch, like deploying to a new AWS region or provisioning a fresh environment for a pull request, there was no automated way to achieve it.

We knew we wanted to move to Azure, where most of Octopus’s infrastructure lives, and that the migration was an opportunity to put the infrastructure management in a better shape. Automating the provisioning and deprovisioning of infrastructure was something we knew we had to get right, particularly if we were going to support ephemeral environments and speed up deployments.

What we changed

The pain points above are just the surface. Each one has a dedicated article that goes into the details, the decisions, the trade-offs, and the edges we hit along the way. If you’ve felt any of this in your own work, the deeper articles are worth your time.

New articles in this series are being published over the coming weeks. As each one goes live, the topics below will link through to it.

The copy-paste problem across both GitHub Actions and Octopus was solved by leaning into templating. Twenty workflow files spread across five repositories were consolidated into a set of reusable GitHub workflows. In Octopus, five individual deployment processes were consolidated into a single versioned process template that all five microsites now consume, along with templated runbooks for provisioning and deprovisioning infrastructure. Both solved the copy-paste problem, but if you read the reusable GitHub workflows article, you might come away thinking Octopus has set a pretty high bar for what templating should look like.

Slow deployments and the pnpm thin workaround that papered over them were addressed during the migration to Azure. We evaluated several hosting options before landing on Blob Storage, swapped out the file transfer tooling for AzCopy, and in doing so, made pnpm thin unnecessary. The full story is in finding the right hosting for a static site.

Ephemeral environments were the catalyst for this project, and while Octopus does much of the heavy lifting, it requires a mature foundation underneath. Every pull request now gets its own hosted environment, with a preview URL posted as a comment on the PR. Along the way, we hit some interesting edges, including managing Terraform state for short-lived environments and a subtle race condition with short-lived PRs that took a pragmatic workaround to resolve.

Two of the more interesting conversations in the project weren’t really about tooling at all. Where ownership lines fall between a platform team and the teams consuming what they build is something many practitioners wrestle with, and it came up repeatedly here. And rather than installing tools at deployment runtime - something I’ve seen a lot of teams do in the field - we took an approach of keeping workers lean with execution containers, which made deployments faster and more predictable.

Finally, not every pull request needs a full environment spun up. Based on some conversations I had at conferences and internal feedback, we made the whole thing optional using a simple GitHub PR label, without compromising branch protection rules.

Wrapping up

None of the pain points in this article were dramatic on their own. A few copied workflow files, a slow deployment, a staging environment people stopped using. But together, they made it harder to improve, harder to trust what we were shipping, and harder for contributors to see their changes before they went live.

It’s also worth thinking about what happens as these patterns scale. Five microsites is a small footprint, but the same copy-paste approach across dozens or hundreds of applications makes it close to impossible to answer simple questions like “are all our deployments following the same process?” or “has everyone picked up the latest security change?” That’s where platform thinking earns its keep. Shared templates, consistent infrastructure, and guardrails built into the process mean compliance and risk are handled by default, rather than relying on every team to remember to do the right thing based on a knowledge base article people rarely revisit.

For us, the realization was that the tooling we rely on had moved on, and our pipelines hadn’t kept up. Taking the time to step back, look at the whole delivery cycle, and bring it in line with what modern software delivery looks like today has been worth it. If some of this sounded familiar, it might be worth taking that same step back with your own pipelines.

Happy deployments!

Read the whole story
alvinashcraft
17 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Vector stores in Agent Framework for Python: familiar foundations, streamlined for agents

1 Share

Vector stores have been part of the Python experience in Semantic Kernel for some time. With Microsoft Agent Framework, we have ported that work forward rather than starting over.

This post focuses on Python. For the .NET equivalent, see Use vector stores in .NET AI apps.

The foundation remains familiar: typed record definitions, stores and collections, batch storage and retrieval, embedding generation, vector search, and a growing set of integrations. If you have used Semantic Kernel’s vector-store APIs, you will recognize the overall model.

At the same time, this is more than a straight port. We streamlined model registration and the batch APIs, updated filters and embedding options, and made vector stores fit naturally into how agents receive and use context.

The most important additions are the context providers. They expose your data to agents through tools, or use a vector store to keep conversation history searchable without continually expanding the model context.

The vector-store APIs remain experimental, and the integration packages are alpha or beta. Use each implementation’s documentation for setup and its supported capabilities.

Define the data your application already owns

Agent Framework starts with your data model rather than a database SDK model. Mark the key, data, and vector fields with VectorStoreField, and @vectorstoremodel derives the collection definition:

from dataclasses import dataclass
from typing import Annotated

from agent_framework import (
    Filter,
    InMemoryCollection,
    VectorStoreField,
    vectorstoremodel,
)
from agent_framework.openai import OpenAIEmbeddingClient


@vectorstoremodel(collection_name="documents")
@dataclass
class Document:
    id: Annotated[str, VectorStoreField("key")]
    category: Annotated[str, VectorStoreField("data", is_indexed=True)]
    text: Annotated[str, VectorStoreField("data")]
    embedding: Annotated[
        list[float] | str | None,
        VectorStoreField(
            "vector",
            dimensions=1536,
            distance_function="cosine_similarity",
        ),
    ] = None


async def index_and_search() -> None:
    documents: InMemoryCollection[str, Document] = InMemoryCollection(
        Document, embedding_generator=OpenAIEmbeddingClient(model="text-embedding-3-small"),
    )
    await documents.ensure_collection_exists()

    await documents.upsert([
        Document(
            id="release-checklist",
            category="operations",
            text="Verify rollback, monitoring, and owner sign-off.",
            embedding="Verify rollback, monitoring, and owner sign-off.",
        )
    ])

    results = await documents.search(
        "What should I check before a release?",
        filter=Filter("category", "eq", "operations"),
        top=3,
    )

    async for result in results:
        print(result["record"].text, result["score"])

Set OPENAI_API_KEY before running the code. The records are stored in-memory (so this will not survive a restart), while OpenAI generates embeddings during both upsert and search. You can swap out the InMemoryCollection with any of the other implementations without changing anything else, see the list below for the ones we currently support.

Dataclasses are only one option. The same model layer supports Pydantic models, msgspec structs, plain classes, dictionaries, and externally owned types through custom codecs.

Turn vector search into an agent tool

As in Semantic Kernel, you can expose search directly as a first-class agent tool. create_vector_search_tool wraps a searchable collection and lets you control its name, description, result shape, filters, and approval behavior.

Param now makes custom search tools easier to build. Place a Param inside a Filter or FilterGroup, and the helper exposes it as a typed argument the model can supply. Given a collection configured to embed text queries, the tool definition stays small:

from typing import Literal

from agent_framework import Filter, Param, create_vector_search_tool

category = Param(
    "category",
    Literal["engineering", "operations", "product"],
    required=False,
    description="Only return documents from this category.",
)

search_documents = create_vector_search_tool(
    collection,
    name="search_documents",
    description="Search the document collection.",
    filter=Filter("category", "eq", category),
    result_mapper=lambda result: result["record"].text,
)

The resulting tool always accepts a natural-language query and optionally accepts category. The Literal values become an enum in the tool schema, giving the agent a clear set of valid choices. If the model omits the optional parameter, that filter is omitted. More complex tools can compose several parameters through FilterGroup.

The key addition: make collections available through context providers

When an agent needs more than one focused search tool, VectorCollectionContextProvider provides the higher-level experience.

VectorCollectionContextProvider can add upsert, get, delete, and search tools to an agent, together with instructions for using them. You can expose a read-only knowledge collection, let an agent manage records with approvals, scope the available data, or replace the default search with specialized tools.

Embedding setup stays specific to the operation: ingestion can use document embedding options while search uses query embedding options. Generated tools forward those settings to the same collection.

A context provider for searchable conversation history

Conversation histories keep growing, increasing the context sent to the model on every turn. VectorStoreHistoryProvider persists the full conversation in a searchable vector collection while loading only a compacted working set into the model context.

When include_search_tool=True, the agent also receives a tool for searching the complete scoped transcript. The active context can therefore remain bounded without losing access to an older decision, requirement, or user preference: the agent retrieves the relevant part of the conversation when it needs it instead of carrying every prior message on every invocation.

Embeddings and search without hard-coding a provider

Applications can supply precomputed vectors or have Agent Framework call an embedding service during upsert and search. They can generate every vector field or only selected fields, and pass different provider options for document ingestion and query generation. Vector dimensions stay with the model definition, so Agent Framework can validate generated and supplied vectors before they reach the backing store.

Search uses the same collection API across integrations, with portable data filters and keyword-hybrid search where supported. Each implementation documents its supported indexes, filters, metrics, and search modes.

Available integrations

The current Python integrations are:

  • Azure AI Search
  • Azure Cosmos DB for NoSQL
  • Azure DocumentDB
  • MongoDB
  • Oracle AI Database 26ai
  • Redis
  • SQL Server and Azure SQL
  • PostgreSQL with pgvector
  • DuckDB
  • Qdrant
  • The core in-memory store for tests and local development

The same model and core operations work across integrations, while each database keeps its own features and constraints.

Get started

Start with the runnable vector-store samples for models, search, and agent tools. The vector-store-backed conversation sample demonstrates bounded working context with search over the full conversation.

Install the integration package for the store you want to use, and use its README for setup and supported capabilities. We would especially like feedback on the context-provider experience, searchable history, and which integrations you need next. Open an issue in the Microsoft Agent Framework repository and tell us what you are building.

 

The post Vector stores in Agent Framework for Python: familiar foundations, streamlined for agents appeared first on Microsoft Agent Framework.

Read the whole story
alvinashcraft
44 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Triaging Dependabot With a Copilot Canvas

1 Share

Dependabot is one of those tools that I really appreciate, right up until I open a repository and see a pile of pull requests waiting for me. And this is particularly noticeable when working on a monorepo, such as the Aspire Community Toolkit.

Individually, each PR is pretty simple: a dependency moves from one version to another, the lockfile changes, CI runs, and if all is well you merge it. The problem is that they don’t arrive individually. They arrive as a queue, and that queue has relationships that GitHub’s PR list doesn’t really make obvious. Which PRs fix security alerts? Which ones touch the same lockfile and will conflict with each other once I merge the first one? Are there alerts with no PR fixing them at all?

I found myself doing that analysis in my head (or across a bunch of browser tabs) whenever I sat down to tidy up dependencies. So I built a canvas for the Copilot app to help: the Dependabot Triage Canvas.

Dependabot Triage Canvas

Looking at the queue differently

Canvases are apps that you host in a session within the Copilot app that let you work with something other than chat - a kanban board, an external system dashboard, or in this case, a purpose-built view of your Dependabot PRs.

Rather than listing PRs in the order they were opened, the canvas groups them. By default the grouping is by dependency, but you can also group by directory, ecosystem or status. Each PR shows the dependency update (including whether it’s a patch, minor or major version bump), the state of its checks (is CI passing), whether it’s mergeable, and its review state.

To make this concrete, I opened the canvas against this blog’s repository while writing this post. It found 7 open Dependabot PRs, all in the npm ecosystem, and 6 of them are linked to security alerts. They include a mixture of updates:

  • brace-expansion 1.1.11 → 1.1.21 (patch)
  • js-yaml 4.3.1 → 4.3.2 (patch)
  • sharp 0.35.3 → 0.35.4 (patch)
  • browserslist 4.21.1 → 4.28.9 (minor)
  • @babel/core 7.18.6 → 7.29.7 (minor)
  • A PR with several major @graphql-codegen updates

It also told me something less pleasant: all 7 PRs have failing checks. Clearly this repository needs a bit of attention! 😅 (The failing checks are to do with deploying PR branches to Azure Static Web Apps, which has been broken for a while, so no biggy 😜)

That’s the sort of thing I wanted the canvas to surface quickly. I don’t need to open each PR to discover that there’s a broader problem worth investigating before I start merging.

Connecting PRs and alerts

The thing that pushed me to build this was the disconnect between Dependabot pull requests and Dependabot security alerts.

The canvas loads open security alerts and matches them against the open PRs. When a PR fixes one or more alerts, you can see that relationship from the PR itself. The alerts view also highlights the alerts that don’t have an open PR addressing them.

For this repository, there are 62 open alerts and 53 of them are not addressed by an open Dependabot PR. Many of those are in the /api project’s package-lock.json, including alerts for packages such as undici, ws, form-data and lodash.

That’s a very different picture to “there are 7 PRs to review”. The PR queue is only part of the work; there’s also a bunch of security work that needs a different approach, such as updating a parent dependency or retiring an old part of the project.

Spotting overlaps

Another thing that the canvas looks for is overlap between PRs.

This is something that we hit a lot in the Community Toolkit. Since it’s a monorepo design, we have a lot of package.json files spread across a lot of folders. If a security alert comes out for one of the packages, we could end up with 50+ PRs updating the same package simultaneously. The canvas helps identify these overlaps, and allows us to combine them into one single PR. Doing this means that we have less merges happening, and fewer CI jobs that need to run when the merge happens to the main branch.

Taking action without giving up control

The canvas isn’t only a read-only dashboard. It supports a number of actions that I’d otherwise perform on GitHub:

  • Approve a PR
  • Approve and merge a PR, optionally with auto-merge
  • Close a PR
  • Post supported @dependabot commands like rebase, recreate and ignore

These can be done one at a time or in bulk. But I was very deliberate about where the guardrails sit.

Approving, merging, closing and commenting only happen after I confirm the action in the canvas. The agent-facing actions can read, filter and select PRs, and they can record results like review verdicts, but they can’t mutate the PRs. Merges also use --match-head-commit, so if a PR has changed since I looked at it, it won’t be merged.

In other words, Copilot can help me understand the queue and get the right PRs selected, but I’m still the one pulling the trigger.

Bringing agents into the workflow

Because the canvas is running inside the Copilot app, it can also hand work off to agents.

Review with agent sends one or more PRs to an agent for review. The agent records a verdict - safe, caution or blocked - along with a summary, and that verdict appears back in the canvas. If the PR’s head changes later, the review is marked as stale so I’m not relying on an old assessment. This can be particularly helpful if there’s a failing CI job, as the agent can help dig through the CI logs to see if the failure is transient or related to the dependency that is being bumped.

The other workflow is what I mentioned above, Combine into one PR. I can select a set of Dependabot PRs and ask an agent to fold them together. The agent works in a separate git worktree, so my checkout isn’t disturbed. It cherry-picks the PRs, regenerates conflicting lockfiles, pushes a new branch and opens a combined PR that lists what it supersedes.

Reviews and combines run in background agents by default, so multiple tasks can be in progress while my main session stays available. The Agents tab shows what’s running, which PRs each agent is working on, and its latest response.

This is where a canvas becomes more interesting than a dashboard. It gives the agent structured context - the PRs, their diffs, the related alerts, and the overlaps - and gives me a structured way to see what the agent did.

Trying it out

The canvas is available as a plugin in the Copilot app:

  1. Open Customise in the Copilot app and navigate to plugins.
  2. Search for “Dependabot Triage Canvas” under Awesome Copilot and enable it.
  3. Ask Copilot to “open the Dependabot triage canvas”.

By default it’ll use the current repository, but you can target a different one by passing { "repo": "owner/name" } when opening the canvas.

You’ll need the GitHub CLI installed and authenticated, as all of the GitHub calls go through gh. Read access is enough for triage, but you’ll need write access to approve, merge, close or comment, and security alert access to see the Dependabot alerts.

Wrapping up

The biggest change for me isn’t any individual button in the canvas. It’s that I can see the shape of the Dependabot work before I start doing it.

Opening it on my blog repo immediately showed me three things: every Dependabot PR is failing checks, most of the open security alerts don’t have a PR, and most of the PRs are going to collide in the same lockfile. That’s a much better starting point than working through PRs one by one.

If you’re dealing with Dependabot queues of your own, give it a try and let me know what you think. The source is on GitHub, and I’d love to hear what other triage workflows would be useful.

Read the whole story
alvinashcraft
4 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Are We Losing Our Ability to Think in the Age of AI?

1 Share
Photo by Glen Carrie on Unsplash
AI is making us more productive than ever. But are we becoming better thinkers, or simply getting better at finding answers?

We are living in an extraordinary time. For the first time, we have technology that can write code, analyze information, explain complex concepts, solve mathematical problems, and help us make decisions within seconds. Tasks that once required hours of research, experimentation, and problem-solving can now be completed with a few well-written prompts.

As someone who has spent years in software engineering and now works extensively with AI-assisted development, I find this transformation fascinating. I use AI regularly, and I see firsthand how much it can improve productivity. It has fundamentally changed the way I approach certain problems, explore ideas, and build software.

But the more I use these tools, the more I find myself thinking about something that doesn’t get enough attention.

What happens to our ability to think critically when getting an answer becomes easier than understanding the problem?

I am not particularly concerned that AI will suddenly make people incapable of thinking. Human intelligence doesn’t work that way. What concerns me is something more gradual. We may become so accustomed to receiving immediate, polished answers that we stop exercising some of the mental processes that helped us develop our knowledge and judgment in the first place.

And unlike the dramatic doom’s day predictions that we hear about with AI, this change could happen quietly, through everyday habits.

The Thinking That Happens Before the Answer

Let’s think about how we used to approach problem solving before AI.

When I was learning to code, finding a solution often meant reading documentation, going through forums, experimenting with different implementations in trial and error, debugging errors, and spending hours trying to understand why something wasn’t working.

It wasn’t always efficient. There were plenty of frustrating moments, and I am certainly not suggesting that we should return to a world where developers spend half their day searching Stack Overflow.

But something valuable happened during that process.

Exploring a problem and getting to the solution took time. And when we reached the answer, there was an Ah! Ah! moment that used to give us a dopamine hit! That the hours of hard work paid off.

We also developed an understanding of why certain solutions worked and others didn’t. We began recognizing patterns. We learned how to troubleshoot unfamiliar problems because we had experienced the process of investigating them.

More importantly, we developed judgment.

Judgment is difficult to acquire simply by reading an answer. It develops through experience, comparison, mistakes, reflection, and making decisions when the correct approach isn’t immediately obvious.

Today, AI can eliminate much of that process. You describe a problem and receive a solution. If the solution doesn’t work, you paste the error back into the chat and receive another one. The cycle continues until something works.

The task gets completed, often much faster than before. But there is an important distinction between completing a task and developing the ability to complete similar tasks independently.

And I think that distinction is going to matter alot in the years ahead.

When Convenience Starts Replacing Curiosity

Photo by Aaron Lefler on Unsplash

Technology has always made certain mental tasks easier.

Calculators reduced the need for manual arithmetic.

GPS changed how we navigate.

Search engines transformed how we find information.

Each of these technologies allowed us to spend less effort on certain activities and redirect our attention elsewhere.

AI is different in one important way. It doesn’t simply retrieve information or perform predefined calculations. It can produce arguments, interpret evidence, generate solutions, recommend decisions, and explain its reasoning.

In other words, it can perform activities that look remarkably similar to the processes we associate with thinking.

This makes it incredibly useful, but it also creates an interesting problem.

When AI provides a clear, convincing explanation, we may feel as though we understand something simply because the explanation makes sense while we are simply reading it.

Understanding an explanation and being able to independently reason through a problem are not necessarily the same thing.

Imagine reading an AI-generated explanation of a complex architectural decision. It discusses scalability, performance, maintainability, and trade-offs. Everything sounds reasonable, so you accept the recommendation and move forward.

But what happens when the situation changes? What happens when an unexpected constraint appears, or the proposed solution fails in production?

Can you evaluate what went wrong? Can you challenge the original assumptions? Can you develop an alternative without relying entirely on another generated response?

These are the questions that reveal whether we have gained understanding or simply borrowed it.

What Research Suggests About AI and Critical Thinking

In a 2025 study titled The Impact of Generative AI on Critical Thinking, researchers from Microsoft Research and Carnegie Mellon University examined how knowledge workers perceived their use of critical thinking while working with generative AI.

The study involved 319 participants who provided 936 examples of AI-assisted work.

One of the findings was that participants who reported greater confidence in AI tended to report less critical-thinking engagement. Conversely, participants with greater confidence in their own abilities tended to report more critical-thinking engagement.

The researchers also observed a shift in the nature of cognitive work. Rather than spending as much effort generating and analyzing information independently, participants increasingly described their roles as verifying, integrating, and overseeing AI-generated output.

This finding deserves a nuanced interpretation. The study examined self-reported experiences and does not establish that AI directly causes a decline in critical-thinking ability. And reviewing, validating, and integrating information are themselves important forms of critical thinking.

Still, the findings raise a question worth considering.

If our role increasingly becomes evaluating AI-generated work, how do we ensure we develop the expertise necessary to evaluate it well?

Someone who has spent years building software systems may recognize subtle architectural problems in generated code. But what about a developer who has always relied on AI to make those architectural decisions?

If that developer has rarely worked through the trade-offs independently, what foundation will they use to challenge AI’s recommendations?

This is particularly important for people who are still developing expertise. Experienced professionals often have years of knowledge and judgment to draw upon. Beginners are still building that foundation.

AI can accelerate their learning, but only if they use it in ways that encourage understanding rather than simply completing assignments or tasks.

Why Critical Thinking Will Matter Even More for Software Engineers

Photo by Marija Zaric on Unsplash

Software engineering is already experiencing this transformation at an extraordinary pace.

The amount of code a developer can produce in a short period has changed significantly.

But aworking implementation doesn’t automatically mean we have a good design. Passing tests doesn’t guarantee that we have considered all the relevant scenarios. An application that performs well during a demonstration may still have security vulnerabilities, accessibility problems, scalability limitations, or long-term maintenance challenges.

Imagine a developer using AI to build an authentication system. The generated code works, the login flow appears functional, and the basic tests pass. From the outside, everything looks complete.

But has the developer considered session expiration, authorization boundaries, credential storage, token handling, account recovery, and potential attack scenarios? Does the implementation follow the organization’s security requirements? Are the assumptions behind the solution appropriate for the system being built?

The quality of the outcome depends on more than the quality of the generated code. It also depends on the developer’s ability to ask the questions that the AI may not have considered.

This is why I often think about how engineering skills are moving upstream. As code generation becomes easier, the ability to define problems, establish constraints, recognize trade-offs, evaluate architectures, and make informed decisions becomes increasingly important.

The engineer who simply accepts generated solutions may become faster at producing code without necessarily becoming better at engineering systems.

The engineer who uses AI to explore approaches, challenge assumptions, validate decisions, and deepen understanding has the opportunity to become both faster and more capable.

We Need to Preserve the Ability to Struggle With Problems

Photo by Glen Carrie on Unsplash

One of the things I worry about most is our growing discomfort with not having an immediate answer.

Remember the dopamine hit we talked about.

The hours of trying to solve the problem, the hardwork, frustrations, highs and lows invovled in the process. And when you finally are done solving the problem, or achieving your task, the crazy dopamine hit! Well we won’t have that anymore.

AI has made it possible to resolve uncertainty almost instantly. When we encounter something unfamiliar, we can ask a question and receive an explanation in seconds.

That is a wonderful capability. But uncertainty also serves a purpose in learning.

When we don’t immediately know the answer, we begin exploring. We form hypotheses, compare possibilities, test ideas, and revise our understanding. That process is often where deeper learning takes place.

I am not suggesting that frustration is inherently valuable or that spending hours on a simple problem is somehow a superior way of working. There is no intellectual virtue in performing unnecessary repetitive work.

But there is a difference between removing unnecessary friction and removing all opportunities for independent reasoning.

When we are learning a new concept, for example, asking AI for hints rather than a complete solution may be more beneficial. When debugging an unfamiliar issue, attempting to understand the root cause before requesting a fix can help build knowledge that becomes useful later.

And when evaluating a significant decision, spending time developing our own initial perspective before consulting AI may help us avoid becoming overly influenced by its first recommendation.

This requires intentional effort because AI offers something incredibly tempting: a way around the discomfort of not knowing.

But sometimes that discomfort is exactly what motivates us to think more deeply.

How I Think We Should Use AI to Strengthen Critical Thinking

The answer isn’t to stop using AI or artificially make every task more difficult. Instead, I believe we need to become more deliberate about the moments when we allow AI to take over and the moments when we remain actively involved.

One habit I find particularly valuable is developing an initial opinion before asking AI for its recommendation. If I am evaluating an architectural decision, I want to understand the problem, identify the major constraints, and consider potential approaches before asking an AI assistant to analyze the options.

This gives me something to compare its response against. I am not approaching the answer as a blank slate, instead I am more likely to notice when its assumptions differ from mine.

Another important habit is asking AI to challenge ideas rather than simply support them.

Instead of asking whether an approach is good, I can ask what might be wrong with it, what assumptions haven’t been validated, and what circumstances would cause it to fail.

That turns AI into something closer to an intellectual sparring partner.

I also think we should develop a stronger habit of distinguishing between facts, assumptions, and recommendations. AI-generated responses often combine all three into a polished narrative, making them appear equally credible.

But a fact supported by evidence is different from an assumption about how something might behave. And both are different from a recommendation based on a particular interpretation of the situation.

Being able to separate these elements is fundamental to good decision-making.

For consequential decisions, independent verification remains essential. We should examine original sources, review documentation, run tests, validate performance claims, and consult relevant expertise when necessary. Asking AI whether its own answer is correct is not a substitute for verification.

Perhaps one of the simplest practices is to occasionally step away from AI and explain what we have learned in our own words.

If AI helped us understand a new technical concept, can we explain it without reopening the conversation? If it recommended an architectural decision, can we articulate the trade-offs and defend the choice? If it generated code, do we understand how it behaves and why it is appropriate?

If the answer is no, we may need to spend more time developing our understanding.

These practices aren’t about proving that we can work without AI. They are about ensuring that AI continues to expand our capabilities rather than becoming a substitute for them.

What Happens to the Next Generation of Thinkers?

Photo by Robert Collins on Unsplash

New developers are entering a different world.

They may have access to advanced AI coding assistants from their very first programming assignment. They can generate functioning applications before fully understanding the underlying languages, frameworks, or architectural patterns.

That creates an extraordinary educational opportunity. Someone who previously struggled to begin learning software development can now receive personalized explanations, interactive examples, and immediate feedback.

But it also creates a challenge.

If beginners consistently use AI to bypass the reasoning involved in solving problems, they may complete more work while developing less independent expertise than their apparent productivity suggests.

This isn’t an argument against teaching students to use AI. Quite the opposite.

AI literacy should become part of modern education and professional development.

But we need to teach more than prompt engineering.

We need to teach people how to evaluate generated answers, recognize uncertainty, challenge assumptions, and determine when additional evidence is necessary.

We also need learning experiences that require students to demonstrate understanding rather than merely produce a completed artifact.

A student who can generate a working application has accomplished something. A student who can explain its architecture, diagnose its failures, identify security risks, and adapt it to new requirements has demonstrated a different level of mastery.

In an AI-enabled world, that distinction becomes increasingly important.

The Future Belongs to People Who Know What to Question

Photo by Zhimai Zhang on Unsplash

We often talk about AI as a technology that is changing the skills people need to succeed.

I agree with that, but I don’t think the transformation is simply about learning new tools or becoming better at writing prompts.

The deeper change is in where human value is created.

When information becomes easy to retrieve and content becomes easy to generate, knowing what to do with that information becomes more important.

When code becomes easier to produce, understanding which code should be written and why becomes more valuable.

When recommendations are available instantly, the ability to evaluate their consequences becomes essential.

And when nearly everyone has access to powerful AI systems, the differentiator may not be who can generate the most answers. It may be who can ask the most meaningful questions, identify the weakest assumptions, and make the best decisions.

I don’t believe AI will inevitably weaken human intelligence. I think it gives us a choice about how we use our own intelligence.

For me, the goal has never been to do everything without AI. I want to use the technology to its fullest potential. I want to automate repetitive work, accelerate development, explore ideas faster, and spend more time on problems that deserve human attention.

But I also want to retain the curiosity to question an answer, the confidence to challenge a recommendation, and the ability to reason independently when necessary.

AI can help us ask better questions, develop deeper understanding, and become better thinkers; if we choose to use it that way.

Research reference: Hao-Ping Lee et al., “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers,” ACM CHI 2025. https://doi.org/10.1145/3706598.3713778

Read the whole story
alvinashcraft
4 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Linus Torvalds let Claude Code fix the Linux page cache. Then reviewers found a race.

1 Share
Circuit board

Linux creator and longtime lead maintainer Linus Torvalds turned to Anthropic’s Claude Code to tackle a kernel performance problem that developers have struggled with for more than two years. On Tuesday, he submitted a 10-patch series that delivered a 23-fold throughput improvement in a synthetic benchmark running on his Ryzen Threadripper.

Torvalds had spent 18 months running a broken version of his own fix for the page-cache reference-counting bottleneck, while other developers had encountered problems with their approaches. The new patches eliminate folio reference-count operations from a common buffered-read path, although Torvalds warned that the benchmark gains would be considerably smaller in real-world workloads.

Reviewers have since identified a race condition and raised concerns about memory reclamation. Torvalds has handed further development to Kiryl Shutsemau, a Linux kernel developer who previously worked on the same bottleneck.

Torvalds had spent 18 months running a broken version of his own fix for the page-cache reference-counting bottleneck, while other developers had encountered problems with their approaches.

Page cache contention problem

On Torvalds’ Core i7-8565U laptop, folio reference-count contention accounted for 99.92% of cross-core load HITMs in eight-thread tests, according to Hardware Busters.

The bottleneck was flagged in 2024 by Matthew Wilcox, an Oracle engineer and longtime Linux kernel memory-management developer, after it surfaced in a customer workload.

Claude Code writes the patches

On Sunday, Phoronix’s Michael Larabel spotted an experimental branch named noref-filemap-read-with-llm in Torvalds’ tree, which rarely contains anything outside the main development branch.

The eight-patch series, based on Linux 7.3-rc6, came with a clear account of what Claude Code did. Torvalds said the assistant wrote the implementation, tests, and initial documentation, and reviewed the code under his direction. He ran the benchmarks himself, despite having previously criticized developers for claiming AI writes 99% of their code.

Earlier approaches copied data without holding a folio reference and checked its validity afterward, leaving races involving reclamation and replacement. The subsequent 10-patch rcu-free-v3 series instead uses a new PG_rcu_free flag and RCU-delayed reclamation to protect the folio through the userspace copy, provided it doesn’t fault.

Earlier approaches copied data without holding a folio reference and checked its validity afterward, leaving races involving reclamation and replacement.

Threadripper benchmark results

The original eight-patch branch nearly tripled throughput on Torvalds’ laptop, pushing eight-thread performance from 10 million to as many as 33 million reads per second while cutting kernel cycles per read from roughly 2,950 to 725. The revised series delivered a much larger gain under heavy contention on his 64-thread Threadripper, where throughput jumped from 13.3 million to 304 million reads per second and cycles per read fell from nearly 20,000 to 735.

The original eight-patch branch nearly tripled throughput on Torvalds’ laptop, pushing eight-thread performance from 10 million to as many as 33 million reads per second while cutting kernel cycles per read from roughly 2,950 to 725.

The performance gains came with trade-offs. In the first implementation, delayed reclamation forced an expedited grace period whenever a CPU queued more than 1,024 pages, according to Hardware Busters. Reads into unmapped user pages also triggered an additional fault before falling back to the existing path, pushing a 16-byte single-thread read from about 4,150 to 6,550 cycles. Whether that regression persists in rcu-free-v3 remains unclear.

Maintainers question the reclamation design

Wilcox questioned the new PG_rcu_free flag and proposed using a page type instead, along with reclaiming inactive-list folios after an RCU grace period.

Torvalds rejected both approaches, arguing that the flag must be set while a folio can still be mapped into userspace and that inactive-list reclamation wouldn’t cover truncate and hole-punch operations. Shutsemau subsequently suggested restricting the flag to 64-bit systems, an approach supported by Arm engineer and Linux memory-management maintainer David Hildenbrand.

Shutsemau also identified a race in filemap_read_rcu_batch(), where folio replacement followed by an extension of i_size could expose data that had never been written. Torvalds recommended moving the file-size read earlier and consolidating the folio checks, which Shutsemau incorporated into a revised patch.

The post Linus Torvalds let Claude Code fix the Linux page cache. Then reviewers found a race. appeared first on The New Stack.

Read the whole story
alvinashcraft
4 hours ago
reply
Pennsylvania, USA
Share this story
Delete

Why Is My Kubernetes Job Still Pending? Tracing Kueue Admission with Headlamp

1 Share

You submit a Kubernetes Job and wait for its Pods to appear. A minute passes. Then another. The Job is still suspended, there are no application logs, and there may not even be a Pod to inspect.

In a cluster managed by Kueue, this can be completely normal. Kueue may still be deciding whether the Job can use the requested resources. The useful question is not only “Why is the Pod pending?” It is first “Has Kueue admitted the Workload?”

The Kueue plugin for Headlamp brings that investigation into the Headlamp UI. Instead of reconstructing the admission path from several commands and large YAML objects, you can begin with the Workload, read its condition message, and follow links through the LocalQueue, ClusterQueue, ResourceFlavor, and Cohort.

This article follows that Headlamp-first path. The command line appears only in a short optional section for reproducing the example.

Before you begin

You need a Kubernetes cluster with Kueue installed, a Headlamp installation, and the 0.1.0-alpha Kueue plugin. Your Kubernetes identity must be allowed to read the Kueue resources used in this article.

The alpha release provides read-only views for Kueue v1beta2 Cohorts, ClusterQueues, LocalQueues, ResourceFlavors, and Workloads. It does not add its own create, edit, stop, resume, or delete workflows. Headlamp still provides its standard YAML and resource actions where your permissions allow them.

First, separate admission from Pod scheduling

Kueue and the Kubernetes scheduler answer different questions.

  • Kueue admission: Should this Workload start now, considering queue policy, quota, ResourceFlavors, admission checks, and other waiting Workloads?

  • Pod scheduling: After the Workload is admitted and the Job resumes, which nodes can run its Pods?

That boundary changes the investigation. If the Workload is not admitted, stay in the Kueue section of Headlamp. If it is admitted but its Pods are still Pending, move to Headlamp's normal Job and Pod views and inspect scheduler events.

A Kubernetes Job becomes a Kueue Workload. Kueue evaluates its LocalQueue, ClusterQueue, ResourceFlavor, and optional Cohort before resuming the Job. Headlamp reads these resources through the Kubernetes API.

Kueue admission architecture

The Headlamp-first troubleshooting path

The quickest investigation starts with the Workload, not with a shell.

A Headlamp troubleshooting flow that starts with Kueue Workloads, checks admission conditions, follows LocalQueue and ClusterQueue links when admission is blocked, and switches to Job and Pod views after admission.

Tracing a pending Job through Headlamp

Step 1: Open the Workloads view

In Headlamp, open Kueue → Workloads. Select the namespace that contains the Job, then find the Workload created for it.

Kueue creates a Workload to represent the resource request and admission state of a Job. In the alpha plugin, owner references are shown as Kind/name text but are not yet clickable. The Workload name normally contains the Job name, so the namespace filter and table search are the simplest ways to locate it.

Before opening the detail page, the list already gives you a useful first scan:

  • LocalQueue

  • priority and priority class

  • active state

  • admitted state

  • completion state

  • a readable current status such as Admitted, Finished, or Inadmissible

  • age

If there is no matching Workload, check three things before going further: the selected namespace, whether the Job was submitted to a LocalQueue, and whether your Kubernetes account can list Workloads. The plugin performs permission checks and displays an access message when the current identity cannot read a resource.

Step 2: Read status, conditions, and events

Open the Workload. Start with its current status and conditions, not with the full YAML.

For admission troubleshooting, the most important condition types are QuotaReserved and Admitted.

  • Admitted=True means Kueue has admitted the Workload. The Job can resume, so any remaining Pending Pods belong to the scheduling branch of the investigation.

  • QuotaReserved=False with a reason such as Pending means Kueue has not reserved quota. The condition message explains what prevented the reservation.

  • An Inadmissible status means the Workload currently cannot fit the available queue configuration. A missing covered resource, unsuitable ResourceFlavor, stopped queue, or admission check can all lead you here.

For example, a condition message may report that Kueue could not assign a ResourceFlavor because the ClusterQueue has insufficient CPU quota. That message is the first real answer to “Why is my Job still pending?” The rest of the Headlamp journey explains the configuration behind it.

Also check the Events section. Conditions describe the current state; events help show what recently changed or what Kueue attempted.

Step 3: Compare the Workload's request with its assignment

Stay on the Workload detail page and inspect the PodSets and resource requests. The plugin shows the requested Pod count, CPU, memory, and other resources that Kueue evaluates together.

This matters because a Job with parallelism: 3 and a one-CPU request per Pod asks Kueue for three CPUs in total. Looking at only one Pod's template would underestimate the Workload.

If quota has already been reserved, the same page also shows the assigned ClusterQueue, ResourceFlavors, and admitted resource usage. If those admission assignments are absent, the Workload has not reached that point yet.

In our example, the Workload requests:

  • three Pods

  • three CPUs in total

  • 600 MiB of memory in total

The next question is where that request entered Kueue.

Step 4: Follow the linked LocalQueue

Use the LocalQueue link on the Workload detail page. A LocalQueue is the namespace-level entry point used by the Job. It connects namespaced Workloads to a cluster-scoped ClusterQueue.

The LocalQueue page shows:

  • its namespace and connected ClusterQueue

  • stop policy and current status

  • pending, reserving, and admitted Workload counts

  • Kueue conditions and Kubernetes events

  • related Workloads in the same namespace

This page answers two practical questions: Did the Job enter the queue you expected, and is it the only Workload waiting there?

If the LocalQueue is inactive or stopped, that may already explain the wait. If it is active, follow the linked ClusterQueue to inspect the capacity behind it.

Step 5: Inspect ClusterQueue capacity and usage

A ClusterQueue represents the cluster-level quota and admission policy available to one or more LocalQueues.

On its Headlamp detail page, compare the Workload request with:

  • covered resources

  • nominal quota for each ResourceFlavor

  • current reservations and usage

  • borrowing and lending limits

  • pending, reserving, and admitted Workload counts

  • queueing strategy and stop policy

  • preemption, fair-sharing, flavor-fungibility, and admission-check settings

  • Kueue conditions and Kubernetes events

For the example in this article, the ClusterQueue provides a nominal quota of two CPUs and two Pod slots. The Workload asks for three of each. The condition message and the ClusterQueue page now tell the same story: the Workload does not fit, so Kueue leaves the Job suspended.

This is an admission-capacity problem. It is not evidence that kube-scheduler failed to find a node.

Step 6: Check ResourceFlavor and Cohort when needed

Quota is not always a single pool. A ResourceFlavor can represent a class of compute through node labels, taints, tolerations, or topology settings. Use the ResourceFlavor link from the ClusterQueue when the condition mentions flavor assignment or when the Workload requires a particular hardware class, such as GPU nodes.

If the ClusterQueue belongs to a Cohort, follow that link too. Cohorts let ClusterQueues share unused quota. The Headlamp page shows parent and child Cohorts, member ClusterQueues, referenced ResourceFlavors, resource groups, borrowing and lending limits, and fair-sharing information.

These pages help answer a more subtle question: Is the Workload blocked because the queue has no quota of its own, because it cannot borrow enough, or because no suitable flavor can be assigned?

What we learned without leaving Headlamp

For the sample pending Job, the UI gives us a complete explanation:

  1. The Workload is not admitted.

  2. Its QuotaReserved condition is false and the message points to insufficient quota.

  3. The Workload requests three CPUs and three Pod slots.

  4. Its LocalQueue is active and points to the expected ClusterQueue.

  5. The ClusterQueue provides only two CPUs and two Pod slots.

  6. Kueue therefore keeps the Job suspended until quota becomes available or the queue configuration changes.

The important part is not that Headlamp invents a new status. It reads the same Kubernetes API objects as other clients, then places the related resources and human-readable values in one connected investigation.

If the Workload is admitted, switch views

Suppose the Workload detail page shows Admitted=True, but the application is still not running. Kueue has completed its part of the decision. Continue in Headlamp's standard Kubernetes views:

  1. Open Workloads → Jobs and inspect the Job state and events.

  2. Open Workloads → Pods and filter by the Job name or namespace.

  3. Select a Pending Pod and read its Events section.

At this stage, look for scheduler concerns such as insufficient node capacity, node affinity, taints and tolerations, topology constraints, persistent-volume binding, or image-pull failures.

This handoff is the most useful troubleshooting rule in the article:

  • Not admitted: investigate Kueue resources in Headlamp.

  • Admitted, but Pods are Pending: investigate Job, Pod, node, and scheduler information in Headlamp.

A repeatable Headlamp checklist

When the next Job does not start, use this short path:

  1. Open Kueue → Workloads and select the correct namespace.

  2. Find the Workload associated with the Job.

  3. Check the readable status, Admitted, and QuotaReserved conditions.

  4. Read the condition message and recent events.

  5. Compare PodSet resource requests with any admission assignments.

  6. Follow the LocalQueue link and confirm its status and ClusterQueue.

  7. Follow the ClusterQueue link and compare quota, reservations, usage, and policy.

  8. Inspect ResourceFlavor or Cohort details when the message points to flavor selection or shared quota.

  9. If admitted, move to Headlamp's Job and Pod views.

What Headlamp replaces during this investigation

The plugin does not replace kubectl or kueuectl for scripts, exact object output, or changes. It replaces the repetitive UI-less navigation needed during an interactive investigation.

Question Headlamp path Command-line equivalent
Is the Workload admitted? Kueue → Workloads → status and conditions kubectl describe workload <workload-name> -n <namespace>
Which LocalQueue did it enter? Workload → linked LocalQueue kubectl get localqueue <localqueue-name> -n <namespace> -o yaml
Which ClusterQueue provides quota? LocalQueue → linked ClusterQueue kubectl get clusterqueue <clusterqueue-name> -o yaml
Which flavor or shared quota matters? ClusterQueue → ResourceFlavor or Cohort kubectl get resourceflavor <flavor-name> -o yaml or kubectl get cohort <cohort-name> -o yaml
What happened after admission? Workloads → Jobs and Pods → events kubectl describe job <job-name> -n <namespace> or kubectl describe pod <pod-name> -n <namespace>

For scripts and GitOps workflows, the command line remains the right tool. For a person asking “why is this waiting?”, the linked Headlamp path is usually faster to read.

Optional: reproduce the pending scenario

The plugin repository includes sample Cohort, ClusterQueue, LocalQueue, ResourceFlavor, and Job manifests for local testing. Apply those examples to a cluster where Kueue is installed, then open the Kueue section in Headlamp.

To reproduce the specific quota mismatch used in this article, configure a ClusterQueue with two CPUs and two Pod slots, then submit a three-Pod Job that requests one CPU per Pod. The Job enters its LocalQueue, Kueue creates a Workload, and the Workload remains pending because the combined request is larger than the available quota.

Keeping this setup separate from the main walkthrough is intentional: once the resources exist, the investigation itself should happen in Headlamp.

Current alpha limitations

The 0.1.0-alpha release is an initial read-only view of the Kueue admission path.

  • The plugin does not add its own create, edit, pause, resume, or delete flows.

  • Workload owner references are displayed as text and are not currently navigable.

  • The UI depends on the Kubernetes permissions of the identity used by Headlamp.

  • Metrics and a visual relationship map are possible follow-up areas rather than features of this release.

These limits are useful to state clearly: the plugin's current purpose is to make admission state and resource relationships easier to inspect.

Developed during LFX Mentorship

The plugin was developed during CNCF LFX Mentorship 2026 Term 2 using TypeScript, React, Kubernetes APIs, and Headlamp's plugin system.

Utkarsh Raj implemented the plugin with guidance and review from René Dudfield (@illume) and Kevin Hannon (@kannon92). Feedback from the Headlamp and Kueue communities shaped the resource views, status presentation, navigation, documentation, tests, and first alpha release.

Try it and share feedback

Explore the plugin source and README, read the 0.1.0-alpha release documentation, or share feedback in the release issue.

If a pending Workload is still difficult to explain after following the Headlamp path, that is exactly the kind of feedback that can improve the plugin.

Read the whole story
alvinashcraft
4 hours ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories