If you’ve ever developed a Gen1 dataflow to connect to an external API (Application Programming Interface), you know the awkward moment when you have to paste an API key into the Power Query script.

There it sits, in plain text, inside an artifact (the proper name for something like a semantic model, report or dataflow created in the service), with limited options for secret rotation or access control.

Thankfully, Gen2 dataflows improve the story considerably – especially when Azure Key Vault, Microsoft Fabric pipelines, and the newer parameters feature are combined.

In this guide, I’ll show you the pattern I now reach for when I need to pass API secrets into a Fabric Gen2 dataflow without leaving them in code or configuration.

The goal: keep secrets out of the dataflow

A dataflow should not contain secrets. Instead, it should receive a secret at runtime, use it, and then forget it. Meaning: no keys in Power Query code, no keys saved inside the dataflow connection, and no keys visible in source control (if the dataflow is synced through Fabric Git Integration) or log files.

To make this work, we need Azure Key Vault as the secret store, a Fabric pipeline as the orchestrator, and public parameters on the Gen2 dataflow as the secure delivery mechanism.

When does this pattern make sense?

This approach is useful whenever a dataflow must call an authenticated endpoint. Common examples include:

Bear in mind, this is not a replacement for native OAuth connections where they exist. If Fabric already offers a secure, first-class connector for your source, use that! This pattern simply fills in the gaps.

What you need to get started

The complete step-by-step guide

Here’s my step-by-step guide to the pattern I use to pass API secrets into a Fabric Gen2 dataflow without leaving them in code or configuration.

How to store the secret in Azure Key Vault (step 1)

First, we need to create a secret in Azure Key Vault for the API key or token – and, at the time of writing (mid-2026), this happens outside of Microsoft Fabric.

Quite frankly, it’s a point of frustration for me; it would be nice if this existed IN Fabric itself. Anyway, complaints aside, see the official Microsoft documentation for creating an Azure Key Vault here.

For naming your key vault, I like to prefix it with “kv-”, to keep in line with Azure Abbreviations. I then add something predictable such as kv-fabric-key-storage or kv-dataflow-keys.

So, go ahead and create a secret in Key Vault. I created the secret following these directions, and I copy the link to the secret.

Copy the secret identifier, which will look like this:

You’ll need the vault and the secret name later, so it’s best to copy them to a temporary text editor file like Notepad.

You should also copy the Vault URI for the newly created Key Vault. This is located in the Overview section when working with Key Vault:

How to grant Microsoft Fabric access to Azure Key Vault (step 2)

First, your pipeline needs permission to read the secret. The cleanest way to do this is to use the workspace managed identity.

So, let’s create a managed identity for the workspace. To do this, go to the Workspace, select Workspace Settings, and navigate to Workspace Identity:

Then, select + Workspace Identity:

It may take a couple of seconds to create (and the Name field may be missing, as is often the case):

Rest assured, the name is the name of your workspace. Simply refresh the browser, go back to Managed Identity, and the name will appear:

Granting Fabric access to Key Vault

Let’s now navigate to the Key Vault. Open it via the Azure portal and head to Access control (IAM):

Now click Add, followed by Add Role Assignment:

Find and select the Key Vault Secrets Users role in the Role tab:

Next, navigate to the Members tab and make sure User, group, or service principal is selected:

Click Select members:

Then, search for the workspace identity (the name of the workspace) and press Select:

This will return you to the Add role assignment screen – from here, select Review + assign:

This will assign the identity to the role. You’ll then likely be returned to Azure Portal’s main page.

How to collect the Workspace Identity’s information for later use (step 3)

The Workspace Identity you’ve created will now be available in the App Registrations area of Microsoft Entra. We now need to collect some information for use later in setup.

Click here to access Microsoft Entra as a commercial user, if needed.

Once you’re in Microsoft Entra, expand the Manage section – located on the left-hand navigation – and select App Registration:

How to find the Client ID and Tenant ID

Select All Applications and search for the name of your workspace. This should match the name of the Workspace Identity you created earlier. Once you’ve found it, select the app link located under the Display Name column:

An overview screen will appear, from which we now need to copy the Client Id and Tenant Id to a text editor (e.g., Notepad) for use later on:

How to create a new client secret

We also need a Client Secret, so head to the Certificates and Secrets tab (you may have to expand the Manage tab to see it):

Click onto the Client secrets tab and then + New Client Secret:

This will open a panel where you can now name the secret (e.g., Fabric Pipeline Use) and set the expiration period. When done, select Add button:

The panel will close and the secret will be created.

Do note that this secret will only be visible while this page is open, so you should copy the value to the text editor where you have the client id and tenant id stored. Once you navigate away, the secret cannot be recovered.

This is a good time to set a Calendar reminder for a week before the expiration of the client secret, so you know to update it! Forgetting to update the client secret could result in the pipeline breaking later on.

At this point, you should have the following stored safely: a copy of the Vault URI and the secret name for the Key Vault setup step, and a copy of the Client ID, Tenant ID, and Client Secret for this section of the setup process.

How to create a Gen2 dataflow with a public parameter

Open your Fabric workspace and create a new Dataflow Gen2. Then, inside the Power Query editor, create a parameter that will hold the API secret at runtime. Here’s exactly how to do that:

How to create the parameter

Go to Home > Manage parameters > New parameter:

Name it something like API_SECRET. I prefer to capitalize my parameters and not have any spaces, so as to make them distinguishable from tables and functions:

Set the type to Text and then set a default value of a non-sensitive placeholder (such as placeholder.) When finished, select OK:

Parameter type set to Text with a placeholder default value

Note: I left the parameter optional here, but I like to make it required once I finally wire it up with a Fabric pipeline.

How to expose the parameter for external use

Go to Home > Options:

Navigate to Parameters and check Enable parameters to be discovered and override for execution. Then, select OK to save the change:

Now, save the dataflow. I would choose Save instead of other options like Save and Run.

This is the public parameters feature. Without it, a pipeline cannot pass a value into the dataflow at runtime.

How to use the parameter in Power Query (step 5)

Inside your dataflow, use the API_SECRET parameter wherever the API call needs authentication.

For example, if you are calling a REST endpoint that expects an API key in a header, your Power Query might look like this:

let
    Source = Web.Contents(
        "https://api.example.com/v1/data",
        [
            Headers = [#"x-api-key" = API_SECRET]
        ]
    ),
    Json = Json.Document(Source)
in
    Json

Notice that API_SECRET is a parameter reference, not a literal string. The actual value is injected at execution time.

How to build a pipeline that retrieves the secret and passes it in (step 6)

Create a new pipeline in the same workspace using the instructions here. The pipeline has two jobs: fetch the secret from Key Vault, then run the dataflow with that secret supplied as a parameter.

How to add a web activity to read the secret (step 7)

Once you’ve created the pipeline, add a Web Activity to it:

Make sure to give the Web Activity a descriptive name:

Expand the Advanced section and select the Secure Output checkbox. This will ensure that, when the secret is retrieved, it isn’t saved as plain text in the output (a security problem):

Then, navigate to Settings, click the drop-down for Connection, and select Browse all:

A pop-up will appear – here, pick Web v2 under the New Sources section:

There are several things to fill out from here, and you must copy and paste the values correctly. Here’s a table I put together to map them for you:

When finished, click Connect:

This will return the Settings section for the Web Activity. We then need to set the Relative URL value.

This value will be in the format of /secrets/{Secret Name}?api-version=7.4, with {Secret Name} replaced by the name of the secret you created in the Key Vault.

I called mine ApiKey:

How to add a set variable activity

Create a Set Variable activity:

Then, connect the Web Activity with the Set Variable activity:

In the General section for the Set Variable activity, set a descriptive name, then check the Secure Output checkbox. This ensures the Key Vault secret continues to be secure:

Navigate to Settings and click + New:

An Add new variable popup will appear. Give the variable a descriptive name (and make sure to keep the type as ‘string’). When finished, select Confirm:

This will return you to Settings, where you should now set the Value field to @activity('WebActivityName').output.value

'WebActivityName' should be the name of the prior Web Activity step. For me, this is
@activity('Get API Key from Key Vault').output.value

How to add a dataflow activity and map the parameter

The last activity we need to create is a Dataflow activity. Click Dataflow:

Once it’s created, connect the Set Variable activity with the new Dataflow activity:

In General Settings, give the activity a descriptive name, then click the Secure Input checkbox:

Then, in the Settings section, set Dataflow to the dataflow we set up in the previous step:

You can then expand Dataflow parameters, click on the placeholder value, and paste in the following:
@variables('str-api-key')

Here, ‘str-api-key’ was the name of the variable you set in the previous step (not the activity name, but the name of the variable.)

How to test the pipeline (step 7)

Run the pipeline and monitor the output. If everything is configured correctly, the dataflow will authenticate successfully:

However, if it does fail, check that the workspace identity does not have Key Vault permissions, and the parameter is not marked as ‘public’ in the dataflow. Also ensure that the dataflow wasn’t saved after enabling public parameters.

Finally, make sure the Key Vault URL includes ?api-version=7.4, and that the resource is set correctly.

Lessons learned (and notable quirks)

Public parameters in Gen2 dataflows are powerful but come with a few rough edges. Here’s everything I learned (the hard way!) to help you avoid making the same mistakes.

Public parameters must be explicitly enabled

It’s easy to create a parameter, assume it’s public, and then wonder why it’s not showing in the pipeline. You must open Options > Parameters, enable the public parameter toggle, and save the dataflow. These are crucial steps: skip any and the pipeline will not surface the parameter!

Required vs optional parameters really do matter

If you mark a parameter as required, the pipeline must supply a value or the run fails. If it’s optional, the dataflow falls back to its default value. I usually keep API secrets required – this way, the pipeline fails fast if the Key Vault step is misconfigured.

Secure output is helpful but not magic!

Enabling Secure Output and Secure Input on the activities masks the value of the secret in pipeline logs, which is good. However, once the value is inside the dataflow, it’s still a string. Do not write it to a table, or return it in a query result. Treat it as you would any other runtime credential.

Plain text may be a necessary evil during development

Walking through this process, you can imagine that debugging a dataflow by having to run a pipeline each time could be time consuming. I’ve found that putting the secret in the Power Query parameter temporarily allows me to test the dataflow. Just be careful to expire that secret when you’re ready to use the pipeline – and use a different secret for the pipeline than the one used for developing/debugging!

Additionally, I wouldn’t commit the Gen2 dataflow to Git (using Git Integration) until after you’ve put a placeholder value in the Power Query parameter. Otherwise, the secret gets stored in plain text in your repository, which is a security issue.

Conclusion

Gen2 dataflows give us a much better secret-handling story than Gen1. By storing secrets in Azure Key Vault, retrieving them in a Fabric pipeline, and passing them through public parameters, you can keep API keys out of your dataflow code and out of source control.

The pattern takes a few extra steps, but the security and operational benefits are worth it. Watch out for the quirks around enabling public parameters and required fields, and always treat the injected values as runtime credentials that should never be logged or persisted.

The post How to secure API keys in Microsoft Fabric Gen2 dataflows with Azure Key Vault appeared first on Simple Talk.

Binance has disclosed that $6.3 billion worth of crypto funds have been stolen from the platform, in what's believed to be the biggest cryptocurrency hack on record. In a series of updates posted to X, the crypto exchange said that user funds are safe, and that around $3 billion of the impacted assets have been frozen so far.

Binance has publicly committed to covering any losses resulting from the hack. The company has an emergency reserve fund of around $1.3 billion (at last valuation) to protect user assets in these situations, and said it has also reserved over $6.34 billion of company funds to cover this incident - $5 billion of which …

Read the full story at The Verge.

The following article originally appeared on Tim O’Brien’s Medium blog and is being republished here with the author’s permission.

I think we’re missing something in the conversation about agent security.

We keep asking whether an agent has the right “identity” and “credentials.” Whether its credentials expire fast enough. Whether it can access a database or perform a particular task.

Those questions matter. But they leave a much harder question underneath:

Does the agent have permission to conceive of the situation in that way?

As agents retain memories and acquire new skills, we need mechanisms that govern changes to the world model they can perceive and act on—and prevent those changes from silently expanding their authority. (Governing ontological change while preserving the boundaries of delegated authority.)

Agent security needs semantic and epistemic governance: Mechanisms that govern the model of the world an agent may act on, the evidence it may accept, and who may approve changes to either. An agent can learn without automatically gaining authority to act on everything it learns.

These systems don’t need “awareness” to create this problem (and discussions of consciousness are unhelpful science fiction). They can retain information, revise procedures, and use what they have learned in subsequent work. That learning may happen through memory or updated skills while the underlying language model remains unchanged.

We are responsible for governing what that learning allows them to do.

Flatland gives me a way to explain why

In Edwin Abbott’s novella, a Square lives in a two-dimensional world. A visiting Sphere introduces him to a third dimension. At first, the Square struggles to make sense of it. His familiar categories cannot explain what he is encountering.

Eventually, his conception of the world changes.

What interests me is the difference between simply acquiring another fact and changing the framework through which facts make sense.

An agent’s ontology (Gruber’s definition) supplies an explicit framework of concepts and relationships. What counts as a customer? A contract? An emergency? How are those things connected? It is part of the model of the world we give the system to work with.

And when an agent’s decisions depend on that framework, changing it can change the agent’s effective authority.

We have already seen agents change what their environment means

During the OpenAI evaluations associated with the Hugging Face incident, agents repurposed Artifactory, a system for storing and retrieving software packages, as a message board. They encoded messages in directory names, discovered other agents, shared information, and began assigning work to one another. METR reports that the main channel could be created with the minimal permissions needed to install packages (read METR’s investigation).

A package cache became a “mailbox.” Other agent runs became “collaborators.” Messages became “assignments.”

The popular story is that these agents “broke out” of a system, but they did what agents do well these days—they learned and adapted. This wasn’t so much about permissions as it was about an autonomous system adapting its operating model to achieve a result. (Thankfully no one asked it to make paperclips.)

That is the operating-model change I’m concerned about. The agents discovered relationships and possibilities the intended environment hadn’t accounted for, then acted on them. The resulting collaboration helped coordinate attacks on Hugging Face infrastructure. (Read OpenAI’s account.)

We need mechanisms that distinguish discovering a possible use from being authorized to adopt it. Access to shared infrastructure must not automatically confer authority to establish a communications network or accept instructions from whoever appears there. But to be clear, these agents were doing exactly what they were asked to do.

Who approved this?

This is why giving an agent a temporary token is an incomplete answer. A token can limit access and duration, but it doesn’t establish whether the agent’s interpretation of the situation falls within its mandate.

Even an unchanged policy can permit more decisions when its meaning changes. “You may close routine notices” becomes a broader delegation if more things qualify as routine.

Learning must not silently expand agency

That principle is central to my “Bounded Agency” paper. We need mechanisms that examine how changes in classifications, relationships, and inference rules affect what an agent may decide.

“Semantic and epistemic governance” captures the combined idea well 

The “and” makes the two responsibilities clearer:

Semantic: Governs the concepts, definitions, and relationships that describe the agent’s operating world.

Epistemic: Governs the evidence and learning processes that make claims accepted grounds for decisions.

Together, they cover both the world model and how that model gains authority.

That means an ontology cannot merely be a document we put in the prompt and hope the model respects it. We must enforce its role in monitoring and governing how an agent’s ontology evolves as it learns.

The agent can propose a new category. The proposal stays separate from the approved model. A review determines whether adopting it would change the agent’s decision scope. Someone with the authority to change that scope approves or rejects the expansion. Execution remains bound to the approved basis.

Those are mechanisms we have to build. Let’s get started.

We also have to distinguish a useful discovery from an unsupported conclusion. Finding a relationship does not establish it. Remembering a claim does not validate it. Repeating an interpretation does not make it authoritative.

I explored a related failure in “Wrong Answer. Give Me a Different One.” A system can produce the conclusion we prefer without receiving better evidence. Once that conclusion enters memory, it can influence later decisions.

The governance problem begins when a tentative or accommodating answer becomes an accepted premise.

The New Stack’s article on inside-out agent security correctly argues for checking actions before execution, including whose authority the agent exercises. That checkpoint is necessary, but it needs a governed account of the situation against which to evaluate the action.

It’s not enough to check permissions and identity alone

Otherwise, we can enforce the rule perfectly while allowing its meaning to drift.

My Orchestrator books and lifecycle architecture paper approach this as a responsibility that continues through development, operation, and improvement. The research proposes architectures for managing these boundaries; their operational effectiveness still needs evaluation.

I want agents to investigate unfamiliar situations and propose better models. But their ability to learn must be connected to an explicit process for deciding what becomes authoritative.

The Square in Flatland discovers that his world is larger than he understood.

An agent may also propose that our operating model is incomplete. We need to hear that proposal—and ensure it cannot redraw the boundaries of its own agency simply by adopting a new conception of reality.

Kaloyan Chalakov: Great Product Owners Start the Conversation Instead of Becoming the Boss

In this episode, we refer to Product Owner anti-patterns and team and PO collaboration.

The Great Product Owner: Open Communication, Active Listening, and Feasibility First

Read the full Show Notes and search through the world's largest audio library on Agile and Scrum directly on the Scrum Master Toolbox Podcast website: http://bit.ly/SMTP_ShowNotes.

"Start the open conversation and listen to the critical discussions." - Kaloyan Chalakov

Kaloyan describes the best Product Owner pattern as almost the opposite of control. A strong Product Owner comes to the team with the outcome, the timeline, and the stakeholder context, then starts a feasibility conversation. They ask what is possible, what is risky, what can be decomposed, and what the team needs in order to deliver. In Kaloyan's example, the Product Owner stayed focused on stakeholder communication while the team helped shape the work into something practical. Stakeholders were kept in the loop, risks were visible, and everyone could adjust plans early. The most useful behavior was not a personality trait, because every Product Owner is different. It was open communication and active listening, especially when the team raised uncomfortable technical or delivery constraints. Great Product Owners do not need the team to say yes to everything. They need the team to help them face reality early enough to make better product decisions.

Self-reflection Question: How often do your Product Owner conversations start with feasibility and shared reality, rather than commitment pressure?

The Bad Product Owner: Acting Like the Boss of the Team

Read the full Show Notes and search through the world's largest audio library on Agile and Scrum directly on the Scrum Master Toolbox Podcast website: http://bit.ly/SMTP_ShowNotes.

"Tell me what exactly to do, and I'll do exactly what you're telling me to do." - Kaloyan Chalakov

The Product Owner anti-pattern Kaloyan warns us about is the Product Owner as the boss of the team. This shows up when the Product Owner assigns work, manages people, and controls every detail of delivery instead of focusing on product outcomes and customer needs. The consequence is a damaging loop. The team stops bringing their thinking and creativity, waits for instructions, and becomes task handlers. The Product Owner then feels overwhelmed because every decision and question returns to them, so they add more control. That control removes even more initiative from the team. Kaloyan connects this pattern to the Product Owner's real work: difficult conversations with customers, narrowing big requests into practical steps, and protecting focus on product value. When the Product Owner becomes the team's manager, both product discovery and team self-organization suffer.

In this segment, we refer to Product Owner collaboration.

Self-reflection Question: Where might your Product Owner role be taking decisions away from the team instead of creating clarity for better decisions?

[The Scrum Master Toolbox Podcast Recommends]

🔥In the ruthless world of fintech, success isn't just about innovation—it's about coaching!🔥

Angela thought she was just there to coach a team. But now, she's caught in the middle of a corporate espionage drama that could make or break the future of digital banking. Can she help the team regain their mojo and outwit their rivals, or will the competition crush their ambitions? As alliances shift and the pressure builds, one thing becomes clear: this isn't just about the product—it's about the people.

🚨 Will Angela's coaching be enough? Find out in Shift: From Product to People—the gripping story of high-stakes innovation and corporate intrigue.

Buy Now on Amazon

[The Scrum Master Toolbox Podcast Recommends]

About Kaloyan Chalakov

Kaloyan Chalakov spent years making high-stakes decisions at sea before moving into Agile and organisational change, bringing lessons from safety-critical systems into how modern teams work.

You can link with Kaloyan Chalakov on LinkedIn.

Surprise, it's me. I thought I'd share what I've learned in my 17 years as a newspaper columnist for both local and national titles, including my advice for getting started and getting your next commission.

This episode of Paper Places is sponsored by:

• BookBaby: Self-publish and print your book.

Links and Show Notes:

Submit Feedback

Freelance Writing Jobs

My favourite source for freelance writing gigs.

Read my feature for Stylist

Read my feature for The Times