We’re pleased to announce that Agility SDK 1.620.1 includes a fallback layer for Batched Async Commands, letting developers use the new commands on hardware that doesn’t support them natively. You can find the NuGet package here.
Background
Batched Async Commands were introduced in the Agility SDK 1.720 Preview as a new set of copy, fill, clear, and resolve commands. The legacy versions of these operations are implicitly serialized, since the original resource barrier model has no way to say that two copies don’t depend on each other. The batched versions drop that implicit serialization and take many operations in a single call, so the driver and GPU are free to overlap independent work and you only add enhanced barriers where there’s an actual dependency. They also fill in some long-standing gaps in the API. ClearTextureSubresources can clear a texture without an RTV, UAV, or descriptor heap (including block-compressed formats), FillBuffers can write formatted or raw patterns into buffers without any UAV clear setup, and ClearBoundRenderTargetViews and ClearBoundDepthStencilView clear whatever is currently bound, even in the middle of a render pass.
The downside is that the feature is only supported on newer GPUs. At the 1.720 Preview launch, that meant AMD Radeon RX 7000 and 9000 series, Intel Arc B-Series, and NVIDIA RTX GPUs. There’s a lot of hardware still in use that isn’t on that list, and if you want to ship on all of it, you’d have to keep your existing copy and clear code around alongside the new path.
That’s where the fallback layer comes in.
Batched Async Commands Fallback
The goal of the fallback layer is to let you move to the new commands now, without having to maintain two separate systems while the hardware ecosystem catches up. When the fallback is enabled on a device that doesn’t support Batched Async Commands, the runtime implements the commands itself using operations that every driver already supports. On devices with native support, the calls go straight to the driver as usual, so the same code takes the fast path wherever it’s available.
The fallback is opt-in, for a couple of reasons. First, it isn’t a perfect match for native behavior, and there are some cases it can’t handle at all (covered below). Second, it has a cost even if you never call one of the new commands. To implement fallback support for commands like FillBuffers and the bound clears, the runtime needs to know the barrier state of the resources involved and has to put back your pipeline state, root signature, bindings, and render targets when it’s done. That means tracking all of that on every direct, compute, and bundle command list while the fallback is enabled.
The Performance section below covers what that costs. None of it is huge on its own, but it adds up over a frame, and apps that don’t use Batched Async Commands shouldn’t have to pay for it.
The fallback itself requires Enhanced Barriers support, so on a device without Enhanced Barriers or native support, the batched commands aren’t available.
Example Usage
// Opt in to the fallback on a device factory, before creating the device
ComPtr<ID3D12DeviceFactory> pFactory;
D3D12GetInterface(CLSID_D3D12DeviceFactory, IID_PPV_ARGS(&pFactory));
ComPtr<ID3D12AsyncCommandsFallback> pFallback;
if (SUCCEEDED(pFactory->GetConfigurationInterface(CLSID_D3D12AsyncCommandsFallback, IID_PPV_ARGS(&pFallback))))
{
pFallback->SetFallbackLayerEnabled(TRUE);
}
ComPtr<ID3D12Device> pDevice;
pFactory->CreateDevice(pAdapter, D3D_FEATURE_LEVEL_12_0, IID_PPV_ARGS(&pDevice));
// Check whether you got native support, the fallback, or neither
D3D12_FEATURE_DATA_ASYNC_COMMANDS asyncCommands = {};
pDevice->CheckFeatureSupport(D3D12_FEATURE_ASYNC_COMMANDS, &asyncCommands, sizeof(asyncCommands));
if (asyncCommands.Impl == D3D12_ASYNC_COMMANDS_IMPL_NOT_SUPPORTED)
{
// Batched Async Commands unavailable, keep using the legacy path
return;
}
The setting applies to devices that factory creates, and you can read it back as D3D12_DEVICE_FLAG_ASYNC_COMMANDS_FALLBACK_ENABLED in ID3D12DeviceConfiguration::GetDesc. If you create your device with D3D12CreateDevice instead, you can get the same interface from D3D12GetInterface(CLSID_D3D12AsyncCommandsFallback, ...). That sets the option globally, for devices created with D3D12CreateDevice. Like other global settings, set it before creating any devices. It doesn’t carry over to device factories unless you call InitializeFromGlobalState on them.
If you tried Batched Async Commands in the preview SDK, note that D3D12_FEATURE_DATA_ASYNC_COMMANDS has changed. The BOOL Supported field is now D3D12_ASYNC_COMMANDS_IMPL Impl, which tells you whether you got the native implementation, the fallback, or neither. Since D3D12_ASYNC_COMMANDS_IMPL_NATIVE is 2, code that compares the old field against TRUE will treat native support as unsupported, so check against D3D12_ASYNC_COMMANDS_IMPL_NOT_SUPPORTED instead.
Where the Fallback Differs from Native
Most of the batched commands map pretty directly onto existing operations. Batched copies, tile copies, and resolves run as a loop over their legacy equivalents, so you get the same results as native, just without the overlap. Fills and clears are where the fallback has to do real work, and where there are a few differences to be aware of.
When the fallback can’t handle a call, it fails rather than silently skipping the work. The command list is removed and Close returns an error, usually E_FAIL. With the debug layer enabled, the rejected call also produces a debug-layer error that names the API the fallback couldn’t record, so if Close starts failing after you’ve switched to one of the new commands, check the debug output first. The fallback also validates its inputs more strictly than the runtime does when passing calls to a native driver, so a call that works natively can still be rejected on a fallback device.
ClearTextureSubresources
Multisampled textures can only be cleared on direct command lists; on compute command lists the call fails, though single-sample clears work fine there. The fallback is fastest when it knows a texture’s current layout, which it learns from the barriers you record earlier in the same command list. A barrier that only covers some of a texture’s subresources makes the layout unknown until the next whole-resource barrier, and until then clears take a slower copy-based path. There are also a few combinations the fallback can’t handle, and these fail outright:
- Partial clears of depth/stencil textures (single-sample or multisampled) and of multisampled color textures, on devices without programmable sample positions, when the texture can’t be cleared in place. For example, this happens on compute command lists, or when the texture’s layout isn’t known.
- Multisampled clears in protected sessions or on protected textures.
- Multisampled clears with a format and sample count the device can’t render to.
ClearBoundRenderTargetViews and ClearBoundDepthStencilView
Bound clears are mostly implemented with internal draws. Afterwards, the runtime restores the state those draws changed: your pipeline (a pipeline state object, a state object set with SetPipelineState1, or a generic program set with SetProgram), root signature and root arguments, viewports, scissors, and other related state.
In a few cases the fallback can’t reconstruct that state, for example if a bundle changed state in a way the fallback can’t track, or if ExecuteIndirect used an argument type the fallback doesn’t track. What happens next depends on whether the call is inside a render pass:
- Outside a render pass, the fallback uses legacy clears instead of draws.
- Inside a render pass, the clear fails, since ending the pass for a legacy clear would defeat the purpose of clearing inside it.
Native drivers can record bound clears in bundles, but the fallback doesn’t support that yet, so a bound clear in a bundle fails on a fallback device. A few other differences to keep in mind:
- The internal draws count toward occlusion and pipeline statistics queries that are open at the time, so query results won’t match native.
- If the bound outputs are different sizes and a clear region extends past the smallest one, or if they have different sample counts, the clear fails inside a render pass. Outside of a render pass, the call is lowered to legacy clears instead.
- Depth clear values are normalized. NaN, negative values, negative zero, and denormals become 0, and anything above 1 (including +infinity) becomes 1.
- View information is captured when you call
OMSetRenderTargets or BeginRenderPass. The resource needs to stay alive until the clear is recorded, and the runtime doesn’t check this.
Inexact Clear Values
A few paths go through a legacy view clear, which takes its clear value as floats. When a legacy view clear can’t produce your exact value, the fallback uses the closest value it can instead of failing. This applies to ClearTextureSubresources on multisampled textures, and to bound clears that fall back to legacy clears outside of a render pass, as described above. In those cases:
UINT and SINT values are converted to float, so values with a magnitude above 2^24 can round.- Float subnormals are flushed to zero, keeping the sign.
- NaN values are passed through, but the stored NaN payload isn’t guaranteed.
- Depth is clamped to [0, 1].
The debug layer doesn’t warn when this happens. Single-sample ClearTextureSubresources calls never approximate: if a view clear can’t produce the exact value, the fallback uses its copy-based path instead.
For the full list of differences, see the Async Commands fallback spec.
Testing the Fallback on Hardware with Native Support
If all of your development machines support Batched Async Commands natively, you’ll never actually exercise the fallback. To help with that, you can use d3dconfig to force the fallback on, even on hardware with native support. Like other d3dconfig device settings, this applies to the apps in d3dconfig’s app list, so add your app first:
d3dconfig apps --add MyGame.exe
d3dconfig device force-async-commands-fallback=true
This is also an easy way to compare native and fallback performance for your own workload on a single machine.
Performance
To get a sense of how the fallback compares with native support and with doing this work yourself, we measured each command on a GPU with native support at batch sizes of 1, 16, and 256 operations per call, with the fallback forced on for the fallback runs. For ResolveQueryDataAsync, batch size is the number of queries resolved, and for the bound clears it’s the number of clear rectangles. The legacy versions loop over the equivalent legacy calls and batch their barriers. FillBuffers uses a compute shader fill, ClearTextureSubresources transitions the textures and uses view clears, and the bound clears use ClearRenderTargetView and ClearDepthStencilView with clear rects, recorded outside a render pass. For most commands, the fallback performs about the same as native and the legacy version on the GPU, with some extra CPU overhead for tracking state and restoring the pipeline state and bindings it changes. The main exception is ClearTextureSubresources. When the fallback can’t clear a texture in place, it copies the clear value from a staging buffer, which costs more on both the GPU and the CPU. The bound clears also cost noticeably more CPU time to record.
Keep in mind that these are microbenchmarks. Each one times a batch of commands on its own, so they show the cost of the work itself, not the benefit of running it asynchronously. With native support, the GPU can overlap batched commands with each other and with surrounding work that doesn’t depend on them. The fallback runs them as legacy operations, so it can’t. In a real frame, the difference between the fallback and native may be larger than these numbers suggest, depending on how much independent work there is to overlap. We recommend leveraging the d3dconfig setting to force on the fallback layer to see how performance is impacted for your use case.
Enabling the fallback also adds CPU overhead to recording, even on command lists that never use the new commands. Here’s how much longer common calls took to record with the fallback enabled in our testing. State-setting calls go up by a bigger proportion, but they’re very cheap to begin with:
| Call |
Relative Change |
Extra time per call |
| Draws, dispatches, copies, and legacy clears |
about the same |
— |
ExecuteIndirect |
+20-25% |
6-8 ns |
| Barriers on direct and compute command lists |
+5-20% (up to +75% in our worst cases) |
5-20 ns per barrier (up to 90-100 ns) |
| Barriers on copy command lists |
about the same |
— |
| Setting root arguments |
+20-180% |
1-11 ns (up to 35 ns for root constants in bundles) |
| Setting a pipeline state |
1.5-4x as long |
3-9 ns |
| Changing the root signature |
5-16x as long |
25-30 ns |
| Binding render targets |
+15-70% |
13-15 ns |
| An empty render pass |
about +33% |
about 20 ns |
| Executing a bundle |
about 2.8x as long |
about 100 ns |
| Resetting or closing a bundle |
1.4-2.9x as long |
20-35 ns |
Try It Out
Check out the Batched Async Commands samples, and try them with the fallback forced on and off. We’d love to hear how it works for you.
The post Async Commands Fallback Layer appeared first on DirectX Developer Blog.
Log in
Windows Graphics Drivers
