Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
151513 stories
·
33 followers

mssql-python 1.5: Apache Arrow, sql_variant, and Native UUIDs

1 Share

We're excited to announce the release of mssql-python 1.5.0, the latest version of Microsoft's official Python driver for SQL Server, Azure SQL Database, and SQL databases in Fabric. This release delivers Apache Arrow fetch support for high-performance data workflows, first-class sql_variant and native UUID support, and a collection of important bug fixes.

 

pip install --upgrade mssql-python

Apache Arrow fetch support

If you're working with pandas, Polars, DuckDB, or any Arrow-native data framework, this release changes how you get data out of SQL Server. The new Arrow fetch API returns query results as native Apache Arrow structures, using the Arrow C Data Interface for zero-copy handoff directly from the C++ layer to Python.

This is a significant performance improvement over the traditional fetchall() path, which converts every value through Python objects. With Arrow, columnar data stays in columnar format end-to-end, and your data framework can consume it without any intermediate copies.

Three methods for different workflows

cursor.arrow() fetches the entire result set as a PyArrow Table:

import mssql_python conn = mssql_python.connect( "SERVER=myserver.database.windows.net;" "DATABASE=AdventureWorks;" "UID=myuser;PWD=mypassword;" "Encrypt=yes;" ) cursor = conn.cursor() cursor.execute("SELECT * FROM Sales.SalesOrderDetail") # Get the full result as a PyArrow Table table = cursor.arrow() # Convert directly to pandas - zero-copy where possible df = table.to_pandas() # Or to Polars - also zero-copy import polars as pl df = pl.from_arrow(table)

cursor.arrow_batch() fetches a single RecordBatch of a specified size, useful when you want fine-grained control over memory:

cursor.execute("SELECT * FROM Production.TransactionHistory") # Process in controlled chunks while True: batch = cursor.arrow_batch(batch_size=10000) if batch.num_rows == 0: break # Process each batch individually process(batch.to_pandas())

cursor.arrow_reader() returns a streaming RecordBatchReader, which integrates directly with frameworks that accept readers:

cursor.execute("SELECT * FROM Production.TransactionHistory") reader = cursor.arrow_reader(batch_size=8192) # Write directly to Parquet with streaming - no need to load everything into memory import pyarrow.parquet as pq pq.write_table(reader.read_all(), "output.parquet") # Or iterate batches manually for batch in reader: process(batch)

How it works under the hood

The Arrow integration is built directly into the C++ pybind11 layer. When you call any Arrow fetch method, the driver:

  1. Allocates columnar Arrow buffers based on the result set schema
  2. Fetches rows from SQL Server in batches using bound column buffers
  3. Converts and packs values directly into the Arrow columnar format
  4. Exports the result via the Arrow C Data Interface as PyCapsule objects
  5. PyArrow imports the capsules with zero copy

Every SQL Server type maps to the appropriate Arrow type: INT to int32, BIGINT to int64, DECIMAL(p,s) to decimal128(p,s), DATE to date32, TIME to time64[ns], DATETIME2 to timestamp[us], UNIQUEIDENTIFIER to large_string, VARBINARY to large_binary, and so on.

LOB columns (large VARCHAR(MAX), NVARCHAR(MAX), VARBINARY(MAX), XML, UDTs) are handled transparently by falling back to row-by-row GetData fetching while still assembling the result into Arrow format.

Community contribution

The Arrow fetch support was contributed by @ffelixg. This is a substantial contribution spanning the C++ pybind layer, the Python cursor API, and comprehensive tests. Thank you, Felix Graßl, for an outstanding contribution that brings high-performance data workflows to mssql-python.

sql_variant type support

SQL Server's sql_variant type stores values of various data types in a single column. It's commonly used in metadata tables, configuration stores, and EAV (Entity-Attribute-Value) patterns. Version 1.5 adds full support for reading sql_variant values with automatic type resolution.

The driver reads the inner type tag from the sql_variant wire format and returns the appropriate Python type:

cursor.execute(""" CREATE TABLE #config ( key NVARCHAR(50) PRIMARY KEY, value SQL_VARIANT ) """) cursor.execute("INSERT INTO #config VALUES ('max_retries', CAST(5 AS INT))") cursor.execute("INSERT INTO #config VALUES ('timeout', CAST(30.5 AS FLOAT))") cursor.execute("INSERT INTO #config VALUES ('app_name', CAST('MyApp' AS NVARCHAR(50)))") cursor.execute("INSERT INTO #config VALUES ('start_date', CAST('2026-01-15' AS DATE))") cursor.execute("SELECT value FROM #config ORDER BY key") rows = cursor.fetchall() # Each value comes back as the correct Python type assert rows[0][0] == "MyApp" # str assert rows[1][0] == 5 # int assert rows[2][0] == date(2026, 1, 15) # datetime.date assert rows[3][0] == 30.5 # float

All 23+ base types are supported, including int, float, Decimal, bool, str, date, time, datetime, bytes, uuid.UUID, and None.

Native UUID support

Previously, UNIQUEIDENTIFIER columns were returned as strings, requiring manual conversion to uuid.UUID. Version 1.5 changes the default: UUID columns now return native uuid.UUID objects.

import uuid cursor.execute("SELECT NEWID() AS id") row = cursor.fetchone() # Native uuid.UUID object - no manual conversion needed assert isinstance(row[0], uuid.UUID) print(row[0]) # e.g., UUID('550e8400-e29b-41d4-a716-446655440000')

UUID values also bind natively as input parameters:

my_id = uuid.uuid4() cursor.execute("INSERT INTO Users (id, name) VALUES (?, ?)", my_id, "Alice")

Migration compatibility

If you're migrating from pyodbc and your code expects string UUIDs, you can opt out at three levels:

# Module level - affects all connections mssql_python.native_uuid = False # Connection level - affects all cursors on this connection conn = mssql_python.connect(conn_str, native_uuid=False)

When native_uuid=False, UUID columns return strings as before.

Row class export

The Row class is now publicly exported from the top-level mssql_python module. This makes it easy to use in type annotations and isinstance checks:

from mssql_python import Row cursor.execute("SELECT 1 AS id, 'Alice' AS name") row = cursor.fetchone() assert isinstance(row, Row) print(row[0]) # 1 (index access) print(row.name) # "Alice" (attribute access)

Bug fixes

Qmark false positive fix

The parameter style detection logic previously misidentified? characters inside SQL comments, string literals, bracketed identifiers, and double-quoted identifiers as qmark parameter placeholders. A new context-aware scanner correctly skips over these SQL quoting contexts:

# These no longer trigger false qmark detection: cursor.execute("SELECT [is this ok?] FROM t") cursor.execute("SELECT 'what?' AS col") cursor.execute("SELECT /* why? */ 1")

NULL VARBINARY parameter fix

Fixed NULL parameter type mapping for VARBINARY columns, which previously could fail when passing None as a binary parameter.

Bulkcopy auth fix

Fixed stale authentication fields being retained in the bulk copy context after token acquisition. This could cause Entra ID-authenticated bulk copy operations to fail on subsequent calls.

Explicit module exports

Added explicit __all__ exports from the main library module to prevent import resolution issues in tools like mypy and IDE autocompletion.

Credential cache fix

Fixed the credential instance cache to correctly reuse and invalidate cached credential objects, preventing unnecessary re-authentication.

datetime.time microseconds fix

Fixed datetime.time values incorrectly having their microseconds component set to zero when fetched from TIME columns.

The road to 1.5

ReleaseDateHighlights
1.0.0November 2025GA release - DDBC architecture, Entra ID auth, connection pooling, DB API 2.0 compliance
1.1.0December 2025Parameter dictionaries, Connection.closed property, Copilot prompts
1.2.0January 2026Param-as-dict, non-ASCII path handling, fetchmany fixes
1.3.0January 2026Initial BCP implementation (internal), SQLFreeHandle segfault fix
1.4.0February 2026BCP public API, spatial types, Rust core upgrade, encoding & stability fixes
1.5.0April 2026Apache Arrow fetch, sql_variant, native UUIDs, qmark & auth fixes

Get started today

pip install --upgrade mssql-python

We'd love your feedback. Try the new Arrow fetch API with your data workflows, let us know how it performs, and file issues for anything you run into. This driver is built for the Python data community, and your input directly shapes what comes next.

Read the whole story
alvinashcraft
31 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Issue 747

1 Share

Comment

Tracy Miranda writes about IDEs on the official Swift blog:

You can now write Swift in a broader range of popular IDEs, including Cursor, VSCodium, AWS’s Kiro, and Google’s Antigravity. By leveraging VS Code extension compatibility, these editors tap directly into the Open VSX Registry, where the official Swift extension is now live.

I was only vaguely aware of the Open VSX Registry, but after looking into it in more detail this week, it made me think about what an impact Visual Studio Code has had on the industry. It started as a small point in a long list of Visual Studio announcements from the Microsoft BUILD conference in 2015, and was met with a little skepticism, both for being from Microsoft, and for being Electron-based. Just look at it now, though! The underlying open-source code powers the majority of modern coding editors, and is even more dominant in the flurry of new AI-focused editors, like Cursor, Windsurf, Kiro, Antigravity, Trae, and more.

It’s a great editor, too. I must admit I also have a fondness for Zed, but VS Code is my day to day editor when I don’t need something Xcode-specific. I couldn’t have predicted I’d ever be happy using an Electron-based coding editor, but I am.

It’s great that Microsoft didn’t have to “win” in the traditional sense to get here, too. The heart of the editor remains free and open source, and is so important that Google and Amazon are building their next-generation IDEs on top of it and Apple is writing official setup guides for editors built on top of it.

Yes, the VS Code extension isn’t new, but the continued support of it, and the care taken to make sure it works with the VSX marketplace is important. This, along with all the other Android, Wasm, Windows, and Swift on embedded hardware news from the last year are all signs of an Apple that has changed over the last few years.

If all of that doesn’t say that Swift is bigger than Xcode, I don’t know what does.

– Dave Verwer

News

Introducing Untold Engine

It’s not every day you get to talk about a new game engine written in Swift, but today is one of those days! Harold Serrano made an announcement his new Metal-based game engine this week, and with a focus on XR scenes and with visionOS support already in-place, it’s an interesting one!

Tools

Xcode 26.4 Simulator Paste Is Broken: Here’s the Workaround

Have you been having problems pasting text into the simulator from your Mac? I hit this bug last week and it’s really irritating. Junda Ong first provides a workaround using simctl, and then follows up by vibe coding a little Mac app designed to keep your clipboard in sync again.

Code

What’s that “structured” in Structured Concurrency?

Max Seelemann has spent the past six months writing up his thoughts and techniques on the subject in six lengthy posts (1, 2, 3, 4, 5, and 6), and concludes that:

And this concludes my miniseries on tasks and cancellation. If I learned one thing writing these posts, it’s that this is a complex field with many nuances that are easy to get wrong.

It’s a good read if you’re looking for a series that builds towards some nice real world, practical techniques.


Playing in the Mac App Sandbox

What a great guide to the macOS sandbox from Sarah Reichelt:

My recommendation is to start every project with the sandbox enabled and using all the default settings. Enable extra permissions only as you need them. Add entitlements if the settings do not allow what you need. Only turn off sandboxing if there is no other way to make your app work.

Very detailed and well-written from a true expert. What more could you want?


Challenges with Ancient Dates in Apple SDKs

What a lovely story of some extremely obscure date bugs from Aaron Trickey. Did you know that Foundation’s date arithmetic breaks before 4713 BC? Or that UIDatePicker won’t go further back than 1 AD? If you’ve ever needed convincing that working with dates over long periods of time is challenging, this is for you.

And finally...

Click clack, clack clack, click click….boom, I made something.

Read the whole story
alvinashcraft
31 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

dotnet-1.1.0

1 Share

Changes:

  • 3e864cd .NET: Update version to 1.1.0 (#5204)
  • 14d2ab3 Standardize file skills terminology on 'directory' (#5205)
  • e5f7b9c .NET: Support reflection for discovery of resources and scripts in class-based skills (#5183)
See More
  • 1dd828d CHANGELOG Update with V1.0.0 Release (#5069)
  • 8348584 VerifySamples: Filter projects to net10 only (#5184)
  • 6d6cb84 .NET: Improve resilience of verify-samples by building separately and improving evaluation instructions (#5151)
  • 79afda1 Samples fixes (#5169)
  • a7a02c1 Fix test compat for entity key validation (#5179)
  • 7010dd7 .NET: Support custom types in skill resource and script functions (#5152)
  • e10d448 Fix handoff workflow context management and improve AG-UI demo (#5136)
  • 942cb04 .NET: Fix compaction chat history duplication bug (#5149)
  • e224f06 .NET: Update models used in dotnet samples to gpt-5.4-mini (#5080)
  • 826d8db .NET: fix: Concurrent Workflow Sample (#5090)
  • 4134c74 Add CreateSessionAsync(conversationId) to FoundryAgent (#5144) [ #5138 ]
  • 86b49d8 Fix and simplify ComputerUse sample (#5075)
  • d73c06f .NET: Align skill folder discovery with spec (#5078)
  • 746c7da Revise agent examples in README.md (#5067)
  • d30103f .NET: Fix input signal issue during checkpoint restoration (#5085)
  • 55ae57c .NET: Add Message Delivery Callback Overloads to Executor (#5081)
  • d284d96 fix: 04_MultiModelService sample (#5074)
  • d1a8115 Bump Anthropic from 12.8.0 to 12.11.0 (#5055)
  • 9f0dbe5 .NET: Improve workflow unit test coverage (#5072)
  • 3fc1d00 .NET: skill as class (#5027)
  • e4defad .NET: Add github actions workflow for verify-samples (#5034)

This list of changes was auto generated.

Read the whole story
alvinashcraft
31 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Seeing like an agent: how we design tools in Claude Code

1 Share
Seeing like an agent: how we design tools in Claude Code
Read the whole story
alvinashcraft
33 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Multi-agent coordination patterns: Five approaches and when to use them

1 Share
Multi-agent coordination patterns: Five approaches and when to use them
Read the whole story
alvinashcraft
33 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Announcing Windows 11 Insider Preview Build 28020.1812 (Canary Channel)

1 Share
Hello Windows Insiders, today we are releasing Windows 11 Insider Preview Build 28020.1812 to the Canary Channel. (KB 5083824)

What’s new in Canary Build 28020.1812

Changes and Improvements gradually being rolled out with toggle on*

  • This update includes a small set of general improvements and fixes that improve the overall experience for Insiders running this build on their PCs.

[Touchpad]

  • We’re adding a new setting in Settings > Bluetooth & Devices > Touchpad so you can choose how large the right-click zone size is (this is the portion in the bottom-right of your touchpad where pressing a single finger will result in a right-click), between default, small, medium, and large. Note – this setting only appears for touchpads that have a pressable surface for this. Some manufacturers may provide customization for this in their own apps – if you’ve customized your setting using an app like this, a “Custom” entry will be added to the dropdown, which preserves that value.

Feedback: Share your thoughts in Feedback Hub (WIN + F)under Input and Language > Touchpad.

[Sharing]

  • Based on feedback we’re making some refinements to drag tray, using a smaller peek view to reduce accidental invocation and make it easier to dismiss when interacting near the top of the screen.

[Windows Security app]

  • We have recently started to roll out enhancements to the Windows Security app > Device security > Secure Boot experience, which shows green, yellow, and red icon badges as well as new text that reflects your device’s Secure Boot state and certificate status. Microsoft is updating Secure Boot certificates on consumer devices and some business PCs, and you can now check your certificate status with this new feature. This experience is disabled by default on enterprise IT managed devices and servers.

Feedback: Share your thoughts in Feedback Hub (WIN + F)under Apps > Windows Security.

[Feedback Hub Improvements]

  • Thanks Windows Insiders who have been sharing such great feedback about the new Feedback Hub version. As some of you may have already noticed, we're currently rolling out version 2.2604.101.0 to Dev and Canary. This includes design refinements based on your feedback, such as:
    • Improved the default window size, and now it will remember window size across session.
    • You can now use the mouse back button to go back between pages.
    • If you were seeing less community feedback than usual, that should be addressed now.
    • Upvote buttons for feedback should display correctly for Insiders using the Chinese display language now, … and more.

Please keep the feedback coming! You can share your thoughts under Feedback Hub (WIN + F) under Apps > Feedback Hub.

Reminders for Windows Insiders in the Canary Channel

  • Many features in the Canary Channel are rolled out using Control Feature Rollout technology, starting with a subset of Insiders and ramping up over time as we monitor feedback to see how they land before pushing them out to everyone in this channel.
  • The desktop watermark shown at the lower right corner of the desktop is normal for Windows Insider pre-release builds.
  • Some features may show up in the Dev and Beta Channels first before showing up in the Canary Channel.
  • For Windows Insiders who want to be the first to get features gradually rolled out to you, you can turn ON the toggle to get the latest updates as they are available via Settings > Windows Update*. Over time, we will increase the rollouts of features to everyone with the toggle turned on. Should you keep this toggle off, new features will gradually be rolled out to your PC over time once they are ready.
  • Some features in active development we preview with Windows Insiders may not be fully localized and localization will happen over time as features are finalized. As you see issues with localization in your language, please report those issues to us via Feedback Hub.
  • To get off the Canary Channel, a clean install of Windows 11 will be required. As a reminder - Insiders can’t switch to a channel that is receiving builds with lower build numbers without doing a clean installation of Windows 11 due to technical setup requirements.
  • Check out Flight Hub for a complete look at what build is in which Insider channel.
Thanks, Windows Insider Program Team
Read the whole story
alvinashcraft
34 minutes ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories