Ten years ago, Microsoft released the source for MS-DOS 1.25 and 2.0 to the Computer History Museum, and then later republished them for reference purposes. This code holds an important place in history and is a fascinating read of an operating system that was written entirely in 8086 assembly code nearly 45 years ago. 

Today, in partnership with IBM and in the spirit of open innovation, we're releasing the source code to MS-DOS 4.00 under the MIT license. There's a somewhat complex and fascinating history behind the 4.0 versions of DOS, as Microsoft partnered with IBM for portions of the code but also created a branch of DOS called Multitasking DOS that did not see a wide release. 

MS-DOS 4.00

Explore the source code, binaries, and documents on GitHub

View on GitHub

A young English researcher named Connor "Starfrost" Hyde recently corresponded with former Microsoft Chief Technical Officer Ray Ozzie about some of the software in his collection. Amongst the floppies, Ray found unreleased beta binaries of DOS 4.0 that he was sent while he was at Lotus. Starfrost reached out to the Microsoft Open Source Programs Office (OSPO) to explore releasing DOS 4 source, as he is working on documenting the relationship between DOS 4, MT-DOS, and what would eventually become OS/2. Some later versions of these Multitasking DOS binaries can be found around the internet, but these new Ozzie beta binaries appear to be much earlier, unreleased, and also include the ibmbio.com source.  

Scott Hanselman, with the help of internet archivist and enthusiast Jeff Sponaugle, has imaged these original disks and carefully scanned the original printed documents from this "Ozzie Drop". Microsoft, along with our friends at IBM, think this is a fascinating piece of operating system history worth sharing.  

Jeff Wilcox and OSPO went to the Microsoft Archives, and while they were unable to find the full source code for MT-DOS, they did find MS DOS 4.00, which we're releasing today, alongside these additional beta binaries, PDFs of the documentation, and disk images. We will continue to explore the archives and may update this release if more is discovered.  

Thank you to Ray Ozzie, Starfrost, Jeff Sponaugle, Larry Osterman, Mark Zbikowski, our friends at the IBM OSPO, as well as the makers of such digital archeology software including, but not limited to Greaseweazle, Fluxengine, Aaru Data Preservation Suite, and the HxC Floppy Emulator. Above all, thank you to the original authors of this code, some of whom still work at Microsoft and IBM today!

If you'd like to run this software yourself and explore, we have successfully run it directly on an original IBM PC XT, a newer Pentium, and within the open source PCem and 86box emulators.  

The post Open sourcing MS-DOS 4.0  appeared first on Microsoft Open Source Blog.

As part of the Phi-3 launch Microsoft has released optimized ONNX models as detailed in ONNX Runtime supports Phi-3 mini models across platforms and devices and published the models on HuggingFace 🤗 at Phi-3 Mini-4K-Instruct ONNX models for consumption in for example the ONNX Runtime GenAI. This makes it very easy to run this model locally in just a few lines of C# as I’ll show in this blog post.

Prerequisites

• Install .NET 8 SDK.
• Clone ONNX model repo (use git lfs)

  1
  2
  git lfs install
  git clone https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx
  

• Checkout the model files you would like to use or all with:

  1
  git lfs checkout
  

• This repo has the following directories:

  1
  2
  3
  4
  5
  6
  7
  8
  ├───cpu_and_mobile
  │   ├───cpu-int4-rtn-block-32
  │   └───cpu-int4-rtn-block-32-acc-level-4
  ├───cuda
  │   ├───cuda-fp16
  │   └───cuda-int4-rtn-block-32
  └───directml
      └───directml-int4-awq-block-128
  

  Any of these directories can be used with ONNX Runtime GenAI. It does also mean there is a large ~2GB onnx.data weights file for each of these, and that you need to have each of these present if you want to run either of them depending on available hardware.

• CUDA is required for the cuda models, which I’ll use here, and you need to have a compatible GPU and drivers installed.

Code

Create a new console app with dotnet new console -n OnnxRuntimeGenAiDemo and change to:

1
2
3
4
5
6
7
8
9
10
11
12
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.ML.OnnxRuntimeGenAI.Cuda" 
                      Version="0.2.0-rc4" />
  </ItemGroup>
</Project>

Then change Program.cs to:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
using Microsoft.ML.OnnxRuntimeGenAI;
var modelDirectory = args.Length == 2 ? args[1] :
    @"C:\git\oss\Phi-3-mini-4k-instruct-onnx\cuda\cuda-int4-rtn-block-32";
using var model = new Model(modelDirectory);
using var tokenizer = new Tokenizer(model);
while (true)
{
    Console.Write("Prompt: ");
    var line = Console.ReadLine();
    if (line == null) { continue; }

    using var tokens = tokenizer.Encode(line);

    using var generatorParams = new GeneratorParams(model);
    generatorParams.SetSearchOption("max_length", 2048);
    generatorParams.SetInputSequences(tokens);

    using var generator = new Generator(model, generatorParams);

    while (!generator.IsDone())
    {
        generator.ComputeLogits();
        generator.GenerateNextToken();
        var outputTokens = generator.GetSequence(0);
        var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
        var output = tokenizer.Decode(newToken);
        Console.Write(output);
    }
    Console.WriteLine();
}

And you are good to go and can run it with dotnet run .\OnnxRuntimeGenAiDemo.csproj, which may output something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[0;93m2024-04-28 09:30:55.0208088 
[W:onnxruntime:onnxruntime-genai, session_state.cc:1166 onnxruntime::VerifyEachNodeIsAssignedToAnEp] 
Some nodes were not assigned to the preferred execution providers which may or may not have an negative 
impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
[0;93m2024-04-28 09:30:55.0262077 
[W:onnxruntime:onnxruntime-genai, session_state.cc:1168 onnxruntime::VerifyEachNodeIsAssignedToAnEp] 
Rerunning with verbose output on a non-minimal build will show node assignments.
Prompt: What is the capital of Denmark?

 The capital of Denmark is Copenhagen.

It is the largest city in Denmark and is located on the eastern coast of the
island of Zealand. Copenhagen is not only the political and cultural center of
Denmark but also a major economic hub. The city is known for its beautiful
architecture, historic sites, and vibrant cultural scene. The Copenhagen Opera
House, Tivoli Gardens, and the Nyhavn canal area are just a few of the
attractions that draw millions of visitors each year.

That’s all!

The warnings are from ONNX Runtime and it is not unusual to see such depending on the ONNX model. If you want to use CPU or DirectML you have to switch both nuget package and adjust code accordingly. I could not get the DirectML working in a quick trial of that.

Generally ONNX runtime and GenAI packaging and modularization of these is lacking, despite the architecture and design of ONNX runtime being highly modular and extensible. Ideally you should be able to pick and choose one or multiple execution providers and run on whatever provider is available at runtime. This is what we do for our ML computer vision needs but it has required us to custom built and custom package native packages for the ONNX Runtime. This is part of the NtvLibs packages I have published on nuget and for which I created a .NET tool to create such packages. Including support for splitting large native library files across multiple nuget packages similar to how TorchSharp does this. Something I have yet to detail in a blog post. If I’ll ever find the time.

It’s fairly easy to get Phi-3-mini to output garbage or loop forever. Just prompt test for example. Some of this can be controlled via various parameters (via generatorParams.SetSearchOption) of which some are detailed in the Python example code.

Full solution and project can be downloaded as OnnxRuntimeGenAiDemo.zip

I’ve been quiet lately, and that’s because I’ve joined OpenSesame as Vice President of Engineering. It’s been a fascinating opportunity to rebuild an engineering organization from the inside, and I’m loving every minute. We’re introducing a lot of cutting-edge software development practices, such as self-organizing vertically-scaled teams and Extreme Programming.

As you might expect, introducing these changes to an organization with [REDACTED] number of engineers has been challenging. (I’m not sure if I’m allowed to say how many engineers we have, so let’s just say “lots,” but not “tons.” Bigger than a breadbox, anyway. Enough that I don’t do any coding myself, and the managers that report to me don’t have time to do much either.)

What I’m really doing is changing the engineering culture at OpenSesame. Culture doesn’t change easily. It tends to snap back. True change involves changing hundreds of little day-to-day decisions. That’s hard, even when people want to make those changes, and full buy-in is hard to come by. I’ve hired several XP coaches to help, but even they’re stretched thin.

A Lever for Change

This is where the new career ladder comes in. OpenSesame had a pretty innovative approach to career development before I joined. It involved a spreadsheet where engineers would gather evidence of their skills. Each piece of evidence contributed towards an engineer’s promotion. It did a nice job of being objective (or at least, as objective as these things can be) and clear about expectations.

The new career ladder builds on the ideas of the previous spreadsheet to introduce the changes I want. Where the old spreadsheet focused on individual ownership and investigating new technologies, the new one emphasizes teamwork, peer leadership, and maintainable code. I’m hoping this will help direct people to new behaviors, which will in turn start to change the engineering culture.

The new spreadsheet also replaces the previous evidence-based approach with a simple manager-led evaluation of skills. This makes room for a lot more skills. Too many, possibly. It’s a very fine-grained approach. But I’m hoping that will help provide clarity to engineers and give them the opportunity to pick and choose which skills they want to work on first.

How It Works

Each title has certain skill requirements, which are grouped into skill sets. For example, “Software Engineer” requires these skill sets:

• Basic Communication
• Basic Leadership
• Basic Product
• Basic Implementation
• Basic Design
• Basic Operations

Each skill set includes several skills. For exaqmple, “Basic Design” includes these skills:

• Decompose problem into tasks
• Class abstraction
• Mental model of your team’s codebase
• Mental model of a complex dependency
• Campsite rule
• Fail fast
• Paranoiac telemetry
• Evaluate simple dependencies

(There’s a document that explains each skill in more detail.)

Managers evaluate each engineers’ skills by talking to team members and observing their work. Each skill is graded on this scale:

• None. The engineer doesn’t have this skill.
• Learning. The engineer is learning this skill.
• Proficient. The engineer can succeed at the skill when they concentrate on it, but it isn’t second nature.
• Fluent. The engineer uses the skill automatically, without special effort, whenever it’s appropriate.

When an employee is fluent at all the skills for a particular title (and all previous titles), they’re eligible for promotion to that title.

(We also offer step promotions, such as Software Engineer 1 to Software Engineer 2, which come when the engineer is proportionally far along their way to the next title.)

Submitted for Your Approval

Why tell you all this? Because I want your feedback. We have an early draft that we’re starting to roll out to a handful of engineers. I’m sure there are opportunities for improvement. We’ve probably forgotten some skills, or set the bar too high in some areas, or too low.

So I’d love for you to take a look and share what you think. Maybe you’ll find some of the ideas useful for your own teams, too. You can find the spreadsheet and documentation here:

• Skill documentation
• Career ladder spreadsheet (somewhat mangled)

Please share your feedback in one of these places:

• Mastodon
• LinkedIn
• Hacker News

Full Career Ladder

Here’s the full list of titles and skills. You can find descriptions of each skill in the documentation.

Associate Software Engineers

Associate Software Engineer 1s are at the start of their career. They’re expected to understand the basics of software development, and be able to work in a professional setting, but they’re mostly working under the guidance of more experienced engineers.

• Professionalism

  • Spoken and written English
  • Work ethic
  • Intrinsic motivation
  • Remote attendance
  • In-person attendance
  • Active participation
  • Respectful communication
  • Transparency
  • Team orientation
  • Follow the process
  • Grit
  • Absorb feedback
  • Growth mindset
  • OpenSesame Qualified¹

• Classroom Engineering

  • Object-oriented programming language
  • Pairing/teaming driver
  • Classroom-level debugging
  • Function and variable abstraction

¹“OpenSesame Qualified” is our internal training program.

Software Engineers

Software Engineer 1s still have a lot to learn, but they’re able to contribute to the work of their team without explicit guidance. They’re beginning to demonstrate peer leadership skills and develop their abilities as generalizing specialists.

• Basic Communication

  • Collective ownership
  • Defend a contrary stance
  • “Yes, and...”
  • Try it their way
  • Technical feedback
  • Active listening
  • As-built documentation

• Basic Leadership

  • Basic facilitation
  • Team steward
  • Valuable increment steward
  • Scut work

• Basic Product

  • Your team’s product
  • Your team’s customers and users
  • User story definition

• Basic Implementation

  • Your team’s programming language
  • Your team’s codebase
  • Basic test-driven development
  • Sociable unit tests
  • Narrow integration tests
  • End-to-end tests
  • Manual validation
  • Spike solutions
  • Basic SQL
  • Pairing/teaming navigator
  • Basic algorithms
  • Basic performance optimization
  • Debugging your team’s components
  • Simple dependency integration
  • Unhappy path thinking

• Basic Design

  • Decompose problem into tasks
  • Class abstraction
  • Mental model of your team’s codebase
  • Mental model of a complex dependency
  • Method and variable refactoring
  • Campsite rule
  • Fail fast
  • Paranoiac telemetry
  • Evaluate simple dependencies

• Basic Operations

  • Source control
  • Your team’s release process
  • On-call responsibility
  • On-call triaging
  • Issue investigation
  • Your team’s cloud infrastructure
  • Code vulnerability awareness
  • Cloud vulnerability awareness

Senior Software Engineers

Despite the name, Senior Software Engineer 1s are still fairly early in their careers. However, they have enough experience to take a strong peer leadership role in their teams. They’ve developed broader generalist skills and deeper specialist skills.

• Advanced Communication

  • Clear and concise speaking
  • Clear and concise writing
  • Technical diagramming
  • Explain mental model
  • Ensure everyone’s voice is heard
  • Coalition building
  • Interpersonal feedback
  • Runbook documentation

• Advanced Leadership

  • Peer leadership
  • Comfort with ambiguity
  • Risk management
  • Intermediate facilitation
  • Mentoring and coaching
  • Critique the process
  • Circles and soup

• Advanced Product

  • Ownership
  • Vertical slices
  • Cost/value optimization

• Advanced Implementation

  • All of your team’s programming languages
  • All of your team’s codebases
  • Codebase specialty
  • Code performance optimization
  • Complex dependency integration
  • Retrofitting tests
  • Exploratory testing

• Advanced Design

  • Codebase design
  • Simple design
  • Reflective design
  • Cross-class refactoring
  • Basic database design
  • Mental model of team dependencies
  • Evaluate complex dependencies
  • Simplify and remove dependencies

• Advanced Operations

  • Observability
  • Basic build automation
  • Basic deployment automation
  • Incident leader
  • Incident communicator
  • Incident fixer

• Senior SE Specialty

  • Choose one of the specialty skill sets listed below.

Technical Leads

Technical Leads are the backbone of a team. They combine deep expertise in several specialties with the ability to mentor and coach less experienced team members. They work closely with the team’s other technical leads to advise engineering managers on the capabilities and needs of the team. However, this remains a coding-centric role, and the majority of their time is spent as a player-coach working alongside other team members.

• Team Leadership

  • Personal authority
  • Leaderful teams
  • Leadership specialty
  • Assess technical skills
  • Assess interpersonal skills
  • Assess product skills
  • Technical interview
  • Impediment removal

• Interpersonal Leadership

  • Humility
  • Psychological safety
  • Calm the flames
  • Ignite the spark

• Product Leadership

  • Options thinking
  • Status and forecasting
  • Progress and priorities

• Design Leadership

  • Simple codebase architecture
  • Reflective codebase architecture
  • Risk-driven codebase architecture
  • Architectural refactoring
  • Published API design

• Technical Lead Specialties

  • Choose three(?) additional specialty skill sets.

Staff Engineers

Staff Engineers make a difference to the performance of Engineering as a whole. They rove between teams, cross-pollinating information and ideas. They work hands-on with each team, acting as player-coaches, bringing a breadth and depth of expertise that people are happy to learn from.

These skill sets haven’t been defined yet.

Principal Engineers

This level hasn’t been defined yet.

Specialty Skill Sets

Starting at the Senior Software Engineer level, engineers choose specialty skill sets in additional to the foundational skill sets described above. We haven’t defined these skill sets yet, but here are some of the ones we’re considering:

• Product
• Distributed systems
• Databases
• Security
• Extreme Programming
• Developer Automation
• Algorithms
• Machine Learning
• Front-End
• iOS
• Android

Feedback

Please share your thoughts!

• Mastodon
• LinkedIn
• Hacker News

• Chuck's Resume Template
• Developer Book Club 
• Become a Top 1% Dev with a Top End Devs Membership

• LinkedIn: Santosh Yadav