Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
147884 stories
·
33 followers

Can’t use Claude at work? How I recreated “Skills” in GitHub Copilot

1 Share

When Anthropic introduced Claude Skills, they demonstrated how specialized instruction sets could transform AI assistants into domain experts.

andrew evans claude copilot

But not every developer can use Claude at work. Many companies have IT restrictions or security policies that prevent the use of a large LLM like Anthropic’s. Or they may just want to avoid context-switching between a million different AI tools.

Count me as one of those devs.

As someone who uses GitHub Copilot in my daily workflow, I was curious whether I could bring a Skills-like capability to my development environment.

With Copilot, it‘s not as simple as copying prompt templates, but it can still be achieved. With some additional configuration and usage of Copilot’s Instructions file, I’ve been able to achieve a similar workflow in Copilot to what Claude Skills offers.

In this article, I’ll provide a brief intro of what Claude Skills are and then share my strategy for trying to implement similar behaviors with GitHub Copilot. If you want to see my example project, please check out my sample project on GitHub.

Why build Copilot Skills?

While Claude Skills offer powerful customization within the Claude ecosystem, many development teams have already standardized GitHub Copilot as their primary AI coding assistant. After learning about Claude Skills, I wanted to bring the same level of specialization directly into my existing workflow with Copilot.

Claude Skills allows for making agentic AI tools more efficient through context management. The core idea is to control how much context the AI needs to accomplish a goal. Controlling the context here makes responses faster and also reduces the likelihood of AI hallucination.

LLMs excel at contextual understanding and pattern matching. They start with a broad review of knowledge and then hone in on data specific to a request. As you will see in both Claude Skills and my strategy for Copilot implementation, I make the core work of LLMs more efficient through context management.

The strategy allows for the same beneficial features of Skills within an existing Copilot-based workflow.

What are Claude Skills? And why are they beneficial?

Before going into my Copilot examples, it helps to understand what Claude Skills are at a high level. Claude Skills are specialized instruction sets that extend Claude’s capabilities for specific tasks or domains.

Think of them as expert personas or detailed playbooks that Claude can reference when handling particular types of work. Each Skill contains comprehensive guidelines, best practices, and domain-specific knowledge that help Claude deliver consistently high-quality results for specialized tasks.

Skills are stored as Markdown files that Claude can access and read before tackling a task. When a user’s request aligns with a particular Skill (such as creating a presentation, analyzing a spreadsheet, or writing technical documentation), Claude automatically loads the relevant instructions and follows them throughout the interaction.

Claude Skills improve efficiency with AI tooling through modular context management. Rather than manually providing domain-specific instructions in every conversation, teams can create reusable Skills that Claude loads when relevant. This allows Claude to apply specialized expertise consistently without requiring users to repeatedly share the same context or guidelines.

It helps to understand this through an example. Imagine you’ve created a custom Code Review Skill for your development team. This Skill might contain:

  • Your organization’s specific coding standards and style guidelines
  • Common security vulnerabilities to check for in your tech stack
  • Performance optimization patterns relevant to your application architecture
  • Team-specific conventions for naming, documentation, and error handling
  • Integration with your CI/CD pipeline requirements

A sample SKILL.md file for this Code Review Skill would look like the following:

---
name: code-review
description: Use when the user asks to "do a code review", "review code", "review this file", "review this folder", or any request to analyze code for issues, review pull requests, or provide code feedback against team standards.
---

# Code Review Skill

## Purpose
This skill provides comprehensive code review capabilities aligned with our team's standards, security requirements, and architectural patterns.

## When to Use This Skill
- When reviewing pull requests or code changes
- When analyzing code for potential issues
- When providing feedback on code structure or implementation
- When checking code against team standards

## Code Review Checklist

### 1. Code Standards & Style
- **Naming Conventions**
  - Use camelCase for variables and functions (e.g., `getUserData`, `isValid`)
  - Use PascalCase for classes and components (e.g., `UserService`, `DataProcessor`)
  - Use UPPER_SNAKE_CASE for constants (e.g., `MAX_RETRY_ATTEMPTS`)
  - Avoid single-letter variables except in short loops

- **File Organization**
  - Maximum 300 lines per file
  - Group related functions together
  - Place imports at the top, organized: external libraries, internal modules, types
  - Export statements at the bottom of the file

### 2. Security Vulnerabilities
Check for these common security issues:
- SQL injection risks (always use parameterized queries)
- XSS vulnerabilities (sanitize user input before rendering)
- Hardcoded credentials or API keys (use environment variables)
- Insecure direct object references (validate user permissions)
- Missing authentication/authorization checks on sensitive endpoints

### 3. Performance Patterns
- **Database Queries**
  - Flag N+1 query patterns
  - Suggest eager loading for related data
  - Recommend indexing for frequently queried fields

- **API Calls**
  - Ensure proper caching for repeated requests
  - Check for unnecessary API calls in loops
  - Verify timeout and retry logic exists

### 4. Error Handling
- All async operations must have try-catch blocks
- Errors should be logged with context (user ID, request ID, timestamp)
- User-facing errors should never expose internal details
- Use custom error classes: `ValidationError`, `AuthenticationError`, `DatabaseError`

### 5. Testing Requirements
- Unit tests required for all business logic functions
- Integration tests required for API endpoints
- Minimum 80% code coverage for new code
- Test file naming: `[filename].test.js` or `[filename].spec.js`

### 6. Documentation Standards
- JSDoc comments required for all exported functions
- Include @param, @returns, and @throws tags
- Complex logic should have inline comments explaining the "why"
- Update README.md if public API changes

### 7. Architecture Alignment
- **Service Layer Pattern**: Business logic must be in service files, not controllers
- **Dependency Injection**: Use constructor injection for dependencies
- **Event-Driven**: Use event emitters for cross-module communication, not direct calls
- **Repository Pattern**: Database access only through repository classes

## Review Output Format
Structure your review as follows:
1. **Summary**: Brief overview of the changes and overall assessment
2. **Critical Issues**: Must be fixed before merge (security, bugs, breaking changes)
3. **Suggestions**: Improvements for code quality, performance, or maintainability
4. **Positive Feedback**: Highlight what was done well
5. **Questions**: Any clarifications needed from the author

## Example Review Comments
**Good**: "Line 45: This query could cause N+1 problem. Consider using `include` to eager load the user's orders: `User.findById(id).include('orders')`"
**Avoid**: "This is bad" or "Fix this"

## CI/CD Integration Notes
- All tests must pass before merge
- Linting errors must be resolved (ESLint configuration in `.eslintrc.js`)
- Build must succeed in staging environment
- Database migrations must be reversible

You would then place that SKILL.md file in .claude/skills/<SKILL_NAME>, where in this case <SKILL_NAME> would be code-review. The resulting structure would look something like this:

skill md file

A few notes in this example:

  • The frontmatter at the top is important because it’s what Claude Code uses to determine if/when to use the rules in the SKILL.md file.
  • The structure within the .claude folder is important for it to automatically read in the Skill as an option when doing related tasks.
  • You could ask Claude directly to reference this skill without the built-in structure, but the goal is to use Claude’s Skills and thus follow the conventions of putting it in the appropriate folder.
  • If you wanted additional Skills, you would just create them following the same pattern in the .claude folder.

With the SKILL.md in place, you can then start a Claude Code session and ask, “Can you do a review of the code of the X folder?” and it would read in that SKILL.md file as part of its work.

Here is an example interaction from my sample project where I asked it to do a review of the code in a folder called “JavaScript” in my project (with the SKILL.md) file in place:

❯ can you do a code review of the JavaScript folder?                           

⏺ Skill(code-review)                                                           
  ⎿  Successfully loaded skill                                                 

⏺ I'll start by exploring the JavaScript folder to understand its contents.    

⏺ Explore(Explore JavaScript folder structure)                                 
  ⎿  Done (4 tool uses · 11.6k tokens · 24s)                                   

⏺ Read 1 file (ctrl+o to expand)                                               

⏺ Code Review: JavaScript/auth-controller.js                                   

  Summary                                                                      

  This file is an Express.js authentication controller handling user login. It 
  has several critical security vulnerabilities that must be addressed before  
  this code is used in any environment. The file is short (23 lines) but       
  contains significant issues.                                                 

  ---                                                                          
  Critical Issues                                                              

  1. SQL Injection (Line 8-10)                                                 
  The query directly interpolates user input into the SQL string. This is the  
  most severe vulnerability — an attacker can bypass authentication or         
  extract/destroy the entire database.                                         

  // VULNERABLE                                                                
  `SELECT * FROM users WHERE username = '${username}' AND password =           
  '${password}'`                                                               

  // FIX: Use parameterized queries                                            
  db.query('SELECT * FROM users WHERE username = $1 AND password = $2',        
  [username, password])                                                        

  2. Plaintext Password Storage/Comparison (Line 9)                            
  Passwords are compared directly as plaintext in the SQL query. Passwords must
   be hashed (e.g., with bcrypt) and compared using a constant-time comparison 
  function.                                                                    

  // FIX: Hash lookup + bcrypt comparison                                      
  const user = await db.query('SELECT * FROM users WHERE username = $1',       
  [username]);                                                                 
  if (user &amp;&amp; await bcrypt.compare(password, user.password_hash)) { ... }      

  3. Insecure Token Generation (Lines 19-21)                                   
  generateToken simply concatenates 'token_' with the user ID. This is         
  trivially guessable and provides no security. Use a proper JWT library or    
  cryptographically secure token generator.                                    

  // FIX: Use JWT or crypto                                                    
  const jwt = require('jsonwebtoken');                                         
  function generateToken(id) {                                                 
    return jwt.sign({ userId: id }, process.env.JWT_SECRET, { expiresIn: '1h'  
  });                                                                          
  }                                                                            

  4. Missing Error Handling (Line 5-16)                                        
  The async handler has no try-catch block. If the database query fails, the   
  error will be unhandled, potentially crashing the process or leaking internal
   details to the client.                                                      

  router.post('/login', async (req, res) =&gt; {                                  
    try {                                                                      
      // ... logic                                                             
    } catch (err) {                                                            
      logger.error('Login failed', { error: err.message });                    
      res.status(500).json({ error: 'Internal server error' });                
    }                                                                          
  });                                                                          

  ---                                                                          
  Suggestions                                                                  

  - Input validation — Validate that username and password are present and are 
  strings before querying. Consider using a validation library or middleware.  
  - Rate limiting — Add rate limiting to the login endpoint to prevent         
  brute-force attacks.                                                         
  - Architecture — Per the team's service layer pattern, the business logic    
  (query + token generation) should be extracted into an AuthService class     
  rather than living in the controller.                                        
  - JSDoc — The generateToken function is exported implicitly through usage but
   lacks documentation. Add JSDoc with @param and @returns tags.               
  - SELECT * — Avoid selecting all columns; only retrieve what's needed (e.g., 
  id, password_hash).                                                          

  ---                                                                          
  Positive Feedback                                                            

  - The file is well under the 300-line limit and focused on a single concern. 
  - Uses async/await rather than raw promises.                                 
  - Proper use of Express Router for modular routing.                          

  ---                                                                          
  Questions                                                                    

  - Is there an existing authentication/JWT infrastructure elsewhere in the    
  project that this should integrate with?                                     
  - Is there a shared error-handling middleware this should defer to?   

As you can see from the output, Claude Code initially finds and reads the Code Review Skill, as it recognizes the phrases that indicate it would be needed. After reading the Skill, Claude Code then uses it to conduct the analysis, and responds with the review.

In the next sections, I will show a way to achieve a similar behavior with GitHub Copilot.

Claude Skills with Copilot

Now with an understanding of Claude Skills, let’s dive into my strategy to achieve a similar behavior with GitHub Copilot.

As I mentioned in my earlier section, the goal is to bring Claude Skills to workflows that are centered on Copilot. My strategy achieves bringing a similar workflow to that of Claude Skills, but in a Copilot-centered workflow. The benefit is that teams that are limited to the use of Copilot (or who want to avoid context switching) can utilize a feature similar to Claude Skills.

I first start with creating a .github folder, which is where all of my Copilot Instructions files will live. The copilot-instructions.md file would be read in anytime you start a Copilot chat and serve as an entry point when Copilot chats begin.

Within the Copilot Instructions file, I’ll next guide Copilot to refer to a second “instructions” folder with specific instructions based on what I’m doing.

The files I place in this “instructions” folder also need to have -instructions in the file name to properly be recognized by Copilot. Using these “instructions” files is similar to the way that Claude Skills work with a SKILL.md file within a folder. The only added step is that I have to basically provide an additional system prompt via the copilot-instructions.md file to reference those files when needed.

Here is the part of my copilot-instrucitons.md file where I outline usage of the additional instructions files:

## Language-Specific Guidelines

This repository uses modular Copilot instructions organized by language and framework. Language-specific guidelines are automatically applied via `.instructions.md` files in `.github/instructions/` based on the file being edited:

- **Dotnet/** - .NET Web API guidelines (`dotnet-webapi.instructions.md`)
- **React/** - TypeScript React guidelines (`typescript-react.instructions.md`)

If you’re following along with my sample project, the end project structure looks like:

.github/
├── copilot-instructions.md                              # Main instructions file (always loaded)
└── instructions/                                        # Language-specific guidelines (auto-loaded by applyTo glob)
    ├── dotnet-webapi.instructions.md                    # .NET Web API guidelines → **/Dotnet/**
    └── typescript-react.instructions.md                 # TypeScript React guidelines → **/React/**

With the configuration setup, I now have more control over the system prompts and context that Copilot reads in on every session. Instead of all of the rules for specific tasks being read in on each session, the domain-specific instructions are only read in when they’re needed.

The goal is to manage the context and make Copilot more efficient. Instead of using up the valuable context window, my Copilot sessions can be cleaner and focused on the tasks to be done.

With any agentic AI tool, controlling the context window is essential for efficiency and to have a positive developer experience. Without proper management of context, any agentic AI tool is open to inefficiencies, which leads to poor performance (slow response), hallucinations, poor cost management, and overuse of account tokens.

A visual of the Copilot workflow when asked to “code review a .NET project” with this configuration looks like:

A visual of the Copilot workflow when asked to “code review a .NET project”

If you look at my sample project, you’ll see that I’ve set it to enforce standards based on the programming language. I’ve included special instructions for (1) .NET and (2) TypeScript React as examples. Here is a portion of the .NET instructions file:

---
applyTo: '**/Dotnet/**'
---

# .NET Web API Guidelines

## Architecture &amp; Structure

- Use minimal APIs for simple endpoints, controller-based APIs for complex scenarios
- Organize code using feature folders or clean architecture layers
- Implement dependency injection for all services
- Use `Program.cs` for service registration and middleware configuration

## Best Practices

### Controllers/Endpoints

- Return `IActionResult` or `ActionResult&lt;T&gt;` from controller actions
- Use attribute routing with versioning: `[Route("api/v1/[controller]")]`
- Apply appropriate HTTP method attributes: `[HttpGet]`, `[HttpPost]`, etc.
- Use `[FromBody]`, `[FromQuery]`, `[FromRoute]` for explicit binding

If you notice, I have also included a frontmatter section at the top of the file applyTo: '**/Dotnet/**'. This frontmatter is seen by Copilot when it initially scans the .github folder and connects what is outlined in the copilot-instructions.md file to look for “Dotnet” specific tasks.

I did the same with the React instructions using applyTo: '**/React/**' for React projects.

Now I can share what it looks like with this configuration in my sample project. When I start Copilot chats, it follows the rules when I ask it to do code changes or review. Here is an example where I asked Copilot to create a new endpoint in the .NET project:

asked Copilot to create a new endpoint in the .NET project

Once it started doing the work, it referenced the correct dotnet-webapi.instructions.md file.

Similarly, with the React project, you’ll notice the same behavior if I ask it to add a Date Selector component:

Similarly, with the React project, you’ll notice the same behavior if I ask it to add a Date Selector component:

Note here in this screenshot that Copilot referenced the typescript-react.instructions.md file to conduct its review. If you are following along with the sample project, I encourage you to try to get Copilot to make code changes and see how it is using these special instructions files.

One last example would be to ask Copilot to do a review similar to how I did in the earlier section with Claude. Here is a code review for the React project:

User: Can you do a code review of the React project?

GitHub Copilot: I'll review the React project for you. Let me start by examining the files. 

Read \[\](file:///Users/andrewevans/Documents/projects/copilot-custom-development-skills/React/src/App.tsx)

Read \[\](file:///Users/andrewevans/Documents/projects/copilot-custom-development-skills/React/src/Counter.tsx)

Read \[\](file:///Users/andrewevans/Documents/projects/copilot-custom-development-skills/React/src/main.tsx)

Read \[\](file:///Users/andrewevans/Documents/projects/copilot-custom-development-skills/React/package.json)

Read \[\](file:///Users/andrewevans/Documents/projects/copilot-custom-development-skills/React/tsconfig.json)

## React Project Code Review

Here's my comprehensive review of your React project:

### ✅ Strengths

1. **TypeScript Configuration**: Solid setup with strict mode enabled, ES2020 target, and proper JSX configuration.
2. **Component Structure**: Good separation of concerns with `App` and `Counter` components.
3. **Export Pattern**: Using named exports for `Counter` is better for testability.
4. **React Version**: Modern React 18 with Hooks.
5. **Build Setup**: Vite is a great choice for fast development and optimized builds.

### ⚠ Issues &amp; Recommendations

1. **App.tsx** - Missing CSS file import
   - No styling file included, making the component unstyled. Consider adding CSS modules or Tailwind.

2. **Counter.tsx** - Potential performance issues
   - `increment` function is recreated on every render. Use `useCallback` to memoize it for consistency:
   ```tsx
   const increment = useCallback(() =&gt; setCount(count + 1), [count]);
   const clear = useCallback(() =&gt; setCount(0), []);
   ```

3. **main.tsx** - Unnecessary React import
   - Line 1 imports React but only uses `React.StrictMode`. Since you're using the `jsx: "react-jsx"` compiler option, you don't need this import.

4. **Type Safety**
   - Consider adding prop types to `App` even if it currently has no props, for consistency.

5. **Missing Features**
   - No error boundaries for error handling
   - No test files visible (consider adding Jest/Vitest tests)
   - No `.gitignore` file shown (should exclude `node_modules`, `dist`, `.env`)

6. **index.html** - Not reviewed
   - Unable to check; ensure it has proper accessibility attributes and a non-empty `&lt;div id="root"&gt;`.

### 📋 Suggested Improvements

1. Add CSS file for component styling
2. Remove unused React import from main.tsx
3. Add `useCallback` optimization to Counter
4. Create `.gitignore` if not already present
5. Add unit tests (especially for Counter increment/clear logic)
6. Consider adding a lint configuration (ESLint) - check if one exists in the root

Would you like me to implement any of these fixes?

In all cases, the special instructions files that I have set up are consulted in Copilot sessions. This example project is very simple, but you could imagine expanding this to more detailed tasks for team-specific workflows.

Other ways to achieve Claude-like skills in Copilot

In addition to the strategy I’ve outlined, the Microsoft Copilot ecosystem offers some other options for features similar to Claude Skills.

The Microsoft Copilot ecosystem offers some features that do not require the use of instruction files. These options are for individuals who want to build workflows integrated within Microsoft products, as opposed to software development tasks.

Copilot Studio

Microsoft offers Copilot Studio as a way to create custom AI agents with specialized domain knowledge and behaviors. With Copilot Studio, you can develop workflows for things like document uploads, SharePoint Connections, and linking to websites.

Inside Copilot Studio, you can define custom topics and conversation flows and integrate with Microsoft 365 apps like Teams, Outlook, and SharePoint. You can also create custom actions and plugins using Power Automate or API connectors.

Declarative agents

Declarative Agents for Microsoft 365 Copilot enable lightweight customizations through the use of a manifest file. With Declarative Agents, you can define custom instructions and knowledge sources (like SharePoint or OneDrive) along with integrated apps and actions. Declarative Agents allow you to build personalized experiences for business processes.

Copilot Connectors

Microsoft 365 Copilot Connectors allow you to extend Microsoft 365 Copilot with the use of plugins to external data sources and services. Copilot Connectors can use semantic indexing to provide more meaningful and efficient data retrieval.

A retrospective: What I learned through this exercise

Reflecting on my experience with this strategy, I wanted to callout a few things that I’ve observed when trying to work with this strategy.

First, I had mixed experiences with the instruction files being automatically loaded. Generally, they seemed to load, but there were occasions where Copilot ignored them. In a Copilot chat, you can tell it to directly reference those files in its solutions.

Part of the initial issue was properly naming the files with the -instructions name, and calling them out in the copilot-instructions.md file. Another part of the issue was that at times, Copilot would ignore the instruction files for more generic requests that it may already have knowledge of. Through some trial and error, you could tweak the instruction files to look for more specific phrases that would help steer the AI toward what you wanted done.

Second, the sample project I created is simplistic and only meant as an example. A more realistic solution would include more pointed descriptions in the copilot-instructions.md file.

Development teams will have unique processes that they use for delivery. The point of my sample project was just to showcase how this works. But in the real world, a more robust solution would be required for good results.

Based on my experiences, I generally think this approach would work well if your team is centered around Copilot based workflows.

Both Copilot and Claude provide a good range of customization options and work with many different types of workflows. The workflow I’ve presented takes advantage of that flexibility and provides a good workflow for those who mainly use Copilot but want the benefit of Claude Skills.

Wrapping Up

Overall, I hope you found some value in the strategy and examples I’ve shared in this post. As I worked through this strategy, I learned quite a bit about how Copilot uses system prompts and instruction files.

I also learned more about how to manage context with AI tooling. One of the key things about maximizing Agentic AI tool use is to manage context.

I have additionally written other articles that cover similar concepts, including:

I encourage you to look at those articles for more ways you can improve your experience with Agentic AI tools.

I also encourage you to play with my sample project, and look at the different interactions with the instructions files. Hopefully, this post helped you on your journey of using Agentic AI tools. Thanks for reading!

The post Can’t use Claude at work? How I recreated “Skills” in GitHub Copilot appeared first on LogRocket Blog.

Read the whole story
alvinashcraft
just a second ago
reply
Pennsylvania, USA
Share this story
Delete

Things That Caught My Attention Last Week - February 15

1 Share

caught-my-i

Open-source

Welcome to the Eternal September of open source. Here's what we plan to do for maintainers by Ashley Wolf

.NET

On .NET Live: Lessons Learned Building Source Generators - YouTube by Jason Bock

.NET 11 Preview 1 is now available! by .NET Team

Unlock language-specific rich symbol context using new find_symbol tool by Sinem Akinci

Getting Started with .NET Scheduling in Hangfire by Barret Black

Getting more information in MSBuild binlogs with property tracking by Gérald Barré

ASP.NET Core warmup EF Core by Karen Payne

2 Weeks of Claude Code for Me ΓÇô The Shade Tree Developer by Jeremy D. Miller

How to Extract Structured Data From Images Using Ollama in .NET by Milan Jovanović

Azure

Subscribe E2/2026 - Microsoft Fabric Real-Time Intelligence - YouTube by Clemens Vasters

Deploy to Azure App Service deployment slots with azd by PuiChee (PC) Chan

Maximize Azure Cosmos DB Performance with Azure Advisor Recommendations by Iria Osara

February Patches for Azure DevOps Server by Gloridel Morales

Software Development

Enhanced developer tools on the Microsoft Store by Windows Developer Blog

Soft Delete Is a Workaround by Golo Roden

AI

An AI Agent Published a Hit Piece on Me by Scott Shambaugh

Llms Need Mark as Answer by Ardalis (Steve Smith)

Shortwave: Human-centered AI is not a new idea by Mike Amundsen

Read the whole story
alvinashcraft
19 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Notes on clarifying man pages

1 Share

Hello! After spending some time working on the Git man pages last year, I’ve been thinking a little more about what makes a good man page.

I’ve spent a lot of time writing cheat sheets for tools (tcpdump, git, dig, etc) which have a man page as their primary documentation. This is because I often find the man pages hard to navigate to get the information I want.

Lately I’ve wondering – could the man page itself have an amazing cheat sheet in it? What might make a man page easier to use? I’m still very early in thinking about this but I wanted to write down some quick notes.

I asked some people on Mastodon for their favourite man pages, and here are some examples of interesting things I saw on those man pages.

an OPTIONS SUMMARY

If you’ve read a lot of man pages you’ve probably seen something like this in the SYNOPSIS: once you’re listing almost the entire alphabet, it’s hard

ls [-@ABCFGHILOPRSTUWabcdefghiklmnopqrstuvwxy1%,]

grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz]

The rsync man page has a solution I’ve never seen before: it keeps its SYNOPSIS very terse, like this:

 Local:
     rsync [OPTION...] SRC... [DEST]

and then has an “OPTIONS SUMMARY” section with a 1-line summary of each option, like this:

--verbose, -v            increase verbosity
--info=FLAGS             fine-grained informational verbosity
--debug=FLAGS            fine-grained debug verbosity
--stderr=e|a|c           change stderr output mode (default: errors)
--quiet, -q              suppress non-error messages
--no-motd                suppress daemon-mode MOTD

Then later there’s the usual OPTIONS section with a full description of each option.

an OPTIONS section organized by category

The strace man page organizes its options by category (like “General”, “Startup”, “Tracing”, and “Filtering”, “Output Format”) instead of alphabetically.

As an experiment I tried to take the grep man page and make an “OPTIONS SUMMARY” section grouped by category, you can see the results here. I’m not sure what I think of the results but it was a fun exercise. When I was writing that I was thinking about how I can never remember the name of the -l grep option. It always takes me what feels like forever to find it in the man page and I was trying to think of what structure would make it easier for me to find. Maybe categories?

a cheat sheet

A couple of people pointed me to the suite of Perl man pages (perlfunc, perlre, etc), and one thing I noticed was man perlcheat, which has cheat sheet sections like this:

 SYNTAX
 foreach (LIST) { }     for (a;b;c) { }
 while   (e) { }        until (e)   { }
 if      (e) { } elsif (e) { } else { }
 unless  (e) { } elsif (e) { } else { }
 given   (e) { when (e) {} default {} }

I think this is so cool and it makes me wonder if there are other ways to write condensed ASCII 80-character-wide cheat sheets for use in man pages.

This isn’t a property of the man page itself, but one issue with man pages in the terminal is it’s hard to know what sections the man page has.

When working on the Git man pages, one thing Marie and I did was to add a table of contents to the sidebar of the HTML versions of the man pages hosted on the Git site.

I’d also like to add more hyperlinks to the HTML versions of the Git man pages at some point, so that you can click on “INCOMPATIBLE OPTIONS” to get to that section. It’s very easy to add links like this in the Git project since Git’s man pages are generated with AsciiDoc.

I think adding a table of contents and adding internal hyperlinks is kind of a nice middle ground where we can make some improvements to the man page format (in the HTML version of the man page at least) without maintaining a totally different form of documentation. Though for this to work you do need to set up a toolchain like Git’s AsciiDoc system.

examples for every option

The curl man page has examples for every option, and there’s also a table of contents on the HTML version so you can more easily jump to the option you’re interested in.

For instance the example for --cert makes it easy to see that you likely also want to pass the --key option, like this:

  curl --cert certfile --key keyfile https://example.com

The way they implement this is that there’s [one file for each option](https://github.com/curl/curl/blob/dc08922a61efe546b318daf964514ffbf41583 25/docs/cmdline-opts/append.md) and there’s an “Example” field in that file.

formatting data in a table

Quite a few people said that man ascii was their favourite man page, which looks like this:

 Oct   Dec   Hex   Char                     
 ───────────────────────────────────────────
 000   0     00    NUL '\0' (null character)
 001   1     01    SOH (start of heading)   
 002   2     02    STX (start of text)      
 003   3     03    ETX (end of text)        
 004   4     04    EOT (end of transmission)
 005   5     05    ENQ (enquiry)            
 006   6     06    ACK (acknowledge)        
 007   7     07    BEL '\a' (bell)          
 010   8     08    BS  '\b' (backspace)     
 011   9     09    HT  '\t' (horizontal tab)
 012   10    0A    LF  '\n' (new line)      

Obviously man ascii is an unusual man page but I think what’s cool about this man page (other than the fact that it’s always useful to have an ASCII reference) is it’s very easy to scan to find the information you need because of the table format. It makes me wonder if there are more opportunities to display information in a “table” in a man page to make it easier to scan.

the GNU approach

When I talk about man pages it often comes up that the GNU coreutils man pages (for example man tail) don’t have examples, unlike the OpenBSD man pages, which do have examples.

I’m not going to get into this too much because it seems like a fairly political topic and I definitely can’t do it justice here, but here are some things I believe to be true:

  • The GNU project prefers to maintain documentation in “info” manuals instead of man pages. This page says “the man pages are no longer being maintained”.
  • There are 3 ways to read “info” manuals: their HTML version, in Emacs, or with a standalone info tool. I’ve heard from some Emacs users that they like the Emacs info browser. I don’t think I’ve ever talked to anyone who uses the standalone info tool.
  • The info manual entry for tail is linked at the bottom of the man page, and it does have examples
  • The FSF used to sell print books of the GNU software manuals (and maybe they still do sometimes?)

After a certain level of complexity a man page gets really hard to navigate: while I’ve never used the coreutils info manual and probably won’t, I would almost certainly prefer to use the GNU Bash reference manual or the The GNU C Library Reference Manual via their HTML documentation rather than through a man page.

a few more man-page-adjacent things

Here are some tools I think are interesting:

  • The fish shell comes with a Python script to automatically generate tab completions from man pages
  • tldr.sh is a community maintained database of examples, for example you can run it as tldr grep. Lots of people have told me they find it useful.
  • the Dash Mac docs browser has a nice man page viewer in it. I still use the terminal man page viewer but I like that it includes a table of contents, it looks like this:

it’s interesting to think about a constrained format

Man pages are such a constrained format and it’s fun to think about what you can do with such limited formatting options. I’d be interested to hear about other man pages that you think are well designed and what you like about them, the comments section is here.

Read the whole story
alvinashcraft
32 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Explaining why throughput varies for Postgres with a CPU-bound Insert Benchmark

1 Share

Throughput for the write-heavy steps of the Insert Benchmark look like a distorted sine wave with Postgres on CPU-bound workloads but not on IO-bound workloads. For the CPU-bound workloads the chart for max response time at N-second intervals for inserts is flat but for deletes it looks like the distorted sine wave. To see the chart for deletes, scroll down from here. So this looks like a problem for deletes and this post starts to explain that.

tl;dr

  • Once again, blame vacuum

History of the Insert Benchmark

Long ago (prior to 2010) the Insert Benchmark was published by Tokutek to highlight things that the TokuDB storage engine was great at. I was working on MySQL at Google at the time and the benchmark was useful to me, however it was written in C++. While the Insert Benchmark is great at showing the benefits of an LSM storage engine, this was years before MyRocks and I was only doing InnoDB at the time, on spinning disks. So I rewrote it in Python to make it easier to modify, and then the Tokutek team improved a few things about my rewrite, and I have been enhancing it slowly since then.

Until a few years ago the steps of the benchmark were:

  • load - insert in PK order
  • create 3 secondary indexes
  • do more inserts as fast as possible
  • do rate-limited inserts concurrent with range and point queries
The problem with this approach is that the database size grows forever and that limited for how long I could run the benchmark before running out of storage. So I changed it and the new approach keeps the database at a fixed size after the load. The new workflow is:
  • load - insert in PK order
  • create 3 secondary indexes
  • do inserts+deletes at the same rate, as fast as possible
  • do rate-limited inserts+deletes at the same rate concurrent with range and point queries
The insert and delete statements run at the same rate to keep the table from changing size. The Insert Benchmark client uses Python multiprocessing, there is one process doing Insert statements, another doing Delete statements and both get their work from queues. Another process populates those queues and that other process controlling what is put on the queue is what keeps them running at the same rate.

The benchmark treats the table like a queue, and when ordered by PK (transactionid) there are inserts at the high end and deletes at the low end. The delete statement currently looks like:
    delete from %s where transactionid in
        (select transactionid from %s where transactionid >= %d order by transactionid asc limit %d)

The delete statement is written like that because it must delete the oldest rows -- the ones that have the smallest value for transactionid. While the process that does deletes has some idea of what that smallest value is, it doesn't know it for sure, thus the query. To improve performance it maintains a guess for the value that will be <= the real minimum and it updates that guess over time.

I encountered other performance problems with Postgres while figuring out how to maintain that guess and get_actual_variable_range() in Postgres was the problem. Maintaining that guess requires a resync query every N seconds where the resync query is: select min(transactionid) from %s. The problem for this query in general is that is scans the low end of the PK index on transactionid and when vacuum hasn't been done recently, then it will scan and skip many entries that aren't visible (wasting much CPU and some IO) before finding visible rows. Unfortunately, there will be some time between consecutive vacuums to the same table and this problem can't be avoided. The result is that the response time for the query increases a lot in between vacuums. For more on how get_actual_variable_range() contributes to this problem, see this post.

I assume the sine wave for delete response time is caused by one or both of:
  • get_actual_varable_range() CPU overhead while planning the delete statement
  • CPU overhead from scanning and skipping tombstones while executing the select subquery
The structure of the delete statement above reduces the number of tombstones that the select subquery might encounter by specifying where transactionid >= %d. Perhaps that isn't sufficient. Perhaps the Postgres query planner still has too much CPU overhead from get_actual_variable_range() while planning that delete statement. I have yet to figure that out. But I have figured out that vacuum is a frequent source of problems.


    Read the whole story
    alvinashcraft
    1 minute ago
    reply
    Pennsylvania, USA
    Share this story
    Delete

    Announcing New Optional Windows 11 Insider Preview Build for Canary Channel 29531.1000

    1 Share
    Hello Windows Insiders, today we’re releasing a new optional update to the Windows 11 Insider Canary Channel, continuing our work to bring you early access to platform changes as they take shape. As part of this ongoing development, Canary Channel will move forward on two update paths. This will help us validate platform changes at different stages while continuing to deliver new features and experiences to Insiders in the Canary channel.

    What this means for Canary Channel Insiders

    Today, Windows Insiders in the Canary Channel already have available to preview Windows 11 Insider Preview Build 28020.1611. This build path will continue to focus on previewing new features and experiences in Canary for Windows 11, version 26H1, and moving forward will continue to be identified as a 28000 series build. Available starting today, Insiders can take an optional update under Settings > Windows Update > Advanced options > Optional updates which will move your device to a new active development Build 29531.1000. Once you opt in to this path, future builds will be identified as 29500 series builds and newer. Please note, because of the focus on platform development for this path, you may notice a temporary loss in some features that you have today. These features will return to this new active development build. Going forward, each Canary Channel build will be announced in its own blog post, so you’ll always know exactly what’s new—and what to expect—based on the build you’re receiving. You can always confirm which build you’re running by typing winver into the Windows search box and selecting About Windows. This will show your exact build number and version.

    Choosing the right path for you

    We generally recommend that Insiders continue on the update path they’re already on, as this helps maintain stability and predictability within each build population. That said, some Insiders—especially those who prefer getting the newest platform changes as early as possible—may choose to move to a newer development path when available. This flexibility allows us to experiment, learn quickly, and incorporate feedback before changes move further along in development. As a reminder - Insiders can’t switch to a channel that is receiving builds with lower build numbers without doing a clean installation of Windows 11 due to technical setup requirements. This means Insiders can’t go back to 28000 series builds after taking the optional update.

    What’s new in Canary Build 29531.1000

    Changes and Improvements

    • This update includes platform changes in moving to a new active development build.

    Reminders for Windows Insiders in the Canary Channel

    • These builds can be unstable and may be released with limited documentation. Join Windows Insider – Get early access to Windows 11 features & updates
    • The builds we release to the Canary Channel represent the latest platform changes early in the development cycle and should not be seen as matched to any specific release of Windows. Features and experiences included in these builds may never get released as we try out different concepts and get feedback. Features may change over time, be removed, or replaced and never get released beyond Windows Insiders. Some of these features and experiences could show up in future Windows releases when they’re ready.
    • Many features in the Canary Channel are rolled out using Control Feature Rollout technology, starting with a subset of Insiders and ramping up over time as we monitor feedback to see how they land before pushing them out to everyone in this channel.
    • The desktop watermark shown at the lower right corner of the desktop is normal for Windows Insider pre-release builds.
    • Some features may show up in the Dev and Beta Channels first before showing up in the Canary Channel.
    • Some features in active development we preview with Windows Insiders may not be fully localized and localization will happen over time as features are finalized. As you see issues with localization in your language, please report those issues to us via Feedback Hub.
    • To get off the Canary Channel, a clean install of Windows 11 will be required.
    • Check out Flight Hub for a complete look at what build is in which Insider channel.
    Thanks, Windows Insider Program Team
    Read the whole story
    alvinashcraft
    2 minutes ago
    reply
    Pennsylvania, USA
    Share this story
    Delete

    Mastodon, a decentralized alternative to X, plans to target creators with new features

    1 Share
    Mastodon is looking to grow its open source, decentralized social network with new features aimed at creators.
    Read the whole story
    alvinashcraft
    4 hours ago
    reply
    Pennsylvania, USA
    Share this story
    Delete
    Next Page of Stories