Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
159231 stories
·
33 followers

“Everything is High Priority”

1 Share

In recent times, I’ve personally and vicariously encountered a reoccurring situation concerning competing needs and tasks. It can be summarized to lack of prioritization. Fundamentally, I think we all understand prioritization. If we need to address problems in a house to prepare for a visit, and the house has a door that needs to be repaired, an electrical socket that shorts out, and a fire on the stove it is pretty easy to decide what problem will be addressed first. The consequences of some problems increase with time. The fire will get more unwieldly, a bill will incur fees, the door may allow unwanted water or insects to enter a structure. These consequences differ in severity and growth speed.

The problem I and others encounter is that the use of the word “priority” is being used not to express some ranking of importance, but to express something that they wish to have done. There is a wish for all things to be done coupled with the lack of resources (especially time or manpower resources) to do it and meet deadlines. This creates tensions and problems. There may be times when conditions on a project result in this problem as a temporary and short-lived problem. But there are also groups for which these conditions are their mode of operation. Trends in differences in how I see these problems dealt with correlate to age and work experience among the people through which I’ve observed them. For the younger and less experienced workers, this creates unmanageable conditions is misery as they find their life being consumed by these tasks taking up more hours, after-hours, and weekends. The “solution” to these problems have been to find another job (if there is time to find one). That’s not an entirely unreasonable solution; sometimes these conditions are a reflection of other chaos within an organization.

Older or more experienced employees may do the same, or they may give reasonable pushback. Someone I spoke with was out of town to perform a deployment for a marketing event at some sports event. A lot of time was spent for setup, walking the client through the parts of the event and fixtures, interacting with media, and keeping hired staff in line. While this was going on people in the office, though aware of the event, were sending request for work to be done with deadlines of the next day. The person asked how they should prioritize these requests and got back the “everything is a priority.” The worker let them know “not everything will be done.” The reasonable refusal to satisfy all requests was a forcing function that resulted in the requests getting rankings.

I most recently encountered a similar situation when I was preparing to leave town for a project. For a previous project, a decision was made to change the requirements (something that happened a lot on that project). On the Sunday that I was going to use to finish packing and get things in order at home I found myself on an unscheduled team meeting. I hadn’t planned to be on a 9am meeting on a Sunday, or for that meeting to last 5 hours. At the end of that 5 hours there were changes that I needed to implement and other requests. There was a request that when I get done with the changes, that we have another meeting so that we could live-test them and make alterations during a meeting. I gave a polite refusal and plainly stated that the meeting had already caused disturbances to personal activities and activities for my other project. I told them I wouldn’t be participating in their live testing and let them know that their full list of requests will not be addressed today or this week. This, once again, gave the necessary motivation to think about importance rankings.

My personal and vicarious experiences might not be representative of wider trends. But they do form my views. I am getting the impression that “priority” is being used more as corporate jargon. That it is being used in a way that often differs from the conventional usage of the word is a bit disappointing. But it appears that helping someone understand that all desires being satisfied is not an option (and being steadfast in doing so) is a solution for motivating others to participate in ranking priorities.


Posts may contain products with affiliate links. When you make purchases using these links, we receive a small commission at no extra cost to you. Thank you for your support.

Bluesky: @j2i.net
YouTube: @j2inet
Mastodon: @j2inet@masto.ai
Instagram: @j2inet
Facebook: @j2inet
Telegram: j2inet



Read the whole story
alvinashcraft
35 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Optimising DAX: Practical Examples

1 Share

Hello again, and welcome to the final post in the Optimising DAX series! We've covered VertiPaq storage, encoding, cardinality, relationships, model design, the two engines, data materialisation, and callbacks. Now it's time to put it all into practice with some concrete examples.

The Classic CALCULATE Trap

This was one of the most impactful examples from the workshop. Compare these two expressions:

-- Slow: forces table materialisation
CALCULATE([A Measure], FILTER(Table1, Table1[ColumnA] = "value"))

-- Fast: uses a bitmap filter
CALCULATE([A Measure], Table1[ColumnA] = "value")

The first version uses FILTER with an explicit table reference. The storage engine doesn't know which columns the FILTER might need, so it materialises the entire table and hands it to the formula engine.

The second version uses a direct column filter, which the storage engine handles with an efficient bitmap. It supports <, >, =, !=, and IN operators.

The DAX looks almost identical. The performance difference can be enormous. This is exactly the kind of thing that's very easy to miss if you don't understand the engine architecture - and exactly the kind of thing that suddenly seems obvious once you do.

Variables and IF.EAGER

Watch out for patterns where you end up scanning large tables multiple times. A common case:

IF([Total Sales] > 1000, [Total Sales], BLANK())

Here, [Total Sales] might be evaluated twice - once for the condition and once for the result. Two scans of the same data. You can fix this with a variable:

VAR _sales = [Total Sales]
RETURN IF(_sales > 1000, _sales, BLANK())

Or by using IF.EAGER, which evaluates both branches upfront:

IF.EAGER([Total Sales] > 1000, [Total Sales], BLANK())

Both approaches prevent the duplicate scan.

The Hidden Cost of Slicers

Here's a fun one (or not, depending on your perspective): every time Power BI renders a slicer, it runs a DISTINCT() scan on the underlying column to populate the list of values.

If you're using a flat table model, that scan has to traverse the entire (potentially enormous) table. With a star schema, it only scans the much smaller dimension table.

This is yet another point in favour of star schemas, and it's a cost that's easy to overlook because it happens automatically - for every slicer, on every page load.

Isolating Slow Queries

When you need to actually track down a performance problem, here's the approach that was recommended:

Start in Power BI using the Performance Analyzer to identify slow visuals. The timings show three sections:

DAX Query - the time spent executing the query. This is the bit you can optimise. Visual Display - the time spent actually creating the visual (unavoidable). Other - this is usually by far the longest and is almost entirely time spent waiting in a queue to execute the query. The only real way to reduce this is to reduce the number of visuals on the page.

Move to DAX Studio once you've found a slow visual. Copy the query, enable Server Timings and Query Plan, and start deleting bits until it's fast. This isolates exactly which part is causing the bottleneck. It's a bit tedious, but it works.

Wrapping Up

And that's the series! If you've made it all the way through - well done, and thank you for bearing with me. I really enjoyed this workshop and getting my head around what's going on under the hood. Even in scenarios where I might not need to actively optimise, I think it changes how you think about writing DAX, and that's definitely a good thing!

A doodlegram of me with a cup of tea, having finally finished writing this series



Read the whole story
alvinashcraft
35 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Data Analysis Methods: Qualitative vs. Quantitative

1 Share

Data analysis methods determine how teams extract meaning from raw data such as user behavior logs, application telemetry, and customer feedback.

Two common approaches to analyzing data are qualitative and quantitative analysis. Each method offers different techniques for interpreting and understanding your findings.

This blog post will explore qualitative and quantitative data analysis methods, their strengths and limitations, and how they apply to modern application development. Whether you’re building data-driven applications, selecting a database platform, or designing an analytics architecture, understanding these approaches will help you choose the right analysis techniques and infrastructure to deliver timely, actionable insights.

What are data analysis methods?

Data analysis methods are structured approaches for examining, organizing, and interpreting data to identify patterns, answer questions, and draw meaningful conclusions. The right data analysis methods depend on your goals, the type of data you collect, and how quickly you need insights.

Most data analysis techniques fall into three categories: qualitative, which analyzes non-numerical data such as interviews or open-ended feedback; quantitative, which uses numerical data and statistical analysis; and mixed methods (or hybrid), which combines both approaches for a more complete view. In modern application environments, timing is also a key consideration, with organizations choosing between real-time analysis for immediate insights and batch analysis for processing larger datasets on a scheduled basis.

Why data analysis matters for application teams

For application teams, data analysis turns telemetry, logs, metrics, and user behavior into actionable insights that improve reliability, performance, and user experience. By analyzing data in real time or over longer periods, developers and operators can detect anomalies, troubleshoot issues faster, optimize application performance, and make more informed operational decisions.

A typical data analysis workflow starts with defining the objective, collecting and preparing data, and exploring it for patterns. Teams then apply the appropriate analysis techniques, interpret the results, validate their findings, and iterate as new data becomes available. This process creates a continuous feedback loop for improving applications and services.

Qualitative vs. quantitative data: Key differences

Here are the key differences between qualitative data analysis and quantitative data analysis.

Qualitative dataQuantitative data
Nature of dataConsists of non-numerical or categorical information, such as descriptions, opinions, observations, or narratives. Focuses on capturing subjective or qualitative aspects of a phenomenon.Comprises numerical information that can be measured or counted. Deals with objective or quantitative aspects of a phenomenon.
Data representationTypically represented as words, texts, images, or code, and can be organized into categories, themes, or patterns.Represented as numbers or numerical values, and can be organized into tables, graphs, charts, or statistical summaries.
Data collection methodsCollected through interviews, focus groups, observations, or open-ended survey questions.Collected through surveys, experiments, or structured observations.
Data analysis approachInvolves analyzing data thematically or by identifying patterns, themes, or commonalities. Techniques such as coding, content analysis, and discourse analysis are commonly used.Involves analyzing data using statistical techniques. Focuses on numerical relationships, patterns, or trends, and involves computations, statistical tests, and modeling.
Outcome and generalizabilityProvides in-depth understanding, rich descriptions, and contextual insights. Findings may be specific to the studied context and are not easily generalized to a larger population.Provides numerical measurements, statistical relationships, and quantifiable results. Findings can be generalized to a larger population within a certain level of confidence.
Examples of application dataIncludes user reviews, support tickets, session recordings, interview feedback, and open-ended survey responses.Includes request counts, error rates, latency metrics, response times, resource utilization, and conversion rates.

Mixed-methods approaches combine qualitative data analysis and quantitative data analysis to provide a more complete understanding of how applications perform and how users experience them. For product teams, combining both methods leads to more confident decisions by validating metrics with real user feedback and adding context to operational data. This approach is especially valuable when prioritizing new features, improving user experiences, diagnosing production issues, or measuring the impact of product changes across technical and business outcomes.

Qualitative data analysis methods

Qualitative data analysis involves examining non-numerical or categorical information to uncover patterns, themes, and meanings. Here are some commonly used methods for analyzing qualitative data:

Thematic analysis: Identifies recurring themes or patterns in qualitative data by categorizing and coding the data. Example: Analyzing user feedback, app store reviews, and survey responses to uncover common usability issues or feature requests.

Content analysis: Systematically analyzes textual data by categorizing and coding it to identify patterns and concepts. Example: Tagging support tickets by issue type to identify recurring product defects, documentation gaps, or customer pain points.

Narrative analysis: Examines stories or narratives to understand experiences, perspectives, and meanings. Example: Synthesizing user interviews during product discovery to understand customer workflows, motivations, and challenges.

Grounded theory: Develops theories or frameworks based on systematically collected and analyzed data, guiding theory development through the analysis process. Example: Analyzing customer interviews and product usage observations to develop a new user journey model or identify previously unknown reasons for feature adoption or abandonment.

When understanding why users behave a certain way isn’t enough, quantitative analysis helps measure what is happening, how often it occurs, and whether observed patterns are statistically significant across users or applications.

Quantitative data analysis methods

Quantitative data analysis involves analyzing numerical data to uncover statistical patterns, relationships, and trends. Here are some commonly used data analysis techniques for quantitative data:

Descriptive statistics: Summarizes dataset features using mean, median, mode, standard deviation, and percentages. Example: Calculating average API response times, median page load times, error-rate percentages, or daily active users to understand overall application performance and usage.

Inferential statistics: Draws conclusions about a population based on sample data using hypothesis testing, t-tests, and regression analysis. Example: Evaluating A/B test results to determine whether a new feature significantly improves user engagement, conversion rates, or task completion times.

Data mining: Discovers patterns and correlations in large datasets using algorithms and statistical techniques. Example: Analyzing application telemetry and user behavior data to identify common navigation paths, predict customer churn, or detect anomalous activity.

Experimental design: Designs controlled experiments to determine causal relationships between variables. Example: Running controlled feature rollouts or infrastructure experiments to measure the impact of code changes, UI updates, or caching strategies on performance and user behavior.

Time-series analysis: Examines how metrics change over time to identify trends, seasonal patterns, or anomalies. Example: Tracking latency, request volume, error rates, or API response times over days, weeks, or months to monitor application performance, identify regressions, and support SLA reporting. 

Real-time aggregation: Continuously calculates metrics such as counts, averages, percentiles, or anomaly scores as events are generated. Example: Aggregating live application telemetry to detect traffic spikes, monitor active users, identify increases in error rate, and trigger alerts before issues affect customers.

These are just a few examples of the data analysis methods used for qualitative and quantitative data. The choice of method depends on the research objectives, the type of data, the available resources, and the specific questions to be addressed. Researchers often employ a combination of methods to gain a comprehensive understanding of the data and draw meaningful conclusions.

Data analysis methods for real-time and operational data

Traditional batch analytics processes data after it has been extracted, transformed, and loaded (ETL), making it well suited for historical reporting and long-term trend analysis. Real-time data analytics, by contrast, analyzes data as it is generated, enabling applications to make decisions while transactions are still in progress. Instead of waiting for scheduled jobs to run, operational analytics enables applications to act on live data within milliseconds or seconds.

This shift is especially important for modern applications that depend on immediate insights:

  • User-facing personalization: Recommendation engines, dynamic pricing, and personalized content must analyze live user behavior while meeting strict latency requirements to ensure users receive relevant experiences without added delays.
  • Fraud and anomaly detection: Financial services, e-commerce, and security applications continuously compare incoming transactions against historical patterns and behavioral models to identify suspicious activity before it causes damage.
  • Application observability: Engineering teams aggregate logs, metrics, traces, and events across distributed services in real time to detect performance regressions, troubleshoot incidents, and maintain service-level objectives (SLOs). 

These use cases rely on operational data analytics, with analytical queries running directly against current operational data instead of waiting for data to be copied into a separate analytics platform. This approach reduces data movement, shortens the time between events and insights, and enables faster operational decisions.

Modern database analytics platforms support this model by combining transactional and analytical capabilities. For example, Couchbase Analytics uses a columnar analytics service that runs complex analytical queries on live operational data without affecting transactional workload performance.

Analyzing JSON and semi-structured data

As applications increasingly store data as JSON documents rather than rows and columns, analysis techniques must adapt to more flexible data models. Unlike traditional relational databases, JSON documents can vary in structure from one record to the next, making semi-structured data analytics more dynamic and reducing the need for rigid preprocessing or schema normalization before analysis.

This flexibility changes how teams perform JSON analytics and NoSQL analytics in several important ways:

  • Nested objects and arrays require path-aware queries. Instead of joining multiple normalized tables, analysts query nested fields and arrays directly within JSON documents, simplifying access to related data while requiring query languages that understand document structures.
  • Schema evolution must be expected. As applications evolve, new fields may be added while older documents lack them. Analytical queries need to gracefully handle missing or optional fields without requiring costly schema migrations.
  • Document-level context changes aggregation patterns. Because related information is often stored together in a single document, many analyses require fewer joins compared to relational databases. Instead, aggregations frequently operate across nested collections and document hierarchies to generate insights.

Couchbase supports SQL++ analytics, which extends familiar SQL syntax to work with JSON documents. Rather than forcing developers to flatten data into relational tables, SQL++ enables analysts to query nested objects, arrays, and evolving schemas using SQL-like statements. This makes JSON analytics more approachable while preserving the flexibility of document-oriented data models.

Choosing the right data analysis methodology

The best data analysis methodologies depend on three primary factors: the type of data you’re analyzing, how quickly you need insights, and the kind of outcome you’re trying to achieve. Rather than selecting a single approach, many teams combine multiple data analysis methods to answer different questions throughout the application lifecycle.

Use the following decision framework when selecting data analysis techniques:

Decision criteriaBest fit
Data typeUse quantitative methods for structured, numerical data such as metrics, logs, and telemetry. Use qualitative methods to analyze unstructured information, such as user feedback, support tickets, interviews, and session recordings.
Timing requirementsChoose real-time analysis when applications need immediate decisions, such as fraud detection, personalization, or observability. Choose batch or historical analysis for reporting, trend analysis, forecasting, and long-term planning.
Outcome typeUse qualitative analysis to explore and understand why users behave as they do and to identify emerging themes. Use quantitative analysis when you need statistical measurement, performance benchmarking, hypothesis testing, or validation at scale.

In practice, most production application environments use a mixed-methods approach. Product teams often rely on qualitative analysis to understand customer needs, usability issues, and feature requests, while engineering and operations teams use quantitative analysis to measure application performance, monitor usage patterns, and validate the impact of changes. Combining both approaches provides a more complete picture of application health and user experience than either method alone.

Common data analysis obstacles

You’ll likely encounter obstacles to obtaining accurate and meaningful insights during data analysis. Here are some common issues:

Poor data quality: Avoid critical data quality issues by carefully cleaning and preprocessing your data.

Insufficient or unrepresentative data: If the data collected doesn’t cover the relevant variables or lacks diversity, the insights obtained may be limited or biased. 

Lack of domain knowledge: Data analysis often requires domain knowledge to interpret the results accurately. Without a thorough understanding of the subject matter, it can be challenging to identify relevant patterns or relationships in the data.

Complexity and volume of data: Large, complex datasets can pose challenges for processing, analysis, and interpretation. Analyzing such data requires advanced techniques and tools to handle the volume and complexity effectively.

Schema variability: Semi-structured and JSON data often evolve over time, making rigid preprocessing pipelines difficult to maintain. Flexible data processing and query techniques are essential for analyzing changing schemas without constant rework.

Operational and analytical data silos: Moving operational data into separate analytics systems can introduce latency and increase data management complexity. Database-native analytics can reduce data movement and enable faster insights from live operational data.

Overcoming these obstacles requires careful attention to data quality, ensuring representative data, acquiring domain knowledge, utilizing appropriate tools and techniques, and minimizing data movement. By addressing these challenges, data analysts can enhance the reliability and validity of their data analysis methods, leading to more accurate and insightful results.

Key takeaways 

  • Data analysis methods help organizations turn raw data into actionable insights that improve decision-making, optimize operations, and create better customer experiences.
  • Choosing the right approach depends on your data, goals, and timing requirements. Qualitative methods explain why something is happening, while quantitative methods measure what is happening and how significant it is.
  • Data analysis methods are evolving alongside modern application architectures. Today, real-time, operational, and JSON-native analysis complement traditional qualitative and quantitative techniques to support data-driven applications.
  • Modern applications increasingly rely on the analysis of live operational and semi-structured data, enabling faster decision-making for personalization, fraud detection, observability, and other real-time use cases.
  • Combining the right analysis techniques with modern data platforms helps organizations uncover trends, improve performance, and make more informed business and operational decisions.

Check out the following resources to learn even more about data analysis:

What Is Big Data Analytics?

Enterprise Analytics

Unstructured Data

Semi-Structured Data

What Is Data Management?

What Is a Data Platform?

Database vs. Data Warehouse: Differences, Use Cases, Examples

Couchbase Analytics Product Page

Frequently asked questions about data analysis methods

What are the main data analysis methods? The three main data analysis methods are qualitative, quantitative, and mixed methods. Qualitative analysis identifies themes, patterns, and meaning from non-numerical data, while quantitative analysis measures numerical data using statistical techniques. Mixed-methods analysis combines both approaches to provide a more complete understanding of a problem.

What’s the difference between qualitative and quantitative data analysis? Qualitative data analysis focuses on interpreting meaning, context, and patterns from sources such as interviews, user feedback, support tickets, and observations. Quantitative data analysis measures relationships and trends in numerical data using statistical methods. In application development, qualitative analysis helps explain user behavior, while quantitative analysis measures application performance, usage, and business outcomes.

What data analysis techniques work best for real-time applications? Real-time applications commonly rely on techniques such as real-time aggregation, time-series analysis, and streaming anomaly detection. These methods continuously analyze live operational data to monitor performance, detect issues, and support immediate decision-making. To be effective, they require low-latency query execution against current application data rather than delayed batch processing.

How do you analyze JSON or document database data? Analyzing JSON or document data requires query languages that understand nested objects and arrays, such as SQL++, rather than relying on traditional flat-table queries. Because document schemas can evolve over time, analysis must also accommodate optional or missing fields. Database-native columnar analytics can eliminate the need to move data into a separate warehouse, reducing latency and simplifying the analytics pipeline.

What is operational data analytics? Operational data analytics is the practice of analyzing live transactional data while it is actively supporting application workloads, without first copying it to a separate data warehouse. This enables use cases such as real-time dashboards, personalized user experiences, fraud detection, and immediate anomaly response.

The post Data Analysis Methods: Qualitative vs. Quantitative appeared first on The Couchbase Blog.

Read the whole story
alvinashcraft
36 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

How to Build a Browser-Based PDF Color Overlay Tool Using JavaScript

1 Share

Sometimes you don't want to change the actual content of a PDF. You simply want to add a colored layer over part or all of the document.

This can be useful for creating branded reports, adding colored backgrounds, highlighting printed copies, producing design mockups, applying watermarked color effects, or preparing documents for presentations.

A PDF Color Overlay Tool makes this possible by placing a semi-transparent color layer over PDF pages while preserving the original text, images, and layout beneath it.

Instead of manually editing every page in graphic design software, users can upload a PDF, choose an overlay color, adjust its transparency, select a blend mode, decide where it should appear, preview the result, and download the updated document.

In this tutorial, you'll build this tool using JavaScript. Users will be able to upload a PDF and perform all the actions just mentioned – all without sending the document to a server.

Table of Contents

What This PDF Color Overlay Tool Does and How It Works

A PDF Color Overlay Tool applies a colored layer on top of one or more pages while keeping the original PDF content visible underneath. Unlike a color inverter or grayscale converter, which permanently transform every pixel, a color overlay blends a selected color with the existing page using adjustable transparency and blend modes.

This makes it useful for creating branded documents, adding colored backgrounds, producing presentation-ready PDFs, highlighting sections, creating themed reports, or generating preview versions without modifying the original source document.

In this project, users can upload a PDF, preview every page, choose an overlay color using either a color picker or a hexadecimal value, adjust the overlay opacity, select a blend mode, choose where the overlay should appear, decide which pages should receive the effect, preview the updated document, and download the finished PDF directly from the browser.

Internally, PDF.js renders each page onto an HTML canvas. JavaScript then draws a colored rectangle over the rendered page using the selected transparency and blend mode. Once all selected pages have been processed, PDF-lib assembles the updated pages into a new downloadable PDF.

The overlay color is represented using a hexadecimal value.

const overlay = {
    color: "#667eea",
    opacity: 0.5
};

When drawing the overlay, JavaScript first sets the transparency level.

context.globalAlpha = overlay.opacity;

Next, the selected color is applied.

context.fillStyle = overlay.color;

Finally, the colored rectangle is drawn over the required area.

context.fillRect(0, 0, canvas.width, canvas.height);

Depending on the selected blend mode, the overlay can either gently tint the document, produce darker colors, create dramatic lighting effects, or generate completely different visual styles while preserving the original page underneath.

Project Setup

Before implementing the overlay functionality, let's create a simple project structure.

We'll build the application using HTML, CSS, and JavaScript, together with PDF.js, the Canvas API, and PDF-lib.

Our project structure looks like this:

pdf-color-overlay/
│── index.html
│── style.css
│── script.js
│── pdf.worker.min.js
│── assets/

Separating the HTML, CSS, and JavaScript keeps the project organized and makes future enhancements easier to implement.

Libraries Used

Our PDF Color Overlay Tool relies on three browser technologies that work together to render PDF pages, apply color overlays, and generate a new downloadable document.

PDF.js renders PDF pages directly inside the browser.

The HTML Canvas API draws the color overlay on top of each rendered page using transparency and blend modes.

PDF-lib generates the final PDF after all selected pages have been processed.

Include the required libraries before loading your application:

<script src="https://cdnjs.cloudflare.com/ajax/libs/pdf.js/4.4.168/pdf.min.js"></script>
<script src="https://unpkg.com/pdf-lib/dist/pdf-lib.min.js"></script>
<script src="script.js"></script>

Configure the PDF.js worker.

pdfjsLib.GlobalWorkerOptions.workerSrc = "pdf.worker.min.js";

Using a worker allows PDF rendering to happen in the background, keeping the interface responsive even when opening large PDF files.

Creating the HTML Layout

The application is divided into four main sections:

  • Upload area

  • PDF preview

  • Overlay settings

  • Download section

Create the basic layout.

<section id="uploadSection"></section>
<section id="previewSection" hidden></section>
<section id="settingsSection" hidden></section>
<section id="downloadSection" hidden></section>

Initially, only the upload area is visible. The remaining sections appear after a PDF has been successfully loaded.

Selecting the Main Elements

Store references to the elements used throughout the application.

const uploadSection = document.getElementById("uploadSection");
const previewSection = document.getElementById("previewSection");
const settingsSection = document.getElementById("settingsSection");
const pdfCanvas = document.getElementById("pdfCanvas");

These references allow the application to update the interface without repeatedly searching the DOM.

Uploading and Previewing PDFs

The upload area supports both drag-and-drop and manual file selection.

Before loading the document, verify that the selected file is a PDF.

async function uploadPdf(file) {
    if (!file || file.type !== "application/pdf") {
        alert("Please select a PDF file.");
        return;
    }

    await loadPdf(file);
}

After validation, the PDF is loaded into memory for rendering.

Upload area showing drag-and-drop support and Select PDF button.

Loading the PDF

Convert the uploaded file into an ArrayBuffer before opening it with PDF.js.

async function loadPdf(file) {
    const bytes = await file.arrayBuffer();
    pdfDocument = await pdfjsLib.getDocument({
        data: bytes
    }).promise;
    currentPage = 1;
    renderPage(currentPage);
}

Once the document has loaded successfully, the first page is rendered automatically.

Rendering PDF Pages

PDF.js renders one page at a time onto an HTML canvas.

Retrieve the selected page.

const page = await pdfDocument.getPage(currentPage);

Create the viewport.

const viewport = page.getViewport({
    scale: 1.5
});

Resize the canvas.

pdfCanvas.width = viewport.width;
pdfCanvas.height = viewport.height;

Render the page.

await page.render({
    canvasContext: pdfCanvas.getContext("2d"),
    viewport
}).promise;

After rendering completes, users can view the current page before applying any overlay effects.

PDF preview rendered with PDF.js showing page navigation.

Navigating Between Pages

Most PDF documents contain multiple pages, so the application includes simple navigation controls.

Store the current page.

let currentPage = 1;
let pdfDocument = null;

Move to the previous page.

previousButton.addEventListener("click", async () => {
    if (currentPage > 1) {
        currentPage--;
        await renderPage(currentPage);
    }
});

Move to the next page.

nextButton.addEventListener("click", async () => {
    if (currentPage < pdfDocument.numPages) {
        currentPage++;
        await renderPage(currentPage);
    }
});

Update the page indicator.

pageCounter.textContent = `Page ${currentPage} of ${pdfDocument.numPages}`;

Users can now browse through the uploaded PDF before deciding how the color overlay should be applied.

Building the Overlay Settings

After the PDF has been uploaded and previewed, users can configure how the color overlay should be applied. The settings panel lets users choose an overlay color, adjust its transparency, select a blend mode, specify where the overlay should appear, and decide which pages should receive the effect before generating the final PDF.

Choosing the Overlay Color

The first setting allows users to choose the color that will be placed over the PDF.

The application supports both a color picker and direct hexadecimal input.

Create the color picker.

<input type="color" id="overlayColor" value="#667eea">

Create the hexadecimal input.

<input type="text" id="hexValue" value="#667eea">

Retrieve the selected color.

const overlayColor = document.getElementById("overlayColor").value;

If users enter a hexadecimal value manually, synchronize it with the color picker.

hexValue.addEventListener("input", () => {
    overlayColor.value = hexValue.value;
});

The selected color will later be drawn over the rendered PDF page.

Overlay color picker with hexadecimal color input.

Adjusting the Opacity

Opacity controls how transparent the overlay appears.

Lower values allow more of the original PDF to remain visible, while higher values create a stronger color effect.

Create the opacity slider.

<input type="range" id="opacity" min="0" max="100" value="50">

Retrieve the selected value.

const opacity = Number(document.getElementById("opacity").value) / 100;

This value is later assigned to the canvas transparency before drawing the overlay.

Opacity slider used to control overlay transparency.

Selecting the Blend Mode

Blend modes determine how the overlay color interacts with the original PDF content.

Create the dropdown.

<select id="blendMode">
    <option value="source-over">Normal</option>
    <option value="multiply">Multiply</option>
    <option value="overlay">Overlay</option>
    <option value="soft-light">Soft Light</option>
    <option value="hard-light">Hard Light</option>
    <option value="difference">Difference</option>
</select>

Retrieve the selected blend mode.

const blendMode = document.getElementById("blendMode").value;

Each blend mode produces a different visual effect while preserving the document beneath the overlay.

Blend mode dropdown showing available overlay modes.

Choosing the Overlay Position

The overlay doesn't always need to cover the entire page. Users can apply it only to specific regions if they want.

Create the available options.

<input type="radio" name="position" value="full" checked>
Full Page
<input type="radio" name="position" value="header">
Header Only
<input type="radio" name="position" value="footer">
Footer Only

Retrieve the selected position.

const position = document.querySelector('input[name="position"]:checked').value;

During processing, the application draws the overlay only inside the selected area.

 Overlay position options including Full Page, Header Only, and Footer Only.

Choosing Which Pages to Process

Users can apply the overlay in several different ways:

  • Current page only

  • Entire document

  • Separate overlay for every page

  • Specific pages

Create the page selection controls.

<input type="radio" name="pages" value="current" checked>
Current page only
<input type="radio" name="pages" value="all">
All pages
<input type="radio" name="pages" value="separate">
Separate overlay per page
<input type="radio" name="pages" value="custom">
Specific pages
<input type="text" id="pageRange" placeholder="e.g., 1, 3-5, 10">

Retrieve the selected option.

const pageMode = document.querySelector('input[name="pages"]:checked').value;

Read the custom page range.

const pageRange = document.getElementById("pageRange").value.trim();

This flexibility allows users to apply different overlay strategies depending on the document.

Apply-to-pages options including Current Page, All Pages, Separate Overlay, and Specific Pages.

Applying the Overlay

Once all settings have been configured, users can begin processing the PDF.

Create the action button.

<button id="applyOverlay">Apply Overlay</button>

Start the processing workflow.

applyOverlay.addEventListener("click", async () => {
    await processOverlay();
});
Apply Overlay button.

Starting Over

Users can reset the application at any time and upload another document.

Create the reset button.

<button id="resetTool">Start Over</button>

Reset the tool.

resetTool.addEventListener("click", () => {
    location.reload();
});

The upload area becomes visible again, allowing another PDF to be processed without manually clearing every setting.

Applying Color Overlays to PDF Pages

Now we'll build the main feature of the application: adding a colored overlay to PDF pages.

The process begins by rendering each selected PDF page onto an HTML canvas using PDF.js. JavaScript then draws a semi-transparent colored rectangle over the page using the selected blend mode. Once all selected pages have been processed, PDF-lib generates a new downloadable PDF.

Applying the Overlay Color

Before drawing anything, retrieve the selected color.

const overlayColor = document.getElementById("overlayColor").value;

Set the canvas fill color.

context.fillStyle = overlayColor;

This color will be drawn over the selected portion of each PDF page.

Setting the Overlay Transparency

Opacity determines how much of the original page remains visible beneath the overlay.

Apply the selected transparency.

context.globalAlpha = opacity;

A lower opacity produces a subtle tint, while higher values create a stronger visual effect.

Applying the Blend Mode

Canvas supports several compositing modes that determine how the overlay interacts with the existing page.

Assign the selected blend mode.

context.globalCompositeOperation = blendMode;

Some common modes include:

  • Normal – Places the color directly over the page.

  • Multiply – Produces a darker appearance.

  • Overlay – Increases overall contrast.

  • Soft Light – Creates a gentle lighting effect.

  • Hard Light – Produces a stronger contrast.

  • Difference – Generates an inverted-style appearance based on color differences.

Drawing the Overlay

Once the color, opacity, and blend mode have been configured, draw the overlay on the canvas.

For a full-page overlay:

context.fillRect(0, 0, canvas.width, canvas.height);

If users choose Header Only, draw the rectangle across only the top section.

context.fillRect(0, 0, canvas.width, 120);

For Footer Only, draw the overlay near the bottom of the page.

context.fillRect(0, canvas.height - 120, canvas.width, 120);

These options allow different overlay styles without modifying the underlying PDF content.

Processing the Selected Pages

After configuring the overlay, process only the pages chosen by the user.

Loop through the selected pages.

for (let page = startPage; page <= endPage; page++) {
    await processPage(page);
}

Each processed page is temporarily stored before creating the final document.

If Current Page Only is selected, only the active page is processed. If All Pages is selected, the overlay is applied to the complete document.

Generating the Final PDF

Create a new PDF document.

const outputPdf = await PDFLib.PDFDocument.create();

Convert the processed canvas into an image.

const imageBytes = await canvasToBytes(pdfCanvas);

Embed the image.

const image = await outputPdf.embedPng(imageBytes);

Create a new page.

const page = outputPdf.addPage([
    image.width,
    image.height
]);

Draw the processed image.

page.drawImage(image, {
    x: 0,
    y: 0,
    width: image.width,
    height: image.height
});

Repeat these steps until every selected page has been added to the new PDF.

Saving the Generated PDF

Once all pages have been processed, save the completed document.

const pdfBytes = await outputPdf.save();

Create a downloadable file.

generatedPdfBlob = new Blob([pdfBytes], {
    type: "application/pdf"
});

The new PDF containing the selected color overlays is now ready for preview.

PDF preview after applying the selected color overlay.

Previewing the Result

Before downloading the processed document, users should be able to review the final output. This makes it easy to verify that the selected color, opacity, blend mode, and page selection have been applied correctly.

Load the generated PDF.

let finalPdf = null;
async function showPreview() {
    const bytes = await generatedPdfBlob.arrayBuffer();
    finalPdf = await pdfjsLib.getDocument({
        data: bytes
    }).promise;
    renderFinalPage(1);
}

Render the selected page.

async function renderFinalPage(pageNumber) {
    const page = await finalPdf.getPage(pageNumber);

    const viewport = page.getViewport({
        scale: 1.5
    });

    previewCanvas.width = viewport.width;
    previewCanvas.height = viewport.height;

    await page.render({
        canvasContext: previewCanvas.getContext("2d"),
        viewport
    }).promise;
}

Users can navigate through the processed PDF before downloading it.

Final PDF preview showing the applied color overlay before downloading.

Renaming and Downloading

Before saving the generated PDF, users can customize the output filename.

Create the filename input.

<input type="text" id="outputFilename" value="color-overlay.pdf">

Retrieve the filename.

function getFilename() {
    let filename = outputFilename.value.trim();

    if (!filename) {
        filename = "color-overlay.pdf";
    }

    if (!filename.toLowerCase().endsWith(".pdf")) {
        filename += ".pdf";
    }

    return filename;
}

Display information about the generated PDF.

pageCount.textContent = `${finalPdf.numPages} Pages`;
fileSize.textContent = formatFileSize(generatedPdfBlob.size);

Download the completed document.

downloadButton.addEventListener("click", () => {
    const url = URL.createObjectURL(generatedPdfBlob);
    const link = document.createElement("a");

    link.href = url;
    link.download = getFilename();
    link.click();

    URL.revokeObjectURL(url);
});

Everything happens locally inside the browser, helping users keep their PDF files private.

 Download section showing the output filename, page count, file size, and Download button.

Demo: How the PDF Color Overlay Tool Works

Let's walk through the complete workflow.

Step 1: Upload the PDF

Users begin by dragging a PDF into the upload area or clicking Select PDF.

Upload area with drag-and-drop support and Select PDF button.

Step 2: Preview the Document

The uploaded PDF is rendered page by page, allowing users to review the document before applying any changes.

PDF preview with page navigation controls.

Step 3: Configure the Overlay

Users choose an overlay color, adjust the opacity, select a blend mode, choose the overlay position, and decide which pages should receive the effect.

 Overlay settings panel with color, opacity, blend mode, position, and page selection options.

Step 4: Apply the Overlay

Click Apply Overlay to process the selected pages using the chosen settings.

Apply Overlay button.

Step 5: Review the Processed PDF

The completed PDF appears in the preview window so users can verify the applied overlay before downloading.

Final PDF preview after applying the selected overlay.

Step 6: Rename and Download

Finally, users rename the output file if needed, review the page count and file size, and download the generated PDF.

Download section with filename, page count, file size, and Download button.

Performance Tips

Large PDF files can take longer to process because every selected page must be rendered and updated. Processing only the required pages helps improve performance.

for (const page of selectedPages) {
    await processPage(page);
}

After downloading the file, release temporary resources to reduce memory usage.

URL.revokeObjectURL(downloadUrl);

These small optimizations help keep the application responsive when working with large multi-page PDF documents.

Common Mistakes

One common mistake is applying multiple overlays without first restoring the original page. Always render a fresh copy of the PDF page before applying another overlay.

await renderPage(currentPage);

Another issue is forgetting to restore the default canvas state after changing the opacity or blend mode.

context.globalAlpha = 1;
context.globalCompositeOperation = "source-over";

Finally, using a very high opacity can completely hide the original PDF content. Choosing an appropriate transparency level usually produces a more balanced result.

Conclusion

In this tutorial, you built a browser-based PDF Color Overlay Tool using JavaScript.

You learned how to upload PDF documents, render pages with PDF.js, configure overlay colors, adjust opacity, apply blend modes, position overlays, process selected pages, generate a new PDF with PDF-lib, preview the completed document, rename the output file, and download it directly from the browser.

Because the entire workflow runs locally, users can customize PDF documents without uploading sensitive files to an external server.

You can explore the complete workflow using the PDF Color Overlay Tool.

From here, you can extend the project with gradient overlays, custom overlay shapes, image overlays, reusable color presets, watermark templates, or additional PDF editing features for even greater flexibility.



Read the whole story
alvinashcraft
36 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

The comments that go into code versus those that go into the pull request description

1 Share

When you submit a pull request, there are two places you can use to help explain what you are doing and why you are doing it. One is the pull request description, and another is the code you are modifying. And it’s important to understand the difference between them.

The pull request is where you justify why your change should be accepted. In the title, you spell out the problem you are fixing or the feature you are adding.

Add support for polarity reversal

Fix crash when polarity changes

In a large code base, you may need to be a little more specific.

Add support for widget polarity reversal

Fix widget crash when polarity changes twice in a short time

When somebody is chasing down a regression, they are going to be looking over all of the PRs that went into the branch recently, and having a good title will make it easier for them to identify which changes are likely to be a source of the problem.

For example, if somebody is investigating a doodad crash, they may look into “Add support for widget polarity reversal” because their doodad uses widgets, and maybe the problem is caused by a reverse-polarity widget that their doodad isn’t handling. On the other hand, they can pay less attention to the fix for the crash when widget polarity changes because that’s unlikely to be the reason the doodad is crashing. And if their doodad doesn’t use widgets at all, they may just skip over both of them.

If the PR had used the original titles of “Add support for polarity reversal”, without any mention of widgets, then a team investigating a regression in gadgets would have to dig into the PR (because gadgets also have polarity), only to realize that it’s about widget polarity, not gadget polarity.

The description of the PR talks about the source of the problem and how you fixed and validated it. This is point-in-time information where you justify to your reviewer why the change is needed and why your particular implementation of the change is correct. Discuss alternative designs and why they were rejected (e.g. because they were too risky). Show before-and-after screen shots showing that the problem is fixed. Confirm that associated paperwork has been completed, like unit tests. There might be standard paperwork for this, such as a “checkin template”. (It is often the case that the closer a project comes to release, the more stringent the paperwork. For example, late in the product cycle, you may need to demonstrate that the release management team has deemed that the bug meets the bug bar.)

In other words, the PR description is a point in time statement, providing information that is relevant to the code review itself. It is an exercise in persuasive writing: You are trying to convince the approver that your change should be accepted.

Comments in the code are for talking about the code itself. What is the correct way to call this function? Does it have specific prerequisites? This information is durable: It is information that remains useful even after the pull request completes.

Okay, so let’s do an exercise: I’m going to provide some text, and you tell me where it goes. These are all actual comments (suitably redacted) from PRs I have reviewed.

I have checked all calls to the function, and this was the only one that passed the wrong flag.

This goes into the pull request description. It is justifying why your change is correct, and in particular, it’s answering a question that a reviewer is likely to ask: “It’s great that you’re fixing this one caller of the function, but are there other callers that make the same mistake?” Putting this comment in the code itself would be wrong because the claim is valid only at the time the pull request is made. After the pull request, somebody might add a new call to the function that passes the wrong flag, and it is not true that you validated that new caller.

The JSON schema accepted by this function is documented 〈here〉.

This goes into the code. It is explaining how to use the function correctly. This information is important not just at the time you submit the pull request but also for an indefinite period of time in the future. (At least, until you change the function or the schema.)

The Doodad component will take advantage of polarity reversal.

This goes into the pull request description. It is justifying why you need to implement polarity reversal today. If you put this in the code, the future tense suggests that we are still waiting for Doodad. And future changes to the Doodad might cause them to stop relying on polarity reversal; when they do that, they are unlikely to come and update this comment in somebody else’s component. The comment also suggests that if you confirm with the Doodad team that they don’t need polarity reversal any more, it is safe to remove support for polarity reversal, which might not be the case if other components started using the feature as well.

Still, knowing that Doodad is the intended audience for the feature is worth noting for posterity.

// Polarity reversal was initially added for the benefit of
// the Doodad component.

Bonus chatter: Another thing to consider when making code comments is that the code comment needs to make sense even without the PR description. Suppose you are writing a function with the intention of deprecating an older function that it is replacing. Don’t add this comment to the new function:

// When all clients have migrated to the new function, keep this.

This makes no sense to someone who is seeing the comment without having also seen (and remembered) the PR that introduced it. It sounds like the comment is saying, “When X happens, take no action.”

What you should do is put a comment on the old function:

// When all clients have migrated to the new function, delete this function.
 

The post The comments that go into code versus those that go into the pull request description appeared first on The Old New Thing.

Read the whole story
alvinashcraft
36 minutes ago
reply
Pennsylvania, USA
Share this story
Delete

Why use ORMs if LLMs write code?

1 Share

The return of SQL?

It's no secret that I'm no fan of ORMs. Most people, on the other hand, find them indispensable. As one reader commented:

"I can work with raw SQL ofcourse... but the mapping... oh the mapping..."

This seems to capture something essential. When I discuss ORMs, the most common argument in favour seems to revolve around the amount of boilerplate code required to communicate with a relational database. And indeed, it's significant.

As I've argued, however, I'm not convinced that ORMs solve that problem.

But now that LLMs write code, does it even matter?

In addition to my individual reservations, it strikes me that ORMs come with many issues related to query efficiency. The vibe I'm getting from ORM experts is that if you really know a particular ORM, you can fine-tune the queries it makes. There are, however, various pitfalls to avoid: Anti-patterns to eschew, idioms to follow, particular APIs to keep clear of, certain parameter values to explicitly pass, etc.

Which strikes me as ironic, because wasn't the whole promise of ORMs that you could read from and write to a relational database without getting bogged down in the details of SQL?

So instead of fiddling with a temperamental and implicit ORM API, why not write fine-tuned parametrized SQL queries? Or rather, ask an LLM to do that for you, as well as all the boilerplate code.

You should, of course, remind it to avoid SQL injection vulnerabilities.


This blog is totally free, but if you like it, please consider supporting it.
Read the whole story
alvinashcraft
36 minutes ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories