As a special holiday treat, on December 25th, the social media app Bluesky announced that it has added a new feature to its mobile app: a list of Trending topics that lets you know what subjects are popular among its users.

Screenshot: Bluesky

The new feature can be found by selecting the search icon (the magnifying glass), which appears at the bottom of the screen on the mobile app and on the left sidebar on the web. Lists of Trending and Recommended subjects now appear below the search bar. Tap on any topic, and you will be able to access the associated posts. When I tried it, choices among the top five included Christmas and Nosferatu (not an unexpected selection of topics but an interesting juxtaposition).

If you’d rather not see the list, you can get rid of it via a small “x” in the upper right corner, or go to Settings > Content & Media and toggle off Enable trending topics.

According to the announcement, the new feature is “V1” (it is marked as a Beta on the app) and “we will be iterating with your feedback.” So if you have any objections to Trends appearing under your Bluesky search bar, let them know.

From the 1960s onwards, IBM’s mainframe systems started the era of proprietary hardware and software, which trickled into the PC era. In the early 1990s, Linux broke that chokehold, emerging as an open source alternative for those tired of proprietary operating systems and hardware.

The AI market is traveling the same path, but the surroundings are different. Open AI models on the rise are shaking up the AI market, breaking a stranglehold of proprietary models running on proprietary hardware.

Cloud providers, including Google and Amazon, are rushing to put open models on their proprietary chips. That’s because consumers of AI models want lower cost and flexibility associated with open AI LLMs. The trend aligns with how Linux grew to now run most of the internet.

“If you want to go fast, go alone. If you want to go far, go together,” said Fabrizio Del Maffeo, CEO of Axelera, an AI hardware company.

How Did We Get Here?

The AI era started off much like the PC era, with proprietary Windows software working exclusively on x86 hardware.

The rise of Linux was driven by x86, and “it was Linux plus x86 that became the web stack/LAMP stack,” said David Kanter, founder of AI benchmarking organization MLCommons.

“The reality is Linux has truly taken over the proprietary Unixes. Solaris is gone; HP-UX is gone; Tru64 is definitely gone; AIX is still around,” Kanter said.

Open models such as Meta’s Llama and Google’s Gemma are similarly breaking the dominance of proprietary models driving enterprise AI, such as Google’s Gemini and OpenAI’s GPT-4.

Google’s TPU AI chips previously ran only its proprietary Gemini LLM, but the company earlier this year put its homegrown Gemma open model on the chips.

Amazon at Re:Invent made Meta’s Llama 3.1 model with 405 billion parameters available on its homegrown Trainium2 chip.

“Trainium2 is cheaper than comparable Nvidia instances, so taking Llama2 405B and training it against customer proprietary data to create a custom model is a budget-friendly approach,” said James Sanders, an analyst at TechInsights.

What Are Open Models?

To be sure, open models and open source AI models aren’t the same. In the world of software, you can modify open source code any way you see fit. There are multiple definitions of what it means to be open in AI.

The Open Source Initiative two months ago defined open source AI as “applied to a system, a model, weights and parameters, or other structural elements.” That included all the training data.

Meta’s Llama doesn’t fit the OSI definition, but it is mostly open with some restrictions. Users can use Llama as a pretrained model and finetune it to specific needs. But users can’t access or modify Llama’s pretrained data, as Meta doesn’t want to reveal the sources of data it used to pretrain the model.

Proprietary models like Gemini, Claude and GPT-4 are completely closed.

Like Linux, Lock in Customers

Cloud providers are following the footsteps of Linux OS providers like Red Hat — wrapping the open source OS with proprietary tech and locking customers into the software stack.

The open AI models are a low-cost way to lure customers to cloud services. Once customers are locked into a cloud service provider, it is hard to leave.

“The motive is to get more customers using their services that surround AI like the compute, data management, security and storage,” said Patrick Moorhead, principal analyst at Moor Insights and Strategy.

AWS’s Trainium2 hasn’t set the world on fire, so porting Llama to Trainium2 brings more value to its chips. At Re:Invent, AWS also announced its homegrown Nova models, which will run on Trainium.

AWS wants broad coverage of use cases, said Naveen Rao, vice president of artificial intelligence at Databricks. Rao sold his company, MosaicML, to Databricks in 2023 for $1.3 billion.

“Supporting more models increases the relevance for a piece of hardware, so that’s likely the main reason. And it’s not a huge lift for them,” Rao said.

The Open Edge

Open source and open models are advantageous for cloud-captive AI accelerators.

“The ideal state is to provide a familiar environment with minimal friction at a lower cost,” Sanders said.

Open models also allow creation of open source model derivatives, smaller and optimized, which can better fit industry-specific requirements.

“Adding open models to the catalog allows them to increase customer bases and monetize the services,” Del Maffeo said.

Groq, Sambanova, Cerebras and other small hardware providers are also providing open models as a service at very low token cost.

The Linus of Open AI Models

It can be notoriously difficult to load open AI models on Google’s TPU and AWS’s Trainium. Open models need specialized forks for custom chips.

Open models are typically built on frameworks and toolchains such as PyTorch, JAX or TensorFlow. Developers use the framework’s built-in tools and APIs to measure the performance and fix it with techniques that include optimizations for the architecture and chips.

By comparison, Nvidia’s GPUs are generic AI accelerators that can run just any PC or AI application.

HuggingFace is driving open source AI growth on proprietary hardware. It provides hundreds of open models that have proven similar accuracy and performance to more power-hungry models.

AWS is partnering with Hugging Face to train and deploy models on Trainium.

“Now that the market is accelerating, it’s natural to see Amazon opening access on its infrastructure to any other open source model,” Del Maffeo said.

HuggingFace in July announced AI models were available for deployment on Google Cloud TPUs.

“With more concerns around the increasing power consumption, cooling requirements and cost to train large models, innovations from the community that help alleviate these challenges are welcomed,” Del Maffeo said.

More developers are also gaining experience in machine-learning development, and the community capability can cover the needs of a large part of the AI market.

Changing Definition

Enterprises already use a mix of open and closed source models.

“As of now, the architecture of all models is largely the same,” Rao said.

From a hardware perspective, the difference between open and closed models is much more about the data and training regimen than it is about the architecture of the model, he said.

“You could argue that all models that run on hardware are open source or derivatives of open source architectures. That might change in the future with this whole inference time-scaling idea like GPT-4o1,” Rao said.

The post AI, Hardware and Open Models: Headed in the Linux Direction appeared first on The New Stack.

From deep within the Looker traffic reports of The New Stack, we have unearthed the most viewed posts from 2024 about the subject of Infrastructure as Code (IaC). Collectively, what they show is that, despite IaC’s promise in scaling IT systems, it still has many issues that drive DevOps folks crazy.

“Having used Terraform extensively, I genuinely appreciate the magic of Infrastructure as Code as an accelerant. However, refactoring is a reality of ‘Day 2’ operations and doing this with Terraform is still extremely painful to get right,”  Matt Moore, founder and CTO of security company Chainguard, told TNS.

Cloud services created the need for the practice of “Infrastructure as Code,” as organizations set up their operations on Amazon Web Services and other providers. Declarative, domain specific languages were created by Puppet and Chef and as a way to automate configuration and provisioning work in setting up and maintaining these systems.

And Kubernetes, with its ability to orchestrate microservices, put this practice into overdrive. And so HashiCorp’s Terraform surfaced to manage this next level of cloud provisioning.

But despite the great value Terraform and associated IaC tools have brought, DevOps teams are feeling more frustrated than ever.

The New Stack’s’ 10 most popular IaC stories from 2024 show the frustrations they are feeling, and some possible paths going forward.

1. Infrastructure as Code Is Dead: Long Live Infrastructure from Code

In this contributed post, Asif Awan, co-founder and chief product officer at a company then called appCD but now known as StackGen, noted that managing, maintaining and deploying applications and infrastructure securely and consistently remains an incredibly complicated challenge.

“Just as IaC expanded the ability to deploy to the cloud, it added complexity to that deployment by combining teams with different experiences and expertise and requiring them to find new ways to work together,” Awan noted.

The solution, he suggested was to “generate the infrastructure your application needs based on the version of your application being deployed.”

This approach he called “Infrastructure from Code.”

2. How We Evolved from IaC to Environments as Code

Edan Evantal, CTO of Quali noted that IaC tools were designed for velocity and automation, not as the source of truth for environments. Great for deploying cloud services, they are pretty terrible as a tool for predicting how code changes can change app performance

Also, IaC tools don’t play nicely together.

He noted that Quali rethinks the IaC process, defining everything a developer needs to launch an environment, in such a way that it is easy for machines and humans to understand. Then, teams can use GitOps as a base to launch applications.

3. Terraform Isn’t Dead

Nitric‘s Rak Siva is also a proponent of Infrastructure from Code (IfC).

The problem, Siva wrote, is”when a developer decides to replace a manually managed storage bucket with a third-party service alternative, the corresponding IaC scripts must also be manually updated, which becomes cumbersome and error-prone as projects scale. The desync that occurs between the application and its runtime can lead to serious security implications, where resources are granted far more permissions than they require or are left rogue and forgotten.”

He added, “Infrastructure from Code automates the bits that were previously manual in nature.  Whenever an application changes, IfC can help provision resources and configurations that accurately reflect its runtime requirements, eliminating much of the manual work typically involved.”

Siva noted that the the developer doesn’t write the low-level configuration code, for tasks like setting up IAM roles and permissions, but rather, they just need to know its available.

Nitric offers an open source IfC framework for building in your language of choice and deploying to all the major clouds.

Infrastructure as Code is unidirectional and has an asymmetric experience where changes (writes) and observations (reads) are performed through different tools. What are some consequences of that? Is it really necessary?https://t.co/O1WJX6yhic#infrastructureascode

— Brian Grant (@bgrant0607) December 23, 2024

4. OpenTofu Project Denies HashiCorp’s Allegations of Code Theft

On top of all the manual work that Terraform causes developers, there has been a lot of uncertainty floating around this toolset itself in the past year. In August 2023, HashiCorp moved the software from an open source license to a more restrictive Business Source License (BSL), in an effort to thwart competing service providers.

As a result, a group of these companies, such as Spacelift, forked the last version of the open source code into its own project, which would be called OpenTofu, and quickly backed by the Linux Foundation, which needed an open source IaC tool as part of its Cloud Native Computing Foundation stack.

And licensing wasn’t the only issue either; power users of Terraform complained of HashiCorp being slow at accepting outside bug fixes. A more receptive model of software management was needed, they argued.

Naturally, HashiCorp was not pleased with the fork, and in April, tried to cast the doubt on the effort, by charging the open source collective had stolen code from the now-BSL licensed Terraform. The OpenTofu team quickly debunked the charges and continued on its journey of modernizing OpenTofu.

5. Why Most Companies Are Struggling With Infrastructure as Code

According to a recent survey from StackGen, “Stacked Up: The IaC Maturity Report,” only 13% of organizations have thus far achieved IaC maturity. Most assert that only some of their infrastructure was represented in code, but only 10% have embarked on pilot projects.,

Achieving IaC maturity is hard, noted Arshad Sayyad, co-founder and chief business officer of StackGen, in this post.

Most companies surveyed were still in the early stages, having only a small portion of infrastructure stored in code in pilot projects. Sayyad also recommends IfC, where “the IaC itself is automatically generated from the application code and built with guardrails that align with best practices,” he wrote.

6. Beyond Infrastructure as Code: System Initiative Goes Live

Other companies looked beyond Terraform for answers. This year, Adam Jacob, the former CTO of Chef, launched his own company, System Initiative, with an automation platform where, with a graphical grid-based workspace, an admin can stitch together a system with small, reactive functions, allowing the system to be managed as “living architecture.”

Managing Infrastructure as Code may seem like a good idea but it causes “all sorts of downstream problems,” Jacob told TNS.

“It’s not a single technology problem, but it is the shape, the foundation, the primitives that we’re being asked to use that cause these [negative] outcomes in the vast majority of cases.”

7. Generative AI Tools for Infrastructure as Code

But wait, perhaps AI could help! Dell Technologies‘ Parasar Kodati notes that large language models (LLMs) are great for analyzing error messages and logs to identify the root causes of frequently occurring issues. This approach could apply to any platform too, including Red Hat Ansible Playbooks and Terraform.

In addition to checking errant code, generative AI can also be used to set up your own personalized chatbot for answering questions.

“You can train GPT models with anything, such as a policy document or coding guidelines or an IT infrastructure-size calculator, and have chatbots use these backend models to answer queries from customers or internal stakeholders,” Kodati wrote.

8. For Terraform Deployment, Any CI/CD Can Beat TACOS

One of the first solutions to IaC fragmentation problem was Terraform Automation and Collaboration Software (TACOS), which sought a way to bring IaC under the same governance and collaborative workflows that we use for application code. But admins did not see the hidden cost of TACO implementations, which came in the form of integration challenges, configuration confusion and potential system fragmentation, noted Eran Bibi, co-founder and chief product officer at Firefly in this post.

The answer comes, Bibi argued, in improving tools on the CI/CD pipeline instead. This approach can integrate easily with Policy as Code and Governance as Code initiatives, to provide “a more holistic approach to managing infrastructure, without the need for an additional layer of tools.”

We saw some time ago that strings in Win32 resources are grouped in bundles of 16. Why not have each string be a separate resource? Why are they bundled at all? And why bundles of 16? Why not 15 or 8 or 32?

Recall how resources worked in 16-bit Windows. To load a resource, you allocate a selector to hold the resource data, load the resource data from disk into the selector, and then use the selector to access the resource. The selector remains in memory afterward, but it is marked as “discardable”, so that it can be freed when the system comes under memory pressure.

Windows 1.0’s system requirements did not include a hard drive. You could run it off a two-floppy system.¹ Reducing I/O had a noticeable effect on performance, so issuing a separate I/O for each string was going to be inefficient.

On top of that, Windows 1.0 had a limit of 4096 selectors, so putting each string in its own resource would drain the system of selectors.

On the other hand, you don’t want to load all the strings, because that means doing a large I/O transfer for data, most of which are not going to be used. Furthermore, you increased memory consumption because all of the strings got loaded into memory, creating additional memory pressure, and when the strings were discarded (which would happen more often because of the increased memory pressure), you lost all of your strings.

The decision to bundle strings in groups of 16 was an attempt to balance these two competing performance issues. There’s nothing magical about the number 16. It was a convenient number that gave you a decent amount of batching while still keeping the batches from getting too large.

Grouping your strings into bundles became a performance game similar to segment tuning, where you wanted to put strings that were used at the same time into the same bundle to maximize the value of each I/O operation.

Although 32-bit Windows doesn’t use allocate resources to segments the same way that 16-bit Windows did, the bundling design was nevertheless carried forward. One reason is that bundling expanded the range of string resource IDs from a 16-bit value to a 20-bit value, so the highest resource string ID went from 65535 to a bit over a million. And even though they don’t occupy a segment any more, there is still overhead in the file format to describe each resource. Strings tend to be short, so this overhead ends up being significant. You don’t want a four-character string to come with 24 bytes of overhead. There is still a small memory benefit to not wasting slots in a bundle, though it is not as severe as it was in 16-bit days.

¹ That’s what I did back in the day. The company I worked for at the time had one computer with a hard drive, and wow that hard drive made a big difference.

The post Why are Win32 resources strings bundled at all? And why bundles of 16? appeared first on The Old New Thing.

Two Of My Favorite NOLOCK Demos from Paul White and Aaron Bertrand

This week, I’ve got a coupon that offers an even steeper discount on my training than the one at the bottom of the post.

It brings the total cost on 24 hours of SQL Server performance tuning content down to just about $100 USD even. Good luck finding that price on those other Black Friday sales.

It’s good from December 23rd-27th, or until I remember to turn it off. Hurry along now.

Thanks for watching!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that, and need to solve database performance problems quickly. You can also get a quick, low cost health check with no phone time required.

The holiday season is a time for reflection and celebration, and here at Data Driven, we’re diving into a topic as classic as Christmas carols and as warmly debated as fruitcake: SQL Server Integration Services (SSIS). In this festive podcast episode, Frank and Andy unwrap the enduring significance of SSIS, a stalwart in the data engineering landscape.

The Longevity of SSIS

SSIS, part of the Microsoft SQL Server ecosystem, has been around for nearly two decades. While some might dismiss it as “old school,” Andy Leonard makes a compelling case for why it still has a place in today’s data workflows. With its robust capabilities for data integration and transformation, SSIS continues to solve business problems effectively, especially for organizations with established on-premises infrastructure.

As Andy notes, many businesses aren’t rushing to the cloud despite the buzz surrounding modern tools like Azure Data Factory (ADF) and Fabric Data Factory. This is often due to the sunk costs of existing systems and the high stakes involved in migrating critical workloads. Moreover, SSIS remains a reliable choice for organizations that prioritize accuracy, performance, and security over chasing the latest trends.

Why Dismissing SSIS is a Data Driven Mistake

One of the podcast’s most interesting debates centers around a comment labeling SSIS as “two decades too out of date” for aspiring data engineers. Frank and Andy counter this notion by emphasizing the fundamentals of data engineering. Regardless of whether you’re using SSIS, ADF, or a cutting-edge AI pipeline, the core principles of data integration—extracting, transforming, and loading (ETL)—remain the same.

Andy reminds listeners that tools like SSIS are as relevant as ever, particularly for industries where cloud adoption is either slow or impractical due to compliance or geopolitical concerns. Frank adds that data engineering tools evolve more slowly than consumer technologies, pointing out that many Fortune 100 companies still rely on “legacy” systems like mainframes and SQL Server to power their operations.

A Timeless Career Move

Frank and Andy also touch on career advice for data professionals. In an era dominated by buzzwords like LLMs (large language models) and AI, it’s easy to overlook the value of mastering proven tools like SSIS. The ability to design scalable, maintainable data pipelines is a skill that transcends any one platform, making SSIS experience a strong foundation for broader data engineering expertise.

Key Takeaways

1. SSIS is far from obsolete. It remains a cornerstone of data engineering for many organizations, especially those balancing legacy systems with modern needs.
2. The fundamentals of data integration haven’t changed. Whether you’re using SSIS or the latest cloud-native tools, the principles of accurate, efficient data management are timeless.
3. Career advice for aspiring data engineers: Don’t chase every shiny new technology. Instead, focus on building a strong foundation with versatile tools like T-SQL and SSIS, which can be applied across platforms.

Tune In and Join the Conversation

This episode of Data Driven is a must-listen for anyone passionate about the intersection of tradition and innovation in data engineering. Grab a cup of cocoa, settle in by the fire—or your nearest CPU—and listen to Frank and Andy as they unwrap the legacy and future of SSIS.

Happy Holidays from all of us at Data Driven and Frank’s World!