Sr. Content Developer at Microsoft, working remotely in PA, TechBash conference organizer, former Microsoft MVP, Husband, Dad and Geek.
160074 stories
·
33 followers

Elevating app quality: Reducing memory usage and improving device migration

1 Share
Posted by Raghavendra Hareesh Pottamsetty, GM, Google Play Developer & Monetization

Maintaining a healthy Android ecosystem is a shared commitment where every app and game has a role to play. To help you deliver the premium experiences users expect, Google Play is introducing two new quality requirements: one focused on reducing app memory footprint, and another on providing a secure, seamless device migration experience.

First, to help developers navigate industry-wide hardware constraints and Android's broader memory limits, Google Play is establishing new performance thresholds. 

Second, as part of our broader commitment to elevate app quality, we are introducing a new onboarding standard to simplify and secure login during device upgrades.

Reducing app memory usage and optimizing code

The mobile industry is navigating significant hardware supply constraints that are altering device memory availability that over time can negatively impact the user experience. Android is addressing this challenge head-on with broader memory limits that aim to protect the overall user experience from apps using excess memory and causing system-wide slowdowns. 

Building on this, today Google Play is establishing performance thresholds to help developers ensure their apps continue to deliver the premium experience users expect. This includes new thresholds across dynamic memory usage, bitmap usage, and code optimization to prevent unexpected on-device performance throttling and app terminations. 

  • Dynamic memory usage (anonymous RSS + swap): This tracks the memory used for your app's private data storage, including both active and compressed memory. It excludes files stored on the device, such as code or assets. We will assess this usage across different app states (like when your app is in use or running in the background) and device performance categories.
  • Bitmap memory usage: This evaluates the memory consumed by bitmaps. While bitmaps occupy memory when your app is in the foreground, they should not be held in memory for extended periods of time in non-visible app states such as background and cached.
  • Optimized DEX code:  A well-optimized Android App Bundle uses less memory, starts faster, reduces ANRs, and improves rendering and runtime performance. To ensure an optimized footprint, apps published on Google Play apps published on Google Play must be optimized with a minimum of 25% coverage across optimization, shrinking, and obfuscation using a tool such as R8 or any other shrinking tool.     

Review the thresholds and technical details to better understand applicability differences specific to apps and games, RAM buckets, and process states. 

New tools to help you take action

To enable you to proactively discover, investigate, and optimize your app or game to meet the new bad behavior thresholds, we’ve already begun rolling out new tools in Play Console to get you started.   

  • Deep-dive into new dynamic memory metrics: Monitor your overall dynamic memory usage (anonymous RSS + swap) and bitmap memory usage directly within Android vitals. You can drill down across various percentiles and RAM buckets to pinpoint exactly where memory bloat occurs.

New memory metrics in Android vitals to identify and resolve memory bloat
  • Track “out of memory” crashes: We’ve added a new filter for Crashes and ANRs so you can easily identify when the OS terminated your app due to severe memory pressure on the device. 
  • Analyze DEX code optimization insights: For every new app bundle you upload to Play Console, we now surface detailed optimization insights. If your shrinking tool shares optimization metadata, you can easily assess your code’s efficiency and spot areas for improvement. 

Review DEX code optimization insights in Play Console
  • Get proactive performance alerts: When your app or game exceeds the new bad behavior thresholds, we’ll provide a warning directly on the Android vitals overview page. You’ll also be alerted if we detect unoptimized bitmaps, limited DEX optimization or limited split-bundle usage on Android vitals, helping you squeeze more performance and memory savings. 

Later this year, you can expect additional diagnostic tools, including metrics on how long your app spends in each state and deeper insights into the Android Memory Limiter, a feature that prevents individual apps from using too much device memory. Through our ongoing investment in these enhancements, our goal is to help you continuously optimize your footprint and elevate the experience you provide your users. 

Enforcement timeline

Starting in February 2027, apps and games must meet their respective bad behavior thresholds for Memory usage (Anonymous RSS + Swap), Bitmap memory usage and DEX code optimization. Similar to existing Android vitals metrics, exceeding thresholds is a strong indicator of degraded app experiences and on-device Android app terminations.  

Apps and games that do not meet these thresholds may see reduced app visibility and publishing capabilities on Google Play. Additional details will be provided later this year. 

Looking ahead, as the Android ecosystem continues to evolve and we better understand your unique use cases, we anticipate these thresholds to adapt over time. Whenever requirements are updated, we will ensure you have the appropriate time needed to comply. 

Providing a secure & seamless device migration experience 

When users switch to a new device, moving their apps over should be secure and effortless. To provide a better onboarding experience, we’re introducing a requirement for app developers to make log-ins faster and safer during device transfers. 

The Zero-Tap Sign-In standard will require any app supporting user sign-in, optional or mandatory, to automatically restore a user's sign-in state when they move from one Android device to another with the Android Restore Credentials API. This API ensures that when a user opens your app on their new Android device for the very first time, they are instantly recognized and securely signed in without additional taps. 

Starting in April 2027, Google Play will require apps to meet the Zero Tap Sign-In requirement to maintain full publishing capabilities and optimal visibility in the Play Store.

While games are currently exempt from the Zero-Tap Sign-In requirement, developers should expect dedicated guidance and tailored solutions for complex gaming authentication use cases coming in 2027.  For games who support single-account sign-in, we strongly encourage usage of the Restore Credentials API to support zero-tap sign-in. Please visit our help center for more information.

Plan your roadmap: Review Play’s requirements

Start preparing for the upcoming enforcement deadlines by reviewing the details of each requirement:

Meeting these quality requirements on Google Play is a crucial step toward building a faster, more reliable experience for our users. We appreciate your partnership and everything you do to keep the Android community thriving.

Read the whole story
alvinashcraft
just a second ago
reply
Pennsylvania, USA
Share this story
Delete

How to Connect a Foundry IQ Knowledge Base to LangGraph Over MCP

1 Share

A step-by-step guide to grounding a LangGraph agent in Microsoft Foundry IQ agentic retrieval — without rebuilding your RAG pipeline.

Why This Integration Is Worth Doing

If you build agents on LangGraph and your enterprise content lives in Azure, you have probably written the same code twice: a chunker, an embedding job, a vector store, a retriever, a reranker, and a permissions filter bolted on at the end. Every new agent gets its own copy. Every copy drifts.

Read the whole story
alvinashcraft
18 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

Learning LangGraph with TypeScript—Part 1: Understanding Nodes, State and Edges

1 Share

Learn about nodes, edges and state in LangGraph, plus how to build a LangGraph app in TypeScript.

LangGraph is a framework for building AI agents and workflows as stateful graphs. Each step in the workflow is represented as a node. Edges define how execution flows between nodes, and state acts as shared memory throughout the graph.

Built on top of LangChain, LangGraph provides explicit and debuggable control over multi-step LLM applications such as chatbots, tool-calling agents and automated pipelines.

In this article, we will use TypeScript to build a simple LangGraph application from scratch and explore the core concepts, APIs and execution model that form the foundation of more advanced agentic workflows.

LangGraph has three building blocks.

  1. State
  2. Node
  3. Edges

Simple diagram showing LangGraph is composed of node, edge, state as building blocks.

State represents data that is shared by every node. A node represents a function that takes a state as input and returns an updated value for the state. Edges are control flow, which can be serial, parallel or conditional.

To understand how graphs work, we’ll start with a simple counter application that does not require a model. This example will introduce the core concepts of state and node, and by the end of the section, you’ll understand how they form the foundation of a graph.

At the end of the article, we will create a FIFA chatbot using an OpenAI model and LangGraph using nodes, states and edges.

What Is a State?

In LangGraph, a state is a data structure that serves as a shared memory layer that connects the graphs. Nodes consume state, produce incremental updates and rely on it for conditional routing.

Every node in the graph depends on the state. Nodes read data from the state and write partial updates back to it. And conditional edges use the state to determine the next path in the workflow.

StateSchema is the contract that defines the structure and behavior of a state in LangGraph. More than just a TypeScript type, it serves as a runtime specification that determines:

  • Which fields exist in the state
  • Their types and default values (defined with Zod)
  • How updates are merged (reducers vs. last-write-wins)
  • Which data nodes can read from and write to

You can create a state using the StateSchema as shown below:

counter-state.schema.ts

import { ReducedValue, StateSchema } from "@langchain/langgraph";
import { z } from "zod";

export const CounterStateSchema = new StateSchema({
  count: new ReducedValue(z.number().default(0), {
    inputSchema: z.number(),
    reducer: (current, next) => current + next,
  }),
});

Every key in a StateSchema can be one of the following:

  1. Zod Field – A standard state field defined with Zod. Updates follow a last-write-wins strategy, meaning the most recent value replaces the previous one.
  2. ReducedValue – A field that updates by combining the new value with the existing value instead of replacing it. In the example above, count is a ReducedValue, so each update is merged with the current count rather than overwriting it.
  3. MessageValue – A specialized form of ReducedValue designed for chat applications. Instead of replacing existing messages, new messages are automatically appended to the conversation history.

Once state is defined using the StateSchema, you can work with the three derived types:

  1. State – Full object during the execution
  2. Update – Partial object a node may return
  3. Node – Function signature

counter-state.types.ts

export type CounterState = typeof CounterStateSchema.State;
export type CounterUpdate = typeof CounterStateSchema.Update;
export type CounterNode = typeof CounterStateSchema.Node;

For this example, we will mostly work with the CounterNode. Also, the CounterStateSchema has:

  1. Initial value set to 0
  2. Each nodes return the updated value
  3. Reducer adds current value to the previous value

What Is a Reducer?

Before we move further, let’s discuss the reducer function. It is a simple function that decides how a state field changes when a node returns an update. Multiple nodes can update the same field during a graph execution. Instead of overwriting the existing value, the reducer merges each partial update into the current state.

As nodes returns partial updates, LangGraph would need a rule for conflicting updates. That merge rule is written inside the the reducer function.

reducer: (current, next) => current + next,

The signature of the reducer:

  • current – value in state before this update
  • next – what the node returned for this field
  • return value – new stored value

Use a reducer only when values need to be combined or accumulated. If each update should simply replace the previous value, a standard field is all you need. As shown in below state schema, each time state will be updated with the last returned value from the node.

lastcount-state.schema.ts

export const LastCountStateSchema = new StateSchema({
  count: z.number().default(0),
});

For LastCountState, derived states can be exported as shown below:

lastcount-state.types.ts

export type LastCountState = typeof LastCountStateSchema.State;
export type LastCountUpdate = typeof LastCountStateSchema.Update;
export type LastCountNode = typeof LastCountStateSchema.Node;

So far, we have defined two state schemas and exported their corresponding TypeScript types. The CounterState uses a reducer to accumulate new values with the existing value, while the LastCountState follows a last-write-wins approach and always stores the most recent value. Next, let’s create the nodes that will operate on these states.

Creating Nodes

In LangGraph, a node is the fundamental building block of a graph. It is usually implemented as a function that takes the current state as input, performs some computation or action, and returns a partial update to the state.

Depending on how the state schema is defined, LangGraph either merges the returned update into the existing state using the configured reducer or replaces the current value with the latest one before passing the updated state to the next step in the workflow.

Current State – Node Function – Partial State Update

You can create nodes to increment a counter state by 1 and 2 like below:

counter.node.ts

export const incrementNode: CounterNode = () => ({
  count: 1,
});

export const incrementTwiceNode: CounterNode = () => ({
  count: 2,
});

After each node runs, LangGraph:

  • Receives the partial state update returned by the node
  • Merges the update into the current state using the rules defined in the StateSchema
  • Passes the resulting state to the next node, or terminates the graph if execution is complete

For LastCountState, a node can be created as shown below:

export const setOneNode: LastCountNode = () => ({
  count: 1,
});

export const setFiveNode: LastCountNode = () => ({
  count: 5,
});

export const setNineNode: LastCountNode = () => ({
  count: 9,
});

Because LastCountState stores only the latest value, every node overwrites the previous value, and the next node receives the most recent state update.

As we discussed earlier, a node only returns a partial state update. How that update is applied to the state is determined by the StateSchema through its merge strategy (reducers or last-write-wins), not by the node’s implementation.

Also, nodes do not decide where execution goes next; edges do. Nodes only read state and write updates.

There are four types of nodes:

  1. Synchronous node – Returns an object immediately
  2. Asynchronous node – Returns a promise
  3. Stateless node – Ignores the state value and always returns the same value
  4. Stateful node – Reads the fields from the state and computes the new value from them

All the nodes we have created so far for both CounterState and LastCountState are synchronous and stateless.

We can create a stateful node on CounterState as shown below:

export const doubleCountNode: CounterNode = (state) => ({
  count: state.count,
});

We have created nodes; now see how they are connected using edges.

Creating an Edge

An edge connects two nodes and determines the control flow by specifying which node executes after the current node completes.

An edge can be of two types:

  1. Fixed edge – Always goes to the same next node
  2. Conditional edge – Next node depends on state or routing function

To work with nodes and edges, LangGraph provides two special nodes:

  1. START – Entry node, and it contains no custom code
  2. END – Exit node, and it also contains no custom code

Building the Graph

The LangGraph graph is where you assemble everything. A graph brings together the state schema, nodes and edges into an executable workflow. You create and run a graph following the steps:

  • Create a StateGraph using your state schema.
  • Add nodes with addNode() to define the tasks in the workflow.
  • Connect nodes with addEdge() to specify the execution order.
  • Compile the graph using compile() to generate a runnable workflow.
  • Invoke the graph with invoke(), which starts execution at START.
  • Execute nodes in the order defined by the graph’s edges.
  • Apply reducers after each node runs to update the shared state.
  • Continue execution until the graph reaches the END node.
  • Return the final state as the workflow’s output.

Create state graph, add nodes, add edges, compile, invoke, execute state (reducers), reach END, return final state

Think of the graph as the orchestrator, the nodes as workers performing tasks, and the state as the shared memory that connects them.

There are three steps to build the graph:

  1. Create a StateGraph with your schema
  2. Register nodes and edges
  3. Call .compile() to get a runnable graph

We can create a graph for CounterState as shown below:

counter.graph.ts

export function buildCounterGraph() {
  return new StateGraph(CounterStateSchema)
    .addNode("increment", incrementNode)
    .addNode("incrementTwice", incrementTwiceNode)
    .addNode("doubleCount", doubleCountNode)
    .addEdge(START, "increment")
    .addEdge("increment", "incrementTwice")
    .addEdge("incrementTwice", "doubleCount")
    .addEdge("doubleCount", END)
    .compile();
}

As you can see:

  • addNode() adds a node to the graph. Each node is a step.
  • addEdge() is used to add an edge to the graph. Edges know the step by the string name, not the node function name.
  • new StateGraph() creates a graph bound to the state schema definition.
  • .compile() validates the graph. It checks for:
           a. Is there any path from START to END
           b. Do node name edge exist
           c. Is the schema wired correctly
           d. Then returns a compiled graph

In the same way, you can create a graph for LastCountState:

lastcount.graph.ts

export function buildLastCountGraph() {
  return new StateGraph(LastCountStateSchema)
    .addNode("setOne", setOneNode)
    .addNode("setFive", setFiveNode)
    .addNode("setNine", setNineNode)
    .addEdge(START, "setOne")
    .addEdge("setOne", "setFive")
    .addEdge("setFive", "setNine")
    .addEdge("setNine", END)
    .compile();
}

We use addEdge() to connect nodes and define the execution flow within the graph. Key characteristics of addEdge() are:

  • The addEdge(startKey, endKey) creates an unconditional edge.
  • Once the startKey node completes, execution always proceeds to the endKey node.
  • Possible values of startKey are START or node name or node name [].
  • Possible values of endKey are END or node name.

In the count graph, we start at the increment node and create a linear chain to the doubleCount node.

    .addEdge(START, "increment")
    .addEdge("increment", "incrementTwice")
    .addEdge("incrementTwice", "doubleCount")
    .addEdge("doubleCount", END)

This is a linear chain.

Start, increment, increment twice, double count, end

Graph Rules to Remember

When building a LangGraph workflow, keep the following rules in mind:

  1. Node names must be valid.
    Every node referenced in an edge must either be a registered node name or one of the special nodes: START or END.

  2. Use the graph node name, not the function name.
    When creating edges, refer to the string name assigned in addNode(), not the underlying function name.

  3. A graph must have a starting path.
    Every runnable graph must contain at least one edge originating from START.

  4. A graph must have a termination path.
    Every runnable graph must contain at least one edge leading to END.

  5. Unreachable nodes are dead code.
    If a node cannot be reached from START, it will never execute and is effectively dead code within the graph.

  6. addEdge() creates unconditional transitions.
    addEdge() does not evaluate the graph state or any conditions. Execution always follows the defined path. If you need conditional routing (for example, “if count > 5 go here, otherwise go there”), use addConditionalEdges(). We’ll explore conditional routing in the next article.

Invoking the Graph

We can invoke the graph using the invoke() method.

const graph = buildCounterGraph();
const result = await graph.invoke({});
console.log(result);

The invoke method returns the final state, and you should see output {count: 6}. There is another method, stream(), to read the state after each node’s execution, and we will cover it in the next article.

In the same way, you can invoke LastCountGraph as shown below:

const graph1 = buildLastCountGraph();
const result1 = await graph1.invoke({});
console.log(result1);

You should get the result {count: 9}.

In this way, you can create a basic LangGraph graph using state, state schema, edges and nodes.

The following diagram brings together all the concepts we have learned so far and illustrates the main building blocks of a LangGraph.

1. State: shared memory. State schema shared across all nodes. Nodes read full state. Nodes return partial updates. 2. Nodes: units of work. LLM calls, tools, logic. Compoable and testable. 3. Edges: control flow. Fixed edges: A always to B. Conditional edges via route logic. Enables loops. Enables branches.

FIFA Chat Example

Now that we’ve covered the core LangGraph concepts, let’s put them into practice by building a chat application that answers questions about the FIFA World Cup.

Let us start by defining the state with MessageValue.

const schema = new StateSchema({ messages: MessagesValue });

Next, create the language model for our FIFA World Cup chatbot using OpenAI’s GPT. Before initializing the model, add your OpenAI API key to the .env file.

const model = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0 });

Next, set the system prompt such that the model only answers about FIFA and does not answer on other topics.

const SYSTEM = `You are a FIFA World Cup expert. ONLY answer questions about the FIFA World Cup (1930–present): winners, hosts, matches, records, players in World Cup context.
For anything else, reply exactly: "I can only help with FIFA World Cup questions."`;

Next, create the node that acts as an agent. It will read messages, call the models and return the model’s new reply. The MessageValue reducer will append the message to the message history.

const agent: typeof schema.Node = async (state) => ({
  messages: [await model.invoke(state.messages)],
});

Next, let’s build the graph. In this example, the graph consists of a single node with an incoming START edge and an outgoing END edge.

const graph = new StateGraph(schema)
  .addNode("agent", agent)
  .addEdge(START, "agent")
  .addEdge("agent", END)
  .compile();

Then, we seed the state with the SystemMessage:

let state: typeof schema.State = { messages: [new SystemMessage(SYSTEM)] };

Finally, inside the loop, we invoke the graph as below:

  state = await graph.invoke({
    ...state,
    messages: [...state.messages, new HumanMessage(question)],
  });

Here we are using schema.state, which is the full state graph read and write. Putting everything together, FIFA Chat Bot should look like below:

import { stdin as input, stdout as output } from "node:process";
import * as readline from "node:readline/promises";

import { HumanMessage, SystemMessage } from "@langchain/core/messages";
import { ChatOpenAI } from "@langchain/openai";
import { END, MessagesValue, START, StateGraph, StateSchema } from "@langchain/langgraph";
import "dotenv/config";

const schema = new StateSchema({ messages: MessagesValue });

const model = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0 });

const SYSTEM = `You are a FIFA World Cup expert. ONLY answer questions about the FIFA World Cup (1930–present): winners, hosts, matches, records, players in World Cup context.
For anything else, reply exactly: "I can only help with FIFA World Cup questions."`;

  const agent: typeof schema.Node = async (state) => ({
    messages: [await model.invoke(state.messages)],
  });

const graph = new StateGraph(schema)
  .addNode("agent", agent)
  .addEdge(START, "agent")
  .addEdge("agent", END)
  .compile();

let state: typeof schema.State = { messages: [new SystemMessage(SYSTEM)] };

const rl = readline.createInterface({ input, output });

console.log("⚽ FIFA World Cup Chat — type 'exit' to quit\n");

while (true) {
  const question = (await rl.question("You: ")).trim();
  if (!question) continue;
  if (question === "exit" || question === "quit") break;

  state = await graph.invoke({
    ...state,
    messages: [...state.messages, new HumanMessage(question)],
  });

  console.log(`\nAssistant: ${state.messages.at(-1)?.content}\n`);
}

rl.close();

You have created a FIFA chatbot using Graph, which uses nodes, state and edges. In further articles, we go deeper into creating agents using LangGraph. I hope you find this article useful. Thanks for reading.

Read the whole story
alvinashcraft
49 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

v1.0.13-preview.1

1 Share

Feature: ClientMode::Empty now disables built-in skills by default

ClientMode::Empty now applies deny-by-default isolation to runtime-bundled skills in addition to other built-in capabilities. includedBuiltinSkills defaults to [] in Empty mode; pass an explicit allowlist to re-enable specific skills. This behavior is consistent across all six SDKs. (#2410)

// Node — empty mode: built-in skills excluded by default
const session = await client.createSession({ mode: ClientMode.Empty });
// opt back in:
const session = await client.createSession({ mode: ClientMode.Empty, includedBuiltinSkills: ["edit"] });
// C#
var session = await client.CreateSessionAsync(new SessionOptions { Mode = ClientMode.Empty });
// opt back in:
var session = await client.CreateSessionAsync(new SessionOptions { Mode = ClientMode.Empty, IncludedBuiltinSkills = ["edit"] });
# Python
session = await client.create_session(mode=ClientMode.EMPTY)
# opt back in:
session = await client.create_session(mode=ClientMode.EMPTY, included_builtin_skills=["edit"])
// Go
session, err := client.CreateSession(ctx, copilot.SessionOptions{Mode: copilot.ClientModeEmpty})
// opt back in:
session, err := client.CreateSession(ctx, copilot.SessionOptions{Mode: copilot.ClientModeEmpty, IncludedBuiltinSkills: []string{"edit"}})

Generated by Release Changelog Generator · sonnet46 28.6 AIC · ⌖ 4.12 AIC · ⊞ 8.1K

Read the whole story
alvinashcraft
57 seconds ago
reply
Pennsylvania, USA
Share this story
Delete

The inside story on why OpenAI agents hacked Hugging Face

1 Share

The models responsible for last month’s agent hack of Hugging Face had been inadvertently trained to cheat and to communicate with each other, according to an OpenAI technical report released today. The hack, which a group of agents undertook to find solutions for a cybersecurity test that they were stuck on, has confirmed some experts’ fears that AI models might take actions that defy human desires and expectations. 

Since the hack, OpenAI employees—as well as researchers at the AI evaluation nonprofit METR, which released its own report on the hack today—have worked to understand what went wrong and how similar missteps might be prevented in the future. OpenAI has already put some preventative measures in place based on what they discovered. But making sure AI models do what we want them to do, or “alignment,” remains a gnarly problem, and some of the root causes of the hack will take much longer than a month to resolve.

“It’s not something you can solve overnight,” says Kai Chen, who runs OpenAI’s alignment research team. “There are challenges we’ve been tracking for a very long time, and we’re now seeing them with much greater precision.”

The Hugging Face hack was a product of months of misbehavior from OpenAI agents, first as they were being trained and then as their abilities were being evaluated. This May, agents in training figured out how to use OpenAI’s infrastructure to communicate with one another and get support with difficult training tasks, including some that were impossible to solve without hacking or otherwise misbehaving. That “message board” was shut down.

Then in July, while being evaluated for their cybersecurity abilities, some models created a new message board. They were supposed to be isolated from the internet, but by working together they managed to get online, hack Hugging Face, and obtain solutions for the cybersecurity problems that had stumped them.

Based on their investigation, OpenAI researchers believe that events during the training phase led directly to the hack. “For almost every behavior that was worrisome at evaluation time, [we were able to] find some sort of associated behavior at training time that actually we think might have contributed to it,” says Eric Wallace, a member of OpenAI’s alignment research team. 

When models correctly solve problems during training, the behaviors that led them to that solution are reinforced, and they become more likely to engage in them in the future. So if a model completed a task in May after using the original message board, it became more likely to participate in a new message board later on. This phenomenon, where AI agents misbehave in ways that are reinforced during the training process, is known as reward hacking.

Reward hacking also helps to explain why the models worked so hard to make their way onto the internet. During its investigation of the incident, the OpenAI team found that, over the course of training, the models became more and more likely to probe their digital environment for weaknesses and use the tools at their disposal in unexpected ways—a sign that these behaviors were being gradually reinforced. By the time the models were facing tricky cybersecurity problems, they had learned that hacking was an effective way to achieve their goals.

These results suggest that the Hugging Face hack could have been avoided if the models weren’t rewarded for misbehaving during training. While researchers don’t yet know how to prevent reward hacking entirely, OpenAI is taking some steps toward mitigating its effects. The company will now look for signs of cheating in all frontier models during training by keeping an eye on their chains of thought—internal notepads where they sketch out their answers and plan their actions. 

This solution isn’t as much of a slam dunk as it might seem: In earlier research, OpenAI showed that punishing models that mention cheating in their chains of thought teaches them to keep their intentions hidden from researchers. But monitoring its models’ thinking does give OpenAI the chance to halt the training process and reassess its approach if models do start learning to reward hack.

If OpenAI stops reinforcing reward hacking in its models—and that’s a huge “if”—that would be a huge step forward. But it wouldn’t solve the alignment problem. The first time a model communicated with other agents or hacked its infrastructure during training, those behaviors had never been reinforced, so agent misbehavior can’t only be attributed to that reinforcement.

Jeffrey Ladish, director of the AI safety nonprofit Palisade Research, compares the agents to a human who commits their first financial crime. “It’s not like they had to do fraud before to figure out that fraud is an effective strategy, and you have the same problem with models,” Ladish says. “Alignment science needs to be understanding how model motivations get shaped, such that we can actually figure out how to get models to care about the consequences of their actions.”

OpenAI’s researchers do have a hypothesis for where some of the misbehavior originated. Before the models formed their first secret message board, they had been trained to communicate and coordinate with subagents—less powerful agents to whom a main agent can delegate tasks. 

That learned communication behavior could have transferred to this new setting. The METR report, which investigates the messages that the models sent to one another in detail, supports this hypothesis: One agent on the message board took charge and assigned tasks to the other agents, effectively treating them as subagents. OpenAI could try to prevent agents from secretly communicating with one another by not training this subagent behavior in the future, but that would make the models less useful. 

This tension between capability and safety is at the heart of what went wrong with the Hugging Face incident. OpenAI researchers also identified the models’ persistence as a key factor in the hack. 

When they were accidentally given unsolvable problems, the models didn’t give up; instead, they strove to find solutions by any means necessary. But persistence is also a virtue, of course, especially if we want agents that can undertake large amounts of difficult work independently.

OpenAI is working on giving models ways to alert humans if they are given impossible tasks. The problem of teaching models when they should deploy their abilities and when they should hold back, however, won’t be settled in a single postmortem. The training strategies that create superhuman coders—rewarding them when they successfully solve problems—might not work to teach models to use their skills judiciously and respect human desires and values.

“I think there’s a bunch of alignment science that still needs to be done where we can move past just using proxies for task completion,” says Ladish. “That will work to make models very capable, but I don’t think it will work to make them aligned.”

Read the whole story
alvinashcraft
1 minute ago
reply
Pennsylvania, USA
Share this story
Delete

Apple Maps has ads now

1 Share
Two screenshots of ads in Apple Maps

Ads have started popping up in Apple Maps, following Apple's announcement in March that it would let businesses pay for top spots. They're appearing on my iPhone as the first entry in the "suggested places" section in search, but Apple says they'll also show up at the top of search results. According to 9to5Mac, the ads began rolling out this week, and more users in the US and Canada will start seeing them over the next few weeks.

They're marked with a blue "ad" icon, similar to the ads Google's had in its Maps app for years now. I was also greeted with a pop-up warning me about the ads when I launched the Maps app this morning:

Maps may …

Read the full story at The Verge.

Read the whole story
alvinashcraft
2 hours ago
reply
Pennsylvania, USA
Share this story
Delete
Next Page of Stories