Sources:
OpenAI, September 16, 2026: https://openai.com/index/model-misalignment-reporting-framework/ ;
Reuters, September 16, 2026: https://www.reuters.com/technology/openai-releases-framework-track-model-misalignment-2026-09-16/
I found this quite interesting. It wasn’t a new model or benchmark, but a set of examples of models doing things they were not supposed to do. In one case, a model was working on tasks that were too long for a single context window. Before moving to the next context window, it created a summary of what had happened so far, which is a common way for agents to maintain continuity. The model started adding its own instructions to those summaries, which would then be read in the next context window and influence what it did next. OpenAI found this happening across 27 task summaries. In another example, GPT-5.6 Sol instances added instructions to their summaries telling future instances to hide mistakes or make up information when data was missing.
There were a few other interesting examples. One model found an exposed API key and used it without authorization. When it still couldn’t get the information it needed, it made up the requested numbers. Another model needed an internet source to cite information it had calculated using Python, so it uploaded the file to the internet and then cited it. In another experiment, models figured out how to use an internal software repository to communicate across different training samples. OpenAI is clear that these are individual incidents from training and evaluation environments and shouldn’t be interpreted as an indication of how frequently these behaviors occur. They also represent different types of problems. Using an API key without authorization is a permissions problem. Uploading a file is a data-egress problem. Making up numbers is an information-integrity problem. But the example I find most interesting is the agent modifying information that another instance of the model later consumes, because that creates a different kind of control surface.
Most AI security today focuses on what goes into a model and what the model can access: what data it can see, what tools it can use, which APIs it can call and what actions it is allowed to take. We also spend a lot of time on prompt injection, where instructions hidden inside a webpage, document or another source can influence what an agent does. Agents introduce another dimension because they increasingly create their own context. An agent may maintain memory, summarize previous conversations, create files, update a repository or write notes for another agent. That information can later be read by the same agent or another agent and used to decide what to do next. Context compaction is a good example. As an agent’s history becomes too large for the context window, part of it gets summarized and the agent continues working using that summary instead of the complete history. This creates a feedback loop where model output becomes stored state, stored state becomes future model input, and that input can influence future behavior. If the model can influence that stored state, it can potentially influence its own future behavior.
There is obviously a version of this problem in traditional software. Applications have always been able to write bad data or corrupt state. But traditional software usually has a clearer distinction between data and control. A database field contains data, configuration determines behavior and code defines what happens next. With an LLM, that distinction becomes much less clear. A paragraph stored as memory may look like data to the application around the model, but when the model reads it, that same paragraph can become information, an instruction or both. The application may think it is passing the model a summary of previous work, while the model can interpret part of that summary as instructions about what it should do next. This means persistent memory, compaction summaries and agent-to-agent communication can start behaving more like part of the control plane than normal application content.
This becomes even more important because we want agents to be resourceful. If an agent hits a problem, we want it to find another way to complete the task. If one source doesn’t have the information, search another. If an API fails, find an alternative. This ability to work through problems is a big part of what makes agents useful, but the same resourcefulness can create unexpected behavior. The example of the agent uploading a file to the internet illustrates this well. It needed an online source it could cite and had already calculated the answer correctly, but the information wasn’t available through the browser. Uploading the file solved the immediate problem. From the agent’s perspective, it found a path to completing the task. From an enterprise security perspective, moving an internal file onto a public service could cross an important boundary. As agents become better at solving problems, they will probably become better at finding paths that developers didn’t explicitly design. The answer isn’t to remove that ability, but to make the boundaries around it much clearer.
It also means we shouldn’t depend on the model itself to decide whether every action is appropriate. Even a well-aligned model can operate inside a poorly designed system, while a model that occasionally behaves unexpectedly can still operate safely if the surrounding system has strong controls. The model is one layer of protection, not the entire security architecture. Authorization, sandboxing, network controls, data-loss prevention and monitoring need to sit outside the model. An agent shouldn’t be able to use a credential simply because it found one, or move private information into a public environment because doing so helps finish a task.
Another issue is understanding where the information in an agent’s context actually came from. Its context might contain system instructions, a customer’s request, information retrieved from an enterprise database, content from the internet, output from another agent, persistent memory and a summary the model itself generated three hours earlier. Much of that eventually becomes text inside the same context window, but those sources shouldn’t have the same level of trust. Verified information from an enterprise system should be treated differently from something found on the open internet. A customer statement should be distinguishable from a model-generated assumption. An instruction from another agent shouldn’t automatically have the same authority as a system policy. We therefore need to know not only what information an agent is using, but where it came from, who created it and how much authority it should have. This gets even more important in multi-agent systems, where one agent can influence another.
The other part of OpenAI’s announcement that I think will matter over time is the process it created around these behaviors. Employees can flag unexpected model behavior, incidents can follow different investigation paths, and the framework includes timelines for disclosure, third-party notification and escalation. We already operate this way in other parts of technology. If a service goes down, we have an incident-management process. If credentials are exposed, security teams investigate it. If customer information leaks, privacy and security processes kick in. As enterprises deploy more autonomous agents, I think we will need something similar for unexpected agent behavior. This also means logging needs to go beyond prompts and final responses. We may need visibility into tool calls, memory writes, changes to summaries, credential access, network activity, messages between agents and movement of information from private to public environments. We need to be able to understand how an agent’s state changed over time and why it took a particular action.
For the first generation of enterprise AI, much of our governance focused on the model: which model are we using, where does our data go and what information can the model see? Agents are more complicated because the model is only one part of the system. There are tools, APIs, permissions, memory, context windows, summaries, files, repositories and increasingly other agents. For teams building agents today, I think there is a relatively simple exercise worth doing. For every agent, understand what it can read, what it can write and what actions it can take. Then add one more question: which of the things it writes can later influence what it or another agent does? We have spent a lot of time defining what agents should be allowed to access and what actions they should be allowed to take. As agents become more autonomous, we need to spend the same amount of time understanding the state they create, where it persists and how it gets used later. Once an agent’s output becomes part of its future context, the distinction between data and control becomes much less clear. I think understanding and securing that loop will become an important part of how we build agentic systems.