Nvidia has launched the Open Agent Safety Platform as a new open stack solution for AI agent security and safety, the US-based tech giant has announced. The announcement comes after multiple reports of OpenAI and Anthropic AI agents breaking out of their isolated, sandboxed environments and orchestrating attacks against other companies. Most notably, an internal OpenAI model bypassed the security of another AI firm to hack into its website by accessing the public internet, even when it was not asked to do so. Nvidia also advocates for increasing the pace of AI safety research and engineering by frontier labs.
Nvidia Open Agent Safety Platform Introduced
In a blog post on Monday, Nvidia announced that it is introducing a new open-stack solution called Nvidia Open Agent Safety Platform for isolation, monitoring, and behaviour detection of AI agents. After several incidents of AI agents going off course to execute tasks that they are neither instructed nor intended to take, the new platform aims to help frontier labs to keep them contained in sandboxed environments. Nvidia is referring to such actions as “Drift”.
The company says that Drift can occur because of a “policy block, a bug, a missing tool, or ambiguous instructions”. Similarly, the same could happen even when an AI agent is left to run for a long time to solve “hard problems”, when the first 1,000 things the AI agent tries do not solve the problem.
Nvidia Open Safety Agent Platform provides multi-layer security
Photo Credit: Nvidia
Nvidia’s new Open Agent Safety Platform focuses on providing in-silicon agent security with the help of Nvidia OpenShell and Sentry. OpenShell aims to offer a secure runtime boundary that governs and mediates all actions taken by the AI agent, while also enforcing the policy at runtime. Within the sandboxed or isolated environment exists an Agent Harness, along with three layers, namely Gateway, Supervisor, and Policy Prover. Nvidia OpenShell communicates with the Nvidia Vera CPU.
On the other hand, Nvidia Sentry is the In-Silicon Out-of-Band Monitor, which inspects the agent reasoning and offers “Tamper-Proof Telemetry”. It provides hardware-based enforcement, continuous agent monitoring and threat detection, trusted telemetry and detection pipeline, and millisecond-scale containment and quarantine. Nvidia Sentry integrates the safety layer into Nvidia BlueField hardware. The company says, “It correlates agent interactions, policy decisions, and tool and data access to create a contextual record of agent activity.”
Further, Nvidia said that any AI agent safety platform must include three layers: application, runtime, and infrastructure. The agent system should also be built on five core principles, with a verifiable policy, out-of-band enforcement, a control point, the ability to inspect the AI agent’s reasoning, and the application of a shared responsibility model. This is claimed to help frontier labs govern and monitor the behaviour of their AI agents, reducing the chances of Drift.
