NewsCSO Online

    OpenAI’s Lockdown Mode is trying to solve the problem that it created

    Share
    OpenAI’s move to implement a Lockdown Mode that tries to limit data exfiltration by shutting down external capabilities is being seen as making the best out of a bad situation. But Lockdown Mode doesn’t block exfiltration as much as it slightly reduces it, and the reality of enterprises using multiple AI vendors for their agentic models further complicates an already dicey governance strategy. When activated within OpenAI products’ settings, Lockdown Mode limits web browsing to cached content, limits image support, disables Deep Research and Agent Mode, denies users the ability to approve Canvas-generated code to access the network, and prevents ChatGPT from downloading files for data analysis, though it can still operate on manually uploaded files, OpenAI said in a blog post . The company did not respond to a request for comment. That post included a frequently-asked-questions section in which OpenAI wrote its own questions. and then answered them. One notably asked “Is prompt injection a major risk?” with the response, “Prompt injection is not currently a major risk, but its impact could grow as attackers develop more sophisticated methods.” Consultants found that sentence baffling. “OpenAI’s own posture is telling. It calls prompt injection a frontier research problem, hard enough to warrant a containment mode, while saying in the same breath that it is not currently a major risk,” said Sanchit Vir Gogia , chief analyst at Greyhound Research. “A vendor does not build a panic room for a house it believes is safe. Lockdown Mode is the admission itself.” And the risk of AI-enabled data exfiltration was illustrated recently when some Instagram users’ personal data was stolen after Meta had turned over control of password changes for accounts to an AI agent. Still allows some exfiltration Gogia added that the Lockdown Mode is porous, as it will still allow some data exfiltration; he called the OpenAI effort “a model carrying a trusted user’s authority while acting on instructions hidden in untrusted content. Data can leave by a side door rather than be announced in the chat.” Tom Findling , CEO of Conifers.ai, also questioned whether OpenAI could block all of what it claims it can block. “It is yet to be seen whether [Lockdown Mode] can be breached or not. Is it Nirvana? Probably not, but this is likely the best they could have done, given the infrastructure they have today.” An executive with a major agentic cybersecurity firm, who asked to be not named, agreed with Findling: Lockdown Mode “is not going to be validated until someone tries breaking it. Almost every sandboxing solution out there, AI has been able to break out of,” he said. Debate over who has control Analysts and consultants disagreed over whether enterprises should use the OpenAI capabilities for isolation or use the enterprise’s own restrictions. “The question I immediately asked myself was whether organizations need OpenAI to do this for them. The answer, in my opinion, is no,” said Erik Avakian , technical counselor at Info-Tech Research Group. “Security professionals have been implementing similar concepts for years through control areas like network segmentation, least privilege, applying Zero Trust concepts and principles, application controls, and ‘air-gapping’ some environments.” Flavio Villanustre , CISO for the LexisNexis Risk Solutions Group, also has doubts. “So long as the LLM and associated components are provided as a service by OpenAI, customers can only partially control where those systems can reach out, so this lockdown mode seems to be the answer to that,” he said. “Yes, customers could use a secure gateway,” he added, “but if the LLM and/or agent sitting at OpenAI premises accesses other third party services, there would not be a way for the IT and/or cybersecurity team from the customer to restrict this. The most secure approach is always the deployment of the AI infrastructure on premises, but that’s just not viable for the majority of organizations.” Dennis Xu , a research VP with Gartner, flatly stated that enterprises need to rely on AI vendor provided cutoffs. “This is not something end user clients can do on their own. As this controls how traffic flows from OpenAI infrastructure, the ChatGPT application, going outbound, only OpenAI has the ability to control that flow. ChatGPT is a web/SaaS based application that cannot be air gapped,” Xu said. “In the shared responsibility model, this falls under provider responsibility. End user clients will need to rely on what is available from providers such as OpenAI. Without that, they have no control over this data flow. So if they like this OpenAI feature, they need to raise this as a feature request with other providers for them to implement into their solution.” That can get exponentially more complex if all AI vendors deploy such shutoff valves in different ways. Gogia noted that vendor-specific controls are useful tactically and weak strategically, because each vendor can only constrain its own product. “OpenAI can limit OpenAI but it cannot govern a local model in a business unit or an assistant embedded elsewhere,” he said. “Its own model shows the limit: in managed workspaces, apps and connectors remain governed by role-based access and Lockdown Mode does not automatically disable every app. The hard work does not vanish. It moves into governance.” Villanustre added that the result will be that customers may need to deal with “a patchwork of controls” until independent third party governance tools come to the rescue and support this cross-vendor management model. As well, Avakian said, “rather than relying on a single AI platform, organizations will likely use multiple models from multiple vendors, in which each will serve different business functions. We might soon find ourselves talking about AI trust zones, AI segmentation, AI least privilege, and AI governance frameworks the same way we talk today about network segmentation and Zero Trust architectures.” However, Carmi Levy , an independent technology analyst, said that the OpenAI move is an improvement, albeit an incremental one. “It is not a replacement for pre-existing best practices within any organization. Rather, it enables greater in-model protections before organizational limitations can be imposed. With different vendors incorporating different lockdown modes into their models, IT is challenged to update its own protocols to integrate with an increasingly diverse vendor landscape,” he said. “There’s no getting around the fact that this will add ongoing overhead to IT and cybersecurity operations, as different vendors continue to evolve their own protection-focused regimes.” Humans are the problem One of the reasons that Lockdown Mode can’t halt all exfiltration, even if it works perfectly, is the human factor, coupled with the tendency of autonomous agents to bypass rules. For example, let’s say that an end user works for a large publicly-held American company, and the user asks the agent to gather financial details about an upcoming quarter’s revenue and net income. Security and Exchange Commission (SEC) rules in the US make it illegal to selectively share that unannounced data with the public. If the agent finds a way to access internal emails and documents from Finance and shares the answer with the end user, and that end user then copies and pastes that information into an email sent to some investors, or possibly even a financial journalist, the user is in contravention of the rule; the model that supplied the data may not have even known that this disclosure was prohibited. Expands the attack surface Justin Greis , CEO of consulting firm Acceligence, noted that the most interesting thing about Lockdown Mode is that it acknowledges a reality many organizations are wrestling with: AI’s value often comes from its ability to connect to systems, access data, browse the web, and take action. “Those same capabilities also expand the attack surface. As AI becomes more integrated into critical business processes, the conversation shifts from maximizing capability to balancing capability with control,” he said. “The broader implication is that we’re likely moving toward a world where AI systems have configurable operating modes based on business context, data sensitivity, user privileges, and risk tolerance. That’s a much more nuanced model than the all-or-nothing approaches we’ve seen so far.” Greis would like the OpenAI option to offer IT granular functionality choices. “IT needs to have the availability to configure it and not just accept the default settings from OpenAI,” he said. For example, IT might want to customize based on connectors, or GPTs, or models, or zones, or regions. Another Gartner VP analyst, Nader Henein , said that OpenAI created Lockdown Mode “with a narrow set of clients in mind, specifically for non-classified government use, potentially for specific governments, the reason being that if an enterprise client has this level of concern regarding data sensitivity, they are not likely going to trust any provider, including OpenAI,” he pointed out. “Those clients are likely to seek on premises large language models, or large language models hosted in secure, trusted environments.”

    More on OpenAI’s

    Other news entries

    Source: https://www.csoonline.com/article/4182650/openais-lockdown-mode-is-trying-to-solve-the-problem-that-it-created.html

    ID