NewsCSO OnlineSMEFinancePublic sector

    What the 3M ChatGPT case reveals about AI governance

    One detail in the Watson Grinding explosion litigation involving 3M changed the way I think about prompt governance. An engineering expert retained by 3M had been using ChatGPT while developing his analysis, and among the conversations that later surfaced was a prompt telling the system to “show how 3M is 0% at fault.” The reporting does not establish that 3M instructed the expert to use ChatGPT or directed him to enter that prompt, and that distinction matters. What matters just as much is what happened when the underlying AI interaction itself became relevant. Plaintiffs’ attorney Will Moye described questioning the expert during his deposition after encountering material that appeared to have been generated with ChatGPT. According to Moye, the deposition went off the record while he demanded the underlying prompts. Roughly three hours later, more than 350 pages of previously unproduced ChatGPT material were provided. The issue was no longer limited to what appeared in the expert’s finished report. The conversation behind it had become part of the inquiry. That immediately felt familiar to me. I have spent much of my career working in environments where, when something goes wrong, or a decision is challenged, the final document is rarely enough to explain what happened. You reconstruct the story from the surrounding evidence: an approval, a system log, a ticket, a message that changed the direction of the work, or a process step that was supposed to happen but did not. The official artifact may tell you where the process ended, but the surrounding trail often provides the context needed to understand how it got there. AI is now adding another layer to that trail. The prompt can become part of the decision record For years, enterprise conversations about generative AI risk have focused heavily on inputs. Organizations have warned employees not to paste proprietary code into ChatGPT, upload sensitive customer information, expose personally identifiable information or enter protected intellectual property into public AI tools. Those controls are necessary, and my own experience in regulated technology environments reinforces why organizations started there. But protecting the input addresses only part of the problem. AI interactions can also preserve information about the process that produced the final answer. Consider an engineer using an AI assistant to compare two architectures and repeatedly changing assumptions until the preferred option wins. A procurement analyst could ask AI to construct the strongest rationale for a vendor that has effectively already been selected. A manager could use it to help document an employment decision after that decision has been made, while an auditor or compliance professional might continue revising prompts until a control deficiency sounds less serious. None of those scenarios requires the AI to hallucinate or malfunction. The technology could perform exactly as designed while the interaction captures assumptions, preferred outcomes, rejected alternatives and lines of inquiry that never appear in the final artifact. That does not mean a prompt proves someone’s reasoning or intent. People use AI to test arguments, play devil’s advocate, challenge their own assumptions or explore positions they may ultimately reject. A single prompt taken out of context can therefore be misleading. But that is also why the interaction history may matter: it can provide evidence or context about how an analysis developed that the polished output alone does not provide. The American Bar Association has already examined AI chat histories as an emerging source of discovery material , in part because those conversations can preserve questions, abandoned theories and lines of reasoning that never make their way into a finished work product. From an operational-governance perspective, I see something broader than a discovery issue. AI interactions can become part of the evidence surrounding a consequential decision, yet many organizations have not decided when to retain that evidence, who owns it or how to govern it. We protected the input but overlooked the lifecycle The Watson Grinding case changed one of the questions I now ask when I think about enterprise AI. I still want to know what information an organization is giving the model, but I also want to know what record is being created while people use it. That second question moves the conversation beyond acceptable-use policies and into information lifecycle management. An AI interaction does not necessarily end when the answer appears on the screen. There is the conversation itself, the generated output, how the material is shared, where it is stored, how long it remains available, who can retrieve it and what happens when someone eventually deletes it. Most organizations have spent far more time thinking about the beginning of that lifecycle than the end. Even something as ordinary as a shared conversation illustrates the problem. OpenAI’s guidance on ChatGPT shared links explains that anyone with a shared link can view the associated conversation. OpenAI removed the separate search-engine discoverability option for shared conversations in 2025, but the broader governance issue remains: information created inside what feels like an individual workspace can become accessible outside it through an ordinary product feature. Multiply that across an enterprise where employees may be using ChatGPT, Copilot, Claude, Gemini and specialized AI applications with different retention settings, administrative controls, logging capabilities and sharing options, and the challenge quickly extends beyond basic AI acceptable use. I do not think the answer is to preserve everything. One lesson I have learned from working with governance and operational processes is that more documentation does not automatically create more control. Organizations can accumulate enormous amounts of evidence and still struggle to determine what matters, who owns it or how it should be used. Saving every prompt indefinitely would create its own privacy, security, discovery and operational risks. The better approach is to let the consequence of the work drive the level of governance. An employee asking AI to make an email clearer should not be treated the same way as an engineer using AI to support a safety analysis, an auditor evaluating a control, a manager making an employment decision or an executive relying on AI to inform a major business decision. As the potential consequence increases, so does the need to understand AI’s material role in the process. For higher-risk uses, that may mean retaining enough provenance to reconstruct what happened: the material prompts and outputs, the AI system used, evidence of meaningful human review and enough context to understand how AI contributed to the final decision. The goal is not to archive every iteration of someone’s thinking. It is to preserve enough of the process to explain what happened when accountability matters. That also requires clear ownership. AI teams should not be making records-management decisions on their own, and legal or compliance teams should not discover how an AI platform retains information only after litigation or an investigation begins. CIOs need defined responsibility across technology, records management, legal, security and compliance for deciding what is retained, what intentionally expires, what can be shared, what can be recovered and how higher-risk AI interactions fit into legal holds or investigations. Those principles may sound more like information governance than prompt engineering. That is the point. As AI moves deeper into consequential enterprise work, the governance challenge is no longer confined to whether someone used an approved model or entered prohibited information. Organizations also need to understand what evidence the interaction itself can create. Could you reconstruct the decision a year later? One habit I have developed working around operational systems is to think backward from the future investigation. I do not assume that every process will fail, but asking what evidence would be needed six months or a year later forces a clearer conversation about accountability. If someone challenged a decision after the fact, could the organization establish what information was available, what role AI played, what the human accepted or rejected, and who ultimately owned the decision? The legal treatment of AI conversations is still developing, and not every prompt will automatically become discoverable evidence. Privilege, work-product protection, relevance, possession, proportionality and the circumstances of a particular case can all affect whether material must be produced. A New York court, for example, recently rejected an effort to obtain a litigant’s ChatGPT records , finding the material at issue was protected legal research. That uncertainty should not encourage organizations to ignore the issue. It should encourage them to make deliberate governance decisions before an auditor, regulator, investigator or opposing counsel forces the question. The Watson Grinding litigation did not convince me that every AI prompt is a record. It convinced me that organizations need to know when an AI interaction becomes consequential enough to treat like one. My colleague Josh Copeland, a cybersecurity professional, put the accountability risk memorably: “AI won’t testify for you; it won’t do jail time for you; it won’t pay your fines; but it will absolutely testify against you.” That leaves one question I think every organization using AI in consequential work should be able to answer: If this decision were challenged a year from now, could we reconstruct how it was actually made?

    More on ChatGPT

    Other advisories

    Other news entries

    Source: https://www.csoonline.com/article/4221273/what-the-3m-chatgpt-case-reveals-about-ai-governance.html

    ID