Black-box problem
Also known as: Black-Box-Problem, non-auditable AI output
A situation in which an Artificial Intelligence (AI) system delivers a result without the path to it being traceable – forensically useless even if the result is statistically correct.
Forensics rests on chain of custody and repeatability, Artificial Intelligence (AI) on probability and context. A result with 99 per cent accuracy is worthless in court if nobody can show the steps behind it. The European Union Agency for Law Enforcement Cooperation (incl. EC3) (Europol) report therefore demands structured logging of every AI action and the ability to reproduce the result manually. Auditors place the same demand on AI-assisted controls in an internal control system.