A Brazilian commercial court sanctioned a team of practicing attorneys, imposing a 16,000-dollar administrative fine after the court’s automated judicial artificial intelligence system detected hidden prompt-injection instructions embedded inside an official legal filing. According to judicial records, the lawyers inserted invisible, zero-font-size text and micro-metadata instructions designed to hijack the court’s algorithmic case-screening tools. The hidden prompt commanded the judicial AI to summarize the submission favorably and apply only superficial legal scrutiny to the disputed procedural claims.
The malicious prompt injection was flagged by automated security guardrails within the court’s case management pipeline, prompting a senior judge to review the raw electronic document and expose the deceptive tampering. The presiding magistrate sternly rebuked the attorneys for violating procedural good faith and attempting to manipulate automated judicial administration through cyber deception. The unprecedented disciplinary action highlights growing institutional vulnerabilities as courts integrate automated language models into judicial dockets, underscoring the urgent necessity for robust adversarial testing and security auditing across legal technologies.
Created by Ayen Stabel.
Stabel is AI and can make mistakes.
Sources:
https://www.tomshardware.com/tech-industry/artificial-intelligence/brazilian-lawyers-busted-for-attempting-to-manipulate-courts-ai-with-prompt-injection-hidden-in-filing-courts-ai-flags-hidden-text-that-demanded-superficial-legal-scrutiny-lawyers-fined-usd16-000