My content engine published a client's billing figure at 9:12 on a Monday morning.
Monday morning my weekly content engine did exactly what it was told. It harvested an internal note about a client engagement, and it published what that client pays me. First to the blog, then to LinkedIn at 9:12am. Wrong number too, which somehow made it worse.
Here is the part I own. The writer prompt literally invited concrete revenue figures, because real numbers beat adjectives and I had trained it to reach for them. Nothing anywhere said client billing was off limits. So the machine reached, found a dollar amount in a note it never should have seen, and put it on the internet under my name.
I scrubbed the articles and deleted the scheduled posts inside a few minutes. But cleanup is not a fix. So I put in three layers. The harvester now redacts billing figures before the writer ever sees them. Every content prompt carries the ban in plain words. And there is a fail-closed check that throws before anything commits or schedules, so if the guard is missing, nothing goes out at all.
Here is what I keep relearning. A prompt is not a guardrail. A prompt is a suggestion the model is free to ignore the moment your instructions point the other way. The only thing that actually stops a bad publish is a tripwire that fails closed, code that refuses to run when the safety is absent instead of hoping the model behaves.
If you have an automation writing anything customer-facing, ask one question. What happens when it does the thing you never explicitly told it not to do? If the answer is it publishes, you do not have a guardrail. You have a rumor waiting for a Monday.
AI Diagnostic | All insights