Do you allow one-off prompt edits in provider consoles?

Someone fixed an urgent refusal issue by editing a prompt in a provider console. The change helped the target case but it never passed source control, our eval suite or the deployment record. A day later, structured output broke for one locale because the edited instruction changed how the model handled missing fields. Oncall could see schema failures but could not tell which customers received which template. the rollback plan was a message in chat, which is not much of a plan This exposed a gap in prompt provenance that ordinary service releases do not cover. We need immutable prompt versions, environment promotion and trace metadata that records the exact template for each call. I also want canary cohorts with schema adherence checks before full release. The part I’m still sorting out is emergency access. A strict path can slow a real incident but an invisible console edit creates configuration drift and makes the next incident harder. How do you handle urgent prompt changes while preserving review, evals, cohort history and a reliable rollback? Is the console locked down entirely, or does an emergency edit automatically create a versioned change elsewhere? submitted by /u/TheIncredibleCustard [link] [comments]