On April 10, 2026, a planned dependency upgrade triggered a full authentication outage on Atmos Pro. Every authenticated request returned an unauthorized error, the dashboard failed to load, and sign-in was broken for all users. Production was rolled back within minutes, the root cause was identified later the same day, and a fix was deployed with regression test coverage. No customer data was lost or compromised.
A planned upgrade to an internal payload validation library produced stricter error behavior than the prior version. The stricter behavior exposed three pre-existing defects in adjacent systems:
Authentication error handling. A defect in how our auth callbacks propagated errors caused validation failures to escape the controlled error path. When the new validation behavior produced errors, the authentication layer returned null sessions, causing every authenticated request to fail.
Dashboard response completeness. A prior change had added required fields to an internal data schema without corresponding updates to the dashboard service. The stricter validation rejected the incomplete response.
Identity provider configuration drift. An earlier fix introduced a configuration that caused our authentication library to attempt an unsupported discovery flow against our identity provider.
