History

Elevated errors and response times in Auth0

Elevated errors and response times in Auth0

Current Status: Resolved | Last updated at October 8th 2021, 22:35 UTC

Affected environments: US-1 Preview, US-1 Production

A root-cause analysis for this issue has been performed and is now available at https://cdn.auth0.com/blog/2021-09-24&2021-09-28_Extensibility_Service_Disruption_in_US-1_and_EU.pdf


History for this incident

September 24, 202120:29 UTC

Resolved

Service has been restored. A root cause analysis (RCA) will be posted here within 14 days. We apologize again for any impact this had on you and your users.

September 24, 202120:15 UTC

Monitoring

System performance has returned to normal and we are continuing to monitor. Barring any changes in performance, our next update will be in 30 minutes.

September 24, 202120:00 UTC

Identified

Our Engineers have identified the issue and are continuing to work towards resolution. Error rates and latency have significantly improved. We will provide our next update in 15 minutes.

September 24, 202119:45 UTC

Investigating

We are seeing reduced error rates to our extensibility services due to a mitigating action taken by our Engineering team. Our Engineering team is continuing to discuss additional actions to restore service fully. Next update will be available in 15 minutes.

September 24, 202119:30 UTC

Investigating

We have isolated the problem to our extensibility service. Our team is working on mitigation actions. We apologize for the impact this is having on you and your users. Our next update will be in 15 minutes.

September 24, 202119:15 UTC

Investigating

We are continuing to experience degraded service. We have isolated the problem to our extensibility service. Our team is working on mitigation actions. We apologize for the impact this is having on you and your users. We're shifting to updates every 15 minutes.

September 24, 202118:45 UTC

Investigating

We are continuing to experience degraded service. Our team is investigating and working on mitigation actions. We apologize for the impact this is having on you and your users. Our next update will be posted within 30 minutes.

September 24, 202118:15 UTC

Investigating

Around 17:40 UTC we saw a spike in load resulting in elevated response times in Auth0. Users may have experienced errors or login timeouts. Service levels are back to normal however our team is still investigating. Next update will be provided in 30 min.

September 24, 202118:00 UTC

Investigating

We are seeing elevated response times in Auth0. Our Engineers are investigating.