Elevated errors and response times in Auth0
Current Status: Resolved | Last updated at October 8, 2021, 22:35 UTC
Affected environments: US-1 Preview, US-1 Production
A root-cause analysis for this issue has been performed and is now available at https://cdn.auth0.com/blog/2021-09-24&2021-09-28_Extensibility_Service_Disruption_in_US-1_and_EU.pdf
History for this incident
Resolved
Service has been restored. A root cause analysis (RCA) will be posted here within 14 days. We apologize again for any impact this had on you and your users.
Monitoring
System performance has returned to normal and we are continuing to monitor. Barring any changes in performance, our next update will be in 30 minutes.
Identified
Our Engineers have identified the issue and are continuing to work towards resolution. Error rates and latency have significantly improved. We will provide our next update in 15 minutes.
Investigating
We are seeing reduced error rates to our extensibility services due to a mitigating action taken by our Engineering team. Our Engineering team is continuing to discuss additional actions to restore service fully. Next update will be available in 15 minutes.
Investigating
We have isolated the problem to our extensibility service. Our team is working on mitigation actions. We apologize for the impact this is having on you and your users. Our next update will be in 15 minutes.
Investigating
We are continuing to experience degraded service. We have isolated the problem to our extensibility service. Our team is working on mitigation actions. We apologize for the impact this is having on you and your users. We're shifting to updates every 15 minutes.
Investigating
We are continuing to experience degraded service. Our team is investigating and working on mitigation actions. We apologize for the impact this is having on you and your users. Our next update will be posted within 30 minutes.
Investigating
Around 17:40 UTC we saw a spike in load resulting in elevated response times in Auth0. Users may have experienced errors or login timeouts. Service levels are back to normal however our team is still investigating. Next update will be provided in 30 min.
Investigating
We are seeing elevated response times in Auth0. Our Engineers are investigating.