History

Elevated errors in Auth0

Elevated errors in Auth0

Current Status: Resolved | Last updated at September 14th 2021, 3:19 UTC

Affected environments: US-1 Preview, US-1 Production, US-2 Production

A root-cause analysis for this issue has been performed and is now available at https://cdn.auth0.com/blog/20210831-Incident-RCA.pdf


History for this incident

August 31, 202121:15 UTC

Resolved

Service has been restored. A root cause analysis (RCA) will be posted here within 14 days. We apologize again for any impact this had on you and your users.

August 31, 202120:47 UTC

Monitoring

We have applied workarounds to the networking issues seen by our cloud provider for our US-1 and US-2 environments and are now seeing stable performance in all regions. We will continue to monitor the system.

August 31, 202120:31 UTC

Identified

We have applied a workaround to the networking issues seen by our cloud provider. Our US-1 environment is operating normally at this time. We are continuing to monitor the US-1 region. Our Engineering team continues to work on affected areas of US-2. We are attempting to restore all services as quickly as possible.

August 31, 202120:15 UTC

Identified

Our Engineering team continues to work to restore services as quickly as possible once the networking issues are resolved. We will continue to keep you informed as the situation evolves.

August 31, 202120:00 UTC

Identified

Our Engineering team is working to restore services as quickly as possible once the networking issues are resolved.

August 31, 202119:46 UTC

Identified

We have isolated the issue to networking issues with our cloud provider. We have allocated resources to ensure we restore services as quickly as possible once the networking issues are resolved. We apologize for the impact.

August 31, 202119:31 UTC

Investigating

Our US-1 and US-2 regions are experiencing degraded service. Our Engineering team is working quickly to mitigate this and apologize for the impact this is having on you and your users.

August 31, 202119:25 UTC

Investigating

Our US-1 and US-2 regions are experiencing degraded service. Our Engineering team is working quickly to mitigate this and apologize for the impact this is having on you and your users.

August 31, 202119:01 UTC

Investigating

We are continuing to investigate the elevated error rate, though we are seeing this issue impact customers intermittently. We will provide status updates every 15 minutes.

August 31, 202118:45 UTC

Investigating

We are continuing to investigate the elevated error rate, though we are seeing this issue impact customers intermittently. We will provide status updates every 15 minutes.

August 31, 202118:31 UTC

Investigating

We are currently investigating an elevated error rate in Auth0. Our Engineers are investigating.