Traffic patterns and service have normalized to regular levels observed prior to the maintenance window. We will conduct an incident analysis and write up a blog post for this next week. Thank you for your patience and I'm sorry that this impacted your workflows.
Posted 5 years ago. Apr 16, 2020 - 19:24 PDT
Update
We've implemented a short term workaround to restore service. We're monitoring the service.
Posted 5 years ago. Apr 16, 2020 - 19:13 PDT
Update
We're isolating the issue with authz service's database queries that is taking an abnormally long time to complete.
Posted 5 years ago. Apr 16, 2020 - 18:58 PDT
Update
We're investigating an unexpected elevation in fetches by the authz service from the database.
Posted 5 years ago. Apr 16, 2020 - 17:00 PDT
Investigating
After upgrading PostgreSQL we are seeing database connections and CPU spikes. We're resizing the database to get more system resources and will provide additional updates as we have them.