Traffic patterns and service have normalized to regular levels observed prior to the maintenance window. We will conduct an incident analysis and write up a blog post for this next week. Thank you for your patience and I'm sorry that this impacted your workflows.
Posted Apr 16, 2020 - 19:24 PDT
Update
We've implemented a short term workaround to restore service. We're monitoring the service.
Posted Apr 16, 2020 - 19:13 PDT
Update
We're isolating the issue with authz service's database queries that is taking an abnormally long time to complete.
Posted Apr 16, 2020 - 18:58 PDT
Update
We're investigating an unexpected elevation in fetches by the authz service from the database.
Posted Apr 16, 2020 - 17:00 PDT
Investigating
After upgrading PostgreSQL we are seeing database connections and CPU spikes. We're resizing the database to get more system resources and will provide additional updates as we have them.
Posted Apr 16, 2020 - 16:06 PDT
This incident affected: Hosted Chef (Hosted Chef API, Hosted Chef Web Console).