Teleport Cluster Unavailable
Incident Report for Teleport Cloud
Postmortem

At 12:51:05 UTC on November 25, 2022, a single tenant’s proxy server TLS certificate expired resulting in the Teleport cluster being unavailable. The issue was reported to Teleport support at 18:09 UTC and resolved by 18:33 UTC.

Certificates for Teleport Cloud customers renew automatically. This issue stemmed from renewal automation being disabled in order to pin the Teleport version and configure custom throughput for the tenant. At the time of this incident, both version pinning and custom throughput settings had since been exposed as configurations but the disable flag was not reenabled due to anticipated downtime for the customer. The tenant’s automated certificate renewal was scheduled to be reenabled once the Teleport cluster was upgraded but the original certificate expired beforehand.

The tenant is now running with certificate renewal enabled.

Posted Dec 21, 2022 - 23:13 UTC

Resolved
At 12:51:05 UTC on November 25, 2022, a single tenant’s proxy server TLS certificate expired resulting in the Teleport cluster being unavailable. The issue was reported to Teleport support at 18:09 UTC and resolved by 18:33 UTC.
Posted Nov 25, 2022 - 13:00 UTC