Intermittent access failure for a subset of US hosted production tenants
Incident Report for Zuora
Postmortem

TIMEFRAME:
10:51AM PST 12/01/2019 to 11:25AM PST 12/01/2019

AFFECTED SYSTEMS:
Two (2) Zuora production tenants in US Hosted Production environment

SYMPTOMS:
Database subsystem active writer unresponsive for one tenant shard

ROOT CAUSE:
Database subsystem bug causing instance failure

RESOLUTION
Recovery and restart of failed instance

FUTURE PREVENTATIVE MEASURES:
Upgrades of database subsystem for this instance scheduled for FY20Q1 to address future stability issues

Posted Jan 06, 2020 - 10:51 PST

Resolved
This incident has been resolved.
Posted Dec 01, 2019 - 17:52 PST
Update
We are continuing to monitor for any further issues.
Posted Dec 01, 2019 - 13:48 PST
Monitoring
We have identified US Hosted Production environment was unavailable for a very small subset of tenants. The system is now fully operational.
Posted Dec 01, 2019 - 13:26 PST
This incident affected: AMERICAS - CLOUD 2 (NA2) - www|rest.zuora.com (Production UI, Production API).