Small subset of customers in US Hosted production environment are experiencing failing API, UI, integrations and batch operations.
Incident
Report for Zuora
Postmortem
TIMEFRAME:
28/May/20 1:24 PM PDT to 1:46 PM PDT
SUMMARY:
During the impact timeline, we experienced a hardware failure to the primary database writer host for a subset of customers in our US Hosted Datacenter
IMPACT:
Zuora Billing APIs and Services were unavailable for the duration of the impact timeline for a subset of customers associated with the impacted hardware.
ROOT CAUSE:
Hardware Failure
RESOLUTION
Once incident team determined root cause, Zuora failed over to stand-by db host
FUTURE PREVENTATIVE MEASURES:
• Improved alerting for database host failure
• General health review of database host inventory
Resolved
This incident has been resolved.
Monitoring
A fix has been implemented and we are monitoring the results.
Investigating
A small subset of customers in US Hosted production environment are experiencing failing API, UI, integrations and batch operations. We’re looking at this with highest priority.
This incident affected: AMERICAS - CLOUD 2 (NA2) - www|rest.zuora.com (Production UI, Production API, Production Integrations, Production Batch Operations, Sandbox UI, Sandbox API, Sandbox Integrations, Sandbox Batch Operations).
