Performance Issue on Promo Codes App for a Subset of US Production Customers
Incident Report for Zuora
Postmortem

TIMEFRAME
2021-01-19 5:00 PM PT to 2021-01-20 2:35 AM PT

SUMMARY AND IMPACT
During the above timeframe, the Zuora Promo Codes app experienced performance issues and timeout errors which impacted approximately 3% of overall API traffic to the Promo Codes App

ROOT CAUSE
A previously undetected infrastructure scaling issue was introduced with an earlier deployment

RESOLUTION
Corrected auto-scaling configuration and Promo Code app was redeployed

FUTURE PREVENTATIVE MEASURES
• Improvements and adjustments to 5xx HTTP monitoring and alerting 
• Improvements to configuration management for this service have been implemented

Posted Jan 27, 2021 - 14:21 PST

Resolved
This incident has been resolved.
Posted Jan 20, 2021 - 05:55 PST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Jan 20, 2021 - 04:23 PST
Identified
The issue has been identified and a fix is being implemented.
Posted Jan 20, 2021 - 03:05 PST
Investigating
We are currently investigating an issue in the Promo Code Application where a subset of US Production environment customers are experiencing performance issues.
Posted Jan 20, 2021 - 02:25 PST
This incident affected: MARKETPLACE (CONNECT) APPLICATIONS (Marketplace (Connect) - Production).