Summary from Workday:
While validating the weekly Maintenance Window for Disaster Recovery tenants, we have identified that tenanted services within the DR Site unexpectedly connected to Production databases. As a result, this has left data in an inconsistent state that can cause errors accessing tenants or jobs/integrations/tasks to fail with missing transaction error messages.
Effective immediately, we are placing all WD1 Production tenants into Maintenance while our teams investigate the best recovery path. We currently do not have any ETA for service restoration and so we are recommending that customers with critical business deadlines today execute their business continuity plans.
We will provide updates every 30 minutes while we work to resolve this issue. We sincerely apologize for the disruption.
Update on 11/14 @ 11:02 A.M.
Service Update is completed. Reindexing and tenant release have completed for about 80% of WD1 customers. We expect to release the majority of the remaining customer tenants by 9:30 AM PT. Customers whose tenants will be delayed later than 9:30 AM PT will be notified via a proactive case. We will provide our next update at 9 AM PT.