3:25 PM ET
A scheduled routine change to our DNS configuration was initiated. An error in the execution of this change resulted in misconfigured DNS entries causing domain resolution errors for
blackboard.com domains. This change propagated across the internet over time, causing Blackboard services to be inaccessible.
6:30 PM ET
The Engineering team was able to identify the root cause and restore the DNS configuration using a backup file.
6:40 PM ET
Access to all products was restored.
Incident Root Cause
A change to the DNS configuration was made that resulted in the misconfiguration of several internet facing
Blackboard DNS zones. Once configuration of the DNS zones was corrected access to all impacted Blackboard
services was restored. At no point were the actual systems/environments down, however, they were
inaccessible from the internet for some time. No data loss occurred as a result of this event.
Resolution required restoration of the proper DNS configuration using a backup file.
Recommended or Preventive Action
Blackboard continues to review and validate staffing policies, training, and change management
procedures with our infrastructure maintenance service provider.
Our service provider has executed this change type successfully on multiple occasions.
Corrective action with the responsible staff has been taken by the management team.