Date of Incident: 2/8/25 Time of Incident: Approximately 6:00 p.m. (after hours) Time Restored: 7:35 a.m. Pacific Time
Overview
On the evening of 2/7/25, several of our critical systems experienced a database outage. This incident affected user login capabilities on platforms including CMS, Announcements, and School Manager. Fortunately, the system Connect remained fully operational. Our technical team identified the issue quickly and worked overnight to restore full functionality before business hours.
Impact
-
Affected Systems:
- CMS
- Announcements
- School Manager
-
Unaffected System:
- Connect
Users may have experienced difficulties logging in during the outage period.
Timeline & Root Cause
- ~6:00 p.m.: The server began experiencing issues.
- Root Cause:
Our database server, ran out of disk space, which resulted in the associated systems being disconnected from their databases. As a result, while some systems required a full restart to recover, most were simply locked out due to the database disconnection. - Overnight Efforts:
Our technicians at Mission extended the drive capacity, which allowed the connections to be restored.
Resolution
- 7:35 a.m. Pacific Time: Full operations were restored.
- Post-incident checks have been performed to ensure system stability, and additional measures are being evaluated to prevent similar issues in the future.
Next Steps
-
Monitoring & Preventative Measures:
We are reviewing our disk space monitoring processes and implementing additional safeguards to prevent recurrence of this issue. -
Feedback & Support:
If you have any questions or continue to experience issues, please reach out to our support team