On Monday, May 11 2015 8:30 AM EDT (-0400 GMT) Augend suffered a failed I/O controller, which resulted in dropped data streamed from its hard drives. No data loss was encountered; however, no data was written either during this window effectively rendering Augend inaccessible. The root cause was a bad capacitor embedded in the board. Resolution was further compounded by a faulty spare on-site requiring physical transfer of capacitors from the dead donor controller to the dead controller by desoldering/soldering parts.
Service was restored by 11:20 AM EDT after a few trips between the data center and office to perform repair. Because this was an extended outage window, we will be offering clients a 3-day prorated refund for those affected during this outage. Please open a ticket within the control panel to request a SLA refund.
Additionally, I will be inspecting all servers shortly for signs of failing capacitors (bulges) and replace as preventative maintenance when necessary. This is the third component to fail in 9 months as a result of a bad capacitor.
And always remember to follow @apisnetworks on Twitter. Outages are always communicated over Twitter, en masse, to all clients. You can stay updated with what’s going on real-time and we can work quickly to get your site back up and running.
– Matt