Our colo facility performed maintenance this morning which caused a brief blip in network connectivity. While this only lasted for a few seconds, the interface state change caused one our routers to enter into a bad state. While in this state, we were intermittantly dropping packets and causing Copilot and On Demand services to perform poorly at times.
All services have been restored.
We have isolated the bad NIC, narrowed the problem down to an issue with its driver, and will be replacing this system entirely this week to avoid future problems of this nature.
