Sunday's On Demand maintenance took longer than expected because Microsoft SQL Server's shut-down and startup processes quiesce and scan each database for corruption. The time necessary for this check increases along with the number of databases on the system. In one extreme case, the server took two and and half hours to shut down properly.
Applying the hotfix in question required a stop and start of SQL Server itself, then a reboot. This timing didn't work out with the original window, so were only able to install the hotfix on two of the seven servers that needed it. We're researching ways to reduce the time it takes to restart SQL Server on these heavily populated servers and have already found some promising leads. We'll be rescheduling the remaining five servers soon, most likely coinciding with an upcoming power maintenance.
The good news is that we resolved a number of other issues with this maintenance. Most were internal, but one exciting networking change is the completion of our double-ring switching architecture. This means our network can survive the loss of any single cabinet with minimal impact to the others.
Thanks for your patience as we make our On Demand services better. Look for more in the near future!
