This weekend I'll have to take the Copilot database offline for approximately 20 minutes while I apply a few patches to the database server. We are in the process of scaling out our backend to increase availability, and this maintenance will get us much closer.
We appreciate your patience in the matter, and thank you for your patronage.
We will be performing maintenance on the Copilot service tonight / tomorrow morning at 1 AM Eastern. During this time, Copilot customers may experience intermittent losses of connectivity. We anticipate this outage to end by 2:30 AM.
While this outage is regrettable, we are performing this to make way for a more reliable version of Copilot.
We have rolled back the memory quotas we introduced yesterday after finding that they were causing premature session timeouts, and we're deeply sorry for the inconvenience. We're still working on a fix to prevent the out-of-memory errors some of you saw yesterday, and will post as soon as we have more info.
We had a few reports that some of our On Demand customers experienced Out-of-Memory errors earlier this morning. Our monitoring system caught it, and we immediately removed the problamatic server from the pool.
The offending process has been identified, and a fix is in the works. In the meantime, quotas have been put in place to prevent the process from using up this amount of memory agian.
RIght now it looks like this was due to some difficulty in handling a particular time zone (unrelated to the DST changeover). Developers have rolled out a patch for this bug, and we'll be watching this over night for any recurrence.
This weekend, the New York instance of our FogBugz Maintenance Service began periodically spinning out. OnDemand customers based in that Data Center may see delays in search-indexing and email processing but, as before, no data is lost. We're working on tracking down the root cause and will post again when we've got more information.