Fog Creek Network Status

FogBugz 8.5.122 and IE8 (UPDATE: All Fixed)

UPDATE: FogBugz 8.5.123 has been released to http://shop.fogcreek.com for licensed users of FogBugz. Please contact us at customer-service@fogcreek.com if you continue to experience issues.

UPDATE 2: FogBugz 8.5.123 has been deployed to all On Demand customers who were previously on 8.5.122 (about 2% of On Demand accounts). Please contact us at customer-service@fogcreek.com if you continue to experience issues.

Several customers using IE8 with FogBugz 8.5.122 have reported javascript errors in the browser. From what we can tell, this only affects users of IE8 Corporate Office edition. We've found the bug, have coded the fix, and will be deploying it to affected On Demand customers and licensed customers shortly.

We will only be upgrading On Demand accounts that are currently on 8.5.122 (about 2% of On Demand customers) and licensed FogBugz customers can expect to find a new build of FogBugz at http://shop.fogcreek.com very soon. We'll edit this post as soon as the newest version is available.

All services maintenance on Sunday, 2011-08-07, at 04:00 GMT

This weekend we'll be taking an extended maintenance window to get the aforementioned SQL updates applied to the rest of our database servers. We've made progress on reducing SQL Server's shutdown and startup times but, without actually testing, it's difficult to tell how much. As a result, we're planning this as an extended, six hour maintenance window from 04:00 GMT to 10:00 GMT. We're unhappy about this kind of service interruption but these updates must get installed.

We're also taking this opportunity to implement some major upgrades to our network architecture. While waiting for SQL Server to restart, we will also:

• Install redundant power supplies for all of our switches
• Install a pair of switches in front of our gateways to ensure proper VRRP communication paths
• Upgrade our Spanning Tree configuration to use MSTP to slash convergence time and give us more flexibility in avoiding congestion

Finally, we'll also be upgrading the RAM in two of our most heavily loaded SQL servers and reconfiguring their Maximum Memory settings to eliminate memory pressure. 

Altogether, this maintenance will dramatically improve our infrastructure's reliability and fault tolerance.