Fog Creek System Status

Maintenance Complete!

The database maintenance has been completed and all systems are back online.

Thank you for your patience!

Reminder: Fog Creek Database Maintenance Tonight!

Just a reminder, http://fogcreekstatus.typepad.com/fog_creek_network_status/2008/02/service-impacti.html. 

Quick Blip on Some FogBugz On Demand Accounts

A few of our FogBugz On Demand users may have received an HTTP 500 response when they hit their accounts.  The outage began at approx 5:15 PM EST and was mitigated by 5:20 PM EST.  We bumped into an unexpected memory limitation on one of our application servers.  I have a short-to-mid-term work around in place until we are able to replace all of those boxes with 64-bit systems. 

The ultimate lesson to all of you who desire to go into the application hosting business: run 64-bit on *everything*.      

FogBugz On Demand Outage

We're currently experiencing issues related to search on FogBugz On Demand that may be generating errors for some customers.  We're investigating...

[UPDATE] A server that should not have been in our load balancer, accidentally got pushed into the pool.  So anyone that was getting that server as their web server was seeing errors.  We've taken that server out of the pool now and are doing more investigation, but things are stable again.

Payment Processor Back Up

Payment processor LinkPoint is now claiming they are back up and fully operational.  Orders seem to be going through successfully.

Payment Processor Outage

Our payment processor is currently experiencing an outage.

Orders placed on our shop for FogBugz, CityDesk and job postings, will be delayed.

Copilot day passes are currently offline.  Please call us at 866-364-2733 during the outage and we'll be happy to provide you with a magic credit card to purchase a day pass for free.

Non-Service Impacting: Data Center Power Generator Test on February 20th, 2008 @ 07:00 AM EST

Just a note to everyone: our NYC colo will be performing a power generator test @ 7:00 AM EST on Wednesday, January 20th, 2008.  This is a routine test that ensures that the power system is functioning properly, and we are not expecting any trouble.

Service Impacting: Database Maintenance on February 24th, 2008, 00:01 - 04:00 EST

We will be upgrading a database server this weekend that provides service for a number of applications:

• Copilot
• Fogcreek.com
• Joel on Software (including the discussion groups and job board)

During this time, FogBugz On Demand customers can use their applications as usual, but will not be able to modify payment settings. 

Fog Creek 866 Number Back Online (Again)

All of our phone services have been restored.  It turns out that IDT, our current provider, did not actually increase our volume cap as they stated last week.  We were lost in the shuffle, I suppose.

I am working to move us to another provider ASAP, and have requested quotes.   

Fog Creek 866 Number Offline (Again)

I'm on hold with our toll-free provider right now sorting things out.

In the meantime, you can reach the customer service and sales @ 212-279-2335.

We apologize for the inconvenience.