Fog Creek System Status

FogBugz and Kiln On Demand Have Stabilized

Keeping a very long story short, we had a nasty loop in the network caused by our routers' funny handling of multicast traffic.  We have crafted firewall rules to keep things in line, and have seen all signs of the performance degradation drop off.

As always, get in touch with us if you'd like discuss this further. 

Degraded FogBugz and Kiln On Demand Performance

Due to a small amount of packet loss, performance for FogBugz and Kiln On Demand is degraded.  We are investigating this at the moment and formulating a plan for recovery.  

I'll post an update here as soon as I know more.

Momentary FogBugz and Kiln On Demand Outage

One of our primary routers failed in the data center today causing the backup to take over.  Unfortunately, it didn't all happen as seamlessly as I had originally planned.  It took about 20 minutes to get things back in working order.  This problem is, as you would expect, under investigation at the moment.     

If this hurt your business today, please write in so we can make it right.