Fog Creek Network Status

Kiln On Demand - Can't create repositories or projects

Some of our Kiln On Demand customers may be experiencing difficulties creating projects and repositories within Kiln. Our engineering team is actively working on the problem and we'll put up another post on the status blog as soon as the issue is resolved. 

Issue resolved; All services up

The root cause of today's intermittent Kiln On Demand HTTP errors has been found and the bug squashed. Accounts should have seen no recurrence of the issue since the last post and no further issues are anticipated. We apologize for the inconvenience.

Intermittent Kiln On Demand HTTP failures affecting 10% of customers

For about forty minutes customers on a particular version of Kiln On Demand may have seen intermittent HTTP 502 and 503 errors when when accessing Kiln. We believe we have isolated the problem requests and are working to determine the root cause of the issue. At this time, requests should be flowing normally.

Issue resolved; All services normal

The delayed email and indexing issue above has been resolved. The problem was related to a similar issue we experienced last week. A fix to prevent this from happening again had already been developed and deployed to test accounts, and this week it will be implemented across the board.

FogBugz On Demand delayed emails, index updates

Due to a failure in one of our asynchronous update processes, about half of our FogBugz On Demand accounts are experiencing delayed emails (inbound and outbound) and search indexing. We're actively working on the issue and will post updates as they become available.