Last night at 10pm EDT, some Kiln On Demand accounts were upgraded to a new version of Kiln. Kiln On Demand uses a blue-green strategy for managing deployments: each of our servers runs multiple copies of Kiln, one for each current and upcoming version. During the upgrade, one of our servers failed to swap these accounts to the new version. Thus, requests to these accounts were intermittently served with old code, causing errors and connectivity problems. At 9:43 this morning, the faulty server was removed from service, bringing all accounts back on line. The server has since been repaired and is back in service. In the future, we will be monitoring our servers more closely during upgrades, to verify that all accounts are deployed to the correct Kiln version.
We apologize for any inconvenience caused by this outage. If you have any questions or concerns, please don't hesitate to contact us.
