Heartbleed Security Update

Updated on April 10, 2014 with further precautionary steps in the "What can you do" section below.

On April 7, 2014, a serious vulnerability in the OpenSSL library (CVE-2014-0160) was publicly disclosed. OpenSSL is a cryptography library used for the majority of private communications across the internet.

The vulnerability, nicknamed "Heartbleed", would allow an attacker to steal secret certificates keys, names and passwords of users and other secrets encrypted using the OpenSSL library. As such it represents a major risk for a large number of internet application and services, including AppHarbor.

What has AppHarbor done about this

AppHarbor responded to the announcement by immediately taking steps to remediate the vulnerability:

1. We updated all affected components with the updated, secure version of OpenSSL within the first few hours of the announcement. This included SSL endpoints and load balancers, as well as other infrastructure components used internally at AppHarbor.
2. We re-keyed and redeployed all potentially affected AppHarbor SSL certificates (including the piggyback *.apphb.com certificate), and the old certificates are being revoked.
3. We notified customers with custom SSL certificates last night, so they could take steps to re-key and reissue certificates, and have the old ones revoked.
4. We reset internal credentials and passwords.
5. User session cookies were revoked, requiring all users to sign in again.

Furthermore, AppHarbor validates session cookies against your previously known IP addresses as part of the authorization process. This has reduced the risk of a stolen session cookie being abused. Perfect forward secrecy was deployed to some load balancers, making it impossible to read intercepted and encrypted communication with stolen keys. Forward secrecy has since been deployed to all load balancers hosted by AppHarbor.

What can you do

We have found no indication that the vulnerability was used to attack AppHarbor. By quickly responding to the issue and taking the steps mentioned above we effectively stopped any further risk of exposure. However, due to the nature of this bug, we recommend users who want to be extra cautious to take the following steps:

1. Reset your AppHarbor password.
2. Review the sign-in and activity history on your user page for any suspicious activity.
3. Revoke authorizations for external applications that integrates with AppHarbor.
4. Recreate, reissue and reinstall custom SSL certificates you may have installed, and revoke the old ones. Doing this may revoke the old certificates, so make sure you're ready to install the new certificates.
5. Read the details about the Heartbleed bug here and assess the risks relative to your content.

Updated instructions (April 10, 2014):

While we still have not seen any abuse on AppHarbor as a result of this bug, we now also encourage you to take these precautionary steps:

1. Reset your build URL token.
2. If you're using one of the SQL Server or MySQL add-ons: Reset the database password. Go to the add-on's admin page and click the "Reset Password" button. This will immediately update the configuration on AppHarbor and redeploy the application (with a short period of downtime until it is redeployed).
3. If you're using the Memcacher add-on: Reinstall the add-on by uninstalling and installing it.
4. Rotate/update sensitive information in your own configuration variables.

If you have hardcoded passwords/connection strings for any your add-ons this is a good opportunity to start using the injected configuration variables. You can find instructions for the SQL add-ons here and the Memcacher add-on here. This way your application is automatically updated when you reset the add-ons, or when an add-on provider updates the configuration. If this is not an option you should immediately update your code/configuration files and redeploy the application after the configuration is updated.

Stay tuned

Protecting your code and data is our top priority, and we continue to remediate and asses the risks in response to this issue. We'll keep you posted with any new developments, so stay tuned on Twitter and the blog for important updates. We're of course also standing by on the support forums if you have any questions or concerns.