One of the technical challenges in a fast growing database such as OpenCorporates is that the baseline never stands still. As a result, due to the increasing data storage and traffic demands, we can see that we’re approaching the limit of what our servers can handle. To compound this, we have pretty much run out of rack space in our current location in the data centre.
So a few months ago we took the decision to move to a new location, which allows us to more than double the available rack space. This is the first time we’ve increased it since we started hosting things ourselves, so it’s a big moment for us – it’s also confirmation that we’re on the right track to meeting our aim of being the best source of company data in the world.
We looked at a number of ways of handling the move, including zero- or partial-downtime approaches (e.g. deploying a temporary, read-only version to some other servers in the data centre or in the cloud). However, when we sat down and examined them in detail, it became clear that a short period of scheduled downtime was going to be the best approach for our users – the effort involved with the no-downtime approach was too high, given what we could spend the time doing instead. (Despite its size, behind the scenes OpenCorporates is actually a very small team.)
The bulk of the move should take 1-2 hours, but to allow for unforeseen circumstances we are scheduling the maintenance to last 6 hours, so we’ve scheduled it for 10am-4pm GMT on 8th February 2017.
We’re also communicating to our users, via our multiple channels (website, blog, twitter, status website, slack etc), to give them as much advance warning as possible.
To keep the downtime to a minimum, as much work as possible has been done in advance (in fact our DNS is already being served from the new racks), and during the maintenance users will see a friendly webpage or json/xml response (in the case of the API). During the move we’ll be posting progress updates on our status site (and maybe a few pics on our twitter feed). We also plan to take the opportunity to do some small hardware upgrades we’ve had pending for a while.
On a personal note, this will be the first time some members of the team have visited a data centre, so we’re looking forward to the experience. Even if it does mean an early start to get from London to the data centre in Newport!
blog.opencorporates.com 