Running websites on a Docker Cluster using Rancher
For my company, Ingewikkeld, we are migrating all our services to run on a series of Droplets at DigitalOcean that together form a cluster of Docker containers. This setup should reduce the costs of all the separate VPS’s that we previously had because Docker hosts automatically fill up the available resources of a host to the limit, reducing wasted resources.
In this article, I want to give an overview of our setup without going into too much technical detail. Many of the things I mention here can be found in their respective documentation, and it would make this article verbose and out of date before you can say “supercalifragilisticexpialidocious”.
Setting up your hosts
Let’s get started: to set up a cluster we needed at least two Droplets,
- The Rancher Manager which also runs the Database (I recommend using a Droplet with 2GB of RAM; any less and it may become unresponsive at times).
- A host that can receive client services (I would recommend creating two of these from the Rancher UI).
The first one needs to be created from within the DigitalOcean UI. I recommend enabling Private IPs and to create a Volume and attach that to this Droplet. By having Rancher communicate purely over Private IP addresses you can mitigate some security issues. The Volume is meant to be used for your Database files; this way you can easily migrate your database to another host and/or benefit from backups.
Since you are going to run your Docker Cluster over the Private IP range of your Digital Ocean droplets; don’t forget to change your “Host Registration URL” in your Rancher UI to match the Private IP and port of your Rancher management host before setting up other hosts.
The second type of host that I have mentioned here is one that can be freely cloned once set up, and I would recommend creating it from within the Rancher UI, as is described in the Rancher Documentation. Repeat this process for the number of hosts that you want or need, or use the slider during host setup to create several hosts at once.
When you create a host from the Rancher UI, please keep in mind that you cannot attach your own SSH keys as you would using the DigitalOcean UI. See this documentation page should you want to SSH to them; for example, to install the Digital Ocean monitoring agent (highly recommended!).
Once we had both machines up and running, the first thing we did was to add the host from which the Rancher UI is running as a managed host; but disable it. This trick will allow you to view the status of your rancher host machine from within Rancher but because you disable it no new services will be deployed there.
Adding the basic services
Before adding any services I recommend doing two things first:
- Adding a stack with your database as an External Service
- Adding a stack with your a Load Balancer
Rancher comes out of the box with four types of services: Service, External Service, Service Alias and Load Balancer.
To make it easier to refer to your database (for apps that need it), I recommend adding an External Service called ‘database’ that will serve as a source for Service Aliases. This can be useful if you create a Stack for your application (which I recommend) but inside that stack, you want to have a service “database”. This can be done by adding a Service Alias that refers to you centralized “database” service.
Within a stack, your services can refer to each other by name. This is why it is convenient to have a Service Alias called database. With that alias, you do not need to configure a URL or IP address in your application but just use the DNS address “database”. Similar to how you would do in a Docker Compose setup.
Next up is the Load Balancer. Rancher comes with the option to setup a managed HAProxy by choosing the Load Balancer option when adding a new service (see the dropdown in the UI). Since hosts can only expose ports once, it is important that your load balancer acts as a gateway to your websites. The easiest way to do this is to configure your load balancer to deploy on every host once, and to setup your services not to map to any external ports.
This way, you can configure your load balancer to accept traffic for the hostname that you would like (for example http://www.example.org) and even do SSL Offloading. This all comes out of the box with Rancher.
But before we can add certificates to the load balancer; we need to make sure Rancher knows about them. Luckily, this is easy using the Let’s Encrypt service in Rancher’s catalog. However! For Let’s Encrypt to work we need to make sure the DNS of your domains can be managed through Rancher.
Managing your DNS using Rancher
Awesome, we got something running. If you were to change your DNS address to the IP address of your client hosts then your website should show up fine.
DNS however, is another thing you would like to automate. For (at least) two reasons:
- when you add, remove or change hosts; you would not want to update all DNS settings manually. The basic idea of this setup is that you need not care.
- The Let’s Encrypt service in Rancher uses DNS verification to check for the validity of your request for a certificate. Automating your DNS will automatically make it suitable for this purpose as well.
Thankfully, automating your DNS is an easy task as long as you use a provider that is supported by one of the DNS services in Rancher’s Catalog.
Rancher comes with a ‘Catalog’, which is a library of recipes with which you can easily install specific services.
We have chosen CloudFlare to manage our DNS zones with. With CloudFlare, you can get additional DDOS protections and several benefits that other providers don’t, and Rancher has a CloudFlare service that automatically adds a DNS entry to the selected domain that automatically updates as soon as a host is added that has a public port mapping.
There are two strategies that you can pick now:
- Have a CloudFlare service for each domain
- Have one CloudFlare service and CNAME all your domains to use the DNS address created by that service.
Because we have quite a few domains, we chose for option number 2. This is possible because CloudFlare allows you to CNAME your Top Level address (such as “mikevanriel.com”), something which not all providers allow; and because we choose to accept that all domains are hence linked to each other.
The downside of option number 1 is that it takes a lot more resources since each domain needs to have its own CloudFlare DNS service running in your cluster.
Setting up SSL
One of the services in Rancher’s Catalog is the Let’s Encrypt service. This service will automatically install an SSL certificate in Rancher’s certificate listing, and renew it once it is going to expire. Because this service will automatically register the SSL certificate in Rancher’s certificate listing; you can select this certificate when editing your Load Balancer and have HAProxy take care of your SSL Offloading automatically.
Installing your Let’s Encrypt service is a matter of going to Rancher’s catalog, searching for the Let’s Encrypt service and installing it. The rest is done automatically.
Once your certificate is installed, you can edit your Load Balancer and in the bottom of the screen, you can add the certificate. Don’t forget to add a load balancer rule that uses https and maps from port 443 to your web service’s port (usually 80), and you are all set!
Tips, tricks and gotcha’s
Aside from the various tips and tricks already mentioned above, I would like to mention just a few more quick notes to you.
- Setup health checks – when you add a service: add a health check. This will help rancher in knowing when to kill a container and spawn a new one elsewhere.
- Setup monitoring and alerting of your hosts – using a Docker Cluster may inadvertently overload hosts because it will try to cram as many services possible per host.
- Give your manager node enough RAM – your cluster will continue running without your manager, but it is bloody annoying to you if the UI is down or unstable. Give the machine at least 2GB of RAM or more when your cluster becomes large.
- RAM is your bottleneck – when a machine runs out of memory it will start swapping, and this generally will affect the performance of all containers on that host. Use lightweight services and be mindful that you probably have multiple web servers running on a host instead of one.
- You will do other ops, not necessarily less – DNS, SSL and such things can automatically be managed; but you will need to keep a closer eye on your hosts if they are still healthy.
These are the first things that come to mind, I may think of other things but for now: go experiment and have fun! Setting up a cluster seems like a daunting task but as soon as you get comfortable with this: you can set this up in under an hour’s worth of time.