Vagrant, Packer, Docker
JCrete 2014
Vagrant, Packer, Docker
| Vagrant, Packer, Docker | |||
|---|---|---|---|
| Convenor: Chris Richardson | |||
Participants:
| |||
Summary:
In this session, Chris presented packer, vagrant and docker. packer packer is a tool for the automated building of AMIs (Amazon Machine Images), but it can also build vmware images and VirtualBox images. You can for example start out with a DigitalOcean standard AWS vanilla API with a default Linux, then use packer to install whatever you need onto that vanilla image. You could run this from Jenkins. packer can work with chef for automation. At high level vagrant and packer are very similar; the output of vagrant are one or more running virtual machines; whereas the purpose of packer is to build an AMI. After you have built an AMI you cannot really do anything with it, as you still need to run startup scripts before you can actually use it (like configuring IP, finding cluster members); so packer may save you a bit of time, but not much more. Chris uses packer to create a golden master image, any specific configuration gets parsed in as user data. The packer-built image has your application already installed. Netflix used something similar to packer, which they may have open-sourced, and their AMIs had the application aready installed as a war file. You can also create an instance which when it starts up gets and installs your application.
vagrant is really for launching and configuring one or more virtual machines. vagrant can start up the VM and configure it (e.g. by running shell scripts or chef). Then you have a VM running that is configured according to your needs. You can also use vagrant to launch and configure an EC2 instance. So if you need a particular instance of MySQL and RabbitMQ for your team, you can create a vagrant project and you colleagues just need to type vagrant up in the project folder and a few minutes later they'll have a virtual machine up and running. Latest intelliJ has a vagrant plugin. For example if you are building a Java application you can test it on a VirtualBox which is installed from vagrant, and then intelliJ can test the java application on the VM. juju uses charms to deploy virtual machines with services and log you with the graphical interface to connect to them. One participant would like to use juju with openstack. A participant uses vagrant/openstack to launch virtual machines running on physical hardware. Currently these machines are based on Ubuntu, in the future it will be CoreOS. This will in the future be used to host docker. Good for security (less software, less problems).
docker evolves very rapidly. If you are building an AMI you are usually talking abuot gigabytes on disk, which inherently is slow. Whereas docker images are super-light-weight, because you start with a base image and then your build a new image by adding stuff to it. The disk space which gets taken up is just the delta plus the base image (which is less than having to keep two full base images). Because all of it is incremental, it is pretty light weight, the base images are still big but the app-specific images are pretty light weight. There is a public repository with base images for you to use. A docker file is a sequence of commands which specify the base image and then shell commands (usually bash/Linux sh). For example, these commands could be apt get <some package>. Then when you create an image for a specific Java application, you start with your base Java image and add the rest to it. Then you can add commands which are run when the machine starts up, e.g. commands to startup you application on machine startup. Chris uses Spring Boot to deploy his applications. A Spring Boot jar contains some bootstrap classes and some jar files. If you build an überjar, you may have some conflicts (jars/classes having the same file names). Spring Boot doesn't flatten everything out, you have jars within the jars. In Chris' use case, the whole thing is then automated with Jenkins. all Jenkins has to do is go to the docker folder and then run the docker build. In the production environment you pull down an image and then do docker run to start the service. The output of the deployment pipeline is a docker image
There are some arguments you need to pass in, e.g. for the port mapping. All Chris' applications run on port 8080, and this is possible because each container port number is limited to that container. But then you need to map it to the external port number (on the host). You can pass in external configuration e.g. through environment variables. Mapping a directory on the host machine to a directory inside the container can also be done. Chris has set up his Jenkins instance in that way. Jenkins runs inside a container, but the Jenkins home is on the EC2 instance (the host). So even if the Jenkins container is restarted, Jenkins state is kept. Can you register a docker container with zookeeper? Does a docker container know where it is or do you have to tell it from the outside? Should the application register itself or should there be some helper process which registers it? Cluster manager mesos, lets you have a bunch of physical/virtual machines, and each machine has a defined set of resources. With mesos you can deploy an application, and you can just say well, I want to deploy these applications and each needs so much memory and so much diskspace and then mesos deploys the applications. mesos requires zookeeper. you can also use it for service discovery (zookeeper or mesos) marathon is a framework for long-running applications built on top of mesos. A participant uses an ansible script which downloads the docker images and sets it up. The container is told what it is and where it is, stuff like network routing you'll have to configure outside on the host system. elasticsearch has a master, which a node could discover on UDP. If UDP is not allowed, the node needs to be told where to find the master. Usually networking is not just launching a container, you may need to set up VPNs, and that is outside the container as well. If you deploy on EC2 do you just use AMIs or do you use docker on top. When you just use EC2 instances, you just get all the AWS infrastructure for free, but if you go down the docker route you'll have to find some other technology that'll do the same thing as AWS for you. If you're not running on EC2, you'll have all the downsides (of not running on EC2) anyway, with or without docker. How do you handle database migration? There is a trend towards decoupling deployment of code from database schema changes. You'd update the schema at some point (but the new columns wouldn't be used yet). At some later time you'd roll out the code which uses the new columns. As tools supporting database management/updates dynamodb and liquidbase were mentioned. | |||
| Recommendations:
Don't use Solaris. Pretty much none of the tools discussed above run on Solaris. | |||
