{"id":449,"date":"2019-11-18T07:30:55","date_gmt":"2019-11-18T13:30:55","guid":{"rendered":"https:\/\/jacobncalvert.com\/?p=449"},"modified":"2019-11-17T21:43:22","modified_gmt":"2019-11-18T03:43:22","slug":"virtualization-for-embedded-systems-series-containers-deep-dive","status":"publish","type":"post","link":"https:\/\/jacobncalvert.com\/blog-archive\/2019\/11\/18\/virtualization-for-embedded-systems-series-containers-deep-dive\/","title":{"rendered":"Virtualization for Embedded Systems Series: Containers Deep Dive"},"content":{"rendered":"\n

In the previous post<\/a>, I looked at several real-world use cases for containers and hypervisors. This post will be a deep dive into containers and a how-to on using them. <\/p>\n\n\n\n

Note: all the code, Dockerfiles, etc. are<\/strong><\/em> archived in a git repo at GitHub.<\/strong><\/em><\/a><\/p>\n\n\n\n

Container History<\/h2>\n\n\n\n

Origins<\/h3>\n\n\n\n

A little history is needed before we jump into building and deploying containers. Docker, which is likely still the largest container technology provider by a longshot, was originally released in 2013. In just 6 years a huge community using containers sprang up around the concept – check out DockerHub<\/a> for a sense of scale in the community. Preceding the Docker phenomenon, we had LXC (Linux Containers) which had been around since around 2008<\/a>, but it really never had the sticking power that Docker has enjoyed. Fast-forward to today, and there are a handful of competing container technologies, all with similar features, but some have better ecosystems or support than others. <\/p>\n\n\n\n

Where We Are Today<\/h3>\n\n\n\n

Recently there has a push for a standardization of the container frameworks that makeup these ecosystems. The Open Container Initiative<\/a> is an open governance body for working towards standardization of the container framework. Many container systems have already adopted this and become OCI compliant or aligned. Docker has also bolstered the community by donating much of its container infrastructure components to the open source world for OCI to utilize. <\/p>\n\n\n\n

Goal of the Open Container Initiative<\/h3>\n\n\n\n

I encourage you to go to the OCI’s website and read their mission documents in whole, but I’d like to provide the brief description here as it relates to our use of containers in embedded systems. The OCI intends to create a standard for representing containers and their basic runtime requirements and interfaces for portability. From their FAQs:<\/p>\n\n\n\n

The mission of the Open Container Initiative (OCI) is to promote a set of common, minimal, open standards and specifications around container technology.<\/p>What is the mission of the OCI? FAQ (https:\/\/www.opencontainers.org\/faq#faq1<\/a>)<\/cite><\/blockquote>\n\n\n\n

This is important to us in embedded engineering because we need this portability and compatibility to fully realize the use cases outlined in the last post. Since the OCI has not reached a sufficiently mature status, I will be sticking with Docker as the container ecosystem in this post. <\/p>\n\n\n\n

Docker Containers: A Hands-On Example<\/h2>\n\n\n\n

Objectives<\/h3>\n\n\n\n

In the following sections we will accomplish the following:<\/p>\n\n\n\n

  1. Create a basic container image from a Dockerfile<\/li>
  2. Create a container to build and test a custom application<\/li>
  3. Create a container to host the custom application<\/li>
  4. Export the container<\/li>
  5. Run the container on a different platform<\/li><\/ol>\n\n\n\n

    Creating a Basic Container<\/h3>\n\n\n\n

    I’m starting with the assumption that Docker is installed and you’ve been able to run the Docker hello-world image. We will use a Dockerfile to compose our basic image. A Dockerfile is a script of sorts which instructs the engine to construct our container piece by piece. I’ve put a (very) basic Dockerfile below. <\/p>\n\n\n

    \n##########################################\n# File: Dockerfile\n# Author: Jacob Calvert <jcalvert@jacobncalvert.com>\n# Date: Nov-08-2019\n# \n# This is a basic Dockerfile \n#\n##########################################\n\n# start from the basic busybox image\nFROM busybox\n\n# put a file in our container\nCOPY hello-world.txt .\n<\/pre><\/div>\n\n\n
    What I have in my workspace on my host machine is as follows:<\/p>\n\n\n
    \njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ ls\nDockerfile  hello-world.txt\njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ cat hello-world.txt\nhello, world!\n<\/pre><\/div>\n\n\n
    To build the Docker image from the Dockerfile, you run a ‘docker build’ command. The ‘-t’ specifies a tag by which we’ll reference this image, and the ‘.’ specifies the directory for the Dockerfile –  in this case, the current directory. <\/p>\n\n\n
    \njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ docker build -t basic .\nSending build context to Docker daemon  3.072kB\nStep 1\/2 : FROM busybox\n ---> 020584afccce\nStep 2\/2 : COPY hello-world.txt .\n ---> 11e3eeda8f8e\nSuccessfully built 11e3eeda8f8e\nSuccessfully tagged basic:latest\njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ \n<\/pre><\/div>\n\n\n
    Let’s view and run our image now.<\/p>\n\n\n
    \njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ docker image ls\nREPOSITORY                        TAG                 IMAGE ID            CREATED             SIZE\nbasic                             latest              11e3eeda8f8e        42 seconds ago      1.22MB\nubuntu                            latest              775349758637        8 days ago          64.2MB\nbusybox                           latest              020584afccce        9 days ago          1.22MB\nhello-world                       latest              f2a91732366c        23 months ago       1.85kB\nquantumobject\/docker-zoneminder   latest              469615ab191d        24 months ago       1.15GB\nmysql\/mysql-server                latest              a3ee341faefb        2 years ago         246MB\njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ docker run -it basic\n\/ # ls\nbin              dev              etc              hello-world.txt  home             proc             root             sys              tmp              usr              var\n\/ # cat hello-world.txt \nhello, world!\n\/ # exit\njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ \n<\/pre><\/div>\n\n\n
    What are we looking at here? First we list our available local images with the docker image ls<\/em> command. We can see that I have several images on my development machine, including basic<\/strong> which was created only a few moments ago from our build<\/em> command. Next I run the image we created with docker run -it <image><\/em>. The -it <\/em>flags are for –interactive and –tty. These two together essentially present the container’s console as a pseudo-tty device in your terminal, in interactive mode. It is as if you are sitting in front of another machine. <\/p>\n\n\n\n
    Next, you can see that I have a different prompt. This is the container’s prompt. I type ls<\/em> and we can see our hello-world.txt is there on the filesystem of our container as we desired via the COPY command in the Dockerfile. Lastly, we type exit<\/em> which ends the busybox process and exits the container. <\/p>\n\n\n\n
    Now let’s look at the persistent state parts of Docker. <\/p>\n\n\n
    \njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ docker system info\nClient:\n Debug Mode: false\n\nServer:\n Containers: 1\n  Running: 0\n  Paused: 0\n  Stopped: 1\n Images: 6\n Server Version: 19.03.2\n Storage Driver: overlay2\n  Backing Filesystem: extfs\n  Supports d_type: true\n  Native Overlay Diff: true\n Logging Driver: json-file\n Cgroup Driver: cgroupfs\n Plugins:\n  Volume: local\n  Network: bridge host ipvlan macvlan null overlay\n  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog\n Swarm: inactive\n Runtimes: runc\n Default Runtime: runc\n Init Binary: docker-init\n containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb\n runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f\n init version: fec3683\n Security Options:\n  apparmor\n  seccomp\n   Profile: default\n Kernel Version: 4.4.0-165-generic\n Operating System: Linux Mint 18\n OSType: linux\n Architecture: x86_64\n CPUs: 4\n Total Memory: 15.55GiB\n Name: jacob-aspire-mint\n ID: Z2GE:5Y2A:K4LP:UC4O:QV3A:4JMR:5RLW:2DHQ:WANN:2RA3:VKJ2:UZMI\n Docker Root Dir: \/var\/lib\/docker\n Debug Mode: false\n Registry: https:\/\/index.docker.io\/v1\/\n Labels:\n Experimental: false\n Insecure Registries:\n  127.0.0.0\/8\n Live Restore Enabled: false\n<\/pre><\/div>\n\n\n
    Using docker system info<\/em> we can see that there is 1 container and that it is stopped (not running). How do we see what that container is?<\/p>\n\n\n
    \njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ docker ps -a\nCONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                     PORTS               NAMES\nc76e0de1a530        basic               "sh"                4 minutes ago       Exited (0) 3 minutes ago                       silly_kalam\n\n<\/pre><\/div>\n\n\n
    We can use the ps<\/em> command to see information about our containers. Notice the STATUS<\/strong> column. Our container’s execution has exited. This begs the question: can we restart a container and pick up where we left off? The answer of course is yes!<\/p>\n\n\n
    \njacob@jacob-aspire-mint \/media\/jacob\/jacob\/Documents\/Workspaces\/Blog\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ docker container start -i silly_kalam \n\/ # ls\nbin              dev              etc              hello-world.txt  home             proc             root             sys              tmp              usr              var\n\/ # \n\n<\/pre><\/div>\n\n\n
    Notice the command difference this time around. I am using docker container start -i <name><\/em> where the name<\/strong> is a name given to the container at run time. The container can be referenced by its human-friendly name as I have done here, or by its ID (the long UUID). Also note the -i flag; this is to open it as an interactive session again. No -t is needed since the TTY device has already been allocated. So can we work inside this running container like a real machine and see the state persist? Indeed we can as well! Let’s see it in action. <\/p>\n\n\n
    \n\/ # cd home\/\n\/home # mkdir -p user\/workspace\/\n\/home # cd user\/workspace\/\n\/home\/user\/workspace # echo "another file!" > another.file\n\/home\/user\/workspace # ls\nanother.file\n\/home\/user\/workspace # exit\n\njacob@jacob-aspire-mint \/workspace\/virtualization-for-embedded-systems-series\/containers-deep-dive\/basic $ docker container start -i silly_kalam \n\n\/ # ls\nbin              dev              etc              hello-world.txt  home             proc             root             sys              tmp              usr              var\n\/ # cd home\/user\/workspace\/\n\/home\/user\/workspace # ls\nanother.file\n\/home\/user\/workspace # cat another.file \nanother file!\n\/home\/user\/workspace # \n\n<\/pre><\/div>\n\n\n
    It may be a little hard to follow, but here’s what has happened. From inside our previously started container named silly_kalam<\/em>, I have created a directory \/home\/user\/workspace<\/strong>. Next, I created a file named another.file<\/strong> and filled it with “another file!”. I then exited the container. Next, I started the container again, changed directory to my created directory, and printed out the contents of another.file<\/strong>. <\/p>\n\n\n\n
    Using this example we can see that the content inside a container is not purely ephemeral, but can persist over many starts\/stops. For more persistent storage check out Docker’s Volume subsystem.<\/a><\/p>\n\n\n\n
    Creating a Development Environment Container<\/h3>\n\n\n\n
    Using the knowledge gained through building a basic container, we will now create a container to use as a development environment for our example application. <\/p>\n\n\n\n
    The Application Specs<\/h4>\n\n\n\n
    We want to build a simple application which demonstrates the portability of containers and also does something we can test. We also want it to be a simple application for demo purposes. With this in mind, our application will be a simple data modem. It will take in data from one source medium and spit it out as another medium. Below is a list of basic requirements codified for this application:<\/p>\n\n\n\n