Docker
Docker
https://www.docker.com/
Docker Hub
Docker Hub https://hub.docker.com/
What Is Docker
Package your application into a standardized unit for software development
- "Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment."
https://www.docker.com/what-docker
--
"Docker is an open-source project that automates the deployment of applications inside software containers."
Docker provides an additional layer of abstraction and automation of operating-system-level virtualization on Linux. Docker uses the resource isolation features of the Linux kernel such as cgroups and kernel namespaces, and a union-capable file system such as OverlayFS and others to allow independent "containers" to run within a single Linux instance, avoiding the overhead of starting and maintaining virtual machines.
The Linux kernel's support for namespaces mostly isolates an application's view of the operating environment, including process trees, network, user IDs and mounted file systems, while the kernel's cgroups provide resource limiting, including the CPU, memory, block I/O and network. Since version 0.9, Docker includes the libcontainer library as its own way to directly use virtualization facilities provided by the Linux kernel, in addition to using abstracted virtualization interfaces via libvirt, LXC (Linux Containers) and systemd-nspawn.
As actions are done to a Docker base image, union file system layers are created and documented, such that each layer fully describes how to recreate an action. This strategy enables Docker's lightweight images, as only layer updates need to be propagated (compared to full VMs, for example).
https://en.wikipedia.org/wiki/Docker_(software)
Removing all containers and images
# Stop the container(s) using the following command: docker-compose down # Delete all containers using the following command: docker rm -f $(docker ps -a -q) # Delete all volumes using the following command: docker volume rm $(docker volume ls -q) # Prune system images, included stopped: docker system prune -a
A Docker Tutorial for Beginners
A Docker Tutorial for Beginners
https://docker-curriculum.com/
Once you are done installing Docker, test your Docker installation by running the following:
# docker run hello-world
hello-world container: https://hub.docker.com/_/hello-world
Example:
root@server:~# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
b8dfde127a29: Pull complete
Digest: sha256:9f6ad537c5132bcce57f7a0a20e317228d382c3cd61edae14650eec68b2b345c
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
Example of Ubuntu container:
root@mserver:~# docker run -it ubuntu bash Unable to find image 'ubuntu:latest' locally latest: Pulling from library/ubuntu c549ccf8d472: Pull complete Digest: sha256:aba80b77e27148d99c034a987e7da3a287ed455390352663418c0f2ed40417fe Status: Downloaded newer image for ubuntu:latest root@4d827b59d198:/#
Try Busybox container:
## Fetch container
docker pull busybox
## List containers
docker images
## Run container (did nothing then exited)
docker run busybox
## Run container with instructions to do something
docker run busybox echo "hello from busybox"
hello from busybox
## List running docker containers:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
## List all dockers containers that we have run:
docker ps -a
## Run in interactive mode:
docker run -it busybox sh
## Run and auto delete on exit
docker run --rm busybox echo "hello"
## Remove docker containers (clean up after yourself
docker rm [CONTAINER_ID CONTAINER_ID ...]
## Remove all exited docker containers:
docker rm $(docker ps -a -q -f status=exited)
# or in later editions
docker container prune
## Remove docker images:
docker rmi [IMAGE_ID IMAGE_ID ...]
## Run a detached instance, and name it
docker run -d --name [NAME] [IMAGE]
# example: docker run -d --rm --name u3 -it ubuntu bash
## Attach to running container: [https://docs.docker.com/engine/reference/commandline/attach/]
docker attach [NAME]
## Detach from a running container: [https://docs.docker.com/engine/reference/commandline/attach/]
CTRL-p CTRL-q
## Stop detached instance
docker stop [NAME]
## Pull a specific version:
docker pull ubuntu:latest
docker pull ubuntu:18.04
docker pull ubuntu:16.04
## Run specific version
docker run ubuntu:16.04
root@server:~# docker pull busybox Using default tag: latest latest: Pulling from library/busybox b71f96345d44: Pull complete Digest: sha256:930490f97e5b921535c153e0e7110d251134cc4b72bbb8133c6a5065cc68580d Status: Downloaded newer image for busybox:latest docker.io/library/busybox:latest root@server:~# docker images REPOSITORY TAG IMAGE ID CREATED SIZE ubuntu latest 9873176a8ff5 13 days ago 72.7MB busybox latest 69593048aa3a 3 weeks ago 1.24MB hello-world latest d1165f221234 3 months ago 13.3kB root@server:~# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 495cf502dde2 busybox "echo 'hello from bu…" 2 minutes ago Exited (0) 2 minutes ago
Create Image
All user images are based on a base image.
Login to Docker Hub:
docker login
Sample Dockerfile:
FROM python:3 # set a directory for the app WORKDIR /usr/src/app # copy all the files to the container COPY . . # install dependencies RUN pip install --no-cache-dir -r requirements.txt # define the port number the container should expose EXPOSE 5000 # run the command CMD ["python", "./app.py"]
Simple Dockerfile:
FROM ubuntu RUN touch test.txt
Build image:
docker build -t [username]/[project] .
Test imagE:
docker run [username]/[project]<:version>
Push image to Docker Hub:
docker push [username]/[project]
See parent hash of image:
docker history --format 'Template:.ID' --no-trunc [USERNAME]/[PROJECT]
Pull Image
docker pull
Hash specific: [3]
docker pull ubuntu@sha256:26c68657ccce2cb0a31b330cb0be2b5e108d467f641c62e13ab40cbec258c68d
Hash specific in Dockerfile: [4]
FROM ubuntu@sha256:26c68657ccce2cb0a31b330cb0be2b5e108d467f641c62e13ab40cbec258c68d
Attach to Running Docker
docker attach [NAME]
If it isn't running bash, you can do something like this:
docker exec -it [NAME] /bin/bash
Running ARM on x86
Problem:
uname -m # Display the host architecture #x86_64
docker run --rm -t arm64v8/ubuntu uname -m # Run an executable made for aarch64 on x86_64 #standard_init_linux.go:211: exec user process caused "exec format error"
Solution:
As expected the instructions are not recognized since the packages are not installed yet. Installing the following packages should allow you to enable support for aarch64 containers on your x86 workstation:
sudo apt-get install qemu binfmt-support qemu-user-static # Install the qemu packages docker run --rm --privileged multiarch/qemu-user-static --reset -p yes # This step will execute the registering scripts
docker run --rm -t arm64v8/ubuntu uname -m # Testing the emulation environment #aarch64
ref: [5]
Static MAC
docker run ... --mac-address 12:34:56:78:9a:bc
ref: [6]
Exporting
Save and Load Image
docker save -o [image.tar] [IMAGE] # or docker save [IMAGE] > [image.tar]
docker load -i [image.tar] # or cat [image.tar] | docker load
Export and Import Container
docker export [CONTAINER_ID_OR_NAME] > [container.tar]
docker import [container.tar] [IMAGE_NAME:VER] # or cat [container.tar] | docker import - [IMAGE_NAME:VER]
Extract Build Steps
docker history [IMAGE_ID] --format 'Template:.CreatedBy' --no-trunc | tac
docker history [IMAGE_ID] --no-trunc | tac
Docker Compose
apt install docker-compose docker-compose-plugin # docker-compose provides "docker-compose" command # docker-compose-plugin provides "docker compose" command
docker compose up -d
Sample docker-compose.yml:
version: "3.0" # compose API version
services:
license-service:
container_name: license
# Docker image name
image: flexlm:v1
build:
context: .
user: root
restart: always
# mac address tied to license file
mac_address: xx:xx:xx:xx:xx:xx
ports:
# 27000 for lmgrd
- 27000:27000
# 27001 for armlmd
- 27001:27001
Proxy
Client:
~/.docker/config.json
{
"proxies": {
"default": {
"httpProxy": "http://proxy.example.com:3128",
"httpsProxy": "https://proxy.example.com:3129",
"noProxy": "*.test.example.com,.example.org,127.0.0.0/8"
}
}
}
ref: Configure Docker to use a proxy server | Docker Documentation - https://docs.docker.com/network/proxy/
Configure Docker Service
systemctl edit docker.service
[Service] Environment="HTTP_PROXY=http://proxy.example.com:3128", Environment="HTTPS_PROXY=httpS://proxy.example.com:3129", Environment="NO_PROXY=localhost,127.0.0.1",
systemctl restart docker.service
Registry
Registry API
HTTP API V2 | Docker Documentation https://docs.docker.com/registry/spec/api/
List available images:
GET /v2/_catalog GET https://registry.example.com/v2/_catalog
Pretty print output:
curl -s https://registry.example.com/v2/_catalog | jq curl -s https://registry.example.com/v2/_catalog | python3 -m json.tool
Pagenation
GET /v2/_catalog?n=<count> # using a name of repo in a previous result: GET /v2/_catalog?n=<count>&last=<name_repo_start_with>
List tags: (pagination also works with tags)
GET /v2/<image_name>/tags/list
Delete image:
# reference must be a digest or the delete will fail. DELETE /v2/<name>/manifests/<reference>
API:
Method Path Entity Description GET /v2/ Base Check that the endpoint implements Docker Registry API V2. GET /v2/<name>/tags/list Tags Fetch the tags under the repository identified by name. GET /v2/<name>/manifests/<reference> Manifest Fetch the manifest identified by name and reference where reference can be a tag or digest. A HEAD request can also be issued to this endpoint to obtain resource information without receiving all data. PUT /v2/<name>/manifests/<reference> Manifest Put the manifest identified by name and reference where reference can be a tag or digest. DELETE /v2/<name>/manifests/<reference> Manifest Delete the manifest or tag identified by name and reference where reference can be a tag or digest. Note that a manifest can only be deleted by digest. GET /v2/<name>/blobs/<digest> Blob Retrieve the blob from the registry identified by digest. A HEAD request can also be issued to this endpoint to obtain resource information without receiving all data. DELETE /v2/<name>/blobs/<digest> Blob Delete the blob identified by name and digest POST /v2/<name>/blobs/uploads/ Initiate Blob Upload Initiate a resumable blob upload. If successful, an upload location will be provided to complete the upload. Optionally, if the digest parameter is present, the request body will be used to complete the upload in a single request. GET /v2/<name>/blobs/uploads/<uuid> Blob Upload Retrieve status of upload identified by uuid. The primary purpose of this endpoint is to resolve the current status of a resumable upload. PATCH /v2/<name>/blobs/uploads/<uuid> Blob Upload Upload a chunk of data for the specified upload. PUT /v2/<name>/blobs/uploads/<uuid> Blob Upload Complete the upload specified by uuid, optionally appending the body as the final chunk. DELETE /v2/<name>/blobs/uploads/<uuid> Blob Upload Cancel outstanding upload processes, releasing associated resources. If this is not called, the unfinished uploads will eventually timeout. GET /v2/_catalog Catalog Retrieve a sorted, json list of repositories available in the registry.
Reducing Image Size
1. Bundle Layers
RUN apt-get update -y \ && apt install -y package_one
2. Avoid extra packages
apt install -y --no-install-recommends
3. Clean up after install
apt ... \ && rm -rf /var/lib/apt/lists/*
4. Pick smaller image
alpine vs ubuntu
5. Use .dockerignore
ref: [7]
Issues
apt getting stuck at configuring tzdata
Configuring tzdata ------------------ Please select the geographic area in which you live. Subsequent configuration questions will narrow this down by presenting a list of cities, representing the time zones in which they are located. 1. Africa 4. Australia 7. Atlantic 10. Pacific 13. Etc 2. America 5. Arctic 8. Europe 11. SystemV 3. Antarctica 6. Asia 9. Indian 12. US Geographic area:
Solution:
FROM ubuntu:20.04 ENV TZ=Asia/Dubai RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone RUN apt update RUN apt install -y tzdata
ref: Fix: tzdata hangs during Docker image build - DEV Community - https://dev.to/grigorkh/fix-tzdata-hangs-during-docker-image-build-4o9m