Docker: Difference between revisions

Latest revision as of 19:24, 18 August 2024

Subpage Table of Contents

Docker

https://www.docker.com/

Docker Hub

Docker Hub
https://hub.docker.com/

Search Hub

docker search [image]

Example:

docker search ubuntu

Usage:  docker search [OPTIONS] TERM

Search Docker Hub for images

Options:
  -f, --filter filter   Filter output based on conditions provided
      --format string   Pretty-print search using a Go template
      --limit int       Max number of search results
      --no-trunc        Don't truncate output

Search Tags

docker-browse tags [image]

Install with:

npm install -g docker-browse

Or local with:

mkdir ~/.src/docker-browse
cd ~/.src/docker-browse
npm install docker-browse
cd ~/.bin
ln -s ../.src/docker-browse/node_modules/.bin/docker-browse

Assuming .bin is in your path

ref: https://www.npmjs.com/package/docker-browse

What Is Docker

Package your application into a standardized unit for software development

"Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment."

https://www.docker.com/what-docker

--

"Docker is an open-source project that automates the deployment of applications inside software containers."

Docker provides an additional layer of abstraction and automation of operating-system-level virtualization on Linux. Docker uses the resource isolation features of the Linux kernel such as cgroups and kernel namespaces, and a union-capable file system such as OverlayFS and others to allow independent "containers" to run within a single Linux instance, avoiding the overhead of starting and maintaining virtual machines.

The Linux kernel's support for namespaces mostly isolates an application's view of the operating environment, including process trees, network, user IDs and mounted file systems, while the kernel's cgroups provide resource limiting, including the CPU, memory, block I/O and network. Since version 0.9, Docker includes the libcontainer library as its own way to directly use virtualization facilities provided by the Linux kernel, in addition to using abstracted virtualization interfaces via libvirt, LXC (Linux Containers) and systemd-nspawn.

As actions are done to a Docker base image, union file system layers are created and documented, such that each layer fully describes how to recreate an action. This strategy enables Docker's lightweight images, as only layer updates need to be propagated (compared to full VMs, for example).

https://en.wikipedia.org/wiki/Docker_(software)

Removing all containers and images

#  Stop the container(s) using the following command:
docker-compose down
#  Delete all containers using the following command:
docker rm -f $(docker ps -a -q)
#  Delete all volumes using the following command:
docker volume rm $(docker volume ls -q)
#  Prune system images, included stopped:
docker system prune -a

Ref [1] [2]

A Docker Tutorial for Beginners

A Docker Tutorial for Beginners
https://docker-curriculum.com/

Once you are done installing Docker, test your Docker installation by running the following:

# docker run hello-world

hello-world container: https://hub.docker.com/_/hello-world

Example:

root@server:~# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
b8dfde127a29: Pull complete
Digest: sha256:9f6ad537c5132bcce57f7a0a20e317228d382c3cd61edae14650eec68b2b345c
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Example of Ubuntu container:

root@mserver:~# docker run -it ubuntu bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
c549ccf8d472: Pull complete
Digest: sha256:aba80b77e27148d99c034a987e7da3a287ed455390352663418c0f2ed40417fe
Status: Downloaded newer image for ubuntu:latest
root@4d827b59d198:/#

Try Busybox container:

## Fetch container
docker pull busybox

## List containers
docker images

## Run container (did nothing then exited)
docker run busybox

## Run container with instructions to do something
docker run busybox echo "hello from busybox"
    hello from busybox

## List running docker containers:
docker ps
    CONTAINER ID   IMAGE             COMMAND                  CREATED        STATUS       PORTS         NAMES

## List all dockers containers that we have run:
docker ps -a

## Run in interactive mode:
docker run -it busybox sh

## Run and auto delete on exit
docker run --rm busybox echo "hello"

## Remove docker containers (clean up after yourself
docker rm [CONTAINER_ID  CONTAINER_ID  ...]

## Remove all exited docker containers:
docker rm $(docker ps -a -q -f status=exited)
# or in later editions
docker container prune

## Remove docker images:
docker rmi [IMAGE_ID  IMAGE_ID ...]

## Run a detached instance, and name it
docker run -d --name [NAME] [IMAGE]
# example:  docker run -d --rm --name u3 -it ubuntu bash

## Attach to running container: [https://docs.docker.com/engine/reference/commandline/attach/]
docker attach [NAME]

## Detach from a running container: [https://docs.docker.com/engine/reference/commandline/attach/]
CTRL-p CTRL-q

## Stop detached instance
docker stop [NAME]

## Pull a specific version:
docker pull ubuntu:latest
docker pull ubuntu:18.04
docker pull ubuntu:16.04

## Run specific version
docker run ubuntu:16.04

root@server:~# docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
b71f96345d44: Pull complete
Digest: sha256:930490f97e5b921535c153e0e7110d251134cc4b72bbb8133c6a5065cc68580d
Status: Downloaded newer image for busybox:latest
docker.io/library/busybox:latest

root@server:~# docker images
REPOSITORY             TAG            IMAGE ID       CREATED         SIZE
ubuntu                 latest         9873176a8ff5   13 days ago     72.7MB
busybox                latest         69593048aa3a   3 weeks ago     1.24MB
hello-world            latest         d1165f221234   3 months ago    13.3kB

root@server:~# docker ps -a
CONTAINER ID   IMAGE                            COMMAND                  CREATED         STATUS                     PORTS                         NAMES
495cf502dde2   busybox                          "echo 'hello from bu…"   2 minutes ago   Exited (0) 2 minutes ago

Create Image

All user images are based on a base image.

Login to Docker Hub:

docker login

Sample Dockerfile:

FROM python:3

# set a directory for the app
WORKDIR /usr/src/app

# copy all the files to the container
COPY . .

# install dependencies
RUN pip install --no-cache-dir -r requirements.txt

# define the port number the container should expose
EXPOSE 5000

# run the command
CMD ["python", "./app.py"]

Simple Dockerfile:

FROM ubuntu

RUN touch test.txt

Build image:

docker build -t [username]/[project] .

Test imagE:

docker run [username]/[project]<:version>

Push image to Docker Hub:

docker push [username]/[project]

See parent hash of image:

 docker history --format 'Template:.ID' --no-trunc [USERNAME]/[PROJECT]

Pull Image

docker pull

Hash specific: [3]

docker pull ubuntu@sha256:26c68657ccce2cb0a31b330cb0be2b5e108d467f641c62e13ab40cbec258c68d

Hash specific in Dockerfile: [4]

FROM ubuntu@sha256:26c68657ccce2cb0a31b330cb0be2b5e108d467f641c62e13ab40cbec258c68d

Run Image

docker run -it --rm [image:tag] /bin/bash

If you want to force a specific entry point:

docker run -it --rm --entrypoint /bin/sh [image:tag]

Attach to Running Docker

docker attach [NAME]

If it isn't running bash, you can do something like this:

docker exec -it [NAME] /bin/bash

Running ARM on x86

Problem:

uname -m # Display the host architecture
#x86_64

docker run --rm -t arm64v8/ubuntu uname -m # Run an executable made for aarch64 on x86_64
#standard_init_linux.go:211: exec user process caused "exec format error"

Solution:

As expected the instructions are not recognized since the packages are not installed yet. Installing the following packages should allow you to enable support for aarch64 containers on your x86 workstation:

sudo apt-get install qemu binfmt-support qemu-user-static # Install the qemu packages
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes # This step will execute the registering scripts

docker run --rm -t arm64v8/ubuntu uname -m # Testing the emulation environment
#aarch64

ref: [5]

Static MAC

docker run ... --mac-address 12:34:56:78:9a:bc

ref: [6]

Exporting

Save and Load Image

docker save -o [image.tar] [IMAGE]
# or
docker save [IMAGE] > [image.tar]

docker load -i [image.tar]
# or
cat [image.tar] | docker load

Export and Import Container

docker export [CONTAINER_ID_OR_NAME] > [container.tar]

docker import [container.tar] [IMAGE_NAME:VER]
# or
cat [container.tar] | docker import - [IMAGE_NAME:VER]

Extract Build Steps

docker history [IMAGE_ID] --format 'Template:.CreatedBy' --no-trunc | tac

docker history [IMAGE_ID] --no-trunc | tac

docker history --no-trunc $argv | tac | tr -s ' ' | cut -d " " -f 5- | sed 's,^/bin/sh -c #(nop) ,,g' | sed 's,^/bin/sh -c,RUN,g' | sed 's, && ,\n  & ,g' | sed 's,\s*[0-9]*[\.]*[0-9]*\s*[kMG]*B\s*$,,g' | head -n -1

ref: [7]

Docker Compose

apt install docker-compose docker-compose-plugin
# docker-compose provides "docker-compose" command
# docker-compose-plugin provides "docker compose" command

docker compose up -d

Sample docker-compose.yml:

version: "3.0"  # compose API version
services:
  license-service:
    container_name: license
    # Docker image name
    image: flexlm:v1
    build:
      context: .
    user: root
    restart: always
    # mac address tied to license file
    mac_address: xx:xx:xx:xx:xx:xx
    ports:
      # 27000 for lmgrd
      - 27000:27000
      # 27001 for armlmd
      - 27001:27001

Docker BuildX

buildx:

https://github.com/docker/buildx

apt package:

docker-buildx/jammy-updates,jammy-security 0.12.1-0ubuntu1~22.04.2 amd64
  Docker CLI plugin for extended build capabilities with BuildKit

Install buildx with apt:

sudo apt install docker-buildx

Set buildx as default builder:

docker buildx install

To remove buildx alias:

docker buildx uninstall

You can build multi-platform images using three different strategies that are supported by Buildx and Dockerfiles:

Using the QEMU emulation support in the kernel
Building on multiple native nodes using the same builder instance
Using a stage in Dockerfile to cross-compile to different architectures

If not using Docker Desktop:

docker run --privileged --rm tonistiigi/binfmt --install all

Add additional platforms:

docker buildx create --use --name mybuild node-amd64
docker buildx create --append --name mybuild node-arm64
docker buildx build --platform linux/amd64,linux/arm64 .

Proxy

Client:

~/.docker/config.json

{
 "proxies": {
   "default": {
     "httpProxy": "http://proxy.example.com:3128",
     "httpsProxy": "https://proxy.example.com:3129",
     "noProxy": "*.test.example.com,.example.org,127.0.0.0/8"
   }
 }
}

ref: Configure Docker to use a proxy server | Docker Documentation - https://docs.docker.com/network/proxy/

Configure Docker Service

systemctl edit docker.service

[Service]
Environment="HTTP_PROXY=http://proxy.example.com:3128",
Environment="HTTPS_PROXY=httpS://proxy.example.com:3129",
Environment="NO_PROXY=localhost,127.0.0.1",

systemctl restart docker.service

Registry

Registry API

HTTP API V2 | Docker Documentation
https://docs.docker.com/registry/spec/api/

List available images:

GET /v2/_catalog
GET https://registry.example.com/v2/_catalog

Pretty print output:

curl -s https://registry.example.com/v2/_catalog | jq
curl -s https://registry.example.com/v2/_catalog | python3 -m json.tool

Pagenation

GET /v2/_catalog?n=<count>
# using a name of repo in a previous result:
GET /v2/_catalog?n=<count>&last=<name_repo_start_with>

List tags: (pagination also works with tags)

GET /v2/<image_name>/tags/list

Delete image:

# reference must be a digest or the delete will fail.
DELETE /v2/<name>/manifests/<reference>

API:

Method	Path	Entity	Description
GET	/v2/	Base	Check that the endpoint implements Docker Registry API V2.
GET	/v2/<name>/tags/list	Tags	Fetch the tags under the repository identified by name.
GET	/v2/<name>/manifests/<reference>	Manifest	Fetch the manifest identified by name and reference where reference can be a tag or digest. A HEAD request can also be issued to this endpoint to obtain resource information without receiving all data.
PUT	/v2/<name>/manifests/<reference>	Manifest	Put the manifest identified by name and reference where reference can be a tag or digest.
DELETE	/v2/<name>/manifests/<reference>	Manifest	Delete the manifest or tag identified by name and reference where reference can be a tag or digest. Note that a manifest can only be deleted by digest.
GET	/v2/<name>/blobs/<digest>	Blob	Retrieve the blob from the registry identified by digest. A HEAD request can also be issued to this endpoint to obtain resource information without receiving all data.
DELETE	/v2/<name>/blobs/<digest>	Blob	Delete the blob identified by name and digest
POST	/v2/<name>/blobs/uploads/	Initiate Blob Upload	Initiate a resumable blob upload. If successful, an upload location will be provided to complete the upload. Optionally, if the digest parameter is present, the request body will be used to complete the upload in a single request.
GET	/v2/<name>/blobs/uploads/<uuid>	Blob Upload	Retrieve status of upload identified by uuid. The primary purpose of this endpoint is to resolve the current status of a resumable upload.
PATCH	/v2/<name>/blobs/uploads/<uuid>	Blob Upload	Upload a chunk of data for the specified upload.
PUT	/v2/<name>/blobs/uploads/<uuid>	Blob Upload	Complete the upload specified by uuid, optionally appending the body as the final chunk.
DELETE	/v2/<name>/blobs/uploads/<uuid>	Blob Upload	Cancel outstanding upload processes, releasing associated resources. If this is not called, the unfinished uploads will eventually timeout.
GET	/v2/_catalog	Catalog	Retrieve a sorted, json list of repositories available in the registry.

Reducing Image Size

1. Bundle Layers

RUN apt-get update -y \
  && apt install -y package_one

2. Avoid extra packages

apt install -y --no-install-recommends

3. Clean up after install

apt ... \
&& rm -rf /var/lib/apt/lists/*

4. Pick smaller image

alpine vs ubuntu

5. Use .dockerignore

ref: [8]

Issues

apt getting stuck at configuring tzdata

Configuring tzdata
------------------

Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.

  1. Africa      4. Australia  7. Atlantic  10. Pacific  13. Etc
  2. America     5. Arctic     8. Europe    11. SystemV
  3. Antarctica  6. Asia       9. Indian    12. US
Geographic area:

Solution:

FROM ubuntu:20.04

ENV TZ=Asia/Dubai
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

RUN apt update
RUN apt install -y tzdata

ref: Fix: tzdata hangs during Docker image build - DEV Community - https://dev.to/grigorkh/fix-tzdata-hangs-during-docker-image-build-4o9m