Docker: be efficient

Vm mount data disk and increased from from 4 gb to 8 gb.

learning-docker/README-0-mount.md at main · spawnmarvel/learning-docker (github.com)

0 Getting started

learning-docker/1-getting-started-guide/README.md at main · spawnmarvel/learning-docker · GitHub

End notes part 10:

Container orchestration

“Running containers in production is tough. You don’t want to log into a machine and simply run a docker run or docker compose up.“

* Why not?

* Well, what happens if the containers die?

* How do you scale across several machines?

Container orchestration solves this problem.

Tools like Kubernetes, Swarm, Nomad, and ECS all help solve this problem, all in slightly different ways.

k8s, swarm and nomad

1 Version

Docker compose v2 (the cli plugin) doesn’t require the version anymore, in fact it ignores it and always uses the latest schema. It also favors the filename compose.yml over docker-compose.yml. Though, you can use both, and even have both in the same folder.

Example 2024

Do you need to put the version at the top of docker-compose.yml file? – Open Source Projects / Compose – Docker Community Forums

2 Reference and guidelines

Compose file version 3 reference | Docker Docs

3 daemon.json

# /etc/docker

# override volume, move from OS disk to managed data disk
cat daemon.json
{
   "data-root": "/datadrive"
}

Docker daemon configuration overview | Docker Docs

4 Volume

The standard data location used for docker is /var/lib/docker. Because this directory contains all containers/images/volumes, it can be large.

Docker volume data move

Move it to a mounted data disk so you have control over usage learning-docker/1-getting-started-guide/README-2-docker-volume.md at main · spawnmarvel/learning-docker · GitHub

5 Portainer and compose

https://ip:9443

cli is fun, get to know it, when you know it add the layer of portainer for administration when you are tired of running the same cli commands.

Portainer can be deploy inline with the containers in compose.yml or inside its own compose.yml.

Deploy it as standalone learning-docker/prod-ish/portainer/README.md at main · spawnmarvel/learning-docker · GitHub

You can view containers, networks, images, volumes, etc and start/stop/create/rm and more.

for each container you can:

6 Compose changes

With reference to the portainer deploy, it has the following amount of max resources:

deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 500m

docker stats --no-stream

Lets change that in dir to 600m

# we have mounted volume, so user and pass is the same

docker compose down

sudo nano compose.yml

docker compose up -d

get the stream stats again

7 Dockerfile new build and redeploy

Lets create a new container on the fly with compose

rmq learning-docker/prod-ish/rmq/rmq-non-ssl at main · spawnmarvel/learning-docker · GitHub

copy all the files to a dir

docker compose up -d

http://ip:15672(1)

View it in portainer

stop and start it, view logs, or docker exec -it name sh/bash, no, just enter it from the icon.

It could be an error from container console

even if logs work

then just login

# example
docker ps
docker exec -it 21f31a49b21a bash

lets update the rabbitmq.conf to a new value, a new username

docker compose down

management.tcp.port = 15673 # in rabbitmq.conf

- 15673:15673 # compose.yml

# move all new files to the server

docker rmi -f name-or-id

# maybe you want to keep the volume for reuse else:
docker volume rm $(docker volume ls -qf dangling=true)

docker compose up -d

Then it will rebuild the image with the new properties and the volume.

http://ip:15673(1), lets view it now, nice.

Lets do the same but now change the log.file.rotation to 6 and keep the volume and also set back the ports to 15672.

docker compose down

log.file.rotation.count= 6 # in rabbitmq.conf
management.tcp.port = 15672 # in rabbitmq.conf

- 15673:15673 # compose.yml

docker rmi -f containername-or-id

# move all new files to the server

# maybe you want to keep the volume for reuse else:
# docker volume rm $(docker volume ls -qf dangling=true)

docker compose up -d

Then it will rebuild the image with the new properties and keep the volume.

Use portainer and connect to the container

So updates to the docker file = remove the image and build it again with compose, otherwise you are just starting the same image. The volume you can keep for data..

8 The –build Flag
When the docker-compose up command is used, the –build flag instructs Docker to rebuild a container. When this flag is set, Docker will immediately rebuild the container in light of any changes made to the container’s files.

Lets run some changes the rmq container again.

docker compose down
# edit to 10
log.file.rotation.count= 10 # rabbitmq.conf

docker compose up -d --build

Enter the container from portainer

9 Network and ports

Networking overview | Docker Docs

Map port 8080 on the Docker host to TCP port 80 in the container.
-p 8080:80

Specify user-defined bridge for production

learning-docker/1-getting-started-guide/README-3-docker-network.md at main · spawnmarvel/learning-docker · GitHub

Advanced


# So here we've assigned the 'default' network for this stack to be net_messaging
# This applies to all the services which do not have their own networks: variable

networks:
  default:
    name: net_messaging
    ipam: 
      config:
        - subnet: 172.50.0.0/24

# we have control over ip range, ping and dns is avaliable also.

learning-docker/1-getting-started-guide/README-3.1-docker-network-advanced.md at main · spawnmarvel/learning-docker · GitHub

Example with portainer, if you deploy it with no networks in compose its uses server ip xxx.xxx as prefix for default.

If you deploy it with a network, it uses docker ip’s.

10 depends_on

This is useful for the order of deploy, example, learning-docker/prod-ish/rmq/rmq-x2 at main · spawnmarvel/learning-docker · GitHub

Here we deploy two rabbitmq’s, one is the client with a shovel and the other is the server.

The client depends on the server, else it will just throw errors and warnings.

rmq-app:
    build:
      context: .
      dockerfile: Dockerfile_client
    depends_on:
      - rmq-app-server

As always copy all files and run docker compose up -d

Then the client gets an instant success for the shovel towards the server

commands

docker version

docker –version

docker compose version

docker compose up -d

docker compose up -d –build

docker compose down

docker ps

docker ps -a

docker rmi -f containername-or-id

docker rmi -f $(docker images -aq)

docker volume rm -f volumername-or-id

docker volume rm $(docker volume ls -qf dangling=true)

docker network prune

docker stats –no-stream

Tagged