5.2. Podman Based Migration

This guide details the steps to update your containers to the latest version if you are running S4DBs on a native podman installation (without docker compose, podman-docker or podman-compose).

5.2.1. from v2.0.0 to v2.1.0

The migration from v2.0.0 to v2.1.0 is way less difficult than from v1.8.X to v2.0.0. All container images got an upgrade for improvements and security fixes. So please take care to upgrade the TimeScaleDB and NginX image too!

0. OPTIONAL: Load Images from tar.gz archive

If your system does not have access docker hub or another registry serving the images, you can download them from our download portal, copy them to your machine and load them into local image store with the following podman command.

podman load --input s4dbs_2.1.0_linux_<amd64/arm64>_all_images.tar.gz
1. Stop Existing Containers

First, stop the currently running containers.

podman stop s4dbs_postgres s4dbs_collector s4dbs_service s4dbs_grafana s4dbs_frontend s4dbs_reverse
2. Remove Existing Containers

Next, remove the stopped containers.

podman rm s4dbs_postgres s4dbs_collector s4dbs_service s4dbs_grafana s4dbs_frontend s4dbs_reverse
3. Start New Containers with Updated Tags

Now, run the containers with the new 2.1.0 tag. Change run command if you changed them during first install.

TimeScaleDB/PostreSQL
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=POSTGRES_PASSWORD --name s4dbs_postgres -v s4dbs_postgres:/var/lib/postgresql/data docker.io/itgainis/s4dbs_timescaledb:2.1.0
Note
 If you had added speical environment variables such as POSTGRES_OPTIONS, please don’t forget to add them here! The Command above is just an example!
Collector
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=PDB_PASS --name s4dbs_collector -e PDB_USER=postgres -e PDB_HOST=s4dbs_postgres -e PDB_PORT=5432 -e PDB_DB_NAME=speedgain -e SPGLOGLEVEL=INFO docker.io/itgainis/s4dbs_collector:2.1.0
Service
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=PDB_PASS --name s4dbs_service -e PDB_USER=postgres -e PDB_HOST=s4dbs_postgres -e PDB_PORT=5432 -e PDB_DB_NAME=speedgain -e SPGLOGLEVEL=INFO docker.io/itgainis/s4dbs_service:2.1.0
Grafana
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=PDB_PASS --name s4dbs_grafana -e PDB_USER=postgres -e PDB_HOST=s4dbs_postgres -e PDB_PORT=5432 -e PDB_DB_NAME=speedgain -e GF_SERVER_SERVE_FROM_SUB_PATH=true -e "GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/" docker.io/itgainis/s4dbs_grafana:2.1.0
Frontend
podman run -d --network s4dbs_network --name s4dbs_frontend -e FRONTEND_CONFIG_MODE=false docker.io/itgainis/s4dbs_frontend:2.1.0
Reverse Proxy
podman run -d --network s4dbs_network --name s4dbs_reverse -p 80:80 -v /<PathToMyNginx.conf>/nginx.conf:/etc/nginx/nginx.conf docker.io/nginx:1.29.1-alpine

5.2.2. from v1.8.0 to V2.0.0

With Speedgain for Databases Version 2.0.0 we introduce an own TimescaleDB image. The new TimescaleDB image will automatically migrate from PostgreSQL 13 to PostgreSQL 16 and also upgrades the TimescaleDB plugin to the latest version 2.18.2! The upgrade steps are similar to the upgrade from V1.7.X to V1.8.0.

Note
 The initial start of the s4dbs_postgres service will take way longer because of the DBMS upgrade from PostgreSQL 13 to 16.
Database upgrade information

PostgreSQL 13 will be out of maintenance soon. So its necessary to upgrade it. The new Speedgain for Databases TimescaleDB image will do that automatically! For that the pg_upgrade utility will be used internally. To prevent unnecessary data copies or pg_dump operations, the so called "link mode" of pg_upgrade will be used.

The S4DBS TimescaleDB image will reuse and migrate the new data files on the existing storage volume. To use "link mode" its an requirement to use the same filesystem, so don’t change it! After the successful upgrade to PostgreSQL 16 there will be a new directory inside of the s4dbs_postgres container called "/var/lib/postgresql/data/16" which is the new database directory.

Note
 The old database directory inside the s4dbs_postgres container and its data under "/var/lib/postgresql/data" will remain in its place! Do not delete the old database files and don’t move it! pg_upgrade will create links between the old and new data files to prevent unnecessary file copy or correction operations. Another advantage is that the old database directory can be used to start again PostgreSQL 13 even after the run of pg_upgrade.
Special Switches & Configurations

Under certain conditions your PDB is eventually created with special parameters or configurations. For example these changes could be:

  • custom WAL segement size

  • custom Min WAL size

  • custom Max WAL size

  • etc.

Under these conditions the default S4DBs migration to V2.0.0 will run into errors. To address these issues its possible to override the defaults and use your individual parameters and configurations. This mechanism works via two environment variables that are passed to the migration process or the startup of the PostgreSQL server inside of the container. These two environment variables are:

  • POSTGRES_INITDB_ARGS

  • POSTGRES_OPTIONS

The following is a detailed explanation of each variable and their use cases.

1. POSTGRES_INITDB_ARGS

This variable derives from the official PostgreSQL Container image environment variable (see: https://hub.docker.com/_/postgres#postgres_initdb_args). The variable will be passed to the initial creation of the new data directory under PostgreSQL 16 were the new PDB will be serve and is target of the internal migration.

Note
 This variable must be set before you start the s4dbs_postgres container with the new s4dbs_timescaledb:2.0.0 image the first time! For instance if the WAL segment size was changed in your V1.8.X PDB and you omit POSTGRES_INITDB_ARGS incl. configuration of the corresponding "wal-segsize" parameter, the migration will fail. In that case, you have to manually cleanup your new data directory under the path of environment variable NEW_DATA_DIR! Take care to not delete the old/source data directory (OLD_DATA_DIR). By default NEW_DATA_DIR is set to /var/lib/postgresql/data/16. You need to delete only the folder "16" in that case.

You only have to define the variable with the corresponding values. Please include the double hyphen before each initDB parameter and seperate them by a space. For example:

Docker-Compose

  s4dbs_postgres:
    image: docker.io/itgainis/s4dbs_timescaledb:2.0.0
    ...
    environment:
      ...
      - POSTGRES_INITDB_ARGS="--wal-segsize=256"
    ...

Podman

podman run -d ... -e POSTGRES_INITDB_ARGS="--wal-segsize=256" docker.io/itgainis/s4dbs_timescaledb:2.0.0
2. POSTGRES_OPTIONS

With this variable you can override configuration parameters that are stored by default in the postgresql.conf file. POSTGRES_OPTIONS is an alternative way to override those parameters. The common way to do this is to set a new command within your Container configuration, for example:

Docker-Compose

  s4dbs_postgres:
    image: docker.io/itgainis/s4dbs_timescaledb:2.0.0
    ...
    command: postgres -c max_connections=200 -c max_wal_size=1GB -c max_val_size=5GB -c max_locks_per_transaction=256
    ...

Podman

podman run -d ... docker.io/itgainis/s4dbs_timescaledb:2.0.0 postgres -c max_connections=200 -c max_wal_size=1GB -c max_val_size=5GB -c max_locks_per_transaction=256

The following is the same example but using POSTGRES_OPTIONS. Please note that you can omit the "-c" before each parameter.

Docker-Compose

  s4dbs_postgres:
    image: docker.io/itgainis/s4dbs_timescaledb:2.0.0
    ...
    environment:
      ...
      - POSTGRES_OPTIONS="max_connections=200 max_wal_size=1GB max_val_size=5GB max_locks_per_transaction=256"
    ...

Podman

podman run -d ... -e POSTGRES_OPTIONS="max_connections=200 max_wal_size=1GB max_val_size=5GB max_locks_per_transaction=256" docker.io/itgainis/s4dbs_timescaledb:2.0.0
0. OPTIONAL: Load Images from tar.gz archive

If your system does not have access docker hub or another registry serving the images, you can download them from our download portal, copy them to your machine and load them into local image store with the following podman command.

podman load --input s4dbs_2.0.0_linux_<amd64/arm64>_all_images.tar.gz
1. Stop Existing Containers

First, stop the currently running containers. If you use SystemD Units to control Speedgains containers you have to also use SystemD to stop them.

systemctl stop container-s4dbs_collector.service container-s4dbs_frontend.service container-s4dbs_grafana.service container-s4dbs_postgres.service container-s4dbs_reverse.service container-s4dbs_service.service

If you use podman directly to stop the containers SystemD will automatically restart them causeing problems during the upgrade later.

Only in case you decided to just use podman to control the containers, pleace stop all containers with the following command. But again, only if you don’t use SystemD!

podman stop s4dbs_collector s4dbs_service s4dbs_grafana s4dbs_frontend s4dbs_postgres
2. Remove Existing Containers

Next, remove the stopped containers.

podman rm s4dbs_collector s4dbs_service s4dbs_grafana s4dbs_frontend s4dbs_postgres
3. Start New Containers with Updated Tags

Now, run the containers with the new 2.0.0 tag. Change run command if you changed them during first install. At last restart reverse proxy.

TimescaleDB
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=POSTGRES_PASSWORD --name s4dbs_postgres -v s4dbs_postgres:/var/lib/postgresql/data docker.io/itgainis/s4dbs_timescaledb:2.0.0

IMPORTANT If you had changed the default PostgreSQL data directory by using the PGDATA environment variable, you have to provide this directory to the upgrade process by specifying it with the OLD_DATA_DIR environment variable. Because the upgrade process uses "link mode" the new and old data directory have to be on the same filesystem. Thus adjustment of NEW_DATA_DIR is also required! The following example shows this case where the PGDATA directory was changed.

podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=POSTGRES_PASSWORD --name s4dbs_postgres -e OLD_DATA_DIR=/path/to/your/postgres/data_dir -e NEW_DATA_DIR=/path/to/your/postgres/data_dir/16 -v s4dbs_postgres:/path/to/your/postgres/data_dir docker.io/itgainis/s4dbs_timescaledb:2.0.0

IMPORTANT Now you have to wait! Depending on the amount of monitored objects and your retention settings, the upgrade process can take a very long time.

To monitor the upgrade process, watch the logs of the s4dbs_postgres container:

podman logs -f s4dbs_postgres

As said the DBMS upgrade will take a while! Look in the log for the following messages:

Upgrade TimeScaleDB plugin and then PostgreSQL data!
...
>> Migrate TimeScaleDB plugin in database speedgain
...
Performing Consistency Checks
...
Performing Upgrade
...
Upgrade Complete
...
ALTER EXTENSION
                                              List of installed extensions
    Name     | Version | Schema |                                      Description
-------------+---------+--------+---------------------------------------------------------------------------------------
 timescaledb | 2.18.1  | public | Enables scalable inserts and complex queries for time-series data (Community Edition)
...
Migration done! :-)

PostgreSQL Database directory appears to contain a database; Skipping initialization

<date> <time> UTC [<pid>] LOG:  starting PostgreSQL 16.7 on x86_64-pc-linux-musl, compiled by gcc (Alpine 14.2.0) 14.2.0, 64-bit
<date> <time> UTC [<pid>] LOG:  listening on IPv4 address "0.0.0.0", port 5432
<date> <time> UTC [<pid>] LOG:  listening on IPv6 address "::", port 5432
<date> <time> UTC [<pid>] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
<date> <time> UTC [<pid>] LOG:  database system was shut down at 2025-03-21 06:50:14 UTC
<date> <time> UTC [<pid>] LOG:  database system is ready to accept connections
<date> <time> UTC [<pid>] LOG:  TimescaleDB background worker launcher connected to shared catalogs
...

When you find all these messages the PostgreSQL upgrade was successful and you can proceed with the other containers! If you get drastically different log messages, please export your s4dbs_postgres container log and contact support@itgain.de for further assistance!

podman logs s4dbs_postgres > s4dbs_postgres.log
Note
 There are some expected errors in the log that can be ignored. For example 
...
ERROR:  TimescaleDB background worker connected to template database, exiting
...
FATAL:  terminating background worker "TimescaleDB Background Worker Scheduler" due to administrator command
...
FATAL:  terminating connection due to administrator command
...
WARNING:  failed to launch job <number> "Compression Policy [<pid>]": out of background workers
...
FATAL:  terminating background worker "Compression Policy [<pid>]" due to administrator command

These error messages are triggered by the planned restarts of the PostgreSQL server and that connections are blocked during the upgrade process.

Collector
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=PDB_PASS --name s4dbs_collector -e PDB_USER=postgres -e PDB_HOST=s4dbs_postgres -e PDB_PORT=5432 -e PDB_DB_NAME=speedgain -e SPGLOGLEVEL=INFO docker.io/itgainis/s4dbs_collector:2.0.0
Service
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=PDB_PASS --name s4dbs_service -e PDB_USER=postgres -e PDB_HOST=s4dbs_postgres -e PDB_PORT=5432 -e PDB_DB_NAME=speedgain -e SPGLOGLEVEL=INFO docker.io/itgainis/s4dbs_service:2.0.0
Grafana
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=PDB_PASS --name s4dbs_grafana -e PDB_USER=postgres -e PDB_HOST=s4dbs_postgres -e PDB_PORT=5432 -e PDB_DB_NAME=speedgain -e GF_SERVER_SERVE_FROM_SUB_PATH=true -e "GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/" docker.io/itgainis/s4dbs_grafana:2.0.0
Frontend
podman run -d --network s4dbs_network --name s4dbs_frontend -e FRONTEND_CONFIG_MODE=false docker.io/itgainis/s4dbs_frontend:2.0.0
Reverse Proxy
podman restart s4dbs_reverse
4. OPTIONAL: PostgreSQL Configuration Files
Note
 This step can be ignored if no changes to the PostgreSQL configuration files (postgresql.conf, pg_hba.conf or pg_ident.conf) were done!

The automated database migration uses the PostgreSQL utility pg_upgrade. This utility converts if necessary the binary structure of all data files so the PostgreSQL 13 data files are compatable to PostgreSQL 16. How ever the utility doesn’t handle configuration files! Later PostgreSQL release have for instance new configuration parameters or changed values to adress a different behavior. An automatic merge of the old and new configuration files can result in an unwanted or even faulty behavior of PostgreSQL. Therefore manual intervention is requried!

If you changed something in your postgresql.conf, pg_hba.conf or pg_ident.conf file, you have to move these changes from the old configuration file in the new file! The following table lists the locations of these configuration files. The default location can variy and is configurable via an environment variable. The following table only lists the default paths.

Environement variable Docker & Podman - default path OpenShift, Helm & Kubernetes - default path Configuration file name

NEW_DATA_DIR

/var/lib/postgresql/data/16

/var/lib/postgresql/data/pgdata/16

postgresql.conf
pg_hba.conf
pg_ident.conf

OLD_DATA_DIR

/var/lib/postgresql/data

/var/lib/postgresql/data/pgdata

If you have to move a lot of changes from the old configuration file to the new one, it is maybe easier to extract the config files form the container and do the merging in your favorite merge editor (for example, delta, WinWerge, VS-Code, meld, kdiff3, etc.). You can make a copy of the configuration file by using the podman command:

podman cp s4dbs_postgres:/<new-path-see-above->/<config-file-see-above> <new-local-copy-of-config-file>
podman cp s4dbs_postgres:/<old-path-see-above->/<config-file-see-above> <old-local-copy-of-config-file>

Now do your merges with your favorite merge utility. Is this step completed you have to copy the merged configuration files back into the s4dbs_postgres container. This can be done with the following command:

podman cp <merged-local-copy-of-config-file> s4dbs_postgres:/<new-path-see-above->/<config-file-see-above>

You have to restart the s4dbs_postgres container so the changes are applied.

podman restart s4dbs_postgres

5.2.3. from v1.7.0/v1.7.1 to v1.8.0

0. OPTIONAL: Load Images from tar.gz archive

If your system does not have access docker hub or another registry serving the images, you can download them from our download portal, copy them to your machine and load them into local image store with the following podman command.

podman load --input s4dbs_1.8.2_linux_<amd64/arm64>_all_images.tar.gz
1. Stop Existing Containers

First, stop the currently running containers.

podman stop s4dbs_collector s4dbs_service s4dbs_grafana s4dbs_frontend
2. Remove Existing Containers

Next, remove the stopped containers.

podman rm s4dbs_collector s4dbs_service s4dbs_grafana s4dbs_frontend
3. Start New Containers with Updated Tags

Now, run the containers with the new 1.8.2 tag. Change run command if you changed them during first install. At last restart reverse proxy.

Collector
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=PDB_PASS --name s4dbs_collector -e PDB_USER=postgres -e PDB_HOST=s4dbs_postgres -e PDB_PORT=5432 -e PDB_DB_NAME=speedgain -e SPGLOGLEVEL=INFO docker.io/itgainis/s4dbs_collector:1.8.2
Service
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=PDB_PASS --name s4dbs_service -e PDB_USER=postgres -e PDB_HOST=s4dbs_postgres -e PDB_PORT=5432 -e PDB_DB_NAME=speedgain -e SPGLOGLEVEL=INFO docker.io/itgainis/s4dbs_service:1.8.2
Grafana
podman run -d --network s4dbs_network --secret=s4dbs_pdbpass,type=env,target=PDB_PASS --name s4dbs_grafana -e PDB_USER=postgres -e PDB_HOST=s4dbs_postgres -e PDB_PORT=5432 -e PDB_DB_NAME=speedgain -e GF_SERVER_SERVE_FROM_SUB_PATH=true -e "GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/" docker.io/itgainis/s4dbs_grafana:1.8.2
Frontend
podman run -d --network s4dbs_network --name s4dbs_frontend -e FRONTEND_CONFIG_MODE=false docker.io/itgainis/s4dbs_frontend:1.8.2
Reverse Proxy
podman restart s4dbs_reverse

← Previous: General Migration Terms | ↑ Up: HowTo | ⌂ Home: Speedgain for Databases | Next: Docker Based Migration →