Speedgain for Databases

5.1. Migrate to newest release

Speedgain for Databases migration is an easy task when a container based installation is used. In general terms: only the image tag numbers have to be changed to current release + some additional tasks have to be done. The following chapters describe which tasks these are when migrating from one version to another.

5.1.1. general

New collectors will not automatically be enabled in pre-existing policies. Have a look for new collectors in every release and activate them if needed - see chapter Change / Add monitoring policies for a how-to and look at Collectors for a list of all available collectors and the version of their availability.

5.1.2. from v1.3.5 to v1.4.0

When migrating from version 1.3.5 to 1.4.0, at first the image tags in the Docker Compose file have to be changed (except for timescale and nginx), new images pulled or loaded, and containers recreated.

Next step is to re-set grafana file and folder ownerships (was root (0) and is now grafana(472)).

An optional step is to add healthchecks to the containers and update the dependencies if you are using healthchecks. This optimizes the startup routines.

You can always find the latest version of a complete Docker Compose example at Docker Compose file.

1. stop running containers

docker compose stop

2. change docker-compose.yml to meet current versions

change the image tags in the docker-compose.yml from

    ...
    image: itgainis/s4dbs_collector:1.3.5
    ...
    image: itgainis/s4dbs_service:1.3.5
    ...
    image: itgainis/s4dbs_grafana:1.3.5
    ...
    image: itgainis/s4dbs_frontend:1.3.5
    ...

    ...
    image: itgainis/s4dbs_collector:1.4.0
    ...
    image: itgainis/s4dbs_service:1.4.0
    ...
    image: itgainis/s4dbs_grafana:1.4.0
    ...
    image: itgainis/s4dbs_frontend:1.4.0
    ...

Note	Timescale and nginx image version tags are not changing.

3. pull or load the images

docker compose pull

docker load --input s4dbs_v<version>_all_images.tar.gz

pulling and loading container images is described in detail in Install Step-by-Step chapter.

4. reset grafana ownerships

To reset grafana ownerships you can use one of these 3 options:

Option a) run a sidecar container to reset ownerships and restart grafana container

docker run --volumes-from $(docker ps -a --format "{{.ID}} {{.Names}}" | grep 's4dbs_grafana_1\|s4dbs_grafana-1'| cut -d " " -f1) -i -u 0 -t --rm nginx:1.23 chown -R 472:root /var/lib/grafana;

Option b) copy grafana.db from container, remove container, recreate container, copy grafana db in container, set ownership of database, run container

docker compose cp s4dbs_grafana:/var/lib/grafana/grafana.db grafana.db.backup
docker rm -v $(docker ps -a --format "{{.ID}} {{.Names}}" | grep 's4dbs_grafana_1\|s4dbs_grafana-1'| cut -d " " -f1)
docker volume prune
docker compose create s4dbs_grafana
docker compose cp grafana.db.backup s4dbs_grafana:/var/lib/grafana/grafana.db
docker run --volumes-from $(docker ps -a --format "{{.ID}} {{.Names}}" | grep 's4dbs_grafana_1\|s4dbs_grafana-1'| cut -d " " -f1) -i -u 0 -t --rm nginx:1.23 chown -R 472:root /var/lib/grafana/grafana.db

Option c) delete the grafana container and all of its volumes (will lose any user driven data like grafana user and passwords, additional dashboards, new datasources etc.) and recreate the container

IMPORTANT:

This removes all user driven data in Grafana and should only be used if you do not have any of that!
Be aware, this removes all unused local volumes. Unused local volumes are those which are not referenced by any containers!
If you are unsure about one of the points above, please don’t use this option!

docker rm -v $(docker ps -a --format "{{.ID}} {{.Names}}" | grep 's4dbs_grafana_1\|s4dbs_grafana-1'| cut -d " " -f1)
docker volume prune --all

5. add healtchecks and update dependencies

Every service in the compose file gets its own healthcheck command and an update in the dependencies section to enhance the startup routine. You can adjust the settings based on your environment. See latest version of a complete Docker Compose example at Docker Compose file

s4dbs_postgres:

    healthcheck:
      test: ["CMD-SHELL", "su postgres -c \"psql -h localhost -c '\\q'\""]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 30s

s4dbs_collector:

    healthcheck:
      test: ["CMD-SHELL", "ps -efa | grep \"java\" | grep \"Speedgain_for_Databases.jar\""]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 30s
    depends_on:
      s4dbs_postgres:
        condition: service_healthy

s4dbs_service:

    healthcheck:
      test: ["CMD-SHELL", "curl -f localhost:8080/Speedgain_for_Databases_Service"]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 60s
    depends_on:
      s4dbs_postgres:
        condition: service_healthy

s4dbs_grafana:

    healthcheck:
      test: ["CMD-SHELL", "ps -efa  |  grep \"grafana-server\" | grep \"packaging=docker\" "]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 10s
    depends_on:
      s4dbs_postgres:
        condition: service_healthy

s4dbs_frontend:

    healthcheck:
      test: ["CMD-SHELL", "curl -f localhost:8080/s4dbs-ui/"]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 60s
    depends_on:
      s4dbs_service:
        condition: service_healthy
      s4dbs_grafana:
        condition: service_healthy

s4dbs_reverse:

    healthcheck:
      test: ["CMD-SHELL", "curl -f localhost:80"]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 10s
    depends_on:
      s4dbs_grafana:
        condition: service_healthy
      s4dbs_frontend:
        condition: service_healthy
      s4dbs_service:
        condition: service_healthy

6. Add new frontend environment variable

The frontend got a new environment variable called FRONTEND_CONFIG_MODE to control its behavior in Kubernetes and OpenShift clusters. This variable must be added to your compose file under the s4dbs_frontend service entry. Existing variables like "SERVICE_PORT" etc, stay in place as they are.

  s4dbs_frontend:
  ...
    environment:
      - FRONTEND_CONFIG_MODE=true
      ...

Setting the FRONTEND_CONFIG_MODE to the value "false" is only suggested for Kubernetes and OpenShift environments.

7. create/re-create containers

docker compose up -d

re-creating containers and further steps are described in detail in Install Step-by-Step chapter.

5.1.3. from v1.3.0 to v1.3.5

When migrating from version 1.3.0 to 1.3.5 just the image tags in the Docker Compose file have to be changed (except for timescale and nginx), new images pulled or loaded and containers recreated. You can always find the latest version of a complete Docker Compose example at Docker Compose file

1. change docker-compose.yml to meet current versions

change the image tags in the docker-compose.yml from

    ...
    image: itgainis/s4dbs_collector:1.3.0
    ...
    image: itgainis/s4dbs_service:1.3.0
    ...
    image: itgainis/s4dbs_grafana:1.3.0
    ...
    image: itgainis/s4dbs_frontend:1.3.0
    ...

    ...
    image: itgainis/s4dbs_collector:1.3.5
    ...
    image: itgainis/s4dbs_service:1.3.5
    ...
    image: itgainis/s4dbs_grafana:1.3.5
    ...
    image: itgainis/s4dbs_frontend:1.3.5
    ...

Note	Timescale and nginx image version tags are not changing.

2. load/pull images and create/re-create containers

loading, starting, re-creating and further steps are described in detail in Install Step-by-Step chapter.

5.1.4. from v1.2.2 to v1.3.0

When migrating from version 1.2.2 to version 1.3.0 the image tags in the Docker Compose file have to be changed at first. With version 1.3.0 a reverse proxy is introduced for single port access to all web services. This setting is optional! If you want to use it, some settings and files have to be added and settings have to be changed. You can always find the latest version of a complete Docker Compose example at Docker Compose file. After changing the configuration the images have to be pulled/loaded and the containers have to be created/re-created.

1. change image tags

change the image tags in the docker-compose.yml from

    ...
    image: itgainis/s4dbs_collector:1.2.2
    ...
    image: itgainis/s4dbs_service:1.2.2
    ...
    image: itgainis/s4dbs_grafana:1.2.2
    ...
    image: itgainis/s4dbs_frontend:1.2.2
    ...

    ...
    image: itgainis/s4dbs_collector:1.3.0
    ...
    image: itgainis/s4dbs_service:1.3.0
    ...
    image: itgainis/s4dbs_grafana:1.3.0
    ...
    image: itgainis/s4dbs_frontend:1.3.0
    ...

Note	Timescale image version tag is not changing.

2. add service for reverse proxy in Docker Compose

  ...
  s4dbs_reverse:
    image: nginx:1.23
    restart: unless-stopped
    logging:
      driver: "json-file"
      options:
        max-size: 10m
        max-file: "5"
    ports:
      - "80:80"     # Disable this line when SSL should be used
      #- "443:443"   # Enable this line to activate SSL
    depends_on:
      - s4dbs_grafana
      - s4dbs_frontend
      - s4dbs_service
    volumes:
      - ./configuration/nginx.conf:/etc/nginx/nginx.conf           # Enable SSL in this config file
      #- ./ssl/ssl_cert.key:/etc/ssl/ssl_cert.key    # Enable this line to activate SSL
      #- ./ssl/ssl_cert.crt:/etc/ssl/ssl_cert.crt    # Enable this line to activate SSL
  ...

Note	The public port does not have to 80, you can choose any free port available. Just change the line from '- "80:80"' to '- "MyPublicPort:80', same applies for 443 if ssl is used.

3. change frontend settings to match new public port

search in your docker-compose.yml for the lines starting with "- SERVICE_PORT=" and "- GRAFANA_PORT=" and change the ports to the public port of your reverse proxy and add "/grafana" on the GRAFANA_PORT setting. An example if port 80 ist used:

  ...
  s4dbs_frontend:
  ...
    environment:
      - SERVICE_PROTOCOL=http                              #use http or https
      - SERVICE_HOST=<enter container host hostname here>
      - SERVICE_PORT=80                                    #use 80 or 443
      - GRAFANA_PROTOCOL=http                              #use http or https
      - GRAFANA_HOST=<enter container host hostname here>
      - GRAFANA_PORT=80/grafana                            #use 80 or 443
  ...

4. change grafana settings to match hostname and sub path serving

  ...
  s4dbs_grafana:
  ...
    environment:
      - PDB_USER=postgres
      ...
      - GF_SERVER_SERVE_FROM_SUB_PATH=true
      - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/
      - GF_SERVER_DOMAIN=<enter container host hostname here>
      ...
    depends_on:
      - s4dbs_postgres

5. remove all port mappings except the reverse proxy mapping

As every http/https communication between your browser and Speedgain for Databases is now served via a nginx proxy (s4dbs_reverse), all other ports can removed or marked as a comment. Example:

  ...
  s4dbs_service:
    ...
    #ports:
    #  - "8088:8080"
    ...
  s4dbs_grafana:
    ...
    #ports:
    #  - "3000:3000"
    ...
  s4dbs_frontend:
    ...
    #ports:
    #  - "8080:8080"
  ...

6. copy nginx.conf to configuration directory

create a subfolder called "configuration" besides your docker-compose.yml and place the nginx.conf file (NginX Configuration file) inside of it.

7. load/pull and create/re-create containers

loading, starting, re-creating and further steps are described in detail in Install Step-by-Step chapter.

5.1.5. from <v1.2.2 to v1.2.2

When migrating from a version lower than 1.2.2 just the image tags in the Docker Compose file have to be changed, new images pulled or loaded and containers recreated. You can always find the latest version of a complete Docker Compose example at Docker Compose file

1. change docker-compose.yml to meet current versions

itgainis/s4dbs_service:1.2.0 -> itgainis/s4dbs_service:1.2.2
itgainis/s4dbs_grafana:1.2.0 -> itgainis/s4dbs_grafana:1.2.2

2. pull or load the images

loading, starting, re-creating and further steps are described in detail in Install Step-by-Step chapter.