3.1. Docker Compose Installation

You may want to choose a Docker Compose installation if you want a very fast and easy installation. You just have to provide a Docker host system to install and run the images we provide.

DockerImageInstallation
Figure 3. Docker Image Installation

3.1.1. Prerequisites

Please make sure that you provide the following prerequisites:

  1. a valid licence file is needed (get it at speedgain.com)

  2. any platform that can run Docker Compose files

  3. 4 Cores, 16GB RAM and at least 100GB to store data

  4. free ports 80 or 443 are needed to access the web services (5432 for postgresql/timescale is optional)

  5. a modern web-browser

3.1.2. Services

One service can’t be run on its own. It has to be executed in a compose with 5 other services (all available on Docker-Hub and via ITGAIN download portal: itgain.sharefile.eu ). The list of all 6 images:

  1. Timescale - as the repository to store all performance metrics

  2. Speedgain Collector - the core component to pull metrics

  3. Speedgain Service - rest api service

  4. Speedgain Frontend - web-frontend

  5. Speedgain Grafana - delivering all needed dashboards

  6. NginX - as reverse proxy

3.1.3. Install Step-by-Step

To install and run Speedgain for Databases as a Docker Compose installation follow these steps:

  1. Get at least a trial license via speedgain.com

  2. Create a a new folder with the name you want your Speedgain service to run as (e.g. "speedgain")

  3. Create a subfolder named "licence" in it

  4. Copy the licence file in the licence folder

  5. Create a subfolder named "configuration" in it

  6. Copy nginx.conf into that subfolder (NginX Configuration file)

  7. Create and adjust the Docker Compose file (Docker Compose file) to fit your needs - at least set the Postgres password and the hostname of your docker host)

  8. If you have internet access and by that to Docker-Hub, you can pull the images from hub.docker.com:

    bash
    docker pull docker.io/timescale/timescaledb:2.11.2-pg13
    docker pull docker.io/nginx:1.23
    docker pull docker.io/itgainis/s4dbs_collector:1.6.0
    docker pull docker.io/itgainis/s4dbs_grafana:1.6.0
    docker pull docker.io/itgainis/s4dbs_service:1.6.0
    docker pull docker.io/itgainis/s4dbs_frontend:1.6.0
  9. Optional: If you don’t have direct access to Docker-Hub, download images from our download portal ( https://itgain.sharefile.eu/ ) and transfer to desired system and load via

    bash
    docker load --input s4dbs_v1.6.0_all_images.tar.gz
  10. Start all services via (will create all containers for you)

    bash
    docker compose up -d
  11. Check log files (docker compose logs -f) to see a line similar to the following:

    s4dbs-s4dbs_collector_1  | 28.03.22 18:36:44.0157  [INFO] [Controller::PolicyController_1160199488] PolicyScheduler - Number of Collectors 0 | running Collectors 0
  12. Open a web browser and navigate to http://<DockerHostHostname>:80/s4dbs-ui/ or https://<DockerHostHostname>:443/s4dbs-ui/ if you enabled SSL (Setup SSL encryption)

  13. Login with admin/admin (default user and pw) and configure the databases you like to monitor

3.1.4. Persistent Storage

Speedgain for Databases has two persistent volumes:

  • s4dbs_postgres - the volume for the Timescale container to store performance and configuration data

  • s4dbs_grafana - to store persistent objects from Grafana

3.1.5. Customization Options

  • Logging is set to 10m per file and max files 5 for each container. adjust based on your needs

  • All containers will restart automatically if the fail or after a system restart (if the container service itself will restart)

  • Ports can be adjusted but maybe some config has to be changed as well - ask support for more detail and assistance

  • Port forwarding for postgresql is optional and not needed for the services to run correctly

  • timescaledb-tune accepts environment variables to specify your own limits (e.g. if you do not want to use all exisiting cpu and memory) (https://github.com/timescale/timescaledb-docker/)

    1. TS_TUNE_MEMORY

    2. TS_TUNE_NUM_CPUS

  • Custom Dashboard Provisioning can be done via mounting a volume or a hostpath or files to "/s4dbs_custom_boards" in the grafana container

  • Grafana dashboards are now stored on a volume mounted to the path /grafanaprovisioning in the grafana container, so individual dashboard changes can be persisted

  • Frontend environment variable FRONTEND_CONFIG_MODE controls the behavior of Speedgains web frontend, to automatically generate the URL of the Speedgain Service (s4dbs_service) and the Speedgain Grafana server (s4dbs_grafana). This is helpful in Kubernetes and OpenShift environments, were you don’t know the final DNS name of your container cluster. FRONTEND_CONFIG_MODE can have the following values:

    • ( unset / not specified ) - equals the value of "false"

    • true - The manually configured URLs of the Speedgain Service and Grafana server will be used. This is recommended if you are not in a Kubernetes or OpenShift environment.

    • false - Automatically generates URLs for Speedgain Service and Grafana server. Manually configured ports and hostnames for Service and Grafana server will be ignored. The NginX reverse proxy is mandatory for this configuration!

3.1.6. Setup SSL encryption

The Docker Compose and the bundled NginX configuration file already include a disabled configuration template for setting up SSL quickly. Before you start your Docker Compose service stack with the "docker compose up -d" command, follow the instructions below to activate SSL.

  1. To enable SSL you need first a .crt and a .key file with your organisations SSL certificate data.

  2. in the folder of you Speedgain for Databases services (e.g. "speedgain"), create a new folder named "ssl"

    bash
    mkdir ssl
  3. copy your .crt and .key file in this new folder

    bash
    cp company_ssl.key ~/speedgain/ssl/ssl_cert.key
    cp company_ssl.crt ~/speedgain/ssl/ssl_cert.crt

    You can of course choose a different target filename but remember to change the Docker compose and the NginX config file accordingly.

  4. open the nginx.conf file in the configuration subfolder and look for the following lines and enable these 4 lines

    ...
      server {
    ...
    #SSL Config
    # Enable the following 4 lines if SSL should be activated:
    #    listen 443 ssl http2;
    #    listen [::]:443 ssl http2;
    #    ssl_certificate_key /etc/ssl/ssl_cert.key;
    #    ssl_certificate     /etc/ssl/ssl_cert.crt;
  5. open the docker-compose.yml file and look for the following lines

      ...
      s4dbs_frontend:
      ...
        environment:
          - FRONTEND_CONFIG_MODE=true                          #set to false to generate ...
          - SERVICE_PROTOCOL=http                              #use http or https
          - SERVICE_HOST=<enter container host hostname here>
          - SERVICE_PORT=80                                    #use 80 or 443
          - GRAFANA_PROTOCOL=http                              #use http or https
          - GRAFANA_HOST=<enter container host hostname here>
          - GRAFANA_PORT=80/grafana                            #use 80 or 443
     ...

    You have to change the port from "80" to "443" and the protocol name from "http" to "https". The result should look like this:

          - FRONTEND_CONFIG_MODE=true                           #set to false to generate ...
          - SERVICE_PROTOCOL=https                              #use http or https
          - SERVICE_HOST=<enter container host hostname here>
          - SERVICE_PORT=443                                    #use 80 or 443
          - GRAFANA_PROTOCOL=https                              #use http or https
          - GRAFANA_HOST=<enter container host hostname here>
          - GRAFANA_PORT=443/grafana                            #use 80 or 443
  6. Last step is to configure the NginX service. Look again in the docker-compose.yml to find the following lines:

      ...
      s4dbs_reverse:
      ...
        ports:
          - "80:80"      # Disable this line when SSL should be used
          #- "443:443"   # Enable this line to activate SSL
        ...
        volumes:
          - ./configuration/nginx.conf:/etc/nginx/nginx.conf           # Enable SSL in this config file
          #- ./ssl/ssl_cert.key:/etc/ssl/ssl_cert.key    # Enable this line to activate SSL
          #- ./ssl/ssl_cert.crt:/etc/ssl/ssl_cert.crt    # Enable this line to activate SSL

    Disable the forwarding for port 80 and enable the forwarding for port 443. The SSL certificate must be mounted inside of the NginX container to activate SSL encryption. For that enable the two mount config lines. The following shows the corret configuration.

      ...
      s4dbs_reverse:
      ...
        ports:
          #- "80:80"      # Disable this line when SSL should be used
          - "443:443"   # Enable this line to activate SSL
        ...
        volumes:
          - ./configuration/nginx.conf:/etc/nginx/nginx.conf           # Enable SSL in this config file
          - ./ssl/ssl_cert.key:/etc/ssl/ssl_cert.key    # Enable this line to activate SSL
          - ./ssl/ssl_cert.crt:/etc/ssl/ssl_cert.crt    # Enable this line to activate SSL

    Remember, if you use a different name than ssl_cert.key and ssl_cert.crt, you have to change your configuration accordingly!

  7. SSL is now configured and you can start the services by executing the command "docker compose up -d".

3.1.7. Docker Compose file

Use following Docker-Compose (docker-compose.yml) to deploy Speedgain for Databases ( external download-link https://speedgain.info/docker-compose.yml ):

version: "3.9"

services:
  s4dbs_postgres:
    image: docker.io/timescale/timescaledb:2.11.2-pg13
    restart: always
    logging:
      driver: "json-file"
      options:
        max-size: 10m
        max-file: "5"
    volumes:
      - s4dbs_postgres:/var/lib/postgresql/data
    #ports:
    #  - "5432:5432"
    environment:
      - POSTGRES_PASSWORD=<PasswordOfPostgresUser>
    privileged: true
    healthcheck:
      test: ["CMD-SHELL", "su postgres -c \"psql -h localhost -c '\\q'\""]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 30s
  s4dbs_collector:
    image: docker.io/itgainis/s4dbs_collector:1.6.0
    restart: always
    logging:
      driver: "json-file"
      options:
        max-size: 10m
        max-file: "5"
    volumes:
      - ./licence/Speedgain_for_Databases.licence:/SPEEDGAIN/licence/Speedgain_for_Databases.licence # optional, can be uploaded via frontend
    #ports:
    #  - "60110:60110"
    environment:
      - PDB_USER=postgres
      - PDB_PASS=<PasswordOfPostgresUser_specified_in_s4dbs_postgres_section>
      - PDB_HOST=s4dbs_postgres
      - PDB_PORT=5432
      - PDB_DB_NAME=speedgain
      - SPGLOGLEVEL=INFO
    privileged: true
    healthcheck:
      test: ["CMD-SHELL", "ps -efa | grep \"java\" | grep \"Speedgain_for_Databases.jar\""]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 30s
    depends_on:
      s4dbs_postgres:
        condition: service_healthy
  s4dbs_service:
    image: docker.io/itgainis/s4dbs_service:1.6.0
    restart: always
    logging:
      driver: "json-file"
      options:
        max-size: 10m
        max-file: "5"
    #ports:
    #  - "60120:60120"
    environment:
      - PDB_USER=postgres
      - PDB_PASS=<PasswordOfPostgresUser_specified_in_s4dbs_postgres_section>
      - PDB_HOST=s4dbs_postgres
      - PDB_PORT=5432
      - PDB_DB_NAME=speedgain
      - PDB_PROPERTIES=
      - SPGDB_SERVICE_HOME=/opt/spgdb_service_home/
    privileged: true
    healthcheck:
      test: ["CMD-SHELL", "curl -f localhost:8080/Speedgain_for_Databases_Service"]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 60s
    depends_on:
      s4dbs_postgres:
        condition: service_healthy
  s4dbs_grafana:
    image: docker.io/itgainis/s4dbs_grafana:1.6.0
    restart: always
    logging:
      driver: "json-file"
      options:
        max-size: 10m
        max-file: "5"
    volumes:
      - s4dbs_grafana:/var/lib/grafana
      - s4dbs_grafana_provision:/grafanaprovisioning
      #- <localPathToCustomBoardsOrVolume>:/s4dbs_custom_boards # valid json files will be provisioned via script into S4DBs - Custom Folder
    environment:
      - PDB_USER=postgres
      - PDB_PASS=<PasswordOfPostgresUser_specified_in_s4dbs_postgres_section>
      - PDB_HOST=s4dbs_postgres
      - PDB_PORT=5432
      - PDB_DB_NAME=speedgain
      #- GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/grafanaprovisioning/dashboards/s4dbs-spg/S4DBs-Overview.json
      - GF_SERVER_SERVE_FROM_SUB_PATH=true
      - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/
      - GF_SECURITY_ALLOW_EMBEDDING=true
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_USERS_DEFAULT_THEME=light
      - GF_PATHS_PLUGINS=/grafanaplugins/
      - GF_PATHS_PROVISIONING=/grafanaprovisioning/
    privileged: true
    healthcheck:
      test: ["CMD-SHELL", "ps -efa  |  grep \"grafana-server\" | grep \"packaging=docker\" "]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 10s
    depends_on:
      s4dbs_postgres:
        condition: service_healthy
  s4dbs_frontend:
    image: docker.io/itgainis/s4dbs_frontend:1.6.0
    restart: always
    logging:
      driver: "json-file"
      options:
        max-size: 10m
        max-file: "5"
    environment:
      - FRONTEND_CONFIG_MODE=false #set to false to generate frontend config based on calling URL (usefull for Kubernetes & OpenShift)
      ## following parameters only needed if FRONTEND_CONFIG_MODE is set to true
      #- SERVICE_PROTOCOL=http #use http or https
      #- SERVICE_HOST=<enter container host hostname here>
      #- SERVICE_PORT=80 #use 80 or 443
      #- GRAFANA_PROTOCOL=http #use http or https
      #- GRAFANA_HOST=<enter container host hostname here>
      #- GRAFANA_PORT=80/grafana #use 80 or 443
    privileged: true
    healthcheck:
      test: ["CMD-SHELL", "curl -f localhost:8080/s4dbs-ui/"]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 60s
    depends_on:
      s4dbs_service:
        condition: service_healthy
      s4dbs_grafana:
        condition: service_healthy
  s4dbs_reverse:
    image: docker.io/nginx:1.23
    restart: unless-stopped
    logging:
      driver: "json-file"
      options:
        max-size: 10m
        max-file: "5"
    ports:
      - "80:80" # Disable this line when SSL should be used
      #- "443:443"   # Enable this line to activate SSL
    healthcheck:
      test: ["CMD-SHELL", "curl -f localhost:80"]
      interval: 10s
      timeout: 10s
      retries: 10
      start_period: 10s
    depends_on:
      s4dbs_grafana:
        condition: service_healthy
      s4dbs_frontend:
        condition: service_healthy
      s4dbs_service:
        condition: service_healthy
    volumes:
      - ./configuration/nginx.conf:/etc/nginx/nginx.conf # Enable SSL in this config file
      #- ./ssl/ssl_cert.key:/etc/ssl/ssl_cert.key    # Enable this line to activate SSL
      #- ./ssl/ssl_cert.crt:/etc/ssl/ssl_cert.crt    # Enable this line to activate SSL
volumes:
  s4dbs_postgres:
  s4dbs_grafana:
  s4dbs_grafana_provision: #s4dbs_grafana_custom_boards: #only needed if custom boards volume is used in grafana section

3.1.8. NginX Configuration file

The following NginX configuration (nginx.conf) is used to setup basic functionality for the reverse proxy service ( external download-link https://speedgain.info/configuration/nginx.conf ):

user  nginx;
worker_processes  5;  ## Default: 1

events {
  worker_connections   1000;
}

http {
  map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
  }

  upstream grafana {
    server s4dbs_grafana:3000;
  }

  upstream service {
    server s4dbs_service:8080;
  }

  server {
    listen             80;
    root /usr/share/nginx/www;
    index index.html index.htm;

#SSL Config
# Enable the following 4 lines if SSL should be activated:
#    listen 443 ssl http2;
#    listen [::]:443 ssl http2;
#    ssl_certificate_key /etc/ssl/ssl_cert.key;
#    ssl_certificate     /etc/ssl/ssl_cert.crt;

# service
    location /Speedgain_for_Databases_Service/ {
      proxy_set_header Host $http_host;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection $connection_upgrade;
      proxy_pass http://service;
    }

# frontend
    location / {
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_pass http://s4dbs_frontend:8080/s4dbs-ui/;
    }
    location /s4dbs-ui/ {
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_pass http://s4dbs_frontend:8080/s4dbs-ui/;
    }

# grafana
    location /grafana/ {
      proxy_set_header Host $http_host;
      proxy_pass http://grafana;
      proxy_read_timeout 300;
      proxy_connect_timeout 300;
      proxy_send_timeout 300;

    }

    location /grafana/api/live/ {
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection $connection_upgrade;
      proxy_set_header Host $http_host;
      proxy_pass http://grafana;
      proxy_read_timeout 300;
      proxy_connect_timeout 300;
      proxy_send_timeout 300;
    }
  }
}