Skip to content

Running Conductor Using Docker

In this article we will explore how you can set up Conductor on your local machine using Docker compose. The docker compose will bring up the following:

  1. Conductor API Server
  2. Conductor UI
  3. Elasticsearch for searching workflows

Prerequisites

  1. Docker: https://docs.docker.com/get-docker/
  2. Recommended host with CPU and RAM to be able to run multiple docker containers (at-least 16GB RAM)

Steps

1. Clone the Conductor Code

$ git clone https://github.com/conductor-oss/conductor.git

2. Build the Docker Compose

$ cd conductor
conductor $ cd docker
docker $ docker-compose build

3. Run Docker Compose

docker $ docker-compose up

Once up and running, you will see the following in your Docker dashboard:

  1. Elasticsearch
  2. Conductor UI
  3. Conductor Server

You can access the UI & Server on your browser to verify that they are running correctly:

Conductor Server URL

{{ server_host }}

swagger

Conductor UI URL

http://localhost:5000/

conductor ui

4. Exiting Compose

Ctrl+c will exit docker compose.

To ensure images are stopped execute: docker-compose down.

Alternative Persistence Engines

By default docker-compose.yaml uses config-local.properties. This configures the memory database, where data is lost when the server terminates. This configuration is useful for testing or demo only.

A selection of docker-compose-*.yaml and config-*.properties files are provided demonstrating the use of alternative persistence engines.

File Containers
docker-compose.yaml
  1. In Memory Conductor Server
  2. Elasticsearch
  3. UI
docker-compose-dynomite.yaml
  1. Conductor Server
  2. Elasticsearch
  3. UI
  4. Dynomite Redis for persistence
docker-compose-postgres.yaml
  1. Conductor Server
  2. Elasticsearch
  3. UI
  4. Postgres persistence
docker-compose-prometheus.yaml Brings up Prometheus server

For example this will start the server instance backed by a PostgreSQL DB. 

docker-compose -f docker-compose.yaml -f docker-compose-postgres.yaml up

Standalone Server Image

To build and run the server image, without using docker-compose, from the docker directory execute: 

docker build -t conductor:server -f server/Dockerfile ../
docker run -p 8080:8080 -d --name conductor_server conductor:server
This builds the image conductor:server and runs it in a container named conductor_server. The API should now be accessible at {{ server_host }}.

To 'login' to the running container, use the command: 

docker exec -it conductor_server /bin/sh

Standalone UI Image

From the docker directory, 

docker build -t conductor:ui -f ui/Dockerfile ../
docker run -p 5000:5000 -d --name conductor_ui conductor:ui
This builds the image conductor:ui and runs it in a container named conductor_ui. The UI should now be accessible at localhost:5000.

Note

  • In order for the UI to do anything useful the Conductor Server must already be running on port 8080, either in a Docker container (see above), or running directly in the local JRE.
  • Additionally, significant parts of the UI will not be functional without Elastisearch being available. Using the docker-compose approach alleviates these considerations.

Monitoring with Prometheus

Start Prometheus with: docker-compose -f docker-compose-prometheus.yaml up -d

Go to http://127.0.0.1:9090.

Combined Server & UI Docker Image

This image at /docker/serverAndUI is provided to illustrate starting both the server & UI within the same container. The UI is hosted using nginx.

Building the combined image

From the docker directory, 

docker build -t conductor:serverAndUI -f serverAndUI/Dockerfile ../

Running the combined image

  • With interal DB: docker run -p 8080:8080 -p 80:5000 -d -t conductor:serverAndUI
  • With external DB: docker run -p 8080:8080 -p 80:5000 -d -t -e "CONFIG_PROP=config.properties" conductor:serverAndUI

Elasticsearch

Elasticsearch is optional, please be aware that disable it will make most of the conductor UI not functional.

How to enable Elasticsearch

  • Set conductor.indexing.enabled=true in your_config.properties
  • Add config related to elasticsearch E.g.: conductor.elasticsearch.url=http://es:9200

How to disable Elasticsearch

  • Set conductor.indexing.enabled=false in your_config.properties
  • Comment out all the config related to elasticsearch E.g.: conductor.elasticsearch.url=http://es:9200

Troubleshooting

To troubleshoot a failed startup, check the server logss located at /app/logs (default directory in dockerfile)

Potential problem when using Docker Images

Not enough memory

You will need at least 16 GB of memory to run everything. You can modify the docker compose to skip using Elasticsearch if you have no option to run this with your memory options.

To disable Elasticsearch using Docker Compose - follow the steps above.

Elasticsearch fails to come up in arm64 based CPU machines

As of writing this article, Conductor relies on 6.8.x version of Elasticsearch. This version doesn't have an arm64 based Docker image. You will need to use Elasticsearch 7.x which requires a bit of customization to get up and running

Elasticsearch remains in Yellow health

When you run Elasticsearch, sometimes the health remains in Yellow state. Conductor server by default requires Green state to run when indexing is enabled. To work around this, you can use the following property: conductor.elasticsearch.clusterHealthColor=yellow

See Github issue

Elasticsearch timeout

Symptom: Standalone (single node) Elasticsearch has a yellow status which will cause timeout for the Conductor server at startup (Required: Green).

Solution: Spin up a cluster (more than one node) to prevent the timeout or use config option conductor.elasticsearch.clusterHealthColor=yellow.

See Github issue

Changes in config-*.properties do not take effect

Config is copied into the image during the docker build. You have to rebuild the image or better, link a volume to it to reflect new changes automatically.

Unable to access to conductor:server API on port 8080

It may takes some time for conductor server to start. Please check server log for errors.