Skip to main content

Quick Start Guide

Demo​

This guide will get you from repository clone to a running SMOCS system with a Gymnasium RL agent as an example.

Prerequisites​

Before starting, ensure you have:

  • Docker: Version 20.10 or later
  • Git: For cloning the repository

Check your versions:

docker --version
docker compose version

Step 1: Clone and Navigate​

git clone https://github.com/JeffersonLab/SMOCS
cd SMOCS/orchestration

Step 2: Configure Environment​

Copy the example environment file and set minimal configuration:

nano .env

The .env file must contain these essential variables:

# InfluxDB Configuration
INFLUXDB_TOKEN=my-super-secret-auth-token
INFLUXDB_ORG=myorg
INFLUXDB_BUCKET=kafka_data

# Agent MySQL Information
MYSQL_HOST=localhost
MYSQL_PORT=3306
MYSQL_USER=root
MYSQL_DATABASE=agentdb
MYSQL_ROOT_PASSWORD=setapassword

# Docker Compose Profiles
COMPOSE_PROFILES=gymnasium, rl1

Note: For this demo, the default values work out of the box. In production, change the INFLUXDB_TOKEN as well as the MYSQL_ROOT_PASSWORD to a secure value.

Step 3: Launch the Demo​

Start the Gymnasium environment with RL control agent:

docker compose up --build

This command launches:

  • Kafka broker (message queue)
  • InfluxDB (time-series database)
  • Gymnasium controller (Pendulum-v1 demonstration environment)
  • RL Control Agent (TD3 algorithm)

Step 4: Verify It's Working​

You should see logs streaming in your terminal. Look for these key indicators:

✓ kafka-broker
✓ influxdb
✓ gymnasium-kafka-controller
✓ rl-control-agent1

Alternatively you can check Docker Desktop and see the containers running themselves!

Step 5: Monitor the Training​

Open your browser and navigate to:

InfluxDB Dashboard: http://localhost:8086

  • Username: admin
  • Password: admin123 (default in .env please change in production!)

You'll see real-time metrics from the RL training process flowing into InfluxDB.

You can visualize these metrics by navigating to the Data Explorer tab to see the kafka data being generated from the RL controller and the environment.

Step 6: View Training Progress​

SMOCS uses SOCT which utilizes Tensorboard to see monitor agent training progress.

The tensorboard training logs are mounted inside of the SMOCS/orchestration/ directory under tb-logs

This is optional to view but they are available utilizing the command:

# In a new terminal and in an environment that has tensorboard installed:
tensorboard --logdir ./tb-logs

What's Happening in the RL Demo?​

The system is running a complete reinforcement learning loop:

  1. Gymnasium publishes environment states to Kafka
  2. RL agent consumes states and generates actions
  3. Actions are published back to Kafka
  4. Gymnasium executes actions and produces new states
  5. SARSA tuples are stored for training
  6. Agent continuously improves its policy

Step 7: Stop the Demo​

When finished, stop the system:

# Press Ctrl+C in the terminal, then:
docker compose down

To clean up all data and start fresh:

# Press Ctrl+C in the terminal, then:
docker compose down -v

Understanding the Demo Configuration​

The demo uses these key configuration sections from config.yaml:

Gymnasium Environment:

gymnasium:
  environment: "Pendulum-v1"
  input_topic: "gymnasium-action"
  output_topics:
    sarsa: "gymnasium-sarsa"
    state: "gymnasium-state"
  blocking_mode: false
  default_action_strategy: "random"

RL Control Agent:

rl_control_agent1:
  environment: "Pendulum-v1"
  soct_agent_type: "KerasTD3-v0"
  enabled_threads: ['ingest', 'training', 'inference']
  kafka_topics:
    input_sarsa: "gymnasium-sarsa"
    input_state: "gymnasium-state"
    output_action: "gymnasium-action"

You now have SMOCS running! Continue through the documentation to learn more about developing and using SMOCS.