> ## Documentation Index
> Fetch the complete documentation index at: https://docs-v2.ard.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Getting started

> First-time setup, Docker image building, and local testing for ModMS

## Repository Setup

Clone the ModMS repository:

```bash theme={null}
git clone https://git.wthr.ws/weathertech/modms/
cd modms
```

ModMS uses multiple branches for different components. The repository structure uses **git worktrees** to manage separate branches in parallel directories.

## Git Worktree Setup

ModMS has different branches for each component:

* **`aggregator`** branch: Main branch for the aggregator component
* **`indexer`** branch: Indexer component code
* **`server`** branch: Server and receiver components
* **`mams`** branch: MAMS (Model Archive Management System) - uses same code as server

### Setting Up Worktrees

The repository uses git worktrees to allow working with multiple branches simultaneously. Here's how to set them up:

```bash theme={null}
# Ensure you're on the aggregator branch (or main branch)
git checkout aggregator

# Add worktree for indexer branch
git worktree add indexer-worktree indexer

# Add worktree for server branch
git worktree add server-worktree server

# Add worktree for mams branch (uses server code)
git worktree add mams-worktree mams
```

<Note>
  If worktrees already exist, you can remove them with `git worktree remove <path>` and recreate them.
</Note>

### Worktree Structure

After setup, your directory structure will look like:

```
modms/
├── Docker/              # Aggregator Dockerfile
├── source/             # Aggregator source code
├── static_data/        # Static data files
├── indexer-worktree/   # Indexer branch worktree
│   ├── Docker/
│   │   └── Dockerfile
│   └── source/
└── server-worktree/    # Server branch worktree
    ├── Docker/
    │   ├── Dockerfile
    │   ├── Dockerfile.nginx
    │   └── build.sh
    └── server/
```

<Info>
  Worktrees allow you to work on multiple branches simultaneously without switching between them. This is useful when building Docker images for different components.
</Info>

## Building Docker Images

Each ModMS component has its own Dockerfile and build process.

### Aggregator Image

Build the aggregator Docker image from the root directory:

```bash theme={null}
cd modms
docker build -f Docker/Dockerfile -t modms-aggregator:latest .
```

**Dependencies:**

* wgrib2 (GRIB file utilities)
* eccodes (ECMWF codes library)
* LDC compiler (D compiler)
* System libraries: libssl, curl, cdo, etc.

### Indexer Image

Build the indexer Docker image from the indexer worktree:

```bash theme={null}
cd modms/indexer-worktree
docker build -f Docker/Dockerfile -t modms-indexer:latest .
```

**Dependencies:**

* eccodes (ECMWF codes library)
* LDC compiler (D compiler)
* NetCDF libraries
* GDAL libraries

### Server Image

Build the server Docker image using the build script:

```bash theme={null}
cd modms/server-worktree/Docker
./build.sh v1.0.0
```

This script builds two images:

* `registry.docker.devops.arabiaweather.com/modms:v1.0.0`
* `registry.docker.devops.arabiaweather.com/modms-nginx:v1.0.0`

<Note>
  Replace `v1.0.0` with your desired tag. The build script requires a tag parameter.
</Note>

Alternatively, build manually:

```bash theme={null}
cd modms/server-worktree
docker build -f Docker/Dockerfile -t modms:latest .
docker build -f Docker/Dockerfile.nginx -t modms-nginx:latest .
```

### MAMS Image

MAMS uses the same code as the server. See the [MAMS documentation](/data-ai/modms/mams) for build instructions.

## Local Testing

To test ModMS components locally, you'll need to set up directories and configure environment variables.

### Required Directories

Create the following directory structure on your host machine:

```bash theme={null}
mkdir -p /data/downloads    # Downloaded GRIB files
mkdir -p /data/state        # State files for tracking runs
mkdir -p /data/models       # Processed NetCDF files
mkdir -p /data/manifests    # Manifest files from aggregator
mkdir -p /data/temp         # Temporary files
```

<Note>
  You can use custom paths instead of `/data/*`. Just ensure the paths are consistent across your setup.
</Note>

### Environment Variables

ModMS components use several environment variables. Create a `.env` file or export them:

```bash theme={null}
# Container paths (used inside containers)
export DOWNLOAD_DIR=/data/downloads
export LAST_STATE=/data/state/<model_name>
export TEMP_DIR=/data/temp
export MODELS_DIR=/data/models

# Additional model-specific variables may be required
# Check your model configuration for specific requirements
```

### Testing Aggregator - Check Run Ready

Check if a model run is ready for processing:

```bash theme={null}
docker run --rm \
  -v /data/downloads:/data/downloads \
  -v /data/state:/data/state \
  -v /data/temp:/data/temp \
  -v /path/to/modms/static_data:/static_data \
  -e DOWNLOAD_DIR=/data/downloads \
  -e LAST_STATE=/data/state/<model_name> \
  -e TEMP_DIR=/data/temp \
  -e MODELS_DIR=/data/models \
  --env-file .env \
  modms-aggregator:latest \
  ./modms-aggregator-bin \
  --check-run-ready \
  --model <model_name> \
  --runtime <runtime>
```

**Example:**

```bash theme={null}
docker run --rm \
  -v /data/downloads:/data/downloads \
  -v /data/state:/data/state \
  -v /data/temp:/data/temp \
  -v /path/to/modms/static_data:/static_data \
  -e DOWNLOAD_DIR=/data/downloads \
  -e LAST_STATE=/data/state/uk10 \
  -e TEMP_DIR=/data/temp \
  -e MODELS_DIR=/data/models \
  --env-file .env \
  modms-aggregator:latest \
  ./modms-aggregator-bin \
  --check-run-ready \
  --model uk10 \
  --runtime 20251222T120000
```

**Volumes:**

* `/data/downloads`: Downloaded GRIB files (read-write)
* `/data/state`: State tracking files (read-write)
* `/data/temp`: Temporary processing files (read-write)
* `/static_data`: Static data files (read-only)

### Testing Aggregator - Download and Process

Run the aggregator to download and process a model:

```bash theme={null}
docker run --rm \
  -v /data/downloads:/data/downloads \
  -v /data/manifests:/output \
  -v /data/state:/data/state \
  -v /data/temp:/data/temp \
  -v /path/to/modms/static_data:/static_data \
  -e DOWNLOAD_DIR=/data/downloads \
  -e LAST_STATE=/data/state/<model_name> \
  -e TEMP_DIR=/data/temp \
  -e MODELS_DIR=/data/models \
  --env-file .env \
  modms-aggregator:latest \
  ./modms-aggregator-bin \
  --models <model_name> \
  --runtime <runtime> \
  --output-dir /output
```

**Volumes:**

* `/data/downloads`: Downloaded GRIB files (read-write)
* `/data/manifests`: Output manifest files (read-write, mounted as `/output`)
* `/data/state`: State tracking files (read-write)
* `/data/temp`: Temporary processing files (read-write)
* `/static_data`: Static data files (read-only)

**Output:**
The aggregator creates a manifest file in `/data/manifests` with the format: `<MODEL>_<RUNTIME>.json`

### Testing Indexer

Run the indexer to convert GRIB files to NetCDF:

```bash theme={null}
docker run --rm \
  -v /data/manifests:/manifests \
  -v /data/models:/data/models \
  -v /data/downloads:/data/downloads \
  -v /data/temp:/tmp/indexer \
  -e TEMP_DIR=/tmp/indexer \
  -e MODELS_DIR=/data/models \
  -e DOWNLOAD_DIR=/data/downloads \
  modms-indexer:latest \
  ./modms-indexer-bin \
  --manifest /manifests/<manifest_file>.json
```

**Example:**

```bash theme={null}
docker run --rm \
  -v /data/manifests:/manifests \
  -v /data/models:/data/models \
  -v /data/downloads:/data/downloads \
  -v /data/temp:/tmp/indexer \
  -e TEMP_DIR=/tmp/indexer \
  -e MODELS_DIR=/data/models \
  -e DOWNLOAD_DIR=/data/downloads \
  modms-indexer:latest \
  ./modms-indexer-bin \
  --manifest /manifests/ARDJO3_20251206T060000.json
```

**Volumes:**

* `/data/manifests`: Manifest files (read-only, mounted as `/manifests`)
* `/data/models`: Output NetCDF files (read-write)
* `/data/downloads`: Source GRIB files (read-only)
* `/data/temp`: Temporary indexer files (read-write, mounted as `/tmp/indexer`)

### Cleanup Operations

#### Aggregator Cleanup

Clean up old downloaded files:

```bash theme={null}
docker run --rm \
  -v /data/downloads:/data/downloads \
  -e DOWNLOAD_DIR=/data/downloads \
  modms-aggregator:latest \
  ./modms-aggregator-bin \
  --cleanup
```

#### Indexer Cleanup

Clean up old manifest and temporary files:

```bash theme={null}
docker run --rm \
  -v /data/manifests:/manifests \
  -v /data/models:/data/models \
  -v /data/downloads:/data/downloads \
  -v /data/temp:/tmp/indexer \
  -e TEMP_DIR=/tmp/indexer \
  -e MODELS_DIR=/data/models \
  -e DOWNLOAD_DIR=/data/downloads \
  modms-indexer:latest \
  ./modms-indexer-bin \
  --cleanup
```

## MAMS (Model Archive Management System)

MAMS is an archiver system for ModMS NetCDF data. For detailed MAMS documentation, see the [MAMS documentation](/data-ai/modms/mams).

<Note>
  MAMS uses the same codebase as the ModMS server and provides identical query capabilities. It archives model data to distributed storage servers using a registry system.
</Note>

## Next Steps

* Review the [Architecture](/data-ai/modms/architecture) documentation for system design details
* Check the [FAQ](/data-ai/modms/faq) for common issues and troubleshooting
* Explore the [API Reference](/api-reference) for query capabilities