> ## Documentation Index
> Fetch the complete documentation index at: https://docs-v2.ard.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Registry and Archive Process

> Registry system and process for archiving data to distributed servers

MAMS uses a `registry.json` file to track where model data is stored across multiple servers and their date ranges.

## Registry Structure

The `registry.json` file is a JSON array with one object per server:

```json theme={null}
[
  {
    "name": "mams.devops.arabiaweather.com",
    "connection": {
      "host": "mams.devops.arabiaweather.com",
      "port": "8082"
    },
    "archive": {
      "GFS": [["2025-09-01T12:00:00", ""]],
      "UK10": [["2025-01-01T00:00:00", ""]]
    },
    "static": ["ELE", "LSM", "GFSA14", "WRFMEA14"]
  },
  {
    "name": "archive.mams.devops.arabiaweather.com.138.201.49.117",
    "connection": {
      "host": "138.201.49.117",
      "port": "8082"
    },
    "archive": {
      "GFS": [["2024-07-01T00:00:00", "2025-08-31T18:00:00"]],
      "UK10": [["2024-07-01T00:00:00", "2024-12-31T12:00:00"]]
    }
  }
]
```

**Fields:**

* **`name`**: Server identifier
* **`connection`**: Connector service host and port (8082)
* **`archive`**: Model names mapped to date ranges `[start_date, end_date]`
* **`static`** (optional): Static data identifiers (main server only)

**Date ranges:**

* Empty end date (`""`) = ongoing/current data
* Date format: ISO format (e.g., `"2024-07-01T00:00:00"`)

**Location:** `/data/archives/registry.json` on main server

## Archive Process

When the main server gets full, archive data to an archive server:

<Steps>
  <Step title="Copy Data">
    Use `rsync` to copy NetCDF files to archive server:

    ```bash theme={null}
    rsync -avz /modms/nc/<model>/ <archive_server>:/data/archives/<model>/
    ```
  </Step>

  <Step title="Verify Data">
    SSH to archive server and verify data was copied correctly.
  </Step>

  <Step title="Delete from Main Server">
    Remove data from main server after verification:

    ```bash theme={null}
    rm -rf /modms/nc/<model>/<date_range>/
    ```
  </Step>

  <Step title="Update Registry">
    Update `registry.json` to reflect new data locations and date ranges. See [Syncing Registry](#syncing-registry) below.
  </Step>
</Steps>

<Warning>
  Always verify data integrity before deleting from main server.
</Warning>

## Running MAMS Archive

MAMS runs via cron every 3 hours:

```bash theme={null}
0 */3 * * * /root/archive_modms.sh
```

The script runs a Docker container that archives specified models from `/modms/nc` to `/data/archives`.

## Syncing Registry

After moving data between servers, update the registry to match actual data distribution.

### Process

<Steps>
  <Step title="Get Date Ranges">
    For each server, determine date ranges for all models:

    * Find first and last `.meta.json` files in each model directory
    * Extract `runTime` from both files to get the date range
  </Step>

  <Step title="Compare Servers">
    Identify:

    * **Fully moved**: All data on archive server (remove from main entry)
    * **Partially moved**: Data split between servers (update ranges on both)
    * **Only on main**: Keep in main entry only
  </Step>

  <Step title="Backup Registry">
    ```bash theme={null}
    cp /data/archives/registry.json /data/archives/registry.json.backup.$(date +%Y%m%d_%H%M%S)
    ```
  </Step>

  <Step title="Update Entries">
    * **Archive server**: Add/update entry with all models and their date ranges
    * **Main server**: Remove fully moved models, update ranges for split models
  </Step>

  <Step title="Verify">
    Check that entries match actual data on each server and date ranges are correct.
  </Step>
</Steps>

### Update Rules

* **Fully moved**: Remove from main server entry
* **Partially moved**:
  * Archive server: Date range of moved data
  * Main server: Date range starting after archive end date (use `""` for ongoing)
* **Archive entry**: Must list all models present with complete date ranges

<Warning>
  Always backup `registry.json` before changes. Incorrect entries cause query routing failures.
</Warning>

## Related Documentation

* [MAMS Overview](/data-ai/modms/mams) - MAMS system overview
* [MAMS Deployment](/data-ai/modms/mams/deployment) - Deployment configuration
* [MAMS Connector](/data-ai/modms/mams/connector) - Connector service details