> ## Documentation Index
> Fetch the complete documentation index at: https://docs-v2.ard.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Frequently asked questions and troubleshooting for ModMS

<Info>
  Common questions and solutions for working with ModMS. If you don't find your answer here, check the [Architecture](/data-ai/modms/architecture) documentation.
</Info>

## Storage & Infrastructure

<AccordionGroup>
  <Accordion title="Why is GlusterFS used?">
    See the detailed answer: [Why is GlusterFS used?](/data-ai/modms/faq/why-glusterfs)
  </Accordion>

  <Accordion title="The mountpoint `/data` is not working. What should I do?">
    See the troubleshooting guide: [Mountpoint `/data` not working](/data-ai/modms/faq/mountpoint-not-working)
  </Accordion>
</AccordionGroup>

## Adding New Models

<AccordionGroup>
  <Accordion title="How to add a new model?">
    Follow the step-by-step guide: [How to add a new model](/data-ai/modms/faq/adding-new-model)
  </Accordion>
</AccordionGroup>

## Performance & Operations

<AccordionGroup>
  <Accordion title="Why does it take too much time to start the server?">
    Learn about the startup process: [Server startup time](/data-ai/modms/faq/server-startup-time)
  </Accordion>
</AccordionGroup>

## GlusterFS Commands

<AccordionGroup>
  <Accordion title="What are the most useful GlusterFS commands?">
    See the command reference: [GlusterFS Commands](/data-ai/modms/faq/glusterfs-commands)
  </Accordion>

  <Accordion title="Example: Volume Status Output">
    See example output: [Volume Status Example](/data-ai/modms/faq/volume-status-example)
  </Accordion>

  <Accordion title="Example: Volume Info Output">
    See example output: [Volume Info Example](/data-ai/modms/faq/volume-info-example)
  </Accordion>
</AccordionGroup>

## Failures & Downtime

<Info>
  Use this section when ModMS is suspected to be causing Pinpoint 5xx errors, missing data, or slow responses.
</Info>

### Common symptoms

* Pinpoint returns **500** or **502** errors for many locations
* `http://modms.devops.arabiaweather.com/web/` is slow or not loading
* Models appear stale or missing in the ModMS UI
* GlusterFS mount at `/data` is not available on one or more nodes

### Quick health checks

* **ModMS UI**
  * Open `http://modms.devops.arabiaweather.com/web/` and confirm it loads and shows recent models.
* **Models freshness API**
  * Check the latest `last_updated` values:

```bash theme={null}
curl -i "http://modms.devops.arabiaweather.com/q?fields=model%20|%20last_updated&from=models"
```

* **GlusterFS mount**
  * On ModMS servers, verify `/data` is mounted and responsive:

```bash theme={null}
mount | grep /data
df -h /data
```

### Typical root causes

* **GlusterFS issues**
  * Peers not connected, volume problems, or `/data` not mounted
* **Nginx / front-end proxy issues**
  * Nginx not running or misconfigured in front of the ModMS server
* **Indexer / Aggregator problems**
  * Indexer stuck, Aggregator not downloading new runs, or failed jobs

### Step-by-step debugging

1. **Check ModMS UI and freshness**
   * Visit `http://modms.devops.arabiaweather.com/web/`
   * Use the `q` endpoint above to confirm models are updating.
2. **Validate GlusterFS health**
   * On each ModMS node:

```bash theme={null}
gluster peer status
```

* All peers should be in `Peer in Cluster (Connected)` state.
* If not, restart GlusterFS:

```bash theme={null}
systemctl restart glusterd
```

3. **Verify `/data` mount**
   * Ensure `/data` is mounted and accessible on all nodes:

```bash theme={null}
mount | grep /data
df -h /data
```

4. **Check Nginx and ModMS containers/services**
   * Confirm Nginx and ModMS containers are running and healthy (per your deployment stack).
   * Restart only after validating GlusterFS and `/data` mounts.
5. **Look at Aggregator/Indexer**
   * On the Indexer server, use `indexerctl` and the steps in the operations/architecture docs to identify stuck or failed jobs.
   * If required, restart the relevant Aggregator or Indexer components following your standard procedures.