> ## Documentation Index
> Fetch the complete documentation index at: https://docs.datafold.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Monitors as Code

> Manage Datafold monitors via version-controlled YAML for greater scalability, governance, and flexibility in code-based workflows.

> [!TIP]
>
> Use an AI agent with [Datafold MCP](https://docs.datafold.com/api-reference/mcp-server-setup.md) to help generate and manage your monitors YAML configurations.

This is particularly useful if any of the following are true:

* You have (or plan to have) 100s or 1000s of monitors
* Your team is accustomed to managing things in code
* Strict governance and change management are important to you

## Getting started

> [!NOTE]
>
> **INFO**
>
> This section uses GitHub Actions, but the same steps work in any CI system that can run a Python command. Contact us if you need help getting started.

### Generate a Datafold API key

If you've already got a Datafold API key, use it. Otherwise, you can create a new one in the app by visiting **Settings > Account > API keys** and selecting **Create API Key**. The key's user needs the **Edit monitors** permission.

### Create monitors config

In your repository, create a new YAML file where you'll define your monitors config.

For this example, we'll name the file `monitors.yaml` and place it in the root directory, but neither of these choices are hard requirements.

> [!TIP]
>
> For autocomplete and inline documentation, see the [monitors-schema](https://github.com/datafold/monitors-schema) repo. It provides a JSON Schema with setup instructions for VS Code, Cursor, IntelliJ, Neovim, and other editors. Note that Datafold validates this version of the format more strictly than that schema; see [Validation](#validation).

### Add CI workflow

If you're using GitHub Actions, create a new YAML file under `.github/workflows/` using the following template. Be sure to tailor it to your particular setup:

```yaml theme={null}
name: Apply monitors as code config to Datafold

on:
  push:
    branches:
      - main # or master

jobs:
  apply:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: 3.12
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install datafold-sdk
      - name: Update monitors
        run: datafold monitors provision monitors.yaml # use the correct file name/path
        env:
          DATAFOLD_HOST: https://us.datafold.com # the host you log in to
          DATAFOLD_API_KEY: ${{ secrets.DATAFOLD_API_KEY }} # remember to add to secrets
```

To split a large config, keep one file per asset and run `datafold monitors provision` once per file.

## Example

Each entry under `monitors:` is keyed by the monitor's **identifier**: lower-case letters, digits, underscores, and dashes, starting with a letter or digit. Datafold matches monitors by this key, so keep it stable. Renaming a key creates a new monitor.

```yaml theme={null}
monitors:
  replication_test_example:
    name: 'Example of a custom name'
    description: 'Example of a custom description'
    owner: owner@example.com
    type: diff
    enabled: true
    datadiff:
      diff_type: 'inmem'
      dataset_a:
        connection_id: 734
        table: db.schema.table
        materialize: false
      dataset_b:
        connection_id: 736
        table: db.schema.table1
        materialize: true
      primary_key:
        - pk_column
      columns_to_compare:
        - col1
      column_remapping:
        col1: col2
      sampling:
        tolerance: 0.001 # fraction (0-1)
        confidence: 99 # percent
        threshold: 5000
      ignore_string_case: true
    schedule:
      interval:
        every: hour
    tags:
      - finance

  replication_test_example_with_thresholds:
    type: diff
    enabled: true
    datadiff:
      diff_type: 'indb'
      dataset_a:
        connection_id: 734
        table: db.schema.table
        filter: "created_at >= '2026-01-01'"
      dataset_b:
        connection_id: 734
        table: db.schema.table2
        filter: "created_at >= '2026-01-01'"
      primary_key:
        - pk_column
      sampling:
        rate: 0.1
        threshold: 100000
      tolerance:
        float:
          default:
            type: absolute
            value: 50
          column_tolerance:
            A:
              type: relative
              value: 20 # %
            B:
              type: absolute
              value: 30.0
        datetime:
          default: 1ms
    schedule:
      interval:
        every: day
        hour: 8 # 0-23 UTC
    alert:
      different_rows_count: 100
      different_rows_percent: 10
      match_ratio_min: 0.99
```

## Reference

### Monitor fields

| Field | Description |
| - | - |
| `type` | `diff` (the only supported type; may be omitted). |
| `name`, `description` | Display name and description. |
| `owner` | Email of the monitor owner. If no user matches, the owner is left unset. |
| `enabled` | `true` (default) or `false`. |
| `tags` | List of tags. Replaced on every provision, so omitting it clears the existing tags. |
| `schedule` | At most one of `cron: "<five fields, UTC>"`, `interval: {every: hour}`, `interval: {every: day}` (add `hour: <0-23>` to pin the UTC hour; without it the run can happen at any time of day), or `type: none` (run only on demand). Omitting `schedule` is the same as `type: none`. The minimum gap between runs is one hour. |
| `alert.different_rows_count` | Alert when more rows than this differ. |
| `alert.different_rows_percent` | Alert when more than this percentage (0-100) of rows differ. |
| `alert.match_ratio_min` | Alert when the Match Score falls below this value (0-1, e.g. `0.99` = 99%). |

### `datadiff` fields

| Field | Description |
| - | - |
| `diff_type` | `indb` (both datasets in one database) or `inmem` (cross-database). Omit to choose automatically. |
| `dataset_a`, `dataset_b` | `connection_id` plus exactly one of `table` or `query`. Optional `filter` (SQL, tables only) and `materialize`. |
| `primary_key` | List of primary key columns. |
| `columns_to_compare` (alias `include_columns`) / `exclude_columns` | Columns to compare or skip. Use one or the other, not both. |
| `column_remapping` | Map of a column in dataset A to its name in dataset B. |
| `sampling` | One mode: `tolerance` (0-1) with `confidence` (percent), `rate` (0-1), or `max_rows`. `threshold`: tables smaller than this are not sampled. |
| `tolerance.float` | `default` and per-column `column_tolerance`, each `{type: absolute or relative, value}`. A relative value is a percentage. |
| `tolerance.datetime.default` | One of `1s`, `0.1s`, `0.01s`, `1ms`, `0.1ms`, `0.01ms`, `1us`. |
| `column_rules` | Extra equality rules, e.g. `{column: col1, equal_if: "{a} = '1900-01-01' AND {b} IS NULL"}`. |
| `ignore_string_case`, `null_equals_empty_string`, `compare_duplicates`, `sort_array_columns` | Comparison options, `true` or `false`. |
| `empty_tables_match` | Report a 100% match when both datasets are empty (default `false`). |

### Validation

Datafold rejects any key it does not recognize, and names the key in the error. This includes:

* `notifications`: monitors do not send notifications yet. Remove the block and check the monitor's run history instead.
* Monitor types other than `diff` (`metric`, `test`, `schema`) and wildcard blocks.
* `include` and `id`. The map key is the monitor's identity.

To preview changes without applying them, add `--dry-run` to `datafold monitors provision`.

## FAQ

> [accordion] Can I still create/manage monitors in the app if I'm using monitors as code?
>
> Yes, it's not all or nothing. You can still create/manage monitors in the app even if you're defining others in code.

> [accordion] What happens to a monitor in the app if it's removed from the code?
>
> Nothing. `datafold monitors provision` creates and updates monitors, but never deletes them, and the `--dangling-monitors-strategy` flag has no effect. To stop a monitor, set `enabled: false` and provision again. Monitors managed in code can't be deleted in the app; to remove one, contact [support@datafold.com](mailto:support@datafold.com).

> [accordion] Can I use the app to modify monitors managed in code?
>
> No. Monitors created from code show the banner **This monitor is managed in code**, and changes must be made in the YAML. They can still be cloned.

> [accordion] Can I export monitors I've created in the app so I can manage them in code?
>
> Yes. There are two ways to do this:
>
> 1. Exporting all monitors: Navigate to the Monitors list page and click the **View as Code** button
> 2. Exporting a single monitor: Go to the specific monitor and click **Actions** and then select **View as Code**
>
> Monitors without an **Identifier** are left out of the export. Set one in the monitor's settings first. When you provision a file that uses an existing monitor's identifier, that monitor becomes managed in code and keeps its run history.

## Need help?

If you have any questions about how to use monitors as code, please reach out to our team via Slack, in-app chat, or email us at [support@datafold.com](mailto:support@datafold.com).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.