Validating Runtime Data

The data model can be imported and used to validate config loaded from YAML files. Two different approaches to runtime validation are demonstrated:

demos/pydantic/load_config.py

"""Demoing how to use Pydantic to get schema-valid config from a YAML file."""

from pathlib import Path
from pprint import pprint
from typing import Any

import yaml
from pydantic import ValidationError

from config_schema import ConfigV1


def get_config(file: Path = Path.cwd() / "config.yaml") -> ConfigV1:
    """Get validated config as an instance of the data model."""
    with open(file) as f:
        raw_config: dict[str, Any] = yaml.safe_load(f)
    return ConfigV1(**raw_config)


def get_config_as_dict(file: Path = Path.cwd() / "config.yaml") -> dict[str, Any]:
    """Get config as a dictionary that has been validated against the data model."""
    with open(file) as f:
        raw_config: dict[str, Any] = yaml.safe_load(f)
    ConfigV1.model_validate(raw_config)
    return raw_config


if __name__ == "__main__":
    try:
        print("\n(1) config as ConfigV1 object:")
        print("~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~")
        config = get_config()
        pprint(config.model_dump(), indent=2)

        print("\n(2) config as dict:")
        print("~~~~~~~~~~~~~~~~~~~")
        config_dict = get_config_as_dict()
        pprint(config_dict, indent=2)

    except ValidationError as e:
        print(e)

Given a config file,

demos/pydantic/config.yaml

# example config valyes that we will use Pydantic to validate

SCHEMA_VERSION: "0.1"

PROJECT_ID: 012345
PROJECT_ENV: prod
PROJECT_URL: http://foo.com/bar.html

USER_CERT:
  secret_resource_name: http://foo.com/secrets/
  filename: README.md

USERNAME:
  env_var: foo-bar-1

The output from load_config.py is,

(1) config as ConfigV1 object:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
{ 'PROJECT_ENV': 'prod',
  'PROJECT_ID': 5349,
  'PROJECT_URL': 'http://foo.com/bar.html',
  'SCHEMA_VERSION': '0.1',
  'USERNAME': {'env_var': 'foo-bar-1'},
  'USER_CERT': { 'filename': PosixPath('README.md'),
                 'secret_resource_name': Url('http://foo.com/secrets/')},
  'USER_TAG': None}

(2) config as dict:
~~~~~~~~~~~~~~~~~~~
{ 'PROJECT_ENV': 'prod',
  'PROJECT_ID': 5349,
  'PROJECT_URL': 'http://foo.com/bar.html',
  'SCHEMA_VERSION': '0.1',
  'USERNAME': {'env_var': 'foo-bar-1'},
  'USER_CERT': { 'filename': 'README.md',
                 'secret_resource_name': 'http://foo.com/secrets/'}}

Note how config not defined in the schema has been allowed to pass through when valudating the config data held in a dictionary. When instantiating the data model it is silently ignored.

If we manually invalidate a couple of the config values then we can also take a look at how errors are formatted:

(1) config as ConfigV1 object:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 validation errors for ConfigV1
PROJECT_ENV
  Input should be 'dev', 'test' or 'prod' [type=literal_error, input_value='run', input_type=str]
    For further information visit https://errors.pydantic.dev/2.6/v/literal_error
USER_CERT.filename
  Path does not point to a file [type=path_not_file, input_value='README.mdz', input_type=str]