Config Resolution¶

This guide explains how weevr transforms raw YAML files into validated, typed configuration models. The resolution pipeline is a multi-stage process that parses, validates, resolves, inherits, and hydrates configuration.

Introduction¶

When you call Context.run() or Context.load(), weevr runs the full config resolution pipeline internally. The stages are designed to fail fast — syntax errors are caught before variable resolution, schema errors before reference loading, and semantic constraints after all values are concrete.

Key concepts¶

Variable interpolation — ${var} placeholders in YAML values are replaced with concrete values from runtime parameters or config defaults. Default values use the ${var:-default} syntax.
Reference resolution — ref: entries in weave and loom configs load child configuration files recursively. Circular references are detected and rejected.
Inheritance cascade — Defaults flow downward: loom defaults → weave defaults → thread config. The most specific value wins. Replacement is whole-value, not deep merge.
Typed extensions — Files use .thread, .weave, and .loom extensions to declare their config type unambiguously.
Macro expansion — foreach blocks in step lists expand into repeated step sequences with variable substitution.

How it works¶

The pipeline runs in 10 stages, grouped into three phases:

Stage 1–4: Parse and validate structure¶

Parse YAML — Read the file and produce a raw dictionary. File-not-found and YAML syntax errors raise ConfigParseError.
Extract version — Read config_version and parse into a (major, minor) tuple. Unsupported versions raise ConfigVersionError.
Detect type — Determine whether the file is a thread, weave, loom, or params file. Typed extensions (.thread, .weave, .loom) are checked first; if the extension is .yaml or .yml, the structure is inspected.
Validate schema — Run the raw dictionary through a Pydantic model for loose structural validation. At this stage, ${var} placeholders are permitted (the schema uses extra="allow"), so unresolved variables do not cause errors.

Stage 5–6: Resolve values¶

Build parameter context — Merge runtime parameters (highest priority) over the param.* namespace over config-declared defaults over fabric context (lowest priority) into a flat context dictionary. The param.* namespace is populated from either the loom/weave's own declared params: block (for top-level loads only; child configs loaded via ref: do not resolve their own params: block at this stage) or from a ThreadEntry's params: (for thread templates) — see Declared params: and the ${param.x} namespace below and Thread Templates. The ${fabric.*} namespace comes from the fabric layer. Dotted keys like env.lakehouse support nested access.
Resolve variables — Recursively walk all values in the config and resolve ${var} references from the parameter context. Whole-value references return the native type; embedded references coerce to string (see Parameter resolution below). Unresolved variables without defaults raise VariableResolutionError.

Stage 7: Load child configs¶

Resolve references — For looms and weaves, load child config files:

Loom → loads referenced weaves
Weave → loads referenced threads

Each child is itself run through the full pipeline (recursion). A visited set tracks loaded paths to detect circular references. Entry-level overrides like dependencies and condition are preserved alongside the loaded config.

Stage 8–9: Inherit and expand¶

Apply inheritance — For multi-level configs, cascade defaults:
- Loom defaults merge into weave defaults
- Merged defaults merge into each thread config
- Thread values always win (most specific)

Naming configuration cascades separately as a whole block rather than per-field. Audit columns (audit_columns), exports, and connections also cascade separately using additive merge — each level extends the set, and same-named entries at a lower level override the definition from the higher level. Exports with enabled: false are removed after merge.

Expand macros — foreach blocks in thread step lists are expanded into repeated sequences. This only applies when loading a thread config directly; for weave and loom configs, child thread steps are already concrete after reference resolution. Expansion happens after variable resolution so that foreach.values can reference parameters.

Stage 10: Hydrate¶

Validate name and hydrate model — The config name field is validated against the filename stem (mismatch raises ConfigError; missing name is injected from the stem). The fully resolved dictionary is then validated through the typed Pydantic model (Thread, Weave, or Loom). Semantic errors at this stage raise ModelValidationError. Incremental-mode constraints (e.g., CDC requires merge write mode) are checked as post-resolution cross-cutting validations.

Module map¶

Module	Responsibility
`config/parser.py`	YAML parsing, version extraction, typed extension detection
`config/resolver.py`	Variable interpolation, reference loading, circular dependency detection
`config/inheritance.py`	Three-level cascade: loom → weave → thread
`config/validation.py`	Pre-resolution schema validation, post-resolution constraint checks
`config/macros.py`	`foreach` block expansion in step lists
`config/fabric.py`	Builds `${fabric.*}` context from Fabric runtime variables
`config/locations.py`	Filesystem-agnostic location abstraction; routes local paths through pathlib and remote URIs through Hadoop FileSystem
`config/warp.py`	Warp schema contract discovery and drift baseline resolution
`config/__init__.py`	`load_config()` — orchestrates the full pipeline

Deferred variable namespaces¶

Some variable namespaces are not resolved during config loading. They are preserved as literal ${...} placeholders and resolved later at execution time:

Namespace	Resolved at	Used by
`${var.*}`	Weave execution (VariableContext)	Thread steps, sources
`${run.timestamp}`	Thread execution	Audit columns, export paths
`${run.id}`	Thread execution	Audit columns, export paths

The run.* namespace provides per-execution context: ${run.timestamp} is an ISO 8601 UTC timestamp and ${run.id} is a UUID4, both generated once per execute_thread() call. They are available in audit column expressions and export path templates.

Design decisions¶

Pre-resolution loose schema — Schema validation runs before variable resolution, using extra="allow" to tolerate ${...} placeholders. This catches structural errors early without requiring all parameters to be available.
Whole-value replacement — Inheritance uses simple replacement, not deep merge. A thread-level write block replaces the entire inherited write block, not individual fields within it. This keeps behavior predictable.
Typed extensions — .thread, .weave, and .loom extensions eliminate ambiguity about config type without inspecting file contents.
Foreach after resolution — Macro expansion happens after variable resolution so that the values list can come from parameters, but before model hydration so that expanded steps are fully validated.

Parameter resolution¶

Whole-value resolution¶

When an entire YAML value is a single ${param} reference (no surrounding text), the resolver returns the native Python type of the referenced value. The result is not cast to string.

match_keys: ${pk_columns}
# pk_columns: ["mandt", "color"] → list[str]

enabled: ${flag}
# flag: true → bool

limit: ${max_rows}
# max_rows: 1000 → int

A whole-value reference that resolves to None returns null rather than the string "None":

optional_param: ${value}
# value: null → None

Embedded resolution¶

When ${param} appears inside a larger string, standard string interpolation applies. The resolved value is coerced to string and substituted in place.

alias: "SAP.${table_name}"
# table_name: "MARA" → "SAP.MARA"

path: "/mnt/${env}/data/${table_name}"
# env: "prod", table_name: "MARA" → "/mnt/prod/data/MARA"

Embedded references that resolve to None produce the string "None". Use whole-value references when you need null pass-through.

Inline defaults¶

The ${param:-default} fallback syntax always returns the default as a string, even in a whole-value position. For example, limit: ${max_rows:-100} returns the string "100" when max_rows is missing — not the integer 100. To use a typed default, supply the parameter value explicitly at runtime rather than relying on the inline :- fallback.

Declared `params:` and the `${param.x}` namespace¶

A loom or weave that declares a params: block exposes each declared name under the ${param.x} namespace, scoped to that file:

# silver.loom
config_version: "1.0"
weaves:
  - ref: bronze_to_silver.weave

params:
  workspace_id:
    name: workspace_id
    type: string
    required: true
  region:
    name: region
    type: string
    default: eastus

connections:
  bronze:
    type: onelake
    workspace: "${param.workspace_id}"   # required → from runtime
    lakehouse: "lh-${param.region}"      # default → "lh-eastus"

Per-param resolution precedence:

Runtime value supplied via Context(params=...) or load_config(runtime_params=...).
The default declared on the ParamSpec.
ConfigSchemaError if the param is required: true.

Declared params in a loom or weave are reachable as ${param.x} only within the same file's scope, and only when the file is loaded at the top level (via load_config(...) or Context(config=...)). Two related limitations follow from this:

Cross-scope inheritance — child weaves and threads do not automatically inherit a parent loom's declared params; a reference to ${param.x} in a child without a local declaration raises a VariableResolutionError.
Child-config own-scope — a weave that declares its own params: and is loaded as a child via a parent loom's ref: does not have that params: block wired into ${param.x} either. The fix that wires declared params: into the ${param.x} namespace applies only to top-level loads, not to child configs resolved during reference-loading.

Cross-scope cascade and child-config own-scope wiring are planned follow-ups; until then, declare the param in each top-level scope that references it, or use ${var.x} / defaults: for cascading values.

${param.x} references inside a thread template instantiated via ThreadEntry.params (the as: + params: form on a weave's threads: list) are unaffected by this rule — they resolve from the entry's params block as before.