Skip to content

Release Notes — v1.3

Release date: March 2026

This release adds schedule-aware lookup materialization — the engine now defers internal lookups to the correct execution group boundary instead of materializing everything upfront.


Schedule-Aware Lookup Materialization

When a weave-level lookup reads from a table produced by a thread in the same weave, the engine previously materialized all lookups before any thread ran. This could read stale data if the producing thread had not yet written its output.

The planner now builds a lookup schedule that assigns each lookup to the correct materialization point:

  • External lookups (source not produced by a thread in the weave) are materialized before the first execution group, as before.
  • Internal lookups (source produced by thread A in group N) are deferred and materialized at the group N+1 boundary — after the producing thread completes.
Group 0:  dim_product runs, writes Tables/dim_product
          ↓ (lookup dim_product materializes here)
Group 1:  fact_orders, fact_returns run (consume the lookup)

The lookup schedule is included in the ExecutionPlan and visible via explain() in plan mode.

Lookup-Mediated Dependencies

The planner now infers implicit thread dependencies through lookups. If thread B consumes a lookup whose source is produced by thread A, B is placed in a later execution group than A — even without an explicit dependency declaration or overlapping source/target paths.

Compatibility

No breaking changes. Weaves without lookups or with only external lookups behave identically to v1.2. The upfront materialization path is retained as a backward-compatibility fallback when no lookup schedule is computed.

Component Version
Python 3.11
PySpark 3.5.x
Delta Lake 3.2.x
Microsoft Fabric Runtime 1.3