Architecture, distribution and integration
ETL pipeline — sources to marts
A nightly extract-transform-load pipeline in the data-flow idiom: sources, landing, staging, conformance and the marts it feeds.
ddn@1New DDN source
Why use it?
Pipeline reviews fail on invisible handoffs; drawing every read/write makes an unverified hop a visible gap.
When it helps
Batch integration design, data-platform documentation, impact analysis ("who reads this table?").
How to read it
- Activities transform; datasets hold; read/write arrows show which direction data moves on each hop.
- Trace one field from source to mart — every transform it crosses is a place it can change meaning.
What you need
- The sources
- The transforms and their order
- The target marts
Things to check
- Do not draw a dataset-to-dataset arrow with no activity between — the transform is the point.
- A mart nobody declared a consumer for is shelf-ware.
Scope of this diagram
Declared flow in the DFD idiom; job scheduling, data quality and volumes are documented elsewhere.
Work with the converted example
- Open the viewer and select
diagram. Inspect the rendered view and its source. - Open the designer. Keep the selected profile and inspect its legal palette and property controls.
- Change one displayed title or an appropriate data value. Apply the edit and check the resulting diagram.
- Run the verifier and review its diagnostics before moving complete content into review.
- Download your edited DDN source to retain the change.
The example uses the indentation-based ddn-next-1 dialect. The language guide explains reusable definitions, dotted data rows and explicit property blocks.