The SDF engine provides a holistic view of your SQL ecosystem by considering all queries in a warehouse simultaneously. Uncover problems proactively and optimize your data infrastructure with compute, not more.
Code Checks are rules for your SQL. Code checks are powerful and super easy to use on your laptop or in CI/CD. Check out our first blog post for Code Check examples.
SDF Workspaces organize and manage SQL project dependencies. A workspace is a directory defined by a workspace.sdf.yml. The file must start with a Workspace block, and may contain YML blocks for tables, functions, classifiers and more.
Use workspace blocks to define:
Tables can be defined in either SQL or in YML. But only SDF Table blocks specify metadata for SQL tables.
Use Table blocks to:
Classifiers are a powerful extended typing system for data. In SDF they are used to provide more meaning to data, i.e. they can provide units of measurement (e.g. USD, meter), represent the state of data representation (e.g. PII is in clear text or anonymized), or provide the purpose of a table (e.g. this table is public).
Attach a classifier to a table or column; SDF then tracks the flow of a classifier within the whole workspace.
Use Classifier blocks to:
Functions are either builtin or user defined. They are dialect specific. SDF Function blocks specify re-classifications for builtin- and user defined functions.
Use Function blocks to:
SDF profiles are used to specify different configurations for your project. Technically a profile blocks simply overrides anything that appears in a workspace, sources, classifiers, checks. Think of it like a built-in A/B.
Use Profile blocks to:
SDF is a true multi-dialect compiler, supporting: Redshift, Presto, Trino, Big Query, and ANSI SQL.
Label root tables and watch as your classifiers flow through your entire warehouse automatically. Configure rules to tell SDF where to stop propagation or reclassify based on logic you define in YML