Data Catalog¶

In a Kedro project, the Data Catalog is a registry of all data sources available for use by the project. The catalog is stored in a YAML file (catalog.yml) that maps the names of node inputs and outputs as keys in the DataCatalog class.

The kedro-datasets documentation package offers built-in datasets for common file types and file systems.

Concepts¶

These pages explain how the Data Catalog works and the model behind each feature:

• Data Catalog — what the catalog is and how it organises data.
• Dataset factories — generalising catalog entries with patterns.
• Partitioned and incremental datasets — working with data split across multiple files.
• Lazy loading — how Kedro defers dataset instantiation until access.

How-to guides¶

These pages give step-by-step procedures for common catalog tasks:

• How to configure the Data Catalog — load and save arguments, credentials, versioning, environments.
• How to use dataset factories — applying factory patterns to your catalog.
• How to use partitioned and incremental datasets — YAML and Python recipes.
• How to access the Data Catalog in code — programmatic catalog usage.

Examples¶

• Data Catalog YAML examples — a wide range of catalog.yml recipes.

Related¶

For an advanced tutorial that explains how to create your own custom dataset, see Tutorial to create a custom dataset.