Welcome to Kedro’s documentation!

Learn about Kedro

• Introduction to Kedro

• First steps
  • Set up Kedro
  • Create a new Kedro project
  • Kedro concepts

Tutorial and basic Kedro usage

• Next steps: Tutorial
  • Set up the spaceflights project
  • Set up the data
  • Create a data processing pipeline
  • Create a data science pipeline
  • Package an entire Kedro project
  • Spaceflights tutorial FAQs
  • Get help
  • Terminology

• Visualisation with Kedro-Viz
  • Visualise the spaceflights project
  • Publish and share Kedro-Viz
  • Preview data in Kedro-Viz
  • Visualise charts in Kedro-Viz

• Experiment tracking in Kedro-Viz
  • Kedro versions supporting experiment tracking
  • When should I use experiment tracking in Kedro?
  • Set up a project
  • Set up the session store
  • Collaborative experiment tracking
  • Set up experiment tracking datasets
  • Modify your nodes and pipelines to log metrics
  • Generate the run data
  • Access run data and compare runs
  • View and compare plots
  • View and compare metrics data

• Kedro for notebook users
  • Add Kedro features to a notebook
  • Use a Jupyter notebook for Kedro project experiments
• FAQs and resources
  • FAQs
  • Kedro glossary

Kedro projects

• Configuration
  • Configuration
  • Credentials
  • Parameters
  • Migration guide for config loaders
  • Advanced configuration

• The Kedro Data Catalog
  • Introduction to the Data Catalog
  • Data Catalog YAML examples
  • Kedro dataset factories
  • Advanced: Access the Data Catalog in code
  • Advanced: Partitioned and incremental datasets
  • Advanced: Tutorial to create a custom dataset

• Nodes and pipelines
  • Nodes
  • Pipelines
  • Modular pipelines
  • The pipeline registry
  • Micro-packaging
  • Run a pipeline
  • Slice a pipeline

Advanced usage

• Project setup
  • Kedro starters
  • Dependencies
  • Lifecycle management with KedroSession
  • Project settings

• Extend Kedro
  • Common use cases
  • Kedro plugins
  • Kedro architecture overview

• Hooks
  • Hooks
  • Common use cases
  • Hooks examples

• Logging

• PySpark integration
  • Centralise Spark configuration in conf/base/spark.yml
  • Initialise a SparkSession using a hook
  • Use Kedro’s built-in Spark datasets to load and save raw data
  • Spark and Delta Lake interaction
  • Use MemoryDataset for intermediary DataFrame
  • Use MemoryDataset with copy_mode="assign" for non-DataFrame Spark objects
  • Tips for maximising concurrency using ThreadRunner

• Development
  • Set up Visual Studio Code
  • Set up PyCharm
  • Kedro’s command line interface
  • Debugging
  • Automated Testing
  • Code formatting and linting

• Deployment

Contribute to Kedro

• Contribute to Kedro

API documentation

kedro	Kedro is a framework that makes it easy to build robust and scalable data pipelines by providing uniform project templates, data abstraction, configuration and pipeline assembly.
kedro_datasets	kedro_datasets is where you can find all of Kedro's data connectors.

Indices and tables

• Index

• Module Index