Reproducible Data Science Reports in Python with Quarto

TLDR? Get the 99 project ideas and start building!

Why reproducible reports #

Data science almost always results in a report of some kind. Even when you work on a deployed model or system, you want to measure any changes you make to it, both by simulation and experimentation, which are also presented in reports. Never make these reports by copying and pasting plots and tables from notebooks or ad hoc runs. Always ensure that everything is reproducible, that is, produced with code.

This is easy to do using literate programming¹, in which text and code are kept in the same file, separated by markup.

Why not use notebooks? There’s nothing wrong with notebooks, but, they are a roundabout way of writing and interacting with your code. Typically an IDE will give you better editing experience. You can easily get the interactive experience using the REPL (see Running Python code). Reproducible reports also allow for a wide range of output formats (HTML pages, PDF papers, slide decks etc) to be produced, also in a reproducible way.

Setting up #

The simplest way to get started using reproducible reports in Python is using Quarto², specifically by writing Python scripts using the percent format³. Quarto is not a Python package, but a separate program, follow the instructions⁴ to install it.

To follow along with this example, run the following in your terminal:

uv add pandas matplotlib
uv add --dev jupyter

A minimal report #

A minimal script has a metadata section at the top, followed by Python code. Code can be separated into chunks by Markdown that is written in comments, prefixed by # %%, like in this example. In a file called report.py, type or copy the following:

# %% [markdown]
# ---
# title: "Customer growth scenarios"
# format:
#   html:
#     embed-resources: true
# ---
#
# Monthly revenue under each scenario.

# %%
from dataclasses import dataclass

import pandas as pd

@dataclass(frozen=True)
class Scenario:
    new: int
    churn: float
    price: float = 20.0

def simulate(s: Scenario, months: int = 12) -> list[float]:
    customers = []
    current = 0.0
    for _ in range(months):
        current = current * (1 - s.churn) + s.new
        customers.append(current)
    return [c * s.price for c in customers]

scenarios = {
    "base": Scenario(100, 0.05),
    "sticky": Scenario(100, 0.02),
    "bold": Scenario(150, 0.08),
}
table = pd.DataFrame(
    {name: simulate(s) for name, s in scenarios.items()}, index=range(1, 13)
)
table.tail(1)

# %% [markdown]
# Revenue over time.

# %%
table.plot()

The top of the file has metadata, like the title:. The embed-resources: true line means that plots will be embedded in the HTML file instead of saved to a separate directory, so that when you want to share the report you just have to share the one HTML file. See the Quarto documentation⁵ for more output options. Text meant for the report is written in Markdown format in comments. To render the script to a report, run the following (we use uv run so that Quarto can find our Python project, where the dependencies are installed):

uv run quarto render report.py

This will execute the code and render the Markdown, take the results and stitch them together, and apply some additional formatting, and produce an HTML file.

Using this format is especially convenient because with very little boilerplate you can get a nicely formatted, reproducible report. You can write your conclusions right next to the tables and plots, and, they too stay in version control. Never again do you have to scramble to reproduce a plot you shared weeks ago. Of course one caveat is that the data might change if you are pulling it from a live database. Make sure to use explicit date and other filters to at least guard against some of the ways data can drift.

Moving code into a module #

You can also keep most of the code in another module that you import.

# analysis.py
from dataclasses import dataclass

import pandas as pd

# Scenario and simulate() moved here from the report, unchanged

def scenario_table(scenarios: dict[str, Scenario], months: int = 12) -> pd.DataFrame:
    revenue = {name: simulate(s, months) for name, s in scenarios.items()}
    return pd.DataFrame(revenue, index=range(1, months + 1))

and in the report, the cells import and call them:

# %%
from analysis import Scenario, scenario_table

scenarios = {
    "base": Scenario(100, 0.05),
    "sticky": Scenario(100, 0.02),
    "bold": Scenario(150, 0.08),
}
table = scenario_table(scenarios)
table.tail(1)

Conclusion #

Once you start using literate programming in this way, adding your text in Markdown comments, you won’t want to go back. Once you have some experience with this workflow, you can further automate and structure it. I like to have separate directories for each part of the analysis, each generating intermediate reports. Once you have a nice conclusion, you can create a new Quarto file for the final report. This report then just pulls in plots and tables that you have generated in intermediate reports. This way it’s easy to both create a report and slides, using the same tables and plots, with always up to date and accounted for numbers. You can also publish the HTML pages using GitHub Pages or GitLab Pages.