Prerequisite: Git

The most common complaint in data science and ML engineering is “it works on my machine.” A model trains fine on your laptop, fails in CI, and behaves differently in production six months later because numpy silently updated from 1.24 to 1.26. Reproducible environments are the solution - they pin every dependency so that the same code runs identically across machines and across time.

The Dependency Problem

Python packages depend on other packages, which depend on yet more packages. When you pip install torch, you also install filelock, typing-extensions, jinja2, and dozens of others - each at whatever version pip chooses. If a teammate installs the same project a week later, they might get subtly different versions of transitive dependencies that weren’t pinned. The result is a build that differs from yours in ways that are nearly impossible to trace.

The problem worsens for scientific computing. numpy, scipy, and PyTorch link against native libraries - BLAS, LAPACK, and CUDA - that live outside Python’s package manager. A pure pip solution cannot capture those.

pip and requirements.txt

The simplest approach is listing your dependencies in a requirements.txt file with pinned versions:

numpy==1.26.4
pandas==2.2.1
scikit-learn==1.4.2

pip freeze > requirements.txt dumps every installed package and its exact version. The problem is that it captures everything in your current environment, including tools you installed interactively, and it doesn’t distinguish between your direct dependencies and the transitive ones pip pulled in automatically. Anyone reading the file has no way of knowing which packages are truly required.

Virtual Environments

Always work inside a virtual environment. venv is built into Python:

python -m venv .venv
source .venv/bin/activate   # Linux/macOS
.venv\Scripts\activate      # Windows

This creates an isolated Python installation. Packages installed inside it don’t affect the system Python and don’t leak between projects. virtualenv is a third-party alternative with more features and faster creation speed.

Virtual environments solve isolation but not reproducibility - you still need to pin versions.

conda: Handling Native Dependencies

conda is a package and environment manager that handles both Python packages and native dependencies. It can install CUDA, BLAS implementations, and C libraries alongside Python packages, all in a coherent, version-pinned environment.

conda create -n myproject python=3.11
conda activate myproject
conda install numpy scipy pytorch -c pytorch
conda env export > environment.yml

The environment.yml file captures everything, including the native libraries. This is essential for ML projects where the CUDA version must match the PyTorch version exactly. The downside is that conda environments are large (often several gigabytes) and conda’s solver is slow.

Dependency Resolution

When you install packages with conflicting requirements, pip historically would just install the latest compatible version and hope for the best. Modern pip uses a backtracking resolver, but it’s still slow for complex dependency graphs.

conda uses a SAT solver, which guarantees a consistent environment if one exists - but takes longer to compute.

uv is a drop-in pip replacement written in Rust. It resolves dependencies in seconds rather than minutes and produces a lockfile automatically. For most new projects, uv is now the recommended choice.

Lock Files: pip-tools

pip-tools separates the two concerns cleanly. You write a requirements.in with only your direct dependencies:

numpy>=1.24
pandas
torch

Then run pip-compile requirements.in to generate a requirements.txt with every transitive dependency pinned to an exact version. This file is committed to version control. Anyone installing from it gets an identical environment.

pip-compile requirements.in        # generates requirements.txt
pip-sync requirements.txt          # installs exactly what's in the lockfile

pyproject.toml

PEP 517 and 518 introduced pyproject.toml as the modern standard for Python packaging. It replaces setup.py and setup.cfg, and most modern tools (Poetry, PDM, Hatch, uv) use it. Direct dependencies go in [project.dependencies]; tool-specific config goes in [tool.*] sections.

[project]
name = "my-ml-project"
requires-python = ">=3.11"
dependencies = [
    "numpy>=1.26",
    "torch>=2.2",
    "scikit-learn>=1.4",
]

[project.optional-dependencies]
dev = ["pytest", "ruff", "mypy"]

Separating dev dependencies from production dependencies keeps Docker images lean - you don’t need pytest and ruff in the container that serves predictions.

Docker for Full Reproducibility

Python-level pinning doesn’t capture the operating system, system libraries, or the Python interpreter itself. Docker does. A Dockerfile pins the base OS image, the Python version, and all packages:

FROM python:3.11-slim AS base

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY src/ ./src/

Multi-stage builds keep the final image small. The build stage installs compilers and dev tools; the runtime stage copies only the compiled artifacts:

FROM python:3.11-slim AS builder
COPY requirements.in .
RUN pip install pip-tools && pip-compile requirements.in
RUN pip install --user -r requirements.txt

FROM python:3.11-slim AS runtime
COPY --from=builder /root/.local /root/.local
COPY src/ ./src/

Python Version Management

Different projects need different Python versions. pyenv lets you install and switch between Python versions without touching the system Python:

pyenv install 3.11.9
pyenv local 3.11.9     # creates a .python-version file

Combined with a virtual environment, you get exact Python version pinning at the project level.

CI Reproducibility

In CI, always install from the lockfile, never from loose constraints. If your lockfile is a requirements.txt:

- name: Install dependencies
  run: pip install -r requirements.txt

Never run pip install -e . without a lockfile in CI - you’ll get whatever versions are current on PyPI at that moment, breaking reproducibility.

Examples

pyproject.toml + uv Workflow

# Create a new project
uv init my-ml-project
cd my-ml-project

# Add dependencies (uv writes them to pyproject.toml and generates uv.lock)
uv add numpy torch scikit-learn
uv add --dev pytest ruff

# Install exactly what's in the lockfile
uv sync

# Run a script in the project environment
uv run python train.py

The uv.lock file is committed to version control. uv sync installs exactly those versions on any machine, in seconds.

Docker Multi-Stage Build for an ML Project

FROM python:3.11-slim AS builder
WORKDIR /build
RUN pip install uv
COPY pyproject.toml uv.lock ./
RUN uv export --frozen > requirements.txt
RUN pip install --user --no-cache-dir -r requirements.txt

FROM python:3.11-slim AS runtime
WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY src/ ./src/
ENV PATH="/root/.local/bin:$PATH"
CMD ["python", "-m", "src.serve"]

pip-compile Usage

# Install pip-tools
pip install pip-tools

# Write direct deps
cat > requirements.in << EOF
torch>=2.2
numpy
pandas
EOF

# Compile to pinned lockfile
pip-compile --output-file requirements.txt requirements.in

# Sync your environment
pip-sync requirements.txt

Reproducing an environment is not about being paranoid - it is about being professional. A model result that cannot be reproduced is a result that cannot be trusted.

Read Next: Docker