Lavender-2

Appearance

Local development ​

Prerequisites ​

  • Python 3.12 (see .python-version).
  • uv preferred; pip works as a fallback.
  • A CUDA-capable GPU is not strictly required — the dev model config runs on CPU, just slowly. For real pretraining the project targets 8× A100-80GB with FSDP or DeepSpeed ZeRO-3.
  • A HuggingFace account + token for gated datasets (lmsys_chat, arena). Optional otherwise.

First-time setup ​

bash
git clone --recurse-submodules git@github.com:colorful-numbers/Lavender-2.git
cd Lavender-2

# If you cloned without --recurse-submodules:
git submodule update --init --recursive

uv sync                          # authoritative; writes to .venv/
# or: pip install -r requirements.txt

Activate the venv if you're not using uv run:

bash
source .venv/bin/activate

(Optional) HuggingFace token ​

Gated datasets require a token. download_data.py reads HF_TOKEN from the environment or --token hf_xxx on the command line. There's also a file-based convention:

bash
echo "hf_xxx" > hf-token.txt     # .gitignore already covers this path
export HF_TOKEN=$(cat hf-token.txt)

Never commit hf-token.txt or echo its value into notebook output cells. See docs/LLM_CHECK.md §1.2.

Smoke-test the data pipeline ​

bash
python download_data.py --only wikipedia dailydialog --limit 100

Expected outcome: data/wikipedia/ and data/dailydialog/ exist and each loads cleanly:

bash
python -c "
from datasets import load_from_disk
for name in ('wikipedia', 'dailydialog'):
    ds = load_from_disk(f'data/{name}')
    print(name, len(ds), ds.column_names)
"

Run the training notebook ​

bash
jupyter lab full_llm_pipeline.ipynb

The notebook auto-detects a GPU and flips USE_AMP accordingly. On CPU, stick to the dev model config (n_layers=4) or it will not finish in human timescales.

Supervised execution ​

For a host-side supervisor such as training-manager, use the repo's stable wrapper entrypoint instead of calling Jupyter directly:

bash
python3 train.py

The wrapper preserves the repo's uv-only invariant by running uv sync and then executing the chosen notebook through uv run jupyter nbconvert --to notebook --execute --inplace <notebook>. The required nbconvert dependency is part of the project env.

For stronger-host preparation and the recommended manager command for the real pipeline, see rich-machine.md.

Inference ​

After a training run writes to checkpoints/:

bash
python chat.py

Preview the docs site ​

The docs are a VitePress project scoped entirely to docs/ and deployed by Cloudflare Pages. Node 18+ is required for VitePress 1.x (the pinned version lives in docs/.nvmrc).

bash
cd docs
npm install
npm run dev

Live reload at http://localhost:5173/. To check the production build locally:

bash
cd docs
npm run build
npm run preview

See docs/deployment/cloudflare.md for the full hosting pipeline.

Ending a round ​

Before declaring work done, walk through docs/LLM_CHECK.md. It's short and catches the footguns that actually bit earlier rounds.