Lavender-2

Appearance

Rich-machine setup ​

This page is the practical checklist for running the real full_llm_pipeline.ipynb through training-manager or directly on a stronger host.

What this host should have ​

  • Python 3.12
  • uv
  • enough disk for datasets, checkpoints, logs, and .venv
  • enough RAM / VRAM for the intended model configuration
  • a stable network path if datasets or gated sources must be fetched

First-time setup ​

bash
git clone --recurse-submodules git@github.com:colorful-numbers/Lavender-2.git
cd Lavender-2
uv sync

Optional but useful:

bash
uv run python -m ipykernel install --user \
  --name lavender-uv --display-name "Lavender (uv .venv)"

Optional HuggingFace access ​

If gated datasets are required:

bash
echo "hf_xxx" > hf-token.txt
export HF_TOKEN=$(cat hf-token.txt)

Do not commit hf-token.txt.

Direct execution path ​

bash
python3 train.py

That wrapper performs:

bash
uv sync
uv run jupyter nbconvert --to notebook --execute --inplace full_llm_pipeline.ipynb

training-manager path ​

bash
training-manager init --path /absolute/path/to/Lavender-2 --name lavender-full \
  --command "python3 train.py --notebook full_llm_pipeline.ipynb"
training-manager serve

Then open http://127.0.0.1:14514, select lavender-full, and click Start.

What to check first if it fails ​

  • does uv sync complete?
  • does notebook execution begin?
  • do logs appear under .training-manager/logs/train.log if using the manager?
  • is the failure during data access, tokenization, model init, or training?
  • is host memory / GPU memory obviously insufficient for the chosen config?

What to send back after first run ​

Useful first diagnostics:

  • machine CPU / RAM / GPU shape
  • whether uv sync succeeded
  • first meaningful traceback if any
  • how far the notebook got before failing
  • whether the issue is dependency, data, memory, or runtime stability