Skip to content

v0.2.1 — Long-running process visibility

Status: active package release on the v0.2 architecture line.

Version source: pyproject.toml (version = "0.2.1").


What changed

Non-architectural ergonomics added to full_llm_pipeline.ipynb. The tokenizer, model family, training objectives, dataset registry, and checkpoint formats stay on the v0.2.0 architecture line.

  • Quiet pre-flight helpers stay visible. Dataset disk-size scans and load_dataset / load_from_disk calls are wrapped in Heartbeat(...), so blocked metadata or local Arrow loads emit keep-alive messages instead of going silent.
  • Large list scans show progress. Text fingerprinting, train_texts character counting, and effective-mix counting use tqdm where a full-corpus run can spend noticeable wall time.
  • Model setup and restore have heartbeat coverage. Model construction, parameter counting, checkpoint model rebuild, and checkpoint weight loading emit keep-alive messages if production configs run long.
  • Speculative decoding reports progress. The demo adds a token progress bar plus heartbeat status for larger max_new_tokens runs.

Verification

  • Notebook JSON validated with jq empty full_llm_pipeline.ipynb.
  • Notebook code cells compiled with uv run python.
  • uv run python -c "import download_data" passed.