zelestra_AWS_hackathon

Lightweight project skeleton and pipelines used during the ML Ascend Challenge (2nd Edition).

This repository contains preprocessing, CatBoost and NN training, residual modelling and prediction utilities under src/. A single Click-based CLI (main.py) orchestrates the pipeline and the various steps.

Quick overview

• Entrypoint / CLI: main.py (console script zelestra when installed).
• Primary folders: data/, models/, outputs/, src/.
• Designed for local experimentation; uses existing scripts in src/ and applies CLI-provided configuration at runtime.

Requirements

• Python >= 3.10
• Key runtime deps are listed in pyproject.toml (pandas, scikit-learn, catboost, torch, click, ...). Install inside a virtualenv.

Install (editable)

# create and activate venv however you prefer (you mentioned using `uv`)
# using pip directly inside your active virtualenv:
pip install -e .
# After this you'll have the `zelestra` console command available in your environment.

CLI quick-start (no install)

# Show help
python .\main.py --help

# Dry-run preprocessing (global options must go BEFORE the subcommand):
python .\main.py --processed-dir .\data\processed --dry-run preprocess --train-path .\data\raw\train.csv --test-path .\data\raw\test.csv

# Dry-run training (CatBoost):
python .\main.py --dry-run train-model --n-splits 4

# Run full pipeline (WARNING: long-running):
python .\main.py all

Notes on order of options

• Click expects group-level/global options before subcommands. Example (valid):
  • python .\main.py --dry-run preprocess ...
  • python .\main.py preprocess --dry-run ... may not work as expected.

Common global options

• --config PATH : path to a JSON config file (overrides defaults).
• --model-dir PATH, --processed-dir PATH, --output-dir PATH : override common directories.
• --device auto|cpu|cuda : preferred device.
• -v / --verbose : increase verbosity (repeat to increase).
• -n / --dry-run : print actions but don't execute long tasks.
• -y / --yes : assume yes for prompts.

Example minimal config (JSON)

{
  "train_path": "data/raw/train.csv",
  "test_path": "data/raw/test.csv",
  "processed_dir": "data/processed",
  "model_dir": "models",
  "output_dir": "outputs",
  "n_splits": 4,
  "seed": 42,
  "device": "auto"
}

Troubleshooting

• If click is not found: pip install 'click>=8.1' in your environment.
• If a model/data path is missing, the CLI will raise a clear error — check paths and use absolute paths if unsure.
• For GPU issues, the CLI accepts --device and will attempt to use available hardware, but library-specific errors may occur (e.g., torch/cuda mismatch).

Developer notes

• The CLI applies configuration to src/* modules at runtime (module-level attributes) to avoid touching many files inside src/.
• To add tests, consider pytest and click.testing.CliRunner.

License See LICENSE in the repository.