Code

Benchmark Mira LLM Extraction#445

Code

Merged

bgyori merged 13 commits intogyorilab:maingyorilab/mira:mainfrom nanglo123:benchmark_llm_equation_extractionnanglo123/mira:benchmark_llm_equation_extractionCopy head branch name to clipboardMar 12, 2025

Commits

Commits on Mar 5, 2025

• Evaluate llm extraction on 5 biomodels set

  nanglo123

  committed
  8de637d

  Copy full SHA for 8de637d

• Add 2024 hackathon epi scenario 1 model A, add error-free model structure if it differs from correct model structure, add markdown at top explaining notebook

  nanglo123

  committed
  7abe9aa

  Copy full SHA for 7abe9aa

• Add December 2024 hackathon epi scenario 1 model B

  nanglo123

  committed
  3c6d6b9

  Copy full SHA for 3c6d6b9

• Add 2024 hackathon epi scenario 1 model c, delineate between minor and structural correctness edits

  nanglo123

  committed
  d549496

  Copy full SHA for d549496

• Add biomodel 962, standardize wording, improve text, and make variable names more informative

  nanglo123

  committed
  c84bb46

  Copy full SHA for c84bb46

• Add saphire model, start ingesting it

  nanglo123

  committed
  49ef6db

  Copy full SHA for 49ef6db

Commits on Mar 6, 2025

• Evaluate saphire model and rename picture

  nanglo123

  committed
  3755a6d

  Copy full SHA for 3755a6d

• Label each type of error in cells for respective models, add summary at end of notebook

  nanglo123

  committed
  c10d14b

  Copy full SHA for c10d14b

• Add table summarizing results

  nanglo123

  committed
  3c7decd

  Copy full SHA for 3c7decd

Commits on Mar 10, 2025

• Reword text, add initial metrics

  nanglo123

  committed
  5e8fdc4

  Copy full SHA for 5e8fdc4

• Add metrics

  nanglo123

  committed
  0e11ced

  Copy full SHA for 0e11ced

Commits on Mar 11, 2025

• Fix bug with number of models, add error distribution per model for a given error

  nanglo123

  committed
  9859362

  Copy full SHA for 9859362

• Add more information to description, make model error types disjoint in end summary

  nanglo123

  committed
  ab866ed

  Copy full SHA for ab866ed