RAFT and Berkeley Function Calling Leaderboard Updates

[ShishirPatil]

😍 v0.2 release 🚀

Highlights

🎯 Berkeley Function Calling Leaderboard (BFCL): How do models stack up for function calling?

• Now includes latency and cost
• More open-source and closed-source models
• Bug fixes in dataset.

RAFT: Fine-tuning technique to improve LLMs for in-domain RAG!

What's Changed

• Adding APIs of 9 Google Service to API Zoo by @meenakshi-mittal in Adding APIs of 9 Google Service to API Zoo #204
• Github Actions to Maintain API Zoo Index by @ramanv0 in Github Actions to Maintain API Zoo Index #188
• Adding Zoom API to API Zoo by @meenakshi-mittal in Adding Zoom API to API Zoo #221
• API Zoo Index Github Actions Fix by @ramanv0 in API Zoo Index Github Actions Fix #261
• Added Google Forms API by @elva01 in Added Google Forms API #185
• RAFT + readme + small sample dataset by @kaiwen129 in RAFT + readme + small sample dataset #218
• Sample data for RAFT by @ShishirPatil in Sample data for RAFT #264
• Docusign Additions by @dangeo773 in Docusign Additions #194
• [Bug Fix] Fix Executable Exact Match Condition Did not Meet by @Fanjia-Yan in [Bug Fix] Fix Executable Exact Match Condition Did not Meet #251
• [Bug Fix] Fix Error in Parallel Function Possible Answer by @Fanjia-Yan in [Bug Fix] Fix Error in Parallel Function Possible Answer #252
• [Bug Fix] Restrict AST checker on Boolean Variable by @Fanjia-Yan in [Bug Fix] Restrict AST checker on Boolean Variable #256
• Adding 7 Oracle APIs to API Zoo by @meenakshi-mittal in Adding 7 Oracle APIs to API Zoo #205
• Adding Datadog API to API Zoo by @meenakshi-mittal in Adding Datadog API to API Zoo #206
• Added Notion APIs (Block, Page, and Database) to APIZoo by @jennifer818 in Added Notion APIs (Block, Page, and Database) to APIZoo #195
• removed testing code by @kaiwen129 in removed testing code #281
• feat: more type annotations for the functions by @UponTheSky in feat: more type annotations for the functions #283
• [Fix] java, javascript parsers in openfunctions-v2 by @CharlieJCJ in [Fix] java, javascript parsers in openfunctions-v2 #284
• Leaderboard Update April 1 by @HuanzhiMao in Leaderboard Update April 1 #299
• Remove Large File from ./inference by @CharlieJCJ in Remove Large File from ./inference #297
• Typo in raft.py by @danielfleischer in Typo in raft.py #311
• Leaderboard April 3 release by @HuanzhiMao in Leaderboard April 3 release #309
• Support OSS Evaluation for Leaderboard by @HuanzhiMao in Support OSS Evaluation for Leaderboard #318
• Update README.md by @HuanzhiMao in Update README.md #320
• Fix typos by @viniciuslazzari in Fix typos #323
• Correction in BFCL README instruction, fixed path in instructions by @CharlieJCJ in Correction in BFCL README instruction, fixed path in instructions #329

New Contributors

• @elva01 made their first contribution in Added Google Forms API #185
• @kaiwen129 made their first contribution in RAFT + readme + small sample dataset #218
• @jennifer818 made their first contribution in Added Notion APIs (Block, Page, and Database) to APIZoo #195
• @UponTheSky made their first contribution in feat: more type annotations for the functions #283
• @danielfleischer made their first contribution in Typo in raft.py #311

Full Changelog: v0.1...v0.2

---

This discussion was created from the release RAFT and Berkeley Function Calling Leaderboard Updates.