shark-ai: SHARK Modeling and Serving Libraries

SHARK Users

If you're looking to use SHARK check out our User Guide. For developers continue to read on.

Sub-projects

shortfin/

The shortfin sub-project is SHARK's high performance inference library and serving engine.

• API documentation for shortfin is available on readthedocs.

sharktank/

The SHARK Tank sub-project contains a collection of model recipes and conversion tools to produce inference-optimized programs.

• See the SHARK Tank Programming Guide for information about core concepts, the development model, dataset management, and more.
• See Direct Quantization with SHARK Tank for information about quantization support.

sharktuner/

The SHARK Tuner sub-project assists with tuning program performance by searching for optimal parameter configurations to use during model compilation. Check out the readme for more details.

Support matrix

Models

Model name	Model recipes	Serving apps	Guide
SDXL	sharktank/sharktank/models/punet/	shortfin/python/shortfin_apps/sd/	shortfin/python/shortfin_apps/sd/README.md
llama	sharktank/sharktank/models/llama/	shortfin/python/shortfin_apps/llm/	docs/shortfin/llm/user/llama_serving.md
Flux	sharktank/sharktank/models/flux/	shortfin/python/shortfin_apps/flux/	shortfin/python/shortfin_apps/flux/README.md

SHARK Developers

If you're looking to develop SHARK, check out our Developer Guide.