[Tracking] Sentence Embedding Model

[tqchen]

Overview

This is a global tracking issue to bring generic sentence embedding models to MLCEngine.

Action Items

• Add support for mistral based sentence embedding

Links to Related Issues and PRs

[tqchen]

context #1744

[VoVAllen]

bge-m3 is a good candidate since it also supports sparse embedding model. In the e2e pipeline, we found the process turning text into embedding took most of the time. (text->embedding through openai API costs 100ms+ while vector search part only needs 10ms). It would be nice to have an efficient embedding model at local