Skip to content

A simple FastAPI + NLP based project to compare three pre configured tokenizers and send metrics through an API call

License

Notifications You must be signed in to change notification settings

ManavvGarg/TokenizerCompare

Repository files navigation

TokenizerCompare

To run, Simple install the requirements using requirements.txt then run the following command to run the app.py file (FastAPI server)

uvicorn app:app --host 0.0.0.0 --port 8080 --log-level debug

About

A simple FastAPI + NLP based project to compare three pre configured tokenizers and send metrics through an API call

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published