Skip to content

laclouis5/uform-coreml-converters

Repository files navigation

UForm CoreML Converters

CLI for converting UForm models to CoreML.

Quick Start

Download the COCO val 2014 to your disk if you need to evaluate the exported models.

git clone --recurse-submodules https://github.com/laclouis5/uform-coreml-converters.git
poetry env use 3.11
poetry install
poetry shell

Convert the UForm text and image encoders to CoreML:

convert --name uform-vl-multilingual-v2

Convert the UForm text and image encoders to CoreML using 8-bits palettization:

convert --name uform-vl-multilingual-v2 --compression palettization

To evaluate the original and converted models, first cd into the coco-sm directory.

Evaluate the original PyTorch UForm model on the COCO-SM dataset (adapt the val2014 path):

python eval.py --model_name 'uform' --image_dir_path 'val2014' --meta_files_paths 'meta/google.json' --batch_size 16 --report_name 'uform'

You can optionally specify the option --device 'cuda' (or --device 'mps' on macOS) to run the code on GPU.

To evaluate the exported CoreML UForm models:

python eval.py --model_name 'uform_coreml' --image_dir_path 'val2014' --meta_files_paths 'meta/google.json' --batch_size 16 --report_name 'uform_coreml'

You may need to change the models paths hard-coded in coco-sm/modules/uform_coreml.py. Note that evaluating CoreML models only works on macOS.

The UForm generative models are not yet supported (WIP).

Releases

No releases published

Packages

No packages published

Languages