Implementation of this paper
http://slazebni.cs.illinois.edu/publications/yunchao_cca13.pdf
Using CNN features.
You will need to have the MS COCO database and give path to the db folder in some python files.
Make sure to use python 3
Install pycocotools and other python dependencies
git clone https://github.com/pdollar/coco.git && cd coco/PythonAPI && make install && pip3 install -e .
pip3 install --upgrade gensim nltk numpy torch torchvision tqdm Pillow scikit-image
python3 -m nltk.downloader stopwords
mkdir features
python3 Helper/extractTextFeatures.py --dataset-path /path/to/coco
python3 Helper/extractImageFeatures.py --dataset-path /path/to/coco
python3 main.py --output CCA_0
Try:
python3 I2T.py --name CCA_0.npy
# or
python3 T2I.py --name CCA_0.npy