[Solved]: Finetuning model on custom dataset. #51
Unanswered
JaninaMattes
asked this question in
Q&A
Replies: 1 comment
-
Somehow |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi @hpanwar08 I would like to use your pretrained Detectron2 model and fine tune it on my custom datset (same class labels as in the PubLayNet dataset). I use an adjusted from of the ''' tools/train_net_dla.py''' script and also adjusted the DLA_*.yml config file accordingly. If I build the docker file and run it, I keep running into
Error Message
File "finetune_net_dla.py", line 29, in <module> from detectron2.evaluation import ( ImportError: cannot import name 'CityscapesEvaluator' from 'detectron2.evaluation' (/anaconda/lib/python3.8/site-packages/detectron2/detectron2/evaluation/__init__.py)
What I have done so far
I am using Python 3.8.11 on Ubuntu 18.04 Bionic. The site-packages are installed and the command
python -m site
returns me the information:sys.path = [ '/home/appuser/detectron2_repo', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/home/appuser/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/site-packages', ] USER_BASE: '/home/appuser/.local' (exists) USER_SITE: '/home/appuser/.local/lib/python3.8/site-packages' (exists)
And
which python
returns/usr/local/bin/python
.My Docker file
My Docker is not located in the
./docker
, but directly under root directory, which is nameddetectron2_dla
and is structured in accordance to the Docker file from Facebook Research repo:`FROM nvidia/cuda:10.1-cudnn7-devel
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && apt-get install -y
python3-opencv ca-certificates python3-dev git wget
sudo unzip ninja-build &&
rm -rf /var/lib/apt/lists/*
ARG USER_ID=1000
RUN useradd -m --no-log-init --system --uid ${USER_ID} appuser -g sudo
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER appuser
WORKDIR /home/appuser
ENV PATH="/home/appuser/.local/bin:${PATH}"
RUN wget https://bootstrap.pypa.io/get-pip.py &&
python3 get-pip.py --user &&
rm get-pip.py
RUN pip install --user torchvision torchtext==0.9.1 tensorboard cython cmake pyyaml==5.1 gdown
RUN pip install --user torch==1.8.1+cu101 torchvision==0.9.1+cu101 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
RUN pip install --user 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
RUN pip install --user 'git+https://github.com/facebookresearch/fvcore'
RUN mkdir /home/appuser/detectron2_repo
RUN git clone https://github.com/facebookresearch/detectron2.git /home/appuser/detectron2_repo
ENV FORCE_CUDA="1"
ENV TORCH_CUDA_ARCH_LIST="Kepler;Kepler+Tesla;Maxwell;Maxwell+Tegra;Pascal;Volta;Turing"
RUN pip install --user -e /home/appuser/detectron2_repo
ENV FVCORE_CACHE="/tmp"
WORKDIR /home/appuser/detectron2_repo
COPY configs/ /home/appuser/detectron2_repo/configs
COPY finetune_net_dla.py /home/appuser/detectron2_repo
RUN gdown https://drive.google.com/uc?id=XXX
RUN unzip file.zip -d /home/appuser/detectron2_repo/datasets
RUN rm -r file.zip
RUN mkdir /home/appuser/detectron2_repo/pretrained_models
RUN wget https://www.dropbox.com/sh/wgt9skz67usliei/AADGw0h1y7K5vO0akulyXm-qa/model_final.pth -P /home/appuser/detectron2_repo/pretrained_models/
RUN python3 -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
ENTRYPOINT [ "python3", "finetune_net_dla.py" ] `
My adjusted python script is the following:
`
import logging
import os
import datetime
from collections import OrderedDict
import torch
import detectron2.utils.comm as comm
from detectron2.checkpoint import DetectionCheckpointer
from detectron2.config import get_cfg
from detectron2.data import MetadataCatalog
from detectron2.engine import DefaultTrainer, default_argument_parser, default_setup, hooks, launch
from detectron2.evaluation import (
CityscapesEvaluator,
COCOEvaluator,
COCOPanopticEvaluator,
DatasetEvaluators,
LVISEvaluator,
PascalVOCDetectionEvaluator,
SemSegEvaluator,
verify_results,
)
from detectron2.modeling import GeneralizedRCNNWithTTA
from detectron2.data.datasets import register_coco_instances
from detectron2.utils.logger import setup_logger
logger = logging.getLogger("detectron2")
class FineTuner(DefaultTrainer):
def setup(args):
cfg = get_cfg()
cfg.merge_from_file(args.config_file)
cfg.merge_from_list(args.opts)
cfg.freeze()
default_setup(cfg, args)
return cfg
def main(args):
cfg = setup(args)
if name == "main":
args = default_argument_parser().parse_args()
output_dir = os.path.join('./output', datetime.datetime.now().strftime('%Y%m%dT%H%M'))
os.makedirs(output_dir, exist_ok=True)
logger = setup_logger(output=output_dir)
logger.info("Command Line Args:", args)
Do you have any idea what might cause this issue? I have tried fixing it, but couldn't find any working solution. Thank you in advance for any help.
Beta Was this translation helpful? Give feedback.
All reactions