CQ-DINO：用类别查询缓解梯度稀释的大词表目标检测框架
GitHub - FireRedTeam/CQ-DINO: Mitigating Gradient Dilution via Category Queries for Vast Vocabulary Object Detection · GitHub
小红书与武汉大学提出 CQ-DINO，一个基于类别查询（category query）的目标检测框架，用于大词表目标检测，旨在缓解梯度稀释问题。项目基于 MMDetection，推荐 8 张 A100 GPU、CUDA 12.1，支持 V3Det 与 COCO 2017 数据集，训练需先下载第一阶段参数与类别嵌入，并使用 BERT Base 模型。
正文
当前显示原文，中文全文译文尚未就绪。
FireRedTeam; CQ-DINO
Repository navigation
NotificationsYou must be signed in to change notification settings; Fork0 (0); Star10 (10)You must be signed in to star a repository
Repository files navigation
README
Xiaohongshu Inc. Wuhan university
We propose CQ-DINO, a category query-based object detection framework for vast vocabulary object detection.
Table of Contents
Installation; Dataset; Usage
Train; Inference; Citation
Installation
The recommended configuration is 8 A100 GPUs, with CUDA version 12.1. The other configurations in MMDetection should also work.
Please follow the guide to install and set up of the mmdetection.
Dataset
The dataset preparation is the same as the one in mmgrounding of MMDetection, which you can refer to.
V3Det
Please download and prepare V3Det Dataset at V3Det Homepage and V3Det Github. After downloading and unzipping, place the dataset or create a symbolic link to it in the data/v3det directory, with the following directory structure:
Then use coco2odvg.py to convert it into the ODVG format required for training:
After the program has run, two new files v3det_2023_v1_train_od.json and v3det_2023_v1_label_map.json will be created in the data/v3det/annotations directory, with the complete structure as follows:
COCO 2017
Please download it from the COCO official website or from opendatalab. After downloading and unzipping, place the dataset or create a symbolic link to the data/coco directory. The directory structure is as follows:
Then use coco2odvg.py to convert it into the ODVG format required for training:
This will generate new files, instances_train2017_od.json and coco2017_label_map.json, in the data/coco/annotations/ directory. The complete dataset structure is as follows:
Usage
Download the first stage parametres from Google drive in the directory stage1; Download the category embeddings from Google drive.
The complete structure is as follows:
Download the BERT Base model from Huggingface and replace the lang_model_name in the config files. For example:
Train
Inference
Download the checkpoint from Google drive.
Citation