Deformable-DETR模型代码

# Deformable DETR By [Xizhou Zhu](https://scholar.google.com/citations?user=02RXI00AAAAJ), [Weijie Su](https://www.weijiesu.com/), [Lewei Lu](https://www.linkedin.com/in/lewei-lu-94015977/), [Bin Li](http://staff.ustc.edu.cn/~binli/), [Xiaogang Wang](http://www.ee.cuhk.edu.hk/~xgwang/), [Jifeng Dai](https://jifengdai.org/). This repository is an official implementation of the paper [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://arxiv.org/abs/2010.04159). ## Introduction **TL; DR.** Deformable DETR is an efficient and fast-converging end-to-end object detector. It mitigates the high complexity and slow convergence issues of DETR via a novel sampling-based efficient attention mechanism.   **Abstract.** DETR has been recently proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance. However, it suffers from slow convergence and limited feature spatial resolution, due to the limitation of Transformer attention modules in processing image feature maps. To mitigate these issues, we proposed Deformable DETR, whose attention modules only attend to a small set of key sampling points around a reference. Deformable DETR can achieve better performance than DETR (especially on small objects) with 10× less training epochs. Extensive experiments on the COCO benchmark demonstrate the effectiveness of our approach. ## License This project is released under the [Apache 2.0 license](./LICENSE). ## Changelog See [changelog.md](./docs/changelog.md) for detailed logs of major changes. ## Citing Deformable DETR If you find Deformable DETR useful in your research, please consider citing: ```bibtex @article{zhu2020deformable, title={Deformable DETR: Deformable Transformers for End-to-End Object Detection}, author={Zhu, Xizhou and Su, Weijie and Lu, Lewei and Li, Bin and Wang, Xiaogang and Dai, Jifeng}, journal={arXiv preprint arXiv:2010.04159}, year={2020} } ``` ## Main Results | _Method | _Epochs | _AP | _APS | _APM | _APL | _params
(M) | _FLOPs
(G) | _{Total
Train
Time
(GPU
hours)} | _{Train
Speed
(GPU
hours
/epoch)} | _{Infer
Speed
(FPS)} | _{Batch
Infer
Speed
(FPS)} | _URL | | ----------------------------------- | :----: | :--: | :----: | :---: | :------------------------------: | :--------------------:| :----------------------------------------------------------: | :--: | :---: | :---: | ----- | ----- | | _{Faster R-CNN + FPN} | ₁₀₉ | _42.0 | _26.6 | _45.4 | _53.4 | ₄₂ | ₁₈₀ | ₃₈₀ | _3.5 | _25.6 | _28.0 | _- | | _DETR | ₅₀₀ | _42.0 | _20.5 | _45.8 | _61.1 | ₄₁ | ₈₆ | ₂₀₀₀ | _4.0 | _27.0 | _38.3 | _- | | _DETR-DC5 | ₅₀₀ | _43.3 | _22.5 | _47.3 | _61.1 | ₄₁ |₁₈₇|₇₀₀₀|_14.0|_11.4|_12.4| _- | | _DETR-DC5 | ₅₀ | _35.3 | _15.2 | _37.5 | _53.6 | ₄₁ |₁₈₇|₇₀₀|_14.0|_11.4|_12.4| _- | | _DETR-DC5+ | ₅₀ | _36.2 | _16.3 | _39.2 | _53.9 | ₄₁ |₁₈₇|₇₀₀|_14.0|_11.4|_12.4| _- | | **_{Deformable DETR
(single scale)}** | ₅₀ | _39.4 | _20.6 | _43.0 | _55.5 | ₃₄ |₇₈|₁₆₀|_3.2|_27.0|_42.4| _{[config](./configs/r50_deformable_detr_single_scale.sh)
[log](https://drive.google.com/file/d/1n3ZnZ-UAqmTUR4AZoM4qQntIDn6qCZx4/view?usp=sharing)
[model](https://drive.google.com/file/d/1WEjQ9_FgfI5sw5OZZ4ix-OKk-IJ_-SDU/view?usp=sharing)} | | **_{Deformable DETR
(single scale, DC5)}** | ₅₀ | _41.5 | _24.1 | _45.3 | _56.0 | ₃₄ |₁₂₈|₂₁₅|_4.3|_22.1|_29.4| _{[config](./configs/r50_deformable_detr_single_scale_dc5.sh)
[log](https://drive.google.com/file/d/1-UfTp2q4GIkJjsaMRIkQxa5k5vn8_n-B/view?usp=sharing)
[model](https://drive.google.com/file/d/1m_TgMjzH7D44fbA-c_jiBZ-xf-odxGdk/view?usp=sharing)} | | **_{Deformable DETR}** | ₅₀ | _44.5 | _27.1 | _47.6 | _59.6 | ₄₀ |₁₇₃|₃₂₅|_6.5|_15.0|_19.4|_{[config](./configs/r50_deformable_detr.sh)
[log](https://drive.google.com/file/d/18YSLshFjc_erOLfFC-hHu4MX4iyz1Dqr/view?usp=sharing)
[model](https://drive.google.com/file/d/1nDWZWHuRwtwGden77NLM9JoWe-YisJnA/view?usp=sharing)} | | **_{+ iterative bounding box refinement}** | ₅₀ | _46.2 | _28.3 | _49.2 | _61.5 | ₄₁ |₁₇₃|₃₂₅|_6.5|_15.0|_19.4|_{[config](./configs/r50_deformable_detr_plus_iterative_bbox_refinement.sh)
[log](https://drive.google.com/file/d/1DFNloITi1SFBWjYzvVEAI75ndwmGM1Uj/view?usp=sharing)
[model](https://drive.google.com/file/d/1JYKyRYzUH7uo9eVfDaVCiaIGZb5YTCuI/view?usp=sharing)} | | **_{++ two-stage Deformable DETR}** | ₅₀ | _46.9 | _29.6 | _50.1 | _61.6 | ₄₁ |₁₇₃|₃₄₀|_6.8|_14.5|_18.8|_{[config](./configs/r50_deformable_detr_plus_iterative_bbox_refinement_plus_plus_two_stage.sh)
[log](https://drive.google.com/file/d/1ozi0wbv5-Sc5TbWt1jAuXco72vEfEtbY/view?usp=sharing)
[model](https://drive.google.com/file/d/15I03A7hNTpwuLNdfuEmW9_taZMNVssEp/view?usp=sharing)} | *Note:* 1. All models of Deformable DETR are trained with total batch size of 32. 2. Training and inference speed are measured on NVIDIA Tesla V100 GPU. 3. "Deformable DETR (single scale)" means only using res5 feature map (of stride 32) as input feature maps for Deformable Transformer Encoder. 4. "DC5" means removing the stride in C5 stage of ResNet and add a dilation of 2 instead. 5. "DETR-DC5+" indicates DETR-DC5 with some modifications, including using Focal Loss for bounding box classification and increasing number of object queries to 300. 6. "Batch Infer Speed" refer to inference with batch size = 4 to maximize GPU utilization. 7. The original implementation is based on our internal codebase. There are slight differences in the final accuracy and running time due to the plenty details in platform switch. ## Installation ### Requirements * Linux, CUDA>=9.2, GCC>=5.4 * Python>=3.7 We recommend you to use Anaconda to create a conda environment: ```bash conda create -n deformable_detr python=3.7 pip ``` Then, activate the environment: ```bash conda activate deformable_detr ``` * PyTorch>=1.5.1, torchvision>=0.6.1 (following instructions [here](https://pytorch.org/)) For example, if your CUDA version is 9.2, you coul