Wes Robbins

publications

1. CAST: Conditional Attribute Subsampling Toolkit for Fine-grained Evaluation

   Wes Robbins, Steven Zhou, Aman Bhatta, Chad Mello, Vitor Albiero, Kevin W. Bowyer, and Terrance E. Boult

   (to appear) Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Jan 2023

   Bib PDF

   @article{CAST,
     title = {CAST: Conditional Attribute Subsampling Toolkit for Fine-grained Evaluation},
     author = {Robbins, Wes and Zhou, Steven and Bhatta, Aman and Mello, Chad and Albiero, Vitor and Bowyer, Kevin W. and Boult, Terrance E.},
     year = {2023},
     month = jan,
     journal = {(to appear) Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
   }

2. On the Effect of Atmospheric Turbulence in the Feature Space of Deep Face Recognition

   Wes Robbins and Terrance E. Boult

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops Jun 2022

   Abs Bib PDF

   When captured over long distances, image quality is degraded by inconsistent refractive indexes in the atmosphere. This effect, known as Atmospheric Turbulence (AT), leads to lower performance for vision-based biometric systems such as face recognition. To account for AT, the literature has proposed methods to restore face-images from atmospheric turbulence, but has limited success. There is still a need to understand how atmospheric turbulence breaks recognition performance. We offer a first-look in this direction by providing a study on the effect of atmospheric turbulence in the feature space of deep-learning-based face recognition. We present results on recognition performance and feature space transformation caused by a wide range of AT levels. In deep feature space, we find interesting phenomena such as increasing feature magnitudes, which contradicts the expected result from the literature. From our results, we are able to identify an effect that makes face recognition under atmospheric turbulence uniquely difficult, which we call feature defection. In total, our findings suggest several areas of available improvement which can be used as a guideline for further progress in building models that are robust to AT.

   @inproceedings{Robbins_2022_CVPR,
     author = {Robbins, Wes and Boult, Terrance E.},
     title = {On the Effect of Atmospheric Turbulence in the Feature Space of Deep Face Recognition},
     booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
     month = jun,
     year = {2022},
     pages = {1618-1626},
   }

3. Towards Multimodal Vision-language Models Generating Non-generic Text

   Wes Robbins

   AAAI Undergraduate Consortium Feb 2022

   Abs Bib PDF

   Vision-language models can assess visual context in an image and generate descriptive text. While the generated text may be accurate and syntactically correct, it is often overly general. To address this, recent work has used optical character recognition to supplement visual information with text extracted from an image. In this work, we contend that vision-language models can benefit from additional information that can be extracted from an image, but are not used by current models. We modify previous multimodal frameworks to accept relevant information from any number of auxiliary classifiers. In particular, we focus on person names as an additional set of tokens and create a novel image-caption dataset to facilitate captioning with person names. The dataset, Politicians and Athletes in Captions (PAC), consists of captioned images of well-known people in context. By fine-tuning pretrained models with this dataset, we demonstrate a model that can naturally integrate facial recognition tokens into generated text by training on limited data. For the PAC dataset, we provide a discussion on collection and baseline benchmark scores.

   @article{robbins2022,
     title = {Towards Multimodal Vision-language Models Generating Non-generic Text},
     author = {Robbins, Wes},
     journal = {AAAI Undergraduate Consortium},
     year = {2022},
     month = feb,
   }

4. Towards Multimodal Vision-language Models Generating Non-generic Text

   Wes Robbins, Zanyar Zohourianshahzadi, and Jugal Kalita

   International Conference on Natural Language Processing Dec 2021

   Abs arXiv Bib

   Vision-language models can assess visual context in an image and generate descriptive text. While the generated text may be accurate and syntactically correct, it is often overly general. To address this, recent work has used optical character recognition to supplement visual information with text extracted from an image. In this work, we contend that vision-language models can benefit from additional information that can be extracted from an image, but are not used by current models. We modify previous multimodal frameworks to accept relevant information from any number of auxiliary classifiers. In particular, we focus on person names as an additional set of tokens and create a novel image-caption dataset to facilitate captioning with person names. The dataset, Politicians and Athletes in Captions (PAC), consists of captioned images of well-known people in context. By fine-tuning pretrained models with this dataset, we demonstrate a model that can naturally integrate facial recognition tokens into generated text by training on limited data. For the PAC dataset, we provide a discussion on collection and baseline benchmark scores.

   @article{robbins2021,
     title = {Towards Multimodal Vision-language Models Generating Non-generic Text},
     author = {Robbins, Wes and Zohourianshahzadi, Zanyar and Kalita, Jugal},
     journal = {International Conference on Natural Language Processing},
     year = {2021},
     month = dec,
   }