Object Localization and Identification in Maritime Domain

Identifying Maritime Vessels at Multiple Levels of Descriptions using Deep Features

Faculty
Jelena Tešić
Students
David Bo Heyse
Nicholas Warren
Abstract
Varying resolution quality of operational data, size of targets, view occlusions, and large variation in sensors due to nature of overhead systems as compared to consumer devices contribute to degradation of the maritime vessel identification. We exploit the maritime domain characteristics to optimize and refine the deep learning Mask-RCNN framework for training generic maritime vessel classes. Maritime domain, compared to consumer domain, lack alternative targets that would be incorrectly associated as maritime vehicles: this allows us to relax the parameter constraints learned on urban natural scenes in consumer photos, adjust parameters of the model inference, and achieve robust performance and high AP measure for transfer learning scenarios. In this paper, we build upon this robust localization work, and extend our transfer learning work to new domains and datasets. We propose new approach for identifying specific category of maritime vessels and build a refined multi-label classifier that is based on deep Mask-RCNN features. The classifier is designed to be robust to domain transfer (e.g. different overhead maritime video feed), and to the noise in the data annotation (e.g. vessel is not correctly marked or label is ambiguous). We demonstrate superior category classification results of this low shot learning approach on publicly available MarDCT dataset.

Transfer learning of deep neural networks for visual collaborative maritime asset identification

Faculty
Jelena Tešić
Students
Nicholas Warren
Benjamin Garrard
Elliot Staudt
Abstract
Recent advances in deep learning for visual recognition demonstrate high performing pipeline for building and deploying well-performing content models. These advances come with underlying assumptions of the data characteristics pertaining to consumer image and video and availability of the large set of annotated data. In this paper we show how to apply lessons learned in the consumer domain to overhead maritime video corpora. We present how to successfully tune deep learning network to overhead maritime domain and tune parameters to new domain characteristics to achieve high performance metric with smaller set of domain annotations. This approach improves the state-of-the-art metric by 80% on maritime IPATCH data. Next, we present challenges and propose several approaches on user collaboration for maritime asset identification, and introduce the notion of persistent and intermittent models.
Faculty
Jelena Tešić
Abstract
Geospatial analysis is the process of manipulating geographic data as it pertains to terrain, topology, geomorphology, temporal locality, and history, and applying such data against a geographic model to infer upon or better understand an objective reality. Applications of geospatial analysis include, but are not limited to, land use and storm water management, emissions assessment of municipal landfills, preconstruction planning of new development with a focus on graceful spatial integration. Given the broad spectrum of use cases and needs for such a form of analysis, it is imperative that the acquisition and logical ordering of geographic data be streamlined and economical for all entities in need of such data. We have addressed the following aspects of this challenge: 1. Transform ArcGIS annotations into object detection compatible training format 2. Deep learning framework for detecting objects in overhead imagery 3. Object Identification on edge: MaskRCNN on NVIDIA Jetson TX2

Alumni

  • Nicholas Warren
    B.Sc. Dec 2019
  • David Bo Heyse
    B.Sc. Dec 2019(USAA)
  • Brent Redmon
    NSF REU 2019
  • Constance Xu,
    NSF REU 2019
  • Daniel Le,
    B.Sc. May 2020 (DoD)