Rapid Similarity Prediction, Forensic Search & Retrieval in Video

Download Project Report (Phase 2, Year 5).

Project Description

The aim of this project is to leverage machine learning and computer vision methods for surveillance over multi-camera networks. Our objective is to develop methods that are capable of real-time and forensic detection of suspicious activity.

In contrast to conventional automatic video activity recognition approaches, forensic activity detection poses two fundamental challenges. In forensic applications, the type of activity the operators want to identify can be quite complex and a priori unknown until the operator presents a description of the activity. Therefore, we cannot leverage conventional learning approaches that train activity recognition models based on annotated training data, since no such data is a priori available. A second challenge arises from the fact that many surveillance scenarios involve multiple but unrelated co-occurring activities and share common attributes/properties with the desired activity. Therefore, methods that are capable of search over a sub-collection of events are necessary in order to identify the desired activity.


We demonstrated capabilities of our software at the Office of Naval Research and at the NGA National Academic Symposium in 2016. The feedback we have received is that the software capabilities, when fully functional, would significantly enhance current security analyst capabilities.
Year 4 Annual Report
Project Leader
  • Venkatesh Saligrama
    Associate Professor
    Boston University

Faculty and Staff Currently Involved in Project
  • Hanxiao Wang
    Post Doctoral
    Boston University

Students Currently Involved in Project
  • Yannan Bai
    Boston University