innerpage Project Funding
PROJECT & FUNDING SCHEMES

PROJECT & FUNDING SCHEMES

R&D Project Database
Natural Language-Driven Video Archive Searching System

Print

Overview

The "Natural Language-Driven Video Archive Searching System" is a project designed to revolutionise the way users interact with extensive video archives by enabling searches through natural language queries. This initiative addresses the inefficiencies of traditional manual video reviews, which are often time-consuming, labour-intensive, and susceptible to human error.
Project Objectives:
- Human-In-The-Loop Interactive AI System: Develop an AI system that integrates
Computer Vision (CV) and Natural Language Processing (NLP) to facilitate intuitive video archive searches.
- Natural Language Query Support: Allow users to perform searches using everyday language, such as "Show me when someone leaves a bag behind."
- Iterative Conversational Refinement: Implement interactive refinement of search results through user feedback and clarifying questions.
- Cross-Video Correlation: Create algorithms to identify people, objects, or events across multiple camera feeds.
- Camera-Independent Platform: Deliver an edge-based platform compatible with existing CCTV networks, eliminating the need for infrastructure overhaul.Research and Development Methodology:
- Interactive Search Refinement (Human-in-the-Loop): Develop a system that supports iterative refinement through back-and-forth queries, incorporating user feedback to enhance search results progressively.
- Cross-Video Object Search: Implement algorithms to confirm objects or individuals in one video and locate them across multiple videos, addressing challenges like variations in view angles and different times of appearance.
- Activity Search in Archives: Utilize (Generative Adversarial Network) GAN to provide a training dataset, enabling quick identification of specific activities within video archives, reducing manual review time.
Impact and Benefits:
The system enhances efficiency by enabling natural language queries, thereby reducing the time required to review video archives. Its integration of user feedback ensures accurate search results while minimising the risk of missing information. Cost-effective and camera-independent, it works with existing CCTV networks, avoiding costly upgrades. Versatile in application, it enhances monitoring and investigations across security, property management, logistics, and public safety.

More information

Project Reference ITP/015/26LP
Hosting Institution LSCM R&D Centre (LSCM)
Project Coordinator Dr To Bun Ng
Approved Funding Amount HK$ 3.37M
Project Period 30 Mar 2026 - 30 Mar 2028