The "Natural Language-Driven Video Archive Searching System" is a project designed to revolutionise the way users interact with extensive video archives by enabling searches through natural language queries. This initiative addresses the inefficiencies of traditional manual video reviews, which are often time-consuming, labour-intensive, and susceptible to human error.
Project Objectives:
- Human-In-The-Loop Interactive AI System: Develop an AI system that integrates
Computer Vision (CV) and Natural Language Processing (NLP) to facilitate intuitive video archive searches.
- Natural Language Query Support: Allow users to perform searches using everyday language, such as "Show me when someone leaves a bag behind."
- Iterative Conversational Refinement: Implement interactive refinement of search results through user feedback and clarifying questions.
- Cross-Video Correlation: Create algorithms to identify people, objects, or events across multiple camera feeds.
- Camera-Independent Platform: Deliver an edge-based platform compatible with existing CCTV networks, eliminating the need for infrastructure overhaul.Research and Development Methodology:
- Interactive Search Refinement (Human-in-the-Loop): Develop a system that supports iterative refinement through back-and-forth queries, incorporating user feedback to enhance search results progressively.
- Cross-Video Object Search: Implement algorithms to confirm objects or individuals in one video and locate them across multiple videos, addressing challenges like variations in view angles and different times of appearance.
- Activity Search in Archives: Utilize (Generative Adversarial Network) GAN to provide a training dataset, enabling quick identification of specific activities within video archives, reducing manual review time.
Impact and Benefits:
The system enhances efficiency by enabling natural language queries, thereby reducing the time required to review video archives. Its integration of user feedback ensures accurate search results while minimising the risk of missing information. Cost-effective and camera-independent, it works with existing CCTV networks, avoiding costly upgrades. Versatile in application, it enhances monitoring and investigations across security, property management, logistics, and public safety.
R&D Project Database
Natural Language-Driven Video Archive Searching System
| Overview |
More information
| Project Reference | ITP/015/26LP |
| Hosting Institution | LSCM R&D Centre (LSCM) |
| Project Coordinator | Dr To Bun Ng |
| Approved Funding Amount | HK$ 3.37M |
| Project Period | 30 Mar 2026 - 30 Mar 2028 |





