Hashtag
Men's Weekly

PolyU develops novel multi-modal agent to facilitate long video understanding by AI, accelerating development of generative AI-assisted video analysis

HONG KONG SAR - Media OutReach Newswire - 10 June 2025 - While Artificial Intelligence (AI) technology is evolving rapidly, AI models still struggle with understanding long videos. A research team from The Hong Kong Polytechnic University (PolyU) has developed a novel video-language agent, VideoMind, that enables AI models to perform long video reasoning and question-answering tasks by emulating humans' way of thinking.

The VideoMind framework incorporates an innovative Chain-of-Low-Rank Adaptation (LoRA) strategy to reduce the demand for computational resources and power, advancing the application of generative AI in video analysis. The findings have been submitted to the world-leading AI conferences.

A research team led by Prof. Changwen Chen, Interim Dean of the PolyU Faculty of Computer and Mathematical Sciences and Chair Professor of Visual Computing, has developed a novel video-language agent VideoMind that allows AI models to perform long video reasoning and question-answering tasks by emulating humans’ way of thinking. The VideoMind framework incorporates an innovative Chain-of-LoRA strategy to reduce the demand for computational resources and power, advancing the application of generative AI in video analysis.
A research team led by Prof. Changwen Chen, Interim Dean of the PolyU Faculty of Computer and Mathematical Sciences and Chair Professor of Visual Computing, has developed a novel video-language agent VideoMind that allows AI models to perform long video reasoning and question-answering tasks by emulating humans’ way of thinking. The VideoMind framework incorporates an innovative Chain-of-LoRA strategy to reduce the demand for computational resources and power, advancing the application of generative AI in video analysis.

Videos, especially those longer than 15 minutes, carry information that unfolds over time, such as the sequence of events, causality, coherence and scene transitions. To understand the video content, AI models therefore need not only to identify the objects present, but also take into account how they change throughout the video. As visuals in videos occupy a large number of tokens, video understanding requires vast amounts of computing capacity and memory, making it difficult for AI models to process long videos.

Prof. Changwen CHEN, Interim Dean of the PolyU Faculty of Computer and Mathematical Sciences and Chair Professor of Visual Computing, and his team have achieved a breakthrough in research on long video reasoning by AI. In designing VideoMind, they made reference to a human-like process of video understanding, and introduced a role-based workflow. The four roles included in the framework are: the Planner, to coordinate all other roles for each query; the Grounder, to localise and retrieve relevant moments; the Verifier, to validate the information accuracy of the retrieved moments and select the most reliable one; and the Answerer, to generate the query-aware answer. This progressive approach to video understanding helps address the challenge of temporal-grounded reasoning that most AI models face.

Another core innovation of the VideoMind framework lies in its adoption of a Chain-of-LoRA strategy. LoRA is a finetuning technique emerged in recent years. It adapts AI models for specific uses without performing full-parameter retraining. The innovative chain-of-LoRA strategy pioneered by the team involves applying four lightweight LoRA adapters in a unified model, each of which is designed for calling a specific role. With this strategy, the model can dynamically activate role-specific LoRA adapters during inference via self-calling to seamlessly switch among these roles, eliminating the need and cost of deploying multiple models while enhancing the efficiency and flexibility of the single model.

VideoMind is open source on GitHub and Huggingface. Details of the experiments conducted to evaluate its effectiveness in temporal-grounded video understanding across 14 diverse benchmarks are also available. Comparing VideoMind with some state-of-the-art AI models, including GPT-4o and Gemini 1.5 Pro, the researchers found that the grounding accuracy of VideoMind outperformed all competitors in challenging tasks involving videos with an average duration of 27 minutes. Notably, the team included two versions of VideoMind in the experiments: one with a smaller, 2 billion (2B) parameter model, and another with a bigger, 7 billion (7B) parameter model. The results showed that, even at the 2B size, VideoMind still yielded performance comparable with many of the other 7B size models.

Prof. Chen said, "Humans switch among different thinking modes when understanding videos: breaking down tasks, identifying relevant moments, revisiting these to confirm details and synthesising their observations into coherent answers. The process is very efficient with the human brain using only about 25 watts of power, which is about a million times lower than that of a supercomputer with equivalent computing power. Inspired by this, we designed the role-based workflow that allows AI to understand videos like human, while leveraging the chain-of-LoRA strategy to minimise the need for computing power and memory in this process."

AI is at the core of global technological development. The advancement of AI models is however constrained by insufficient computing power and excessive power consumption. Built upon a unified, open-source model Qwen2-VL and augmented with additional optimisation tools, the VideoMind framework has lowered the technological cost and the threshold for deployment, offering a feasible solution to the bottleneck of reducing power consumption in AI models.

Prof. Chen added, "VideoMind not only overcomes the performance limitations of AI models in video processing, but also serves as a modular, scalable and interpretable multimodal reasoning framework. We envision that it will expand the application of generative AI to various areas, such as intelligent surveillance, sports and entertainment video analysis, video search engines and more."


Hashtag: #PolyU #AI #LLMs #VideoAnalysis #IntelligentSurveillance #VideoSearch

The issuer is solely responsible for the content of this announcement.

Business News

Prudential and Prudence Foundation Provide Free Mental Health Support to Vulnerable Communities

In partnership with Intellect, the programme will be launched in Malaysia and Vietnam to make mental healthcare more accessible for vulnerable or at-risk groups HONG KONG SAR - Media OutReach Newswir...

Hashtag.net.au - avatar Hashtag.net.au

Trend Micro Named a Leader in IDC MarketScape for CNAPP

HONG KONG SAR - Media OutReach Newswire - 2 July 2025 - Trend Micro Incorporated (TYO: 4704; TSE: 4704), a global cybersecurity leader, today announced it has been named as a Leader in the IDC Marke...

Hashtag.net.au - avatar Hashtag.net.au

Drug & Medical Device & Aesthetics, 肌顏態 was displayed at the CSD2025, The GeneQueens and 金因敷 launch was grandly held in Changsha

HONG KONG SAR - EQS Newswire - 2 July 2025 - On June 27, 2025, Uni-Bio Science Group Limited (the "Group")grandly held a new medical aesthetics product launch event in Changsha, releasing the high-end...

Hashtag.net.au - avatar Hashtag.net.au

Amazon Singapore Unveils a Week of Prime Day Deals and a Special Public Maze Event for Prime Day 2025

Prime members can access exclusive deals from brands such as Neutrogena, FOREO, Resto, Cricut, Pigeon, PicassoTiles and Coca-Cola from 8 – 14 July. Free Public event: Amazon Prime Day Maze at Suntec...

Hashtag.net.au - avatar Hashtag.net.au

Themed Forum of "Mountain Biodiversity and Our Life" to Kick off "Guizhou Example" Empowering Global Mountain Eco-treatment

GUIYANG, CHINA - Media OutReach Newswire - 2 July 2025 - Recently, Eco Forum Global Guyang 2025 announced at a press briefing that the themed forum "Mountain Biodiversity and Our Life" will be held i...

Hashtag.net.au - avatar Hashtag.net.au

Asia's First Porsche Design Tower Offers Revolutionary Luxury Living Experience to Bangkok's Vibrant Japanese Community

STUTTGART, GERMANY / TOKYO, JAPAN - Media OutReach Newswire - 24 June 2025 - Porsche Design Tower Bangkok emerges as a defining statement of excellence in luxury automotive-inspired living. As Asia's...

Hashtag.net.au - avatar Hashtag.net.au

Health & Wellness

Telehealth Could be the Catalyst for Better Heart Health in Australia

Hashtag.net.au - avatar Hashtag.net.au

Image by stockking on Freepik Heart disease is one of the leading causes of death in Australia despite decades of advancements in medicine and health education campaigns. More than 400 Australians ...

Disability Care in Melbourne: What Families Need to Know in 2025

Hashtag.net.au - avatar Hashtag.net.au

If you’re supporting a loved one with a disability in Melbourne, you’ll know how important it is to find the right care—and how overwhelming it can be to navigate the system. With the National Disab...

Online Medical Certificate Services: Essential Features to Look Out For

Hashtag.net.au - avatar Hashtag.net.au

"Discover top online medical certificate services! Essential features to look for to get trusted and efficient results." Choosing high-quality services is essential for ensuring reliable, efficient, ...