About me

Welcome! I am currently a Research Assistant Professor (PI) in the Department of Electrical and Electronic Engineering (EEE), The Hong Kong Polytechnic University (香港理工大学, PolyU). I am also working in the JC STEM Lab of Machine Learning and Computer Vision with Prof. CHAU Lap Pui. My research interests include Egocentric Vision, Embodied Intelligence, 3D Vision, and Trustworthy AI.

I received BEng in Electronic and Information Engineering and MEng in Signal and Information Processing from Northwestern Polytechnical University (西北工业大学, NPU), China, and earned PhD in the School of Electrical and Electronic Engineering (EEE) from Nanyang Technological University (南洋理工大学，NTU), Singapore, in 2021, supervised by Prof. CHAU Lap Pui (formerly at NTU, now at PolyU).

Prior to joining PolyU, I worked as a Research Fellow in EEE at NTU until March 2023, with PI A/P YAP Kim Hui. Prior to going to Singapore, I was involved in national projects and a robotics center at NPU, China, supervised by Prof. WAN Shuai and Prof. MEI Shaohui.

Recent Embodied AI-related Projects

2026 · Multi-floor Navigation

AERR-Nav

A zero-shot object navigation framework for unknown multi-floor environments, achieving state-of-the-art results on open-world indoor navigation.

2026 · Embodied Social Interaction

FreeAskWorld

A simulation framework for embodied social interaction and scalable data generation, with integration of AI agent interfaces such as OpenClaw, Codex, and Claude Code.

2026 · Vision-Language Navigation

OE-VLN/HiNav

An open-ended vision-language navigation framework, including benchmark, method, and real-world validation on a Unitree Go2 robot.

2025 · Autonomous Driving

OccProphet

A camera-only 4D occupancy forecasting project for autonomous driving, designed to improve efficiency while preserving strong forecasting accuracy.

2025 · Smart Glasses

EgoProceAssist

A novel egocentric procedural AI assistant for smart glasses, tailored to deliver step-by-step support for daily procedural tasks.

News

• [2026.04] I served as Area Chair at NeurIPS 2026 (CCF-A)
• [2026.03] [Nature Communications] (IF: 15.7) Revealing the Intrinsic Ethical Vulnerability of Aligned Large Language Models
• [2026.02] [CVPR_2026] (CCF-A) with 3 papers accepted: Hand-Object Interaction Video Synthesis, Intention-Driven 3D Affordance Grounding(with Code), and Accelerating Diffusion-based Video Editing.
• [2026.01] [ICLR 2026] [Egocentric AI] Interaction-aware Representation Modeling With Co-Occurrence Consistency for Egocentric Hand-Object Parsing.
• [2025.10] [NeurIPS 2025] [LLM Vulnerability] Semantic Representation Attack against Aligned Large Language Models, with code
• [2025.09] I served as Area Chair at ICLR 2026 (CCF-A)
• [2025.09] I was in the role of Associate Editor of Computers and Electrical Engineering Journal (JCR Q1)
• [2025.07] [ACM Multimedia] (CCF-A) with 4 papers accepted: 3D Representation Learning, Open-vocabulary Situation Recognition, Bitstream-corrupted Video Recovery, and Few-Shot Class-Incremental Learning.
• [2025.07] [IEEE TITS] (IF: 8.4) Signeye: Traffic sign interpretation from vehicle first-person view
• [2025.06] [IEEE TMM] (IF: 9.7) 3DGeoDet: General-purpose Geometry-aware Image-based 3D Object Detection
• [2025.05] [IEEE TITS] (IF: 8.4) Occlutrack: Rethinking awareness of occlusion for enhancing multiple pedestrian tracking
• [2025.05] [Knowledge-based Systems] (IF: 7.6) Foundation model-assisted interpretable vehicle behavior decision making
• [2025.05] [Information Fusion] (IF: 15.5) Contrastive learning-based place descriptor representation for cross-modality place recognition, code
• [2025.04] [Machine Intelligence Research] (IF: 8.7) A survey of embodied learning for object-centric robotic manipulation
• [2025.03] [CVPR 2025] [Egocentric Reasoning] ANNEXE: Unified Analyzing, Answering, and Pixel Grounding for Egocentric Interaction, with code
• [2025.02] [ICLR 2025] [Autonomous Driving Perception] OccProphet: Pushing Efficiency Frontier of Camera-Only 4D Occupancy Forecasting with Observer-Forecaster-Refiner Framework, with code
• [2025.01] [IEEE JSTARS] (JCR Q1) We organized a special issue, Scene Analysis and Understanding in the Intelligent Transportation for Urban Area. Deadline: Jul 31, 2025.
• [2024.12] [IEEE TMM] (IF: 9.7) WHANet: Wavelet-based Hybrid Asymmetric Network for Spectral Super-Resolution from RGB Inputs
• [2024.12] [IEEE TMM] (IF: 9.7) ByteNet: Rethinking multimedia file fragment classification through visual perspectives
• [2024.11] [Remote Sensing] (JCR Q1) We organized a special issue, Recent Advances in Multimodal Hyperspectral Remote Sensing. Deadline: Nov 30, 2025.
• [2024.11] [IEEE TCSVT] (IF: 11.1) SGIFormer: Semantic-guided and Geometric-enhanced Interleaving Transformer for 3D Instance Segmentation, with code
• [2024.10] [IEEE TCSVT] (IF: 11.1) Open World Object Detection: A Survey, with code
• [2024.10] Outstanding Reviewer in ACM Multimedia 2024
• [2024.09] [Information Fusion] (IF: 15.5) A survey on occupancy perception for autonomous driving: The information fusion perspective, with github link
• [2024.06] We got the champion of EPIC-Kitchens Challenges Multi-Instance Retrieval Track in CVPR 2024 Champion solution and paper link
• [2024.05] We chaired a special session in ISCAS 2024
• [2024.04] We chaired a special session in ICASSP 2024
• [2024.02] [Knowledge-based Systems] (IF: 7.6) Intra- and Inter-sector Contextual Information Fusion with Joint Self-Attention for File Fragment Classification, with code
• [2024.01] [Knowledge-based Systems] (IF: 7.6) Weakly-Supervised Grounded Image Captioning
• [2024.01] [Remote Sensing] Interactive Change-Aware Transformer Network for Remote Sensing Image Change Captioning
• [2023.10] I am one of Special Session Chair and Review Committee Members of the ISCAS 2024.
• [2023.09] We organized a special issue, AI-powered multimedia computing, in Multimedia Tools and Applications Journal, as Guest Editor
• [2023.09] [NeurIPS 2023] Bitstream-corrupted Video Recovery: A Novel Benchmark Dataset and Method, with dataset
• [2023.09] I have been teaching the subject EIE546 Video Technology in PolyU
• [2023.08] Our group has recruited 8 PhD students, 1 research fellow, and 1 MPhil student, and I am the supervisor/co-supervisor. The lab’s website will be constructed
• [2023.06] I have been working with Prof. CHAU Lap Pui in PolyU and constructing the JC STEM Lab of Machine Learning and Computer Vision
• [2023.05] I was in the role of Associate Editor of The Visual Computer Journal
• [2023.04] I joined the Hong Kong Polytechnic University (PolyU) as Research Assistant Professor
• [2023.03] [CVPR 2023] Bitstream-Corrupted JPEG Images are Restorable: Two-stage Compensation and Alignment Framework for Image Restoration, with code

Recruitment/招生

We are actively recruiting self-motivated PhD and MPhil (self-finance) students with expertise in embodied intelligence, agentic AI, spatial intelligence, and computer vision, to join our research group at the Hong Kong Polytechnic University. Strong candidates can contact me to be nominated for Hong Kong PhD Fellowship Scheme (HKPFS), PolyU Presidential PhD Fellowship, Joint PhD Supervision Leading to PolyU degree, and Joint PhD Supervision (specific universities/institutes). More application information can be found at this link.