If you're attending #UbiComp in Shanghai this week, find my students Baiyu Chen @Br33z3lled, Tahia Tazin, and Hada Melino Muhammad who are presenting their papers. 1. 🚀 Camera-Free Egocentric HAR: Distilling Video Intelligence into Low-Power Wearable Sensors. (see the 1st fig) with COMODO (Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition (IMWUT journal), Baiyu Chen et al. Always-on video in egocentric AI drains batteries, generates high heat, and raises privacy flags.  COMODO is a self-supervised cross-modal distillation framework that transfers the semantic structure of pretrained video representations to IMU representations through similarity-distribution alignment, enabling efficient and generalisable human activity recognition with IMU-only inference. COMODO prevents loss of temporal structure during video-to-IMU translation. By mapping motion data into the same semantic space as video, COMODO enables headworn wearables to track activities without cameras. Why this matters: ⚡ 115x Compute Drop: Slashes operations from 196B to just 1.7B FLOPs and latency to 75.7ms. 🔋 Battery: Camera sensors can safely duty-cycle while the ultra-low-power IMU runs continuously. 👁️ Privacy: Recognises complex activities without recording user spaces. paper: arxiv.org/abs/2503.07259 code: github.com/cruiseresearchgro… I presented this work also in the invite-only @Meta Wearables summit in July. 2. How stressed are ICU nurses during a shift? 🏥 (ISWC 2026) by Tahia Tazin et al. (see the second figure). In our study, we tracked 20 Surgical ICU nurses using non-invasive wearable sensors—measuring brainwaves (EEG), sweat gland activity (EDA), and pulse (BVP) during routine workflows. By using multimodal physiological sensing, the system can predict cognitive load in real time, paving the way for smarter, safer clinical environments. paper: dl.acm.org/doi/10.1145/38307… 3. How few self-reported labels does a mobile-sensing model need to adapt to a users mental health? 📱 by Hada Melino Muhammad et al. (see the 3rd figure) Continuous mental health surveys cause massive user fatigue, leading to high drop-out rates in digital interventions. The Solution: Models can achieve robust screening accuracy by collecting only a few highly strategic baseline labels rather than continuous daily prompts. 💡 Key Takeaways: The 5-Label Plateau: Accuracy gains drop off heavily after the fifth label. Base Rate vs. Case-Finding: Early labels mostly teach the model a user’s baseline symptom rate, leaving macro-F1 and ROC-AUC flat. Zero-Adaptation Baseline: High discrimination is often fully available without personalised tuning. Lets cut survey burden! 🧠 paper: dl.acm.org/doi/10.1145/38307… #ubicomp2026 #sensing #wearables #smartglasses #TSFM #digitalhealth

So proud of this work led by Breeze. Check this out. #neurips2026

Ok we end up with 9 NeurIPS papers. I'll do a post for each of the 9 NeurIPS papers that got through -- I promise! (One done already - GlucoFM). Congrats to my students, postdocs, and collaborators. The list of papers. 1. FOGO: Forgetting-aware Orthogonalization Optimiser - Toan Nguyen, YANG LIU et al. 2. GlucoFM - Zechen Li, Ahmed Abdelhadi Metwally, and the Google team. 3. MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphs - Ruiyi Yang Imran Razzak 4. Robust Residual Correction via Selective Deployment for Time Series Forecasting - Jianxiang Xie, Yuncheng Hua, Hao Xue 5. AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild - Baiyu Chen et al. 6. Discovering Programmatic Policies from Reinforcement Learning-Based Traffic Signal Controllers Lindong Xie 7. Out-of-Distribution Detection in Continual Learning - Nimeshika Udayangani, Sarah M. Erfani, Christopher Leckie (collaboration) And two evaluation track papers 8. When root cause analysis fails: a retrieval-reranking decomposition - Hada Melino Muhammad, Luan (Miles) Pham 9. Beyond high and low: evaluating graded cognitive diversity in LLM Persona simulation - Sion Weatherhead NeurIPS Sydney, here we come. :) #neurips2026 #neurips #ai #foundationmodels #timeseries #llm #tsfm #continuallearning #user #simulation #worldmodel

One dataset, one generator. As datasets multiply, that approach becomes hard to scale. Why should every new dataset or scenario mean another separate model to train and maintain? The problem: synthetic time-series generation is fragmented. Want traffic data? Train a model. Want energy load? Train another. Want to switch between morning-peak, evening-peak, or high-volatility scenarios? That choice should be an explicit input to the generator. UPLOTS takes a different approach. We developed one shared diffusion generator, guided by a pretrained transformer and scenario prompts. Choose a supported dataset and scenario through its prompt, and the same jointly trained model generates samples from the corresponding learned distribution. No separate model for every scenario. No retraining just to switch between supported prompts. 1. Breaking the One-Model-Per-Dataset Pattern UPLOTS brings multiple dataset–scenario pools into a single training framework. At each denoising step, a pretrained transformer combines the selected prompt with the current noisy sequence, producing representations that guide the Diffusion-TS generator. This makes scenario selection part of the generation process itself, while reducing the need to train and maintain separate generators. 2. A Unified Representation Layer Instead of engineering distinct neural architectures for different data behaviors, UPLOTS maps time-series inputs from different datasets into a shared embedding space. Shared Embeddings: From traffic flows to energy-related measurements, time-series inputs are projected into a common representation space. A frozen pretrained language-model backbone provides a shared component across these datasets. Prompt-to-Pattern Architecture: Textual scenario identifiers are embedded alongside the current noisy sequence. These prompts let one model distinguish among learned scenarios, including peak-period, calendar, load-level, and volatility subsets. A simple change of prompt selects a different supported scenario. 3. Multi-Dataset Learning — and a Path Toward Broader Generalisation The shared framework brings different datasets into one model, making joint training and scenario selection central to the design. Train Together, Switch by Prompt: UPLOTS jointly learns from multiple predefined dataset–scenario pools. Curriculum-aware loss modulation and rolling-loss dynamic sampling balance their contributions during training. Once trained, the model switches among these learned scenarios without separate fine-tuning for each one. We evaluate UPLOTS on four real-world datasets, with an extended study of 14 prompt configurations on ETTh and Energy. Downstream forecasting experiments also show the value of its synthetic data, particularly when real training data are scarce. This #CIKM2026 paper will be presented by my student Du Yin. Preprint: arxiv.org/abs/2606.10466 #TimeSeries #GenerativeAI #FoundationModels #MachineLearning #Diffusion #TSFM

🧠 How can a video-language model continually adapt to new tasks and domains without storing a separate set of adaptation tokens for each task? This is especially challenging for multimodal LLMs, where each new task may require different visual reasoning and adaptation, while task-specific prompts and parameters can quickly accumulate over time. I’m very pleased to share another work accepted to the EMNLP 2026 Main Conference! 🎉 DynaTokens: Controlling Token Dynamics for Continual Video-Language Understanding 💡 DynaTokens explores a simple idea: instead of storing a separate set of prompts for every task, can the model learn to generate the right adaptation tokens when they are needed? We build DynaTokens on a frozen LLaMA-2-7B backbone with CLIP ViT-L/14, using LLaMA-Adapter for efficient video-language adaptation. So while the model has a 7B-parameter LLM backbone, we do not fine-tune those 7B parameters. DynaTokens trains only a small token generator and lightweight adapter-side components, around 9.6M trainable parameters, less than 0.15% of the 7B backbone. ⚙️ Dynamic Token Generation: A token hypernetwork generates the task-specific prompts used by LLaMA-Adapter on demand, rather than learning and storing a new prompt block for every task. 🔄 Look-Ahead Regularisation: Anticipates how new-task updates affect past tasks, with a theoretical connection to sharpness-aware optimisation that helps reduce forgetting. 🪶 Very Lightweight: Each new task requires only ~26 KB of storage, around 200× smaller than storing its full generated prompt tensor. 📈 Strong Continual Learning: Higher accuracy and lower forgetting on continual NExT-QA and DramaQA, with strong performance across a 13-task cross-domain sequence. 🌐 Beyond VideoQA: DynaTokens also shows better zero-shot generalisation to unseen video domains and promising results in a challenging ImageQA → VideoQA continual learning setting. An interesting aspect is the efficiency trade-off. We retain the capacity of a large frozen LLaMA-2-7B model, while continual adaptation happens through a very small fraction of trainable parameters and only kilobytes of new storage per task. This work was led by my postdoc @KhanhToanNguye6. 📄 Paper: arxiv.org/abs/2603.06662 Demo: nktoan.github.io/dynatokens-… 💻 Code: github.com/cruiseresearchgro… @KhanhToanNguye6 will present this work at EMNLP 2026 in Budapest! 🇭🇺 #EMNLP2026 #ContinualLearning #MultimodalAI #MachineLearning #VideoUnderstanding #LLM #VLM

Introducing two of our works on robust and efficient agentic retrieval to be presented at EMNLP 2026 by my student Ruiyi Yang, who led three out of our six accepted papers. SPLIT-RAG (Main) When queries sent to an agentic RAG system are too complex and require multi-hop reasoning, retrieving over a large knowledge graph can introduce substantial search overhead, irrelevant evidence, and conflicts across retrieved facts. SPLIT-RAG is a multi-agent framework with: - Question-Driven Graph Partitioning: It uses reasoning patterns observed in training questions to organize a large knowledge graph into compact, reusable partitions aligned with recurring information needs. - Selective Multi-Agent Retrieval: Each query activates only a small set of partition-specialized agents, which retrieve evidence from their assigned graph regions in parallel. - Compatibility-Based Aggregation: Retrieved triples are scored and filtered through a compatibility graph before being passed to the head agent, reducing conflicts across evidence from different agents. SPLIT-RAG achieves the strongest results among all the evaluated KG+LLM methods, and the lowest end-to-end latency under given budget (29.4-33.7s per query). Preprint: arxiv.org/abs/2505.13994 RELOOP (Findings) Multi-hop QA becomes particularly challenging when the required evidence is distributed across text, tables, and knowledge graphs, as different sources have different structures and often require separate retrieval pipelines. RELOOP introduces a structure-aware iterative retrieval framework built on Hierarchical Sequence (HSEQ), providing a unified interface for retrieving and reasoning: - Hierarchical Sequence (HSEQ): converts text, tables, and KGs into a common sequence of typed segments while preserving their original structure and provenance. - Structure-Aware Iterative Retrieval: A lightweight iterator selects and expands evidence through modality-specific structural links, such as paragraph–sentence relations, table rows/columns, and KG neighbourhoods. - Sufficiency-Controlled Stopping: Instead of retrieving for a fixed number of rounds, RELOOP estimates when enough evidence has been collected and stops retrieval accordingly. RELOOP achieves the highest accuracy on all evaluated tasks. RELOOP also substantially reduces iterative retrieval cost: under latency-sensitive inference, it uses roughly 3–4 retrieval steps, compared with 11–17 steps for Think-on-Graph, and > 2x faster than other baselines. Preprint: arxiv.org/abs/2510.20505 This project is supported by @TIIuae. We also thank the support of @sharon__ai for the compute support. #EMNLP2026 #agentic #RAG #LLM #robustness #efficient #retrieval

What if your AI agents could actively fight back against invasive web tracking? Our new paper (EMNLP 2026) introduces GuardianAgent, a privacy framework that evaluates outgoing web traffic against a site's destination policies in real time. Instead of just blocking requests, it automatically sanitises sensitive data or deploys verified adversarial escalation when risk levels spike. Text anonymisation is not simply about removing as much sensitive information as possible. Stronger anonymisation can improve privacy but also unnecessarily destroy useful content. GuardianAgent introduces a policy-conditioned, risk-adaptive anonymisation method that determines how strongly content should be anonymized based on disclosure risk and escalates rewriting only when residual leakage is actually supported by the original text. Core Concepts of GuardianAgent: - Policy-Conditioned Risk Scoring: AMRSF combines policy-violation likelihood with factors such as data sensitivity, recipient, purpose, action context, and policy transparency to produce an explicit risk score. - Risk-Adaptive Anonymization: The risk score determines the initial strength of anonymization through a five-level hierarchy, ranging from partial masking to semantic rewriting. - Verified Adversarial Escalation: An adversarial guesser can trigger stronger anonymization only when its inferred sensitive information is verified against the original text, preventing unsupported guesses from causing unnecessary rewriting. Results: Across TAB, SynthPAI, and PII-Masking-300k, GuardianAgent achieves 0.945, 0.944, and 0.901 privacy, respectively, while maintaining strong utility. It is the only evaluated method to exceed 0.90 privacy across all three domains and lies above or beyond the baseline privacy–utility frontier on each benchmark. Preprint: arxiv.org/abs/2608.29251 Code: github.com/RuiyiYang01/guard… This work is led by my student Ruiyi Yang, funded by UNSW-UTS Trustworthy Digital Society, with the team: Salil Kanhere, Rahat Masood, Gayathri Harshila The genesis of this work was ideated a while ago, described in this Conversation article I wrote with Salil Kanhere. theconversation.com/your-per… We're now in the middle of user testing of a GuardianAgent browser extension with our collaborators. #AI #Privacy #CyberSecurity #DataProtection #MachineLearning

What if a glucose foundation model could capture both long-term metabolic state and short-term responses to meals? This is a collaboration led by my student Zechen Li who's currently interning at Google Research under @aametwally1. Continuous glucose monitoring (CGM) provides a dense view of daily metabolic physiological time series, but general-purpose time-series foundation models are not explicitly designed for its multiscale dynamics, irregular observations, and sensor artifacts. GlucoFM is a lightweight, CGM-specific foundation model built around a state–event dual-stream architecture. • Chronological Grid Alignment: Maps each 24-hour CGM segment onto a standardized five-minute grid while preserving absolute time-of-day information.   • Missingness-Aware Representation: Retains an explicit observation mask to distinguish measured glucose values from unobserved positions.   • Dual-Stream Encoding: Separately models slower glycemic trends through a state stream and short-term deviations through an event stream before fusing them into a unified representation.   • JEPA-Style Pretraining: Combines masked latent prediction over fused daily representations with a Temporal Dynamics objective that models hour-to-hour transitions in both streams, avoiding direct reconstruction of potentially noisy raw glucose values.   • Lightweight Frozen Encoder: Uses a compact 0.72M-parameter encoder that remains frozen during downstream evaluation. GlucoFM builds on the direction introduced by CGM-JEPA (by my student Hada Melino Muhammad, Zechen Li also in close collaboration with Ahmed Abdelhadi Metwally). While CGM-JEPA focuses on masked temporal representation learning—and X-CGM-JEPA extends it with Glucodensity-derived cross-view information—GlucoFM focuses on the internal multiscale structure of the glucose sequence through state–event decomposition and stream-level dynamics modeling. Further, pretrained on 109,066 hours of unlabeled CGM data from 477 subjects, GlucoFM demonstrated strong performance in two complementary settings: • Linear probing: Across four cohorts and seven metabolic phenotype tasks, GlucoFM achieved the strongest average subject-disjoint performance among the evaluated models. • Postprandial glycemic response forecasting: When combined with recent CGM, meal nutrition, and participant context, the same frozen encoder achieved the lowest two-hour PPGR errors across both Dexcom and Libre. To sum up, these results highlight the potential of multiscale, sensing-aware FMs to enable reusable CGM representations for metabolic phenotyping and glucose-response prediction, bringing scalable and personalized metabolic risk assessment closer to practice. GlucoFM: arxiv.org/abs/2605.30865 CGM-JEPA: arxiv.org/abs/2605.00933 #CGM #glucose #T1D #T2D #sensor #timeseries #FM #TSFM #JEPA

🚀 Fine-Tuning LLMs/VLMs with <1.5% of the parameters without Catastrophic Forgetting: Introducing CP-MoE 🧠 When scaling foundation models sequentially, traditional efficient methods (like LoRA) still suffer from catastrophic forgetting. Parameter overwriting and rigid routing load-balancing consistently erase historical task knowledge. CP-MoE (Consistency-Preserving Mixture-of-Experts) solves this. It acts as a post-training framework that governs how LoRA-based experts are routed and updated while keeping the core 7B foundation model entirely frozen. 🔍 The Core Strategy: Assess Then Update Instead of jumping directly into fine-tuning, CP-MoE introduces a temporary probe mechanism: * The Transient Expert: A disposable expert adapts to a new task during a brief warm-up phase. Its trajectory guides the learning process before it is discarded, adding zero long-term overhead. * Consistency-Preserving Routing Bias: CKA-based compatibility score between the transient expert and each stable expert is injected as a bias on the router logits, steering the new task toward the most aligned stable experts * Transient Expert-Guided Regularisation: Parameter importance is calculated dynamically to shield crucial historical knowledge with selective regularisation. 📊 Benchmark Breakthroughs Evaluated on frozen 7B backbones (Llama-2-7B for language, LLaVA-1.5-7B for vision-language), training only the LoRA experts and router. Specifically, by training only 1.48% of total parameters on SuperNI and 0.47% on VQA v2, CP-MoE outperforms traditional CL-MoE and LoRA baselines: * SuperNI (8 sequential language tasks):  Achieved 50.84% average performance with 1.32% forgetting, matching recent LoRA-based continual learning baselines (InfLoRA, O-LoRA, GainLoRA) in-sequence and 35.80% zero-shot transfer on unseen tasks, beating the strongest baseline (GainLoRA at 33.80%). * VQA v2 (10 sequential visual reasoning tasks): Reached 62.30% average accuracy with near-zero forgetting (just 0.35%), outperforming the prior state-of-the-art CL-MoE. CP-MoE proves that we do not need to choose between knowledge transfer and parameter isolation. Dynamic, consistency-guided routing provides a promising path toward scalable and sustainable continual AI training. 🔗 The full paper accepted at CoLLAs 2026: arxiv.org/abs/2605.20247 ⚒️ Github : github.com/cruiseresearchgro… #MachineLearning #LLM #GenerativeAI #MOE #LORA #PEFT #ContinualLearning #CoLLAs2026 #CoLLAs p.s. Very proud of my Master thesis student Yang Liu who did this research as part of his 2-term thesis coursework project, leading to this output in just 8 months, and my postdoc @KhanhToanNguye6 the cosupervisor. This is Yang's first paper as a first-author, and we trained him from scratch to do research, yet with his emergent capabilities, we shall see many more brilliant outputs from him :)

Excited to share that our patent Methods, systems, devices and neural networks for forecasting a time series was granted on July 21, 2026 (US 12,688,375. App no 18/717,983) after nearly a 5-year wait. We were the first to introduce time-series forecasting as a language model prompt/generation task. We filed in 2021 at my former institution RMIT as an Australian patent. Then in 2022 we filed a continuation as a worldwide application, which was continued only as a US patent, with the support of UNSW, not long after me and Hao Xue (my ARC DP postdoc then) moved to UNSW CSE. The link to the patent application reveals the journey: patents.google.com/patent/US… The patent doc: thepatentplace.com/iplibrary… The journey: **before ChatGPT: before prompting was a thing 1. Translating Human Mobility Forecasting through Natural Language Generation** (WSDM 2022). We treat human mobility forecasting as a sequence-to-sequence (seq2seq) language translation task rather than numerical time-series. arxiv.org/abs/2112.11481 2. AuxMobLCast** in "Leveraging Language Foundation Models for Human Mobility Forecasting" (SIGSPATIAL 2022) proposed mapping of TS and contextual semantics into prompts. This was the first known work on prompt-based forecasting. We tried BERT / XLNet / RoBERTa as the encoder, but ultimately selected BERT as the primary encoder due to its strong performance in the bidirectional token embeddings. We used GPT-2 as the decoder to generate forecasts. We also introduced mobility classification task, associated with the [CLS] token of BERT-like encoders, to inject domain-specific spatiotemporal context. arxiv.org/abs/2209.05479 3. PromptCast** (also in 2022, published later in IEEE TKDE): the first large-scale benchmark on prompt-based forecasting. We transformed numerical time-series forecasting into natural language prompts. We introduced PISA dataset containing more than 311K data instances (weather, energy, mobility domain), benchmarked with 10+ statistical and deep learning models and 10+ language models. PromptCast shows a much better generalization ability under zero-shot setting, which was never explored then in the time-series forecasting domain. arxiv.org/abs/2210.08964 4. Utilizing Language Models for Energy Load Forecasting (BuildSys 2023) extends the prompt-based forecasting paradigm specifically to building and urban energy consumption. We introduced autoregressive generation through sentence-to-sentence text generation to predict future load values across varying forecast horizons. arxiv.org/abs/2310.17788 Not on the slide: 2023 onwards: we started working on small LMs and foundation models for time-series and multimodal sensors. 2025 onwards: agentic reasoning paired with the best pre-trained or post-trained models from our lab. Curious? Reach out. We'd like to acknowledge @arc_gov_au (ARC). Without the ARC DP, we wouldn't get here. We also thank @UNSWEngineering @UNSW for their unwavering support.

My school CSE @UNSW is hiring for a total of 12 faculty positions, including 6 Lecturers/Senior Lecturers (Assistant Professors), and 6 Associate Professors in specific areas such as in ML and NLP, neurosymbolic AI, systems (for AI), cybersecurity, software engineering, cryptography. Check out UNSW Careers website. #facultyjobs #academicjobs

🚀 Introducing our #ICLR2026 paper DRIFT-Net: A Spectral-Coupled Neural Operator for PDE Learning! Simulating Partial Differential Equations (PDEs) is computationally expensive. While neural operators help, many PDE foundation models struggle with error drift. Current attention-based models often struggle with error accumulation and drift during long rollouts. DRIFT-Net fixes this using: Dual-Branch Design: It explicitly combines global spectral structure with local spatial detail, with controlled low-frequency mixing. 🌊 Spectral branch for global, low-frequency dynamics. 🔍 Image branch for local, high-frequency details. 🛡️ Stable training with no width inflation via bandwise fusion: The two branches are effectively coupled at every layer. By fusing these branches via bandwise weighting, it avoids the training instability and width inflation caused by naive concatenation. The results? 📉 7%-54% lower relative L1​ error on Navier-Stokes tasks. ⚡ ~15% fewer parameters & higher throughput than scOT! Code is available here 👇 🔗 github.com/cruiseresearchgro… Model & checkpoints on @huggingface : huggingface.co/CRUISEResearc… Paper: arxiv.org/pdf/2509.24868 Whether you are working with fluid dynamics, wave propagation, or phase separation, this modular design can be swapped into existing multi-scale operator backbones for an immediate upgrade. This work was done with my Master student Jiayi Li during his 2-term thesis project at UNSW Computer Science and Engineering. Indeed, it was his first paper submission! We've been exploring ODEs, PDEs, SDEs for a while. If you're going to be in Rio for ICLR, would be great to chat. #AI #MachineLearning #DeepLearning #PDE #physics #neuraloperators #PINNs #ICLR2026 p.s. posting after a while... Giving this platform another go.