avatar

Cheng DENG (Daven)

Research Fellow
Bayes Centre, CSE
University of Edinburgh
Module Lead, PLIN0072
Linguistics, UCL

davendw49 (at) gmail.com
My Blog · All Talks

About Me

I work on efficient large language models, and on extending them into efficient physical AI. My research covers hardware-aware model architecture, scaling laws, efficient attention and inference, and long-context agent systems, and applies them to VLA models and embodied agents that act on robots and vehicles. Before that, my work was in natural language processing and foundation-model training, including the first generation of geoscience LLMs.

A central question across my work is how to build capable AI systems under real constraints on compute, memory, latency, energy, and interaction: from language models running on edge devices to embodied agents acting on robots and vehicles.

Currently, I am a Research Fellow at the Bayes Centre, University of Edinburgh, working in collaboration with Prof. Luo Mai, Prof. Jeff Pan, and Prof. Jun Wang. I also serve as a Visiting Research Fellow at Li Auto, and hold a lecturer-grade teaching appointment in the Department of Linguistics at UCL, where I am the module lead for Advanced Computational Linguistics (PLIN0072) in 2026–2027, responsible for its design, delivery, and assessment. Prior to joining the University of Edinburgh, I was a Research Assistant at the Hong Kong University of Science and Technology (Guangzhou), where I worked with Prof. Lei Chen and Prof. Lionel M. Ni. I obtained my Ph.D. at Shanghai Jiao Tong University, where I was fortunate to be supervised by Prof. Weinan Zhang, Prof. Luoyi Fu, and Prof. Xinbing Wang. During my early research career, I interned with the Data Team at TikTok and worked as an Applied Scientist Intern at Amazon Shanghai AI Lab. In 2021, I was selected for the Wenjun Wu Honored Ph.D. Class.

Research

My current research is organised around three connected directions:

  1. Efficient LLMs & Agents. Hardware-aware model architecture, scaling laws, efficient attention and inference, and long-context agent systems for resource-constrained deployment: PLM, GTA, ContextPilot, MemoryCraft, and the hardware co-design scaling laws.
  2. NLP, Foundation Models & AI for Science. Pre-training, post-training, reasoning, agents, and structured knowledge for language and scientific domains.
  3. Efficient Physical AI & Robotics. VLA and world-action models, embodied agents, and hybrid learned/classical robot systems on edge hardware, deployed on a self-built home robot and on intelligent-vehicle platforms with Li Auto.

My long-term goal is to understand how models, agents, robot skills, and hardware should be co-designed so that increasingly capable AI can operate reliably in the physical world.

News

Selected Research Highlights

⚡ Efficient LLMs & Agents
PLM demo: interacting with objects in front of the device PLM demo: the on-device model's screen output

Hardware Co-Design Scaling Laws NeurIPS 2026 & PLM Preprint

Pareto-optimal scaling laws that choose architectures under latency, memory, and energy constraints (Dancing in Fetters), PLM, a 1.8B peripheral language model co-designed with edge hardware, and RooflineBench for benchmarking on-device LLMs.

ContextPilot system overview MemoryCraft: token cost follows the turn count under harness-issued versus agent-issued retrieval

ContextPilot MLSys 2026 & MemoryCraft Under revision

Long-context agents, from inference to memory: ContextPilot cuts prefill cost through context reuse, ordering, and deduplication; MemoryCraft is a controlled platform for evaluating agent memory systems jointly with their retrieval regimes, backbones, and token cost.

GTA grouped-head latent attention compared with MHA, GQA, and MLA

GTA: Efficient Attention Preprint

Grouped-head latent attention: sharing attention maps across head groups and decoding values from a compact latent to cut attention FLOPs and KV-cache size at matched quality.

💬 NLP, Agents & AI for Science
GAKG (CIKM 2021), K2 (WSDM 2024), and GeoGalactica (AI4X 2024)

GeoGalactica / K2 / GAKG WSDM · AI4X · CIKM

First-generation LLM foundation models for science, covering data acquisition, pre-training, SFT, and RL.

DS-Agent case-based reasoning loop

DS-Agent ICML 2024

Automated data science with LLM agents: case-based reasoning as agent skills and memory in the era of 2024.

Uncertainty-based hallucination detection with keyword focus

Hallucination Detection EMNLP 2023

Uncertainty-based hallucination detection for LLMs with keyword focus and history-aware propagation, improving detection without extra supervision.

🤖 Efficient Physical AI & Robotics
Efficient VLA running on edge hardware

Efficient VLA CoRL 2026

A controlled, latency-aware study of modular VLA design. We identify where additional model capacity actually pays off, derive an efficient VLA recipe, and validate transfer from simulation to real robotic manipulation.

PhysicalAgent operations console: map, camera, and lidar during a navigation task PhysicalAgent: the mobile robot navigating the same task

PhysicalAgent Banbu

An embodied-agent testbed integrating VLA policies, SLAM/navigation, multimodal perception, and task-level agents on an untethered Jetson-powered mobile robot. We use it to study learned/classical skill composition, agent–executor feedback, and resource-aware autonomy in real environments.

Task-level Agent VLA Policy Vehicle / RobotPlatform Hardware co-design scaling law for on-device models

Efficient Embodied AI for Intelligent Vehicles and Robots Industry

VLA policies, agent–executor systems, and hardware-aware deployment for real-world vehicle and robot platforms through industry collaboration with Li Auto.

Platforms. PhysicalAgent mobile base: a self-built wheeled home robot I lead the team on, NVIDIA Jetson Orin on board, lidar, camera, and microphones, ROS 2 navigation, pretrained VLA plus agent loop, running untethered in a real home. Also deployed on Raspberry Pi and consumer phones (PLM), and in-vehicle edge hardware (Li Auto).

Selected Publications [Google Citation]

  1. Luoyang Sun, Guoyang Xia, Fengfa Li, Lei Ren, Xinyu Cui, Haifeng Zhang, Fangxiang Feng, Kaike Zhang, Kun Zhan, Xie Yan, Jun Wang, Cheng Deng*
    Conference on Robot Learning, 2026.
  2. Under Revision Agents · Systems MemoryCraft: How Retrieval Control Reshapes Agent Memory Performance and Cost
    Cheng Deng, Eve Sauvage, Danna Zheng, Wenyu Huang, Luo Mai, Mirella Lapata, Jeff Z. Pan
  3. Siting Wang, Xiaofeng Wang, Zheng Zhu, Minnan Pei, Xinyu Cui, Cheng Deng, Jian Zhao, Guan Huang, Haifeng Zhang, Jun Wang
  4. Luoyang Sun, Jiwen Jiang, Yifeng Ding, Fengfa Li, Yan Song, Haifeng Zhang, Jian Ying, Lei Ren, Kun Zhan, Wei Chen, Yan Xie, Cheng Deng*
    Conference on Neural Information Processing Systems, 2026.
  5. Yinsicheng Jiang, Yeqi Huang, Liang Cheng, Cheng Deng, Xuan Sun, Luo Mai
    International Conference on Machine Learning Systems, 2026.
  6. Zhen Bi, Xueshu Chen, Luoyang Sun, Yuhang Yao, Qing Shen, Jungang Lou, Cheng Deng*
    International Conference on Distributed Artificial Intelligence, 2026.
  7. Siting Wang, Luoyang Sun, Cheng Deng*, Kun Shao, Minnan Pei, Zheng Tian, Haifeng Zhang, Jun Wang
    International Conference on Learning Representations, 2026.
  8. Preprint Efficient ML GTA: Grouped-head latenT Attention
    Luoyang Sun, Cheng Deng, Jiwen Jiang, Xinjian Wu, Haifeng Zhang, Lei Chen, Lionel M. Ni, Jun Wang
  9. Cheng Deng*, Luoyang Sun, Jiwen Jiang, Yongcheng Zeng, Xinjian Wu, Wenxin Zhao, Qingfa Xiao, Jiachuan Wang, Haoyang Li, Lei Chen, Lionel M. Ni, Haifeng Zhang, Jun Wang
  10. Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, Jun Wang
    International Conference on Machine Learning
  11. Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, Luoyi Fu
    Conference on Empirical Methods in Natural Language Processing, 2023.
  12. Zhouhan Lin, Cheng Deng, Le Zhou, Tianhang Zhang, Yi Xu, Luoyi Fu, Weinan Zhang, Junxian He, Chao Ma, Yunqiang Zhu, Xinbing Wang, Chenghu Zhou, et al.
  13. Cheng Deng, Tianhang Zhang, Zhongmou He, Yi Xu, Qiyuan Chen, Yuanyuan Shi, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin, Junxian He
    WSDM, 2024
  14. CIKM2021 AI4Science · Knowledge Graphs GAKG: A Multimodal GeoScience Academic Knowledge Graph
    Cheng Deng, Yuting Jia, Weinan Zhang, Luoyi Fu, Xinbing Wang, Chenghu Zhou, et al.
    CIKM, 2021

* Corresponding author

Teaching

🏫 Courses
📘 Tutorials & Invited Lectures
Overview of the full-stack LLM training tutorial

Hands-on LLM: Full-Stack Training Tutorial · RLChina 2024

An end-to-end, runnable course on training a large language model: data curation, model setup, pre-training, post-training, and deployment, with complete scripts and notebooks.

Lifecycle of reward-guided learning for a physical agent: preparation, functional reuse and refinement, persistence and expansion

From Preference to Experience: Reward-Guided Learning for Physical Intelligence Invited talk · Oxford

An executable talk: how preference and reward signals post-train VLA policies, why flow-based policies need finer steps in online RL, and what transfers to real robots under hardware budgets. Oct 2026.

Accuracy per token across agent memory systems

Trust, Memory, and Reuse: Three Safety Primitives for Agent Systems Invited talk · Oxford

An executable talk on three primitives that make agent failures visible and controllable: trust in inputs and tools, memory evaluated with its retrieval regime, and reuse of context without reuse of errors. Oct 2026.

🎓 Teaching Assistant

Services

Funding

Selected Talks [All Talks]


Powered by Jekyll and Minimal Light theme.