01 / PRE-TRAINING & ARCHITECTURETechnical report
Youtu-LLM
First author. A 1.96B Dense MLA language model pre-trained from scratch, with 128K context and a Commonsense–STEM–Agent curriculum for reasoning and planning.
Explore the workTENCENT YOUTU LAB · SHANGHAI
Junru Lu陆俊如
Foundation Model Researcher
I work on foundation models, efficient long-context modeling, and agent memory.
Core R&D member of YouTu's LLM and MLLM team. Ph.D. in Computer Science from the University of Warwick, advised by Prof. Yulan He.
01 / RESEARCH
Selected work
Pre-training & architecture
Long context & agent memory.
02 / EFFICIENT LONG CONTEXTICML 2026
SSA
Second author. Bidirectional alignment of full and sparse attention outputs addresses training–inference mismatch and incomplete gradient flow in sparse training.
Explore the work03 / AGENT MEMORYEMNLP 2026
ContextPilot
Third author. Planning, long-term memory, and soft offloading keep the working context compact. Branch sampling at key actions assigns RL credit to context-management decisions.
Explore the workRESEARCH TO RELEASE
Open models & deployment
Weights and code for Youtu-LLM (first author) and Youtu-VL (co-author), alongside a team-released MoE base model. Youtu-Tip is an easy-to-use extension of Youtu-Agent, powered by Youtu-LLM / Youtu-VL, with fully offline on-device deployment.
BASE · INSTRUCT · GGUF · 128K
Youtu-LLM-2B
First author of Youtu-LLM. A 1.96B Dense MLA language model pre-trained from scratch with 128K context and a Commonsense–STEM–Agent curriculum. Public releases include Base / Instruct weights, a GGUF branch, and code; Youtu-Tip provides an offline deployment option.
INSTRUCT · GGUF
Youtu-VL-4B
Co-author of Youtu-VL. Unified vision–language autoregressive supervision learns to predict both visual and text tokens. Instruct weights, GGUF, and official code are public; the model also powers Youtu-Tip's offline multimodal deployment.
120B TOTAL · ~7B ACTIVE
Youtu-LLM-A7B-Base
A team-released multimodal MoE base model with approximately 120B total and 7B active parameters. A pre-trained ViT is paired with a randomly initialized language model, followed by 4K, 32K, 128K and 256K context stages over 11.5T multimodal tokens.
Recent updates
Public releases & researchOur co-authored paper PLatour is available as a preprint. It studies prefill-to-autoregressive cross-model transfer and uses PACT-Check to separate genuine communication gains from bridge adaptation. Code ↗ · Weights and data ↗
Our co-authored paper HarnessPro is available as a preprint and remains under review. It studies the performance–cost trade-off in agent harnesses. Code ↗
ContextPilot is available, accepted to EMNLP 2026 Main. Code ↗
We released Toward Native Multimodal Modeling: A Roadmap, covering architectures, training, data, deployment, and evaluation.
Youtu-VL introduces unified autoregressive supervision for visual and language tokens.
We released Youtu-LLM, a lightweight language model pre-trained from scratch.
WRITING / TECHNICAL ESSAYS
Ideas, beyond the paper.
大模型的记忆管理：从记忆外挂到主动管理
介绍记忆结构、检索策略和主动上下文管理的发展脉络。
Read on Hugging Face结合Deepseek代码探讨MLA的改进及收益
结合代码理解低秩投影、KV 缓存和矩阵吸收。
Read on Hugging Face大模型偏好优化技术：DPO及其变种
介绍 DPO 的原理、主要变体和实际应用。
Read on Hugging Face02 / PUBLICATIONS
Papers & technical reports
25 publications & technical reports
2026
On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer
2026
HarnessPro: Recursive Harness Self-Improvement with Pareto Optimization for Performance-Cost Trade-offs
2026
Self-Distillation with Verbal Regularization
2026
Blind Men and the Elephant: Probing the Epistemic Myopia of LLMs under Long-Tail Divergent Knowledge
2026
ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL
2026
Toward Native Multimodal Modeling: A Roadmap
2026
Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesis
2026
Youtu-VL: Unleashing Visual Potential via Unified Vision-Language Supervision
2026
SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space
2025
Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models
2025
OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization
2025
Two Heads Are Better Than One: Dual-Model Verbal Reflection at Inference-Time
2025
RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following
2025
Tell Me What You Don’t Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing
2025
LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Model
2025
FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema
2024
Advancing Video Anomaly Detection: A Bi-Directional Hybrid Framework for Enhanced Single- and Multi-Task Approaches
2024
SNFinLLM: Systematic and Nuanced Financial Domain Adaptation of Chinese Large Language Models
2024
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence
2024
FinVerse: An Autonomous Agent System for Versatile Financial Analysis
2023
MemoChat: Tuning LLMs to Use On-the-fly Text Memos for Consistent Long-Range Conversations
2023
Event Knowledge Incorporation with Posterior Regularization for Event-Centric Question Answering
2023
NapSS: Paragraph-level Medical Text Simplification via Narrative Prompting and Sentence-matching Summarization
2022
Event-Centric Question Answering via Contrastive Learning and Invertible Event Transformation
2020
CHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering
No matching publications. Try another title, author, or topic.
03 / BACKGROUND
Experience & education
EXPERIENCE
Tencent YouTu Lab
May 2023 – present · Shanghai, China
Foundation Model Researcher
Core R&D across model design, pre-training and evaluation.
- Data20TB+ text / 5TB+ multimodal data, quality control and synthesis.
- DesignDense/MoE decisions, sparse attention and native multimodal modeling.
- KernelsFused-kernel R&D and co-design of the new LightningMoBA kernels.
- TrainingLarge-scale pre-training, downstream metrics, model health and stability.
- EvaluationAutomated text, multimodal, long-context and agentic evaluation.
Model series · 2B to 640B
LLM2B / 4B / 30B-A3B
MoE MLLM32B-A4B / 120B-A7B / 640B-A31B
EDUCATION
- 2019–2024
- 2018–2019
New York University
M.S. in Applied Urban Science & Informatics
- 2015–2018
Peking University
Double Bachelor of Economics
- 2014–2018
University of International Relations
B.Eng. in Information Management and Information System
Advisor: Prof. Binyang Li
Earlier notes2017–2019
Earlier notes on NLP, question answering, algorithms, and learning.
Browse the archive