Research Scientist Intern - Neural Representations Compression

In the rapidly evolving fields of AI, Augmented Reality (AR), and Virtual Reality (VR), industry compression research is shifting from human-consumed media to non-human-consumed internal neural model states. These representations include latent tensors, weights, Gaussian splats, feature maps, and depth/alpha buffers exchanged between neural networks. As high-capability models exceed on-device compute limits while cloud execution is constrained by latency, privacy, and power, systems are adopting split inference—partitioning model execution between devices and datacenters connected by neural bitstreams. In this architecture, the compression ratio on the split link serves as a primary determinant of round-trip latency, bandwidth efficiency, and battery performance.

Advancing compression techniques for neural representations expands beyond rendering outputs to critical non-media data streams. Key emergent areas include Feature Coding for Machines (FCM) for intermediate network features, visual and multimodal token pruning/compression, low-latency action and control streams for embodied systems, KV cache compression for distributed LLM prefill and decode stages, model deltas for on-device personalization, and persistent scene state streams for interactive world models. Establishing specialized compression capabilities across these representations directly impacts the operating efficiency and scalability of split-inference AI and XR applications.

The intern will explore several key directions in compression and streaming techniques for neural representations:
1. Latent Vector Compression: Developing efficient quantization, entropy coding, and learned prior models for compact, high-dimensional latent spaces exchanged in split-inference architectures.
2. Gaussian Splat Compression: Creating specialized compression and streaming algorithms that maintain the spatial distribution, opacity, and color parameters of 3D Gaussian splats with high rendering fidelity and low memory footprint.

Throughout the project, the intern will conduct a thorough review of existing literature, implement prototype models, and benchmark them against traditional compression methods. The final goal is to develop a neural compression framework that outperforms current techniques in terms of compression ratio, speed, and fidelity.

This research not only contributes to the academic field but also has direct applications in improving the operational efficiency of AR/VR systems, ultimately pushing the boundaries of what is possible in digital interaction spaces.

Responsibilities

Explore issues involved in processing, compressing and streaming neural and/or learned representations.
Identify, with assistance of the mentor(s), opportunities for significantly improving compression efficiency and/or quality.
Work with peers to build a working prototype of the proposed algorithm.
Communicate ongoing research progress and final results.

Qualifications

Currently has, or is in the process of obtaining, a PhD degree in the field of Image and Video Processing, Signal Processing, Information Theory, Machine Learning, Computer Graphics, or related field
Coding experience in C/C++ and/or Python
Experience in image and video processing
Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as first-authored publications at leading workshops or conferences such as ICIP, DCC (Data Compression Conference), CVPR, IEEE Transactions or Letters, ACM SIGGRAPH, etc Intent to return to a degree-program after the completion of the internship/co-op
Knowledgeable about applications related to neural networks and compression
Knowledgeable about existing image and video codec standards (H.26x, VPx, JPEG, etc)
Knowledgeable about existing Point Cloud compression standards (MPEG PCC)
Knowledgeable about Learning-based compression