11 offres trouvées

Summary In this internship at the London AI Video Lab, the objective is to study fixed-point arithmetic solutions for ensuring bit-exact video compression in AI-based video codecs. Current AI-based video compression models outperform conventional codecs, like HEVC, VVC and AV1. However, AI-based video compression models are trained using floating-point arithmetic. Unfortunately, floating point arithmetic is insufficient to ensure bit-exact execution. Bit-exact execution is needed to ensure encoded bitstreams are universally decodable across any device. Fixed-point arithmetic is a potential solution to this problem. The goal of the internship is to determine a fixed-point arithmetic setup capable of ensuring bit-exactness while maintaining model performance. This work will be seen as one step forward toward the deployment of end-to-end trained AI-based video compression models. The goal will be to study various fixed-point arithmetic setups for layers and components of AI-based...

Summary In this internship at the London AI Video Lab, the objective is to design computationally efficient video decoders in an AI-based video compression codec. Current AI-based video compression models outperform conventional codecs, like HEVC, VVC and AV1. However, this comes at the cost of impractical compute requirements: at decode, current AI-based video compression decoders are several orders of magnitude more complex than conventional video compression decoders. The goal of the internship is to design efficient AI-based decoders that leverage spatial sparsity to reduce their computational complexity. This work will be seen as one step forward toward the deployment of end-to-end trained AI-based video compression models. The goal will be to find and review potential existing methods of spatial sparsity in AI-based video models. In a second step, spatially sparse AI-based decoders will be designed, implemented and integrated into the London AI Video Lab's end-to-end...

Summary Research-oriented internship on multi-user Quality of Experience (QoE) for XR application, the intern will survey the state of the art, identify and formalize QoE metrics specific to multi-user XR, and propose/enhance a composite QoE metric with corresponding instrumentation. The intern will design and run controlled experiments to collect metrics across concurrent users, analyze correlations with perceived experience, and iterate on the model. The work may culminate in planning and conducting user studies in the Rennes lab. Expected outcomes: a literature review, a measurement framework/prototype, practical recommendations for QoE-aware adaptation in multi-user XR, participation in a scientific paper, and potential disclosures. Responsibilities State of the art in QoE in XR, and Multi-User scenarios Propose or refine a composite QoE metric and map each factor to measurable KPIs and thresholds Design and implement instrumentation to capture interaction signals across...

Summary The InterDigital Immersive Video Lab is dedicated to advancing avatar-based communication. With the fast-paced technology and research advancements in view-dependent rendering such as the work done with neural radiance fields (NeRF) and more recently Gaussian splats, one can witness the improvement in visual quality of static and dynamic scenes including avatar models. This internship will contribute to the development of view-dependent realistic human faces. Responsibilities Develop a real-time animation platform based on facial animation using 3DGS technology and MPEG ARF standards Propose representation formats for 3DGS Avatar Representation using both traditional and advanced techniques e.g., mesh based, AI based and 3DGS based Keywords: Avatar, Animation, Neural Networks, 3D Gaussians Splats Expected Outcomes: Patents, Publications and Demos

Summary Short paragraph about the role, accountabilities, and any business until or department specific information The goal of the project is to explore video coding improvement using an additional reference picture generated with a deep neural network (Deep Learning) also known as Deep Reference Frame (DRF). Since a large part of network traffic consists of video content, image compression has become a key challenge to reduce the volume of transmitted data. The project context is VVC video compression. VVC is the latest video compression standard developed by the ISO/IEC and ITU-T standardization bodies. Recently, it has been proposed to use deep learning based image generation (DRF: Deep Reference Frame) to predict certain frames and reduce the bitrate of transmitted video content. Indeed, temporal frame interpolation using Deep Learning techniques is advancing rapidly. Responsibilities List primary responsibilities State-of-art of DRF and analysis of a particular...

Summary The InterDigital Immersive Video Lab is focused on advancing avatar-based communication. In this context, accurate and detailed human 3D data is essential for reliably estimating features such as facial expressions, body posture, skin texture, lighting conditions, background segmentation, hair segmentation, and more. InterDigital has developed a proprietary framework for generating synthetic 3D facial image datasets. The next step is to enhance this framework to support the creation of 4D video sequences featuring full-body characters. This internship will contribute to the development and refinement of this extended capability. Responsibilities The goal of the internship will be to: Study state of the art on face & body analysis models and synthetic data generation Update the framework to full body with body animation Upgrade the framework to output video sequences Create and/or set up avatar elements (clothes, accessories, jewels etc.) Generate video...

Summary In recent years, a variety of volumetric representations have been developed to support New View Generation (NVG)-the process of synthesizing novel perspectives of a scene from a limited set of captured views. Among these, 3D Gaussian Splatting (3DGS) has emerged as a leading technique, offering high-quality rendering of volumetric data with impressive realism and efficiency, marking a significant shift in how 3D scenes are represented. 3DGS models scenes as dense collections of 3D ellipsoidal primitives, each defined by a Gaussian function that supports anisotropic scaling, rotation, and stretching. These primitives also encode view-dependent color and opacity, allowing for nuanced and realistic rendering from multiple angles. However, this rendering quality comes at the cost of a large data footprint, which has sparked growing interest in efficient compression techniques, particularly within MPEG, where standardization efforts are actively progressing. In this context,...

Summary As an applied ML Multimedia Research Intern, you will work closely with researchers and engineers to address cutting-edge challenges in the field. You will receive dedicated technical mentorship and guidance, ensuring continuous learning and skill development throughout the internship. Each day will provide new opportunities to expand your technical expertise while deepening your understanding of the multimedia domain. Beyond technical growth, this internship will allow you to build valuable connections with domain experts and industry professionals. Depending on the outcomes of your work, you may also have the opportunity to contribute to publications, further advancing your academic and professional profile. Responsibilities List of primary responsibilities Survey the recent advances of state-of-the-art ML-based media compression technologies Conduct literature reviews and stay up to date with state-of-the-art techniques in ML-based media compression, multimedia...

Summary Multi-Modal Large Language Models (MLLMs) are increasingly capable of processing and reasoning over diverse input modalities such as text, images, audio, and video. However, running such models in real-time or resource-constrained environments poses significant challenges in terms of bandwidth and compute requirements. Distributing the processing of models between client and server (a "distributed computing" approach) is a promising solution. While traditional distributed inferencing has been applied successfully to DNN models, extending this paradigm to MLLMs is a novel and impactful use case. This internship aims to demonstrate the feasibility of distributed MLLMs inference approach, where MLLM components are distributed across two endpoints, coordinated through an agentic orchestration. Responsibilities The internship will be involved in the following tasks: Survey the recent advances in MLLMs, Select representative models, and a representative agentic...

Summary The System for Enhanced Media Representation team (SEMR) is developing a Unity 3D based platform to demonstrate our technologies regarding Multi-Users Quality of Experience (QoE) for eXtended Reality (XR) applications and test new functionalities. eXtended Reality (XR) is a general term encapsulating augmented, mixed and virtual reality (AR, MR and VR), with a strong emphasis on immersion and interactions between virtual and real world. Multi-Users experiences provide a shared environment for each participant. They rely on network communication to synchronize data and offer inter user interactivity. Quality of Experience (QoE) refers to the subjective perception of quality for a given experience. In the context of a network based XR application, this can be influenced by interactivity level, response time, spatial stability, graphics quality, etc. Maintaining high level of QoE even during network perturbations is an active field of study. During this 6-month...

Summary State-of-the-art video codecs use a very diverse range of coding tools on different parts of the video codec, such as filtering, intra or inter coding, transforms, entropy coding, as well as for different scenarios, such as natural video content, screen content, gaming content, etc. Also, the use of some tools created for screen content videos has proven to be effective on natural content videos. This internship is carried out in the R&I video coding team in Montreal. The student will get familiar with the state-of-the-art video codecs, will understand how screen content tools are used for video coding improvement. The student will analyze various of these tools that exist and work towards applying improved methods to the current and future video coding standards. The work will involve coding in C++ and a good background in digital signal processing is a must. Responsibilities Roles and responsibilities involve: familiarizing with the existing video coding standards...