Audio and Video Software Industry News: 2023 Q2 Review

AI and ML, low latency talks, and new features of video conferencing applications are the highlights of the 2023 spring season. While there were discussions if ChatGPT and WebRTC are considered to be like oil and water, research about A Subjective Dataset for Multi-Screen Video Streaming Applications, Forbes announced the best top five video conference applications of 2023. Apple launched the Apple Vision Pro – a mixed-reality headset that blends digital content with the physical world and transformed iPhone into meeting room cameras. Microsoft introduced an active speaker view and green screen in meetings. While different opinions were shared about hybrid work, Webex and Audi introduced their solution driving hybrid work forward to consumers of modern and dynamic lifestyles. A lot of things are going on and we hope that you will find useful and informative information that you are interested in.

Streaming Media East 18-19 May USA

Streaming Media East is the leading event for streaming video professionals that focuses on the future of digital media and online video. The event brings together industry professionals to learn, share knowledge, and explore disruptive trends in the field. Attendees can expect practical advice, inspiring thought leadership, and in-depth training on various topics such as live streaming, OTT (Over-The-Top) services, enterprise and educational video, encoding and transcoding, next-gen TV, packaging and formats, content delivery, advertising, and content monetization. Streaming Media University workshops, offering exclusive training from expert instructors, will also be available on May 17, along with the Content Delivery Summit 2023. Streaming Media East prides itself as an original and leading event in the streaming media industry, providing cutting-edge information and education to its participants. Here are some highlights of the covered topics:

  • Low-Latency for Live Streaming: The speaker, Robert Reinhardt, discusses low latency for live streaming and emphasizes the importance of understanding the “how” of low latency. Robert Reinhardt is a streaming solutions architect with over 20 years of experience, specializing in live-streaming events. He mentions that the demand for live streaming is increasing, and engagement with live audiences is a key focus. He provides a checklist for determining the required latency level, including factors such as the size of the audience and frequency of broadcasts. The speaker briefly mentions various technologies and protocols for low-latency streaming, including WebRTC, RTMP, HLS, DASH, and NDI. He also discusses different vendors and open-source options available in the market. Interview with Tom Siglin to discuss low-latency streaming and the edge in this interview from Streaming Media East 2023 can be watched here.
  • Choosing a Codec in 2023: Speakers mention that EVC, although a good codec, hasn’t gained commercial traction yet. The Moscow State University test shows H.264 as 145% as efficient than HEVC, X.265 requiring a 43% higher bit rate, AV1 being 35% more efficient than X.265, and VVC slightly outperforming AV1 and HEVC. Factors influencing codec selection include hardware support, compatibility, costs, deployment risks, and potential royalties. The choice should align with the publisher’s specific needs and goals. Implementing a new codec should be carefully considered, as the cost savings mainly benefit the top encoding tier. Limited device support may hinder the adoption of a new codec, and increased costs should be weighed against the benefits. Codec selection is driven by market expansion, technological advancements, and sometimes political motivations. Bandwidth savings alone should not be the sole determining factor, considering challenges like playback compatibility and DRM support. Hardware support and compatibility play crucial roles in codec selection, with HEVC enjoying higher mobile hardware support, except for exceptions like Meta’s support for AV1.
  • Large-Scale Streaming for the Metaverse: Building for a World of Active Virtual Participation: The growth of large-scale virtual streaming has grown exponentially during the pandemic period and has moved beyond esports and Twitch to include virtual platforms such as Roblox and Horizon Worlds. According to Future Market Insights, the esports and games streaming market will likely increase from $2.3 billion USD in 2022 to $7.2 billion USD in 2032. This panel discussion focuses on large-scale streaming for the metaverse and active virtual participation. The panelists discuss the challenges of ultra-low latency infrastructure for streaming across different platforms. They highlight the increasing feasibility of live streaming in 8K, which enhances the immersive experience. The scalability of events and concurrent users in the metaverse is also addressed, emphasizing the importance of interactivity. The conversation touches on the environmental impact of streaming and the need for optimized infrastructure to support efficient and immersive experiences in the metaverse.
  • Meeting the Live-to-VOD Challenge: The discussion is focused on the challenges of repackaging and distributing live content in various formats for different platforms. Four key individuals from different companies introduce themselves: Corey from LiveX, Ben from Boston 25 News, Cal Tao Chan from Alia, and Matthew Santiago from a Global Music Company. The difficulties discussed include capturing content in unique spaces, adapting to technical challenges, and the need for fast turnaround in delivering content for on-demand viewing. Subtitling and dubbing are mentioned as potential requirements for engaging diverse audiences, including those with regional accents or cultural nuances. Content repurposing, accessibility, and the potential for interactive features to enhance the viewing experience were highlighted as crucial factors in the evolving landscape of live and on-demand content.

The 15th International Conference on Quality of Multimedia Experience (QoMEX) 20-22 Jun 2023 Belgium

The 15th International Conference on Quality of Multimedia Experience (QoMEX) brings together leading experts from academia and industry to present and discuss current and future research on multimedia quality, Quality of Experience (QoE), and user experience (UX). QoMEX is held from June 20 to 22, 2023 in Ghent, Belgium. Topics of interest include audio/ visual user experience, new and diverse assessment and evaluation methods, databases for QoE research, etc. Here are some highlights from the event:

  • A Subjective Dataset for Multi-Screen Video Streaming Applications: The paper addresses the lack of open-source datasets that provide subjective scores for the same content when viewed on multiple devices (Mobile, TV, tablet) with different screen sizes, thus limiting a proper evaluation of the existing quality metrics for such multi-screen video streaming applications. It highlights the importance of understanding the differences in video perception across devices and the potential benefits of effectively modeling these differences in terms of better quality metrics and optimized encoding profiles. The authors present a new open-source dataset consisting of subjective ratings for various encoded video sequences of different resolutions and bitrates (quality) when viewed on three devices of varying screen sizes: TV, Tablet, and Mobile. Along with the subjective scores, an evaluation of some of the most famous and commonly used open-source objective quality metrics is also presented. It is observed that the performance of the metrics varies a lot across different device types, with the recently standardized ITU-T P.1204.3 Model, on average, outperforming their full-reference counterparts. The dataset, including the videos, subjective scores, and objective metric evaluations, is freely available on GitHub.
  • Video Quality Sufficiency for Sustainable Video Streaming: With the effects of climate change globally manifesting, all sectors of the economy and society are aiming to reduce their carbon emissions. For the Information and Communication Technology sector, the concept of sufficiency has been proposed as a paradigm to go beyond efficiency improvements toward an absolute reduction of carbon emissions. Yet little is known about the practical implementation of sufficiency and its carbon reduction potential. This study provides a concrete example of the user sufficiency sub-category by considering the quality-energy curves, in order to make tradeoffs between user experience and energy consumption measurable and actionable, and allowing to quantify the carbon reduction potential from this intervention. The study discusses the implementation of user sufficiency in design, such as allowing users to choose quality levels based on consumption contexts and providing information on carbon savings. Future work aims to develop a framework that considers the characteristics of the quality-rate curve for end-to-end optimization.

Video Quality Experts Group (VQEG) meeting 26-30 Jun 2023 USA

The Video Quality Experts Group (VQEG) is a group of experts from various backgrounds and affiliations, including participants from several internationally recognized organizations, working in the field of video quality assessment. VQEG Brings International Experts Together: industry, academia, government Organizations, the International Telecommunication Union (ITU), and other Standard-Developing Organizations (SDOs). VQEG face-to-face meetings are held approximately twice each year. The meeting style is a mixture of conference-style presentations and work related to active VQEG projects. Here are some topics discussed at the VQEG meeting and here’s the full schedule:

  • Panel Session on “Deep learning in video coding and video quality domain” Ioannis Katsavounidis (Meta)
  • Blind visual quality assessment for mobile/edge computing
  • Video Complexity Analysis Tool (VCAv2.0)
  • LS about new work item P.obj-recognition: Object-recognition-rate-estimation model in surveillance video of autonomous driving
  • LS about new work item P.SMAR: Subjective Tests for Evaluating the User Experience for Mobile AR Applications
  • Super Resolution Talk (Marcos V. Conde)
  • Learned Image Compression (Google)
  • Improving Netflix video quality with neural networks
  • Summary of NORM (no reference metrics) successes and discussion of current efforts for improved complexity metric
  • GAMIVAL: Video Quality Prediction on Mobile Cloud Gaming Content
  • Visual Quality Indicators Adapted to Resolution Changes (During Visual Quality Indicators Adapted to Resolution Changes presentation author presents visual quality indicators that have been adapted to accommodate changes in resolution ( full HD -4K Ultra HD, 8K Ultra HD). The selected indicators include blockiness, blur, exposure, contrast, interlacing, noise, slicing etc. Common resolutions ranging from 240p to 2160p are considered in the models. Specific 10-second sequences from various sources such as Cablelabs, Netflix, SJTU_4K, and Twitch are selected for evaluation.)

Other notable conferences and events:

Top Stories

Other Stories

For Developers

Microsoft

Agora

Digital Samba

RingCentral

Threema

TikTok

TrueConf

Vonage

WhatsApp

Webex

Zoom

Besides that, big players in audio and video conferencing introduced many new features, at the same time low latency and quality improvements were on top of discussion in the 2023 spring season. Let us sit back and see what the 2023 summer season brings us!

Linda Kleina
Linda Kleina
Articles: 1