High Dynamic Range (HDR) technology has taken the world of visual entertainment, especially streaming media solutions, by storm. It’s the secret sauce that makes images and videos look incredibly lifelike and captivating. From the vibrant colors in your favourite movies to the dazzling graphics in video games, HDR has revolutionized how we perceive visuals on screens. In this blog, we’ll take you on a technical journey into the heart of HDR, focusing on one of its most popular formats – HDR10. “Breaking down all the complex technical details into simple terms, this blog aims to help readers understand how HDR10 seamlessly integrates into streaming media solutions, working its magic.

What is High Dynamic Range 10 (HDR10)?

HDR 10 is the most popular and widely used HDR standard for consuming digital content. Every TV that is HDR enabled is compatible with HDR10. In the context of video, it primarily provides a significantly enhanced visual experience compared to standard dynamic range (SDR).

Standard Dynamic Range:

People have experienced visuals through SDR for a long time, which has a limited dynamic range. This limitation means SDR cannot capture the full range of brightness and contrast perceivable by the human eye. One can consider it the ‘old way’ of watching movies and TV shows. However, the discovery of HDR has changed streaming media by offering a much wider dynamic range, resulting in visuals that were more vivid and lifelike.

Understanding the basic visual concepts

Luminance and brightness: Luminance plays a pivotal role in our perception of contrast and detail in image processing. Higher luminance levels result in objects appearing brighter and contribute to the creation of striking highlights and deep shadows in HDR content. Luminance, measured in units called “nits,” is a scientific measurement of brightness. In contrast, brightness, in the context of how we perceive it, is a subjective experience influenced by individual factors and environmental conditions. It is how we interpret the intensity of light emitted or reflected by an object. It can vary from person to person.

Luminance comparison between SDR and HDR

Color-depth (bit-depth): Color-depth, often referred to as bit-depth, is a fundamental concept in digital imaging and video solutions. It determines the richness and accuracy of colors in digital content. This metric is quantified in bits per channel, effectively dictating the number of distinct colors that can be accurately represented for each channel. Common bit depths include 8-bit, 10-bit, and 12-bit per channel. Higher bit depths allow for smoother color transitions and reduce color banding, making them crucial in applications like photography, video editing, and other video solutions. However, it’s important to note that higher color depths lead to larger file sizes due to the storage of more color information.

Possible colors with SDR and HDR Bit-depth

Color-space: Color-space is a pivotal concept in image processing, display technology, and video solutions. It defines a specific set of colors that can be accurately represented and manipulated within a digital system. This ensures consistency and accuracy in how colors are displayed, recorded, and interpreted across different devices and platforms. Technically, it describes how an array of pixel values should be displayed on a screen, including information about pixel value storage within a file, the range, and the meaning of those values. Color spaces are essential for faithfully reproducing a wide range of colors, from the deep blues to the rich red colors, resulting in visuals that are more vibrant and truer to life. A color space is akin to a palette of colors available for use and is defined by a range of color primaries represented as points within a three-dimensional color space diagram. These color primaries determine the spectrum of colors that can be created within that color space. A broader color gamut includes a wider range of colors, while a narrower one offers a more limited selection. Various color spaces are standardized to ensure compatibility across different devices and platforms.

CIE Chromaticity Diagram representing Rec.709 vs Rev2020

Dynamic range: Dynamic range relates to the contrast between the highest and lowest values that a specific quantity can include. This idea is commonly used in the domains of signals, which include sound and light. In the context of images, dynamic range determines how the brightest and darkest elements appear within a picture and the extent to which a camera or film can handle varying levels of light. Furthermore, it greatly affects how different aspects appear in a developed photograph, impacting the interplay between brightness and darkness. Imagine dynamic range as a scale, stretching from the soft glow of a candlelit room to the brilliance of a sunlit day. In simpler terms, dynamic range allows us to notice fine details in shadows and the brilliance of well-lit scenes in both videos and images.

Dynamic Range supported by SDR and HDR Displays

Difference between HDR and SDR

Aspect	HDR	SDR
Luminance	Offers a broader luminance range, resulting in brighter highlights and deeper black for more lifelike visuals.	Limited luminance range can lead to less dazzling bright areas and shallower dark scenes.
Color depth	Provides a 10-bit color depth per channel, allowing finer color gradations and smoother transitions between colors.	Offers a lower color depth, resulting in fewer color gradations and potential color banding.
Color space	Incorporates a wider color gamut like BT.2020, reproducing more vivid and lifelike colors.	Typically uses the narrower BT.709 color space, offering a more limited color range.
Transfer function	Utilizes the perceptual quantizer (PQ) as a transfer function, accurately representing luminance levels from 10,000 cd/m^2 down to 0.0001 nits.	Relies on a gamma curve for transfer function, which may not accurately represent extreme luminance levels.

Metadata in HDR10
HDR10 utilizes the PQ EOTF, BT2020 WCG, and ST2086 + Max FALL + Max CLL static metadata. The HDR10 metadata structure follows the ITU Series H Supplement 18 standard for HDR and Wide Color Gamut (WCG). There are three HDR10-related Video Usability Information (VUI) parameters: color primaries, transfer characteristics, and matrix coefficients. This VUI metadata is contained in the Sequence Parameter Set (SPS) of the intra-coded frames.

In addition to the VUI parameters, there are two HDR10-related Supplemental Enhancement Information (SEI) messages. Mastering Display Color Volume (MDCV) and Content Light Level (CLL).

• Mastering Display Color Volume (MDCV):
  MDCV or “Mastering Display Color Volume” is indeed an important piece of metadata within the HDR10 standard, also known as ST2086. This metadata plays a significant role in ensuring that HDR content is displayed optimally on different HDR-compatible screens.
• Max Content Light Level (MaxCLL):
  MaxCLL specifies the maximum brightness level in nits (cd/m²) for any individual frame or scene within the content. It helps your display adjust its settings for specific, exceptionally bright moments.
• Max Frame-Average Light Level (MaxFALL):
  MaxFALL indicates the maximum frame-average brightness level in nits across the entire content, including the brightest frames. It ensures that your display can correctly reproduce the content’s overall brightness. MaxFALL complements MaxCLL by indicating the maximum frame-average brightness level across the entire content. It prevents excessive dimming or over-brightening, creating a consistent and immersive viewing experience.
• Transfer function (EOTF – electro-optical transfer function):
  The EOTF, often based on the ST-2084 PQ curve, dictates how luminance values are encoded in the content and decoded by your display. It ensures that brightness levels are presented accurately on your screen. EOTF defines how luminance values are encoded in the content and decoded by your display.

Sample HDR10 metadata parsed using ffprobe:

Future of HDR10 & competing HDR formats

The effectiveness of HDR10 implementation is closely tied to the quality of the TV used for viewing. When applied correctly, HDR10 enhances the visual appeal of video content. However, there are other HDR formats gaining popularity, such as HDR10+, HLG (Hybrid Log-Gamma), and Dolby Vision. These formats have gained prominence due to their ability to further enhance the visual quality of videos.
Competing HDR formats, like Dolby Vision and HDR10+, are gaining popularity due to their utilization of dynamic metadata. Unlike HDR10, which relies on static metadata for the entire content, these formats adjust brightness and color information on a scene-by-scene or even frame-by-frame basis. This dynamic metadata approach delivers heightened precision and optimization for each scene, ultimately enhancing the viewing experience. The rivalry among HDR formats is fueling innovation in the HDR landscape as each format strives to surpass the other in terms of visual quality and compatibility. This ongoing competition may lead to the emergence of new technologies and standards, further expanding the possibilities of what HDR can achieve.
To sum it up, HDR10 isn’t just a buzzword, it’s a revolution in how we experience visuals in any multimedia solution. It’s the technology that takes your screen from good to mind-blowingly fantastic. HDR10 is very popular because there are no licensing fees (compared to other HDR standards) and is widely adopted by many companies and there are lot of equipment out there already. So, whether you’re a movie buff, gamer, or just someone who appreciates the beauty of visuals, HDR10 is your backstage pass to a world of incredible imagery.
With continuous advancements in technology, we at Softnautics, a MosChip Company, help businesses across various industries to provide intelligent media solutions involving the simplest to the most complex multimedia technologies. We have hands-on experience in designing high-performance media applications, architecting complete video pipelines, audio/video codecs engineering, audio/video driver development, and multimedia framework integration. Our multimedia engineering services are extended across industries ranging from Media and entertainment, Automotive, Gaming, Consumer Electronics, Security, and Surveillance.

Read our success stories related to intelligent media solutions to know more about our multimedia engineering services.

Contact us at business@softnautics.com for any queries related to your solution or for consultancy

[elementor-template id=”14177″]

In the massive world of multimedia, sound stands as a vital component that adds depth to the overall encounter. Whether it’s streaming services, video games, or virtual reality, sound holds a crucial role in crafting immersive and captivating content. Nevertheless, ensuring top-notch audio quality comes with its own set of challenges. This is where audio validation enters the scene. Audio validation involves a series of comprehensive tests and assessments to guarantee that the sound in multimedia systems matches the desired standards of accuracy and quality. Market Research Future predicts that the consumer audio market sector will expand from a valuation of USD 82.1 billion in 2023 to approximately USD 274.8 billion by 2032. This growth trajectory indicates a Compound Annual Growth Rate (CAGR) of about 16.30% within the forecast duration of 2023 to 2032.

What is Audio?

Audio is an encompassing term that refers to any sound or noise perceptible to the human ear, arising from vibrations or waves at frequencies within the range of 20 to 20,000 Hz. These frequencies form the canvas upon which the symphony of life is painted, encompassing the gentlest whispers to the most vibrant melodies, weaving a sonic tapestry that enriches our auditory experiences and connects us to the vibrancy of our surroundings.

There are two types of audio.

• Analog audio
  Analog audio refers to the representation and transmission of sound as continuous, fluctuating electrical voltage or current signals. These signals directly mirror the variations in air pressure caused by sound waves, making them analogous to the original acoustic translating the “analogous” nature of sound into electrical signals. When recorded in an analog format, what is heard corresponds directly to what is stored, maintaining the continuous waveforms but amplitude can differ which is measured in decibels (dB).

• Digital audio
  Digital audio is at the core of modern audio solutions. It represents sound in a digital format, allowing audio signals to be transformed into numerical data that can be stored, processed, and transmitted by computers and digital devices. Unlike analog audio, which directly records sound wave fluctuations, digital audio relies on a process known as analog-to-digital conversion (ADC) to convert the continuous analog waveform into discrete values. These values, or samples, are then stored as binary data, enabling precise reproduction and manipulation of sound. Overall, digital audio offers advantages such as ease of storage, replication, and manipulation, making it the foundation of modern communication systems and multimedia technology.

Basic measurable parameters of audio

Frequency

Frequency, a fundamental concept in sound, measures the number of waves passing a fixed point in a specific unit of time. Typically measured in Hertz (Hz), it represents the rhythm of sound. Mathematically, frequency (f) is inversely proportional to the time period (T) of one wave, expressed as f = 1/T.

Sample rate

Sample rate refers to the number of digital samples captured per second to represent an audio waveform. Measured in Hz, it dictates the accuracy of audio reproduction. For instance, a sample rate of 44.1kHz means that 44,100 samples are taken each second, enabling the digital representation of the original sound. Different audio sample rates are 8kHz, 16kHz, 24kHz, 48kHz, etc.

Bit depth or word size

Bit depth, also known as word size, signifies the number of bits present in each audio sample. This parameter determines the precision with which audio is represented digitally. A higher bit depth allows for a finer representation of the sound’s amplitude and nuances. Common options include 8-bit, 16-bit, 24-bit, and 32-bit depths.

Decibels (dB)

Decibels (dB) are logarithmic units employed to measure the intensity of sound or the ratio of a sound’s power to a reference level. This unit allows us to express the dynamic range of sound, spanning from the faintest whispers to the loudest roars.
Amplitude

Amplitude relates to the magnitude or level of a signal. In the context of audio, it directly affects the volume of sound. A higher amplitude translates to a louder sound, while a lower amplitude yields a softer tone. Amplitude shapes the auditory experience, from delicate harmonies to thunderous crescendos.

Root mean square (RMS) power

RMS power is a vital metric that measures amplitude in terms of its equivalent average power content, regardless of the shape of the waveform. It helps to quantify the energy carried by an audio signal and is particularly useful for comparing signals with varying amplitudes.

General terms used in an audio test

Silence

It denotes the complete absence of any audible sound. It is characterized by a flat line on the waveform, signifying zero amplitude. This void of sound serves as a stark contrast to the rich tapestry of auditory experiences.

Echo

An echo is an auditory effect that involves the repetitive playback of a selected audio, each iteration softer than the previous one. This phenomenon is achieved by introducing a fixed delay time between each repetition. The absence of pauses between echoes creates a captivating reverberation effect, commonly encountered in natural environments and digital audio manipulation.

Clipping

Clipping is a form of distortion that emerges when audio exceeds its dynamic range, often due to excessive loudness. When waveforms surpass the 0 dB limit, their peaks are flattened at this ceiling. This abrupt truncation not only results in a characteristic flat top but also alters the waveform’s frequency content, potentially introducing unintended harmonics.

DC-Offset

It is an alteration of a signal’s baseline from its zero point. In the waveform view, this shift is observed as the signal not being centred on the 0.0 horizontal line. This offset can lead to distortion and affect subsequent processing stages, warranting careful consideration in audio manipulation.

Gain

Gain signifies the ratio of output signal power to input signal power. It quantifies how much a signal is amplified, contributing to variations in its amplitude. Expressed in decibels (dB), positive gain amplifies the signal’s intensity, while negative gain reduces it, influencing the overall loudness and dynamics.

Harmonics

Harmonics are spectral components that occur at exact integer multiples of a fundamental frequency. These multiples contribute to the timbre and character of a sound, giving rise to musical richness and complexity. The interplay of harmonics forms the basis of musical instruments’ distinct voices.

Frequency response

Frequency response offers a visual depiction of how accurately an audio component reproduces sound across the audible frequency range. Represented as a line graph, it showcases the device’s output amplitude (in dB) against frequency (in Hz). This curve provides insights into how well the device captures the intricate nuances of sound.

Amplitude response

It measures the gain or loss of a signal as it traverses an audio system. This measure is depicted on the frequency response curve, showcasing the signal’s level in decibels (dB). The amplitude response unveils the system’s ability to faithfully transmit sound without distortion or alteration.

Types of testing performed for audio

Signal-to-noise ratio (SNR)

Testing the Signal-to-noise ratio (SNR) is a fundamental step in audio validation. It assesses the differentiation between the desired audio signal and the surrounding background noise, serving as a crucial metric in audio quality evaluation. SNR quantifies audio fidelity and clarity by calculating the ratio of signal power to noise power, typically expressed in decibels (dB). Higher SNR values signify a cleaner and more comprehensible auditory experience, indicating that the desired signal stands out prominently from the background noise. This vital audio parameter can be tested using specialized equipment (like audio analyzers and analog-to-digital converters) and software tools, ensuring that audio systems deliver optimal clarity and quality.

Latency

Latency refers to the time delay between initiating an audio input and its corresponding output, plays a pivotal role in audio applications where synchronization and responsiveness are critical, like live performances or interactive systems. Achieving minimal latency, often measured in milliseconds, is paramount for ensuring harmony between user actions and audio responses. Rigorous latency testing, employing methods such as hardware and software measurements, round-trip tests, real-time monitoring, and buffer size adjustments, is essential. Additionally, optimizing both software and hardware components for low-latency performance and conducting tests in real-world scenarios are crucial steps. These efforts guarantee that audio responses remain perfectly aligned with user interactions, enhancing the overall experience in various applications.

Audio synchronization

It is a fundamental element that harmonizes the outputs of various audio channels. This test ensures that multiple audio sources, such as those in surround sound setups, are precisely aligned in terms of timing and phase. The goal is to eliminate dissonance or disjointedness among the channels, creating a unified and immersive audio experience. By validation synchronization, audio engineers ensure that listeners are enveloped in a seamless soundscape where every channel works in concert.

Apart from the above tests we also need to validate audio according to the algorithm used for audio processing.

Audio test setup

Now let us see the types of audio distortions.

Phase Distortion

Phase distortion is a critical consideration when dealing with audio signals. It measures the phase shift between input and output signals in audio equipment. To control phase distortion, it’s essential to use high-quality components in audio equipment and ensure proper signal routing.

Clipping Distortion

Clipping distortion is another type of distortion that can degrade audio quality. This distortion occurs when the signal exceeds the maximum amplitude that a system can handle. To prevent clipping distortion, it’s important to implement a limiter or compressor in the audio chain. These tools can control signal peaks, preventing them from exceeding the clipping threshold. Additionally, adjusting input levels to ensure they stay within the system’s operational range is crucial for managing and mitigating clipping distortion.

Harmonic Distortion

Harmonic distortion introduces unwanted harmonics into audio signals, which can negatively impact audio quality. These harmonics can be odd or even, with “odd” harmonics having frequencies that are an odd number of times higher than the fundamental frequency, and even harmonics having an even number of times higher frequency. To mitigate harmonic distortion, it’s advisable to use high-quality amplifiers and speakers that produce fewer harmonic distortions.

The commonly used test file will have sine tone, sine sweep, pink noise, and white noise.

There are different tools to create, modify, play or analyze audio files. Below are few of them.

Adobe Audition

Adobe Audition is a comprehensive toolset that includes multitrack, waveform, and spectral display for creating, mixing, editing, and restoring audio content.

Audacity
Audacity is a free and open-source digital audio editor and recording application software, available for Windows, macOS, Linux, and other Unix-like operating systems.

There are many devices nowadays involving the application of audio. Devices are headphones, soundbars speakers, earbuds, or devices with audio processors to process different audio algorithms (i.e., Noise cancellation, Voice wake, ANC, etc.).

At Softnautics, a MosChip company, we understand the importance of audio validation in multimedia systems. Our team of media experts enable businesses to design and develop multimedia systems and solutions involving media infotainment systems, audio/video solutions, media streaming, camera-enabled applications, immersive solutions, and more on diverse architectures and platforms including multi-core ARM, DSP, GPUs, and FPGAs. Our multimedia engineering services are extended across industries ranging from Gaming & Entertainment, Automotive, Security, and Surveillance.

Read our success stories related to multimedia engineering to know more about our services.

Contact us at business@softnatics.com for any queries related to your media solution or for consultancy.

[elementor-template id=”13959″]

Embedded Vision and Inferencing are two critical technologies for many modern devices such as drones, autonomous cars, industrial robots, etc. Embedded vision uses computer vision to process images, video, and other visual data, while inferencing is the process of making decisions based on collected data without having to explicitly program each decision step with the help of diverse architectures and platforms including multi-core ARM, DSP, GPUs, and FPGAs to provide a comprehensive foundation for developing multimedia systems. Open standards are essential to the development of interoperable systems, and they allow for transparency in the development process while providing a level playing field for all participants.

The global market for embedded vision and inferencing was valued at USD 11.22 billion in 2021, with a compound annual growth rate (CAGR) of around 7% from 2022 to 2030. It’s worth noting that the market share for embedded vision and inferencing is likely to continue evolving rapidly as these technologies are becoming increasingly pivotal for next-gen solutions across industries.

Adoption of Open Standards

Open Standards have been widely adopted by the industry, with many companies adopting them as their default. The benefits of open standards are numerous and include:

• Reduced cost for development and integration
• Increased interoperability between systems and components
• Reduced time to market

Open Standards for Embedded Vision

Open standards for embedded vision are a critical component of the Internet of Things (IoT). They provide a common language that allows devices to communicate with each other, regardless of manufacturer or operating system. Open standards for embedded vision include OpenCV, OpenVX, and OpenCL.

OpenCV is a popular open-source computer vision library that includes over 2,500 optimized algorithms for image and video analysis. It is used in applications such as object recognition, facial recognition, and motion tracking. OpenVX is an open-standard API for computer vision that enables performance and power-optimized implementation, reducing development time and cost. It provides a common set of high-level abstractions for applications to access hardware acceleration for vision processing. OpenCL is an open standard for parallel programming across CPUs, GPUs, and other processors that provides a unified programming model for developing high-performance applications. It enables developers to write code that can run on a variety of devices, making it a popular choice for embedded vision applications.

Open Standards for Embedded Vision & Inferencing

Open Standards for Inferencing

Open standards for inferencing are also essential for the development of intelligent systems. One of the most important open standards is the Open Neural Network Exchange (ONNX), which describes how deep learning models can be exported and imported between frameworks. It is currently supported by major frameworks like TensorFlow, PyTorch, and Caffe2.

ONNX enables interoperability between deep learning frameworks and inference engines, which is critical for the development of intelligent systems that can make decisions based on collected data. It provides a common format for representing deep learning models, making it easier for developers to build and deploy models across different platforms and devices.

Another important open standard for inferencing is the Neural Network Exchange Format (NNEF), which enables interoperability between deep learning frameworks and inference engines. It provides a common format for deploying and executing trained neural networks on various devices. It allows developers to build models using their framework of choice and deploy them to a variety of devices, making it easier to build intelligent systems that can make decisions based on collected data.

Future of Open Standards

The future of open standards for embedded vision and inferencing is bright, but there are challenges ahead. One of the biggest challenges is the lack of support for open-source software in embedded systems. This means that open-source libraries and frameworks cannot be used on devices with limited memory and processing power. Another challenge is the wide variety of processors and operating systems available, making it difficult to create a standard that works across all devices. However, there are initiatives underway like hardware innovation and algorithm optimization to address these challenges.

Industry Impact of Open Standards

Open standards have a significant impact on the industry and consumers. The use of open standards has enabled an ecosystem to develop around machine learning and deep learning algorithms, which is essential for innovation in this space. Open-source software has been instrumental in accelerating the adoption of AI across industries, including automotive, consumer, financial services, manufacturing, and healthcare.

Open standards also have a direct impact on consumers by lowering costs, increasing interoperability, and improving security across devices. For example, standards enable companies to build products using fewer components than proprietary solutions require, reducing costs for manufacturers and end-users who purchase products with embedded vision technology built-in (e.g., cameras).

Open Standards are the key to unlocking the full potential of embedded vision and inferencing. They allow developers to focus on their applications, rather than on the underlying hardware or software platforms. Open standards also provide a level playing field for all types of companies – from startup to large enterprises – to compete in this growing market. Overall, open standards are crucial for unlocking the full potential of embedded vision and inferencing for designing next-gen solutions across various industries.

At Softnautics, we help business across various industries to design embedded multimedia solutions based on next-gen technologies like computer vision, deep learning, cognitive computing and more. We also have hands-on experience in designing high-performance media applications, architect complete video pipelines, audio/video codecs engineering, applications porting, ML model design, train, optimize, test, and deploy.

Read our success stories related to intelligent media solutions to know more about our multimedia engineering services.

Contact us at business@softnautics.com for your solution design or consultancy.

[elementor-template id=”13562″]

In recent years, the development of video analytics based video solutions have come up as a high-end technology that has changed the way we interpret and analyze video data. Video analytics uses the most advanced algorithms and artificial intelligence (AI) to track the behaviour and understand the data in real-time, allowing to automate necessary actions. This technology has found many applications across different industries, providing valuable insights, intensifying security, improving the safety and optimizing operations. According to Verified Market Research group, video analytics is experiencing rapid market growth, with its global market value to reach USD 35.88 billion by 2030, representing a CAGR of 21.5% from its valuation of USD 5.65 billion in 2021. This growth trend highlights the increasing demand for video analytics solutions as organizations seek to enhance their security and surveillance systems. 

Video analytics is closely related to video processing which is an essential part of any multimedia solution, as it involves the extraction of meaningful insights and information from video data using various computational techniques. Video analytics leverages the capabilities of video processing to analyze and interpret video content, enabling computers to automatically detect, track, and understand objects, events, and patterns within video streams. Video processing techniques are used in a wide range of applications, including surveillance, video streaming, multimedia communication, autonomous vehicles, medical imaging, entertainment, virtual reality, and many more.

In this article, we will see some Industrial applications and use cases of video analytics in different areas.

Industrial Applications of Video Analytics
Automotive
One of the most used and inescapable uses of video analytics is in the automotive industry for Advanced Driver Assistance System (ADAS) in highly-automate vehicles (HAVs). The HAVs use multiple cameras to identify pedestrians, traffic signals, other vehicles, lanes, and other indicators, they are integrated with the ECU and programmed in such a way as to identify the real-time situation and then respond accordingly. Automating this process, requires integration of various system on a chip (SoC). These chipsets help actuators to connect with the sensors through interface and ECUs. It analyses the data with deep learning based machine learning (ML) models that uses neural networks to learn patterns in data. Neural networks are structured with layers of interconnected processing nodes, typically comprising multiple layers. This deep learning algorithms are used to detect and track objects in real-time videos, as well as to recognize specific actions. 

Sports
In the sports industry, video analytics is being utilized by coaches, personal trainers, and professional athletes to optimize performance through data-driven insights. In sports such as rugby and soccer, tracking metrics like ball possession and the number of passes has become a standard practice for understanding game patterns and team performance. Detailed research on a soccer game has shown that analyzing ball possession can even impact the outcome of a match. Video analytics can be used to gain insights into the playing style, strategies, passing patterns, and weaknesses of the opponent team, enabling a better understanding of their gameplay.

Private, public or hybrid, cloud solutions for any business domain are designed to provide the freedom to grow and security for the organization and customer data. For cloud-based multimedia solutions, there is cloud-based custom transcoder IP that supports automated Video-On-Demand (VOD) pipelines. Cloud services offer solutions that ingest source videos, processes video for playback on a wide range of devices using cloud media converter, and store transcoded media files for on-demand delivery to end-users.

Custom IP integration along with other cloud services showcases better feasibility of using Open-Source codec, to use one’s transcoder instead of cloud media-converter for multimedia solutions. In this blog, we will see how an Open-Source codec like AV1 is selected as a custom IP for encoding to integrate over the cloud as a service.

Thus, the video files uploaded on the cloud can be encoded with AV1 codec, without using cloud media-converter service. The solution is automated in such a way that the content provider just needs to upload video on cloud input file storage service and the further encoding happens automatically. It stores the content on cloud storage services after completion and the end-user gets notified about content availability.

Modules’ usage

Local PC can be used to upload input video on target AWS S3 bucket and EC2 instance is used to transcode input video into AV1 codec output. Encoding can be done through FFmpeg as well GStreamer, here FFmpeg is used considering the strong support community and extra features available. EC2 cloud instance can be used on any Linux-based system server. Further, the S3 cloud output file link is integrated into AWS Sumerian, to view it using VR set in 3D scene mode.

To overcome the limitations of cloud media converter, one can have own custom IP i.e. Transcoder solution, which can be used along with other cloud services. It will make faster the encoding or provide the same speed as of cloud media converter with reduced cost per encoding job, as compared to a cloud media converter. It is also easy to integrate any codec and provides a choice of multiple encoders per codec.

Benefits of using AOMedia Video 1 (AV1) codec

• It is an Open-Source, royalty-free video coding format for video transmissions over the Internet
• AV1 Quality and Efficiency: Based on measurements with PSNR and VMAF at 720p, AV1 was about 25% more efficient than VP9 (libvpx). Similar conclusions with respect to quality were drawn from a test conducted by Moscow State University researchers, where it was found that VP9 requires 31% and HEVC 22% more bitrate, than AV1 for the same level of quality
• Comparing AV1 against H.264 (x264) and VP9 (libvpx), Facebook showed about 45-50% bitrate savings using AV1 over H.264 and about 40% over VP9, when using a constant quality encoding mode

At Softnautics, we have incorporated multimedia solutions having market trending features like image overlay, timecode burn-in, bitrate control mode, advertise, rotation, motion image overlay, sub-title, cropping and more. Such features are required to build solutions like end-to-end pipeline orchestration, live and recorded streaming (VOD), transcoding, cloud services, Content Delivery Network (CDN) integration and interactive VR scene creation.

Flow Diagram:

 

Cloud-based-solutions

In the flow diagram of a Virtual Reality solution, the user uploads the video to the Watch Folder of the bucket in AWS S3. The multipart upload complete event will trigger the lambda function, which starts the EC2 instance. Encoding will then be performed through FFmpeg to encode output with the AV1 codec. If encoding is successful, then only the encoded file will be uploaded to the “output” directory in the AWS S3 bucket. If encoding is failed, then the input media file will be deleted from the “input” directory of AWS S3. The content provider will receive an email notification for failure or success of encoding job, using AWS SNS service. AWS SNS will trigger further AWS Lambda function and Lambda will stop the AWS EC2 instance. Lambda will also check whether the trigger is for output file upload or not, if yes, then it will send an email notification to the end user, using AWS SES service, to notify new content’s availability. Further AWS S3 output file link can be integrated into AWS Sumerian, to view it using VR set, in 3D scene mode. Python3 can be used for entire automation scripts.

Using cloud media custom IP-based solution services, one can stream videos to end-users at scale, deliver low-latency content, secure videos from unexpected downloads, remove the complexity of building development steps manually, and construct solutions in own environment for demo purposes.

Softnautics can help media companies design multimedia solutions across various platforms, merging physical reality and digital information in innovative ways, using advanced technologies. Softnautics multimedia experts have experience working on Augmented Reality, Virtual Reality, AV codecs development, image/video analytics, computer vision, image processing, and more.

Read our success stories related to Machine Learning expertise to know more about our services for accelerated AI solutions.

Contact us at business@softnautics.com for any queries related to your solution or for consultancy.

[elementor-template id=”12003″]