Given that the H.266 codec officially became a standard in 2020 and that businesses are already collaborating to look for monetisation potential, it is important to review this technology.
Versatile Video Coding, also referred to as VVC and H.266, is a video compression standard. The VVC codec is a block-based composite codec, making it extremely sophisticated and powerful. The Joint Video Experts Team (JVET) of the International Telecommunication Union developed this standard. JVET began developing VVC codecs in 2015 and finished the final standard in July 2020.
Dolcera discussed the contributors’ insights in the development of the codec in an article in 2021. However, giving a quick recap, VVC facilitates 8K resolutions and has been enhanced for cutting-edge uses such as:
Immersive video;
Augmented reality and virtual reality;
Videoconferencing;
360-degree videos;
Streaming media; and
Gaming.
The standard is geared to provide a desirable level of performance for new video services over 5G networks because it has the best compression efficiency available. For the same perceptual quality, VVC's compression rate is up to 50% better than High Efficiency Video Coding (HEVC) and it supports lossless and subjectively lossless compression. The codec is expected to benefit the players across the value chain.
This article will look into the patenting activity of the contributors and the non-contributing players for VVC.
VVC patent landscape
The following approach has been used to arrive at the finer patenting insights related to VVC:
As with the development of any standard, the codec was formed by mutual agreement to ensure seamless interoperability. HEVC, or H.265, was standardised in 2013 and the committee finished the VVC standard in July 2020.
The searches are formulated to acquire patent information pertinent to VVC. The scope of this study does not include an independent evaluation of the technical relevance of each searched patent record.
To ensure that the data used for the study has a stronger relevance to technologies employed in VVC, the dataset collected through the search is restricted to patent applications with application dates on or after January 1 2013. So the search timeline is limited to the years from January 2013 to July 31 2020 and the search was executed in the Derwent World Patents Index.
The search yielded 6,712 patent applications, translating to 1,872 patent families. The family depiction is valuable because it provides a good estimate of the number of inventions, considering one invention per family.
From a licensing viewpoint, it was determined that the intellectual property trends that were particularly pertinent to each of the specialised technologies in VVC were of particular importance. Therefore, the patent data pertinent to these technologies was mined using relevant keywords and classification codes.
The analysis is concerned with identifying patents that are most likely to be relevant to the standard.
Key insights
It can be seen from Figure 2 that inventive activity related to VVC has surged since 2018. This occurred concurrently with the joint VVC request for proposals from October 2017 and the beginning of the VVC development phase in 2018.
The first milestone for the VVC project took place between December 2017 and April 2018 (it was then that the name ‘Versatile Video Coding’ was settled on). As development of the standard began to take shape around this period, enterprises started filing as many patents as possible in anticipation of owning a standard essential patent in the near future.
Figure 1
The contributor landscape: patent overview
As illustrated by Figure 2, Huawei, LG, and Qualcomm are the leading innovators in terms of the patent application families owned pertaining to VVC. Other significant contributors include ByteDance, MediaTek, Sony, Samsung, and the Electronics and Telecommunications Research Institute.
Figure 2
The contributing entities come from diverse industries spanning semiconductors, electronics, and communications, and include media content firms and conglomerates. While the family depiction is valuable because it provides a good estimate of the number of inventions, considering one invention per family, the number of applications filed gives an indication of the innovative companies’ technology strategies and potential offerings besides the obvious of identifying inventions that may be a potential infringement threat.
The key applicants that take the lead include innovative companies such as Huawei and Qualcomm, which emphasises the futuristic technologies the companies are betting on.
Technology insights
The VVC codec introduces a number of new coding tools, but also builds upon technologies that are based on HEVC and that are improved and refined.
Block-based video codecs include VVC and its predecessors, HEVC and Advanced Video Coding. Every coded video frame is divided into larger or smaller blocks, known as coding tree units (CTUs) in HEVC and VVC.
One of the primary HEVC coding tools is the CTU, which ranges in size from 4 x 4 to 64 x 64 pixels. The partitioning principles used in HEVC include the coding unit, the prediction unit, and the transform unit.
Due to the flexibility of VVC's non-square coding units, extensive modelling of video content is possible. The largest CTU that can be used with VVC is 128 x 128 pixels, and it is divided using the quadtree plus multi-type tree technique.
A look at the top gain performance technologies within VVC shows the various areas in which the leading companies have made progress. It demonstrates that development of the VVC standard has led firms from different backgrounds to come together with complementary technologies to create a game-changing codec. Contributing companies that have filed the majority of the patents around technologies such as intra and inter prediction, transform coding, quantisation, high-level syntax, and in-loop filtering include Huawei, Qualcomm, ByteDance, and LG.
Technologies including 360-degree video coding, block-based affine transform motion compensation prediction, and adaptive loop filtering have received the most attention from the contributing firms when filing patents.
Methods including transform coding, quantisation, intra prediction, in-loop filtering, high-level syntax, and inter prediction have received the most attention. The innovation activity for each of the distinct technical concepts in VVC is depicted in Figure 3.
Figure 3
VVC licensing – the pools
Although various VVC patent owners attempted to form a pool, two exist now (Access Advance pool and MPEG LA), which means implementors are required to license from the respective pools, and negotiate with patent owners that are not part of any pool.
Access Advance’s royalty structure is based on several factors, such as:
Complying with the licence;
Displaying the Access Advance trademark; and
Licensing VVC and HEVC from Access Advance.
Access Advance introduced the Multi-Codec Bridging Agreement (MCBA), according to which, companies will be entitled to reduce their royalty rates for products that include VVC and HEVC, paying the same rate they would pay for a product that only includes VVC. The MCBA does not represent a licence but rather an administrative agreement that facilitates reduced royalty rates centred around the VVC and HEVC codecs. Also, VVC and HEVC pools comprise different groups of patents. Licences would therefore likely be needed from VVC and HEVC pools to avoid violating patents essential to the VVC standard.
With VVC, MPEG LA continues to keep its licensing terms simple, and it offers a royalty rate for hardware, paid software, and free software such as browsers. A few weeks ago, MPEG LA released some new policies to ease the licensing for a potential licensee, including a waiver of royalties for standalone (not in or with hardware) VVC software products sold (with or without compensation or consideration) to an end user that is available to any VVC licensee that commits to becoming a licensor to MPEG LA’s VVC licence if it or its affiliates presently or in the future have the right to license or sublicense VVC essential patents. Also, a VVC royalty discount is available to any VVC licensee that enters into, and is compliant with, MPEG LA’s AVC, HEVC and VVC patent portfolio licences.
Given that the pools have converged more quickly than with HEVC and that the terms are more appealing to potential implementors, what we are witnessing with reference to VVC licensing is more positive than negative.
The evolving market
Although VVC is in its infancy, it is expected to gather momentum over the next few months. The analysis herein provides a quick guide to the potential breadth of the patent portfolios and companies that are most likely to have essential patents.
Qualcomm, Huawei, LG, ByteDance, MediaTek, and the ETRI are among the companies and institutes that own most of the patents that were involved in the development of the VVC codec. However, there are approximately 50 other non-contributing firms and institutes that holds patents pertaining to VVC.
Beyond the technical discussions, the codec has a profound business impact. The way we access and consume content is undergoing a significant change. Customers are downloading or streaming media and attending more meetings remotely.
With regard to streaming and downloading media, over-the-top (OTT) is becoming more significant in this value chain, which is expected to surpass $80 billion in 2022.
Middleware, app, storage, digital rights management, and, most crucially, content delivery network (CDN) fees are the standard breakdowns of OTT operational costs.
The largest line item for OTT operators will be CDN fees. This necessitates the use of an affordable CDN and it has received a lot of attention in the industry. In an effort to combat this, CDN providers added more point-of-presence points, aggressive caching, and oddities. Telephone companies are battling this by focusing on approaches such as 5G.
Beyond the above approaches, it is estimated that codecs such as VVC will reduce the expenditure of OTT enterprises by a 30% margin. It reduces data usage by 50%.
The total bandwidth or consumption, measured in gigabytes per month, is how content owners pay CDNs. Additionally, as consumer demand for quality rises, the load on the CDN also rises and the publisher's costs rise. Thus, there is a direct relationship between the quantity of viewers, the quality (bitrate) consumed, and the duration of viewing. So if the data usage is reduced by approximately 50%, it is a win-win situation for all in the value chain.
In terms of market adoption and deployment, Gary Sullivan, from Microsoft and founding co-chairman of the JVET that created VVC, released a report titled Deployment Status of the VVC standard in October 2022 that described VVC's progress. The table below is an excerpt from the document.
Software decoders and media players | Encoders/Decoders | Services |
|
|
Table 1: VVC implementations
Adoption would be delayed by two to three years if VVC requires hardware for full-frame viewing on desktops and battery-efficient playback on smartphones.
Standards connected to TV benefit from including new codecs as well. The Pentonic 2000, an 8K TV chip with VVC media support, has been released by MediaTek. VVC was also integrated into Brazil's recent TV 3.0 initiative.
In the US, codec adoption is dictated by providers such as Netflix, YouTube, Amazon, and Hulu. It is doubtful that any of these businesses will offer VVC in the foreseeable future. However, organisations such as Alibaba and Tencent have their own cloud distribution systems and seem to favour VVC. The effort behind VVC has undoubtedly been far more concerted. This, along with the swift deployment of patent pools for VVC, is a positive indication.
Simply put, the pools have come together more swiftly than with HEVC and on terms that ought to be more readily acceptable to potential implementors. Even if it does lead to ambiguity with a few IP owners not being part of the patent pools, the fact that both pools have declared attractive terms indicates that VVC is now open for business.
It is reasonable to expect that companies will strongly consider incorporating VVC into future devices based on the present market developments.
