What is HLS, LHLS, LL-HLS? | The Evolution of HLS

HLS (HTTP Live Streaming) is a streaming media transmission protocol developed by Apple Inc. This protocol is widely used today in media players, web browsers, mobile devices, and streaming media servers, providing stable live and on-demand content streams to global audiences.

When designing the HLS protocol, the focus was more on reliability rather than low latency. As a result, HLS live streams have relatively high latency, typically over 10 seconds. This level of delay is too high for live streaming scenarios that involve real-time interaction, such as online education, game streaming, and sports events. It can lead to a poor viewing experience for the audience.

With the development of network technology, low-latency live streaming has become increasingly important. The community initially proposed a low-latency solution for HLS, known as LHLS. Later, Apple introduced its official low-latency solution called LL-HLS. As a result, LHLS was quickly abandoned by the community.

What is HLS?

Apple Inc. released the HTTP-based adaptive bitrate streaming communications protocol called HLS in 2009. HLS achieves streaming media playback by dividing the media stream into small file segments (usually in TS format) and distributing these segments via the HTTP protocol.

The basic principles of HLS are as follows:

1. The source media file (such as MP4, MOV, etc.) is divided into small file segments, typically with durations of 2-10 seconds. These segments can be encoded with different bitrates and resolutions to adapt to varying network conditions and device capabilities.

2. A media playlist (M3U8 file) is generated for each bitrate and resolution segment. The playlist contains the URLs of the segments and other metadata such as segment duration and encoding parameters.

3. A master playlist is created, listing all available media playlists. The master playlist can include media streams with different bitrates and resolutions to achieve adaptive bitrate streaming (ABR).

4. The segmented files and playlists are deployed on an HTTP server, and clients (such as browsers, players, etc.) can request these resources using the HTTP protocol.

5. Clients select the appropriate media playlist based on the master playlist and sequentially download and play the segments. Clients can dynamically switch between different media playlists based on network conditions and device capabilities to achieve adaptive bitrate streaming.

The latency of HLS primarily depends on the duration of the segments, client buffering strategy, and network conditions. Typically, HLS has a latency of at least 2-3 times the total duration of the segments. For example, if each segment has a duration of 2 seconds, the HLS latency may be between 4-6 seconds. Additionally, clients may require additional buffering time to handle network fluctuations, which can result in higher latency.

What is LHLS?

In order to reduce live streaming latency, the community introduced an unofficial low-latency HLS protocol called LHLS, with the goal of reducing latency to 2 seconds or below.

LHLS primarily utilizes the chunked transfer encoding feature of HTTP/1.1. Firstly, a video segment is divided into multiple small chunks, each with a short duration (e.g., 500 milliseconds to 1 second). When a client requests a video segment, the server doesn't need to wait for the entire segment to be generated. Instead, each chunk is transmitted directly to the client for caching as soon as it is generated. During this process, the server maintains a persistent HTTP connection until the entire video segment is transmitted before disconnecting.

LHLS, based on the existing HLS protocol, introduces new features called "prefetch media segments" by adding two custom tags: EXT-X-PREFETCH and EXT-X-PREFETCH-DISCONTINUITY. The EXT-X-PREFETCH tag is followed by a video segment URL, and the server responds to the corresponding video segment request by appending the "Transfer-Encoding: chunked" header, enabling chunked transfer encoding.

If a client supports the custom tags EXT-X-PREFETCH and EXT-X-PREFETCH-DISCONTINUITY, it is LHLS. Clients that do not support these tags will ignore them and operate as standard HLS. Therefore, LHLS is fully compatible with the HLS standard protocol.

LHLS achieves lower live streaming latency and brings streaming playback closer to real-time by utilizing chunked transfer encoding, introducing the prefetch media segments mechanism, and implementing related client-side processing logic. However, LHLS is not an official standard, and different implementations may have compatibility issues.

What is LL-HLS?

In 2019, Apple released an official low-latency HLS specification called LL-HLS. LL-HLS does not adopt the chunked transfer encoding solution but instead introduces a completely new approach. The initial version of LL-HLS utilized the push feature of HTTP/2, which required high implementation requirements for both servers and clients. However, most CDNs did not support this feature, leading to intense debates in the industry and limited adoption of LL-HLS. It was not until 2020 that Apple updated the low-latency HLS specification, abandoning the requirement for HTTP/2 push functionality and introducing a new tag called EXT-X-PRELOAD-HINT as a replacement.

LL-HLS introduces the following five new features to achieve lower latency in live streaming:

1. Partial Segments: Partial segments are a portion of a segment that is being generated and can be transmitted and played before the entire segment is generated. This allows clients to start playing new segments earlier, reducing latency. Partial segments are represented in the playlist using the EXT-X-PART tag.

2. Playlist Delta Updates: Traditional HLS requires clients to periodically reload the entire playlist to obtain new segment information. LL-HLS introduces playlist delta updates, allowing servers to send only the new portions of the playlist to clients without sending the entire playlist. This reduces data transfer and improves efficiency.

3. Blocking Playlist Reload: In traditional HLS, clients need to periodically poll the playlist while waiting for new segments. LL-HLS introduces the blocking playlist reload mechanism, allowing servers to block client requests when there are no new segments until new segments or partial segments become available. This reduces unnecessary polling requests and lowers server load.

4. Preload Hints: Preload hints provide the client with the URL and expected arrival time of the next partial segment or the entire segment. Clients can establish connections and buffer data in advance to reduce latency. Preload hints are represented in the playlist using the EXT-X-PRELOAD-HINT tag.

5. Rendition Reports: Rendition reports allow clients to obtain real-time information about other renditions (different bitrates and resolutions of media streams) without loading their complete playlists. This helps clients make adaptive bitrate streaming (ABR) switching decisions more quickly and efficiently. In traditional HLS, when clients need to switch to another rendition, they need to load the complete playlist of the target rendition, which can cause additional latency and overhead. Rendition reports are represented in the playlist using the EXT-X-RENDITION-REPORT tag.

Low latency is crucial for live streaming, and on the foundation of the standard HLS protocol, two solutions have emerged: the community-based LHLS and the official LL-HLS. Both can effectively reduce live streaming latency. The official LL-HLS solution from Apple offers more features and better performance, and it is supported by Apple devices. The LHLS solution has been deprecated by the community at present.

Tencent MPS Stream

Tencent MPS Stream combines Tencent's years of expertise in audio and video technology platforms with its leading global audio and video AI technology. It opens up the core underlying capabilities of Tencent Cloud's video live streaming to users. It not only provides professional, stable, and efficient basic services such as live stream transcoding, packaging, and transmission to developers worldwide but also offers value-added services such as digital copyright protection and SCTE35 advertising solutions.

In terms of low-latency live streaming, Tencent MPS Stream fully supports the LL-HLS protocol, providing high-quality, stable, and low-latency media transmission services for live streams worldwide. Feel free to contact us for more information.