Output the video in H.265

---

Introduction

H.265 is a high-efficiency video coding, which offers better data compression at the same level of video quality, or substantially improved video quality at the same bitrate.

You can select H.265 as a mixed output format when coding or mixing streams.

The following shows the differences between H.265 and H.264:

Use cases

• Live video streaming, interactive live streaming, live game streaming: With H.265, CDN delivery costs can be reduced by halving the bitrate.

• Video calling, video conference, online education: With H.265, realize better calling effect and more clear image quality at the same bitrate.

Basic concepts

• Video encoding: the process of converting a given video input into a specific format that is compatible with most types of Web players and mobile devices.

• Stream mixing: the process of combining multiple media streams into a single stream on the cloud. ZEGO supports using stream mixing with three methods, manual stream mix, automatic stream mix, and full-automatic stream mix. For details, see Stream mixing.

• Stream forwarding to CDN: the process of forwarding streams from ZEGO Cloud to CDN (Content Delivery Network). For details, see Publishing streams via CDN.

• Co-hosting: a form of interaction between the users in the rooms. Users can call the startPublishingStream method to publish their own streams, and call the startPlayingStream method to play the remote users' streams for interactive communication.

Prerequisites

Before you begin, make sure you complete the following:

• Create a project in ZEGOCLOUD Admin Console and get the AppID and AppSign of your project.

• Refer to the Quick Start doc to complete the SDK integration and basic function implementation.

##1 H.265 ability testing

Test H.265 encoding ability

Some outdated or low-end mobile devices do not support H.265 encoding. You need to call the isVideoEncoderSupported method to test whether the H.265 encoding is supported on your device first.

If supported, call the setVideoConfig method to set the H.265 encoding type before stream publishing; otherwise, the stream publishing will fail.

Test H.265 decoding ability

Some outdated or low-end mobile devices do not support H.265 decoding.

In some cases that you want to play streams with different bitrates, for example, play streams from CDN, you need to call the isVideoDecoderSupported method to test whether the H.265 decoding is supported on your device first.

If supported, you can play the H.265 streams successfully; otherwise, you can only play H.264 streams.

Implement a co-hosting live streaming

Here are two ways for you to implement the co-hosting live streaming using stream mixing, you can choose either of the following based on your real needs:

• Output streams of different formats directly (recommended): Use the stream mixing service to output a single H.265 mixed stream and a single H.264 mixed stream. This method allows you to only transcode once during the stream mixing, which is clearer and more cost-effective than the method below.

• Use the CDN for transcoding after mixed streams: Use the stream mixing service to output an H.265 stream, and then use the CDN for transcoding to get a single H.265 mixed stream and a single H.264 mixed stream.

Output mixed streams of different formats directly (recommended)

In this case, after the ZEGO Real-Time Audio and Video Cloud received the streams published by the host and the guest host, with the stream mixing service, it will output a single H.265 mixed stream and a single H.264 mixed stream, and publish both the two streams to the CDN.

For the audience, they can choose to play the H.265 or H.264 stream from the CDN according to the H.265 video encoding ability of their devices.

Host side

1. Call the constructor ZegoMixerTask method to create a stream mixing task object, and call the instance method to set the input and output parameters.

2. Set the setInputList property of the ZegoMixerTask object to set up the input stream list of stream mixing task (supports H.264 and H.265 formats), 9 channels of streams are allowed by default and set the stream layout manually.

3. Set the setOutputList property of the ZegoMixerTask object to set the output stream list of the stream mixing task. The stream mixing service outputs an H.265 stream and an H.264 stream (that is, the target mixed outputs are in H.264 and H.265 formats).

4. Call the startMixerTask method to start the stream mixing task.

5. Developers send out a notification to the App Server side that new stream published.

// Call the startMixerTask to start a stream mixing task.
String taskID = ""; // Enter the taskID/
ZegoMixerTask task = new ZegoMixerTask(taskID);

// Set up the videoConfig accordingly.
ZegoMixerVideoConfig videoConfig = new ZegoMixerVideoConfig(720, 1280, 15, 1500);
task.setVideoConfig(videoConfig);

task.setAudioConfig(new ZegoMixerAudioConfig());

// Note: The input stream list of the stream mixing task supports H.264 and H.265, and the stream layout and input stream list must be set manually.
ArrayList<ZegoMixerInput> inputList = new ArrayList<>();
task.setInputList(inputList);

// Output 2 mixed streams of different formats. 
// Note: The output target can be streamID or CDN URL, the two are handled differently on the audience side, and stremaID is recommended in this case.
// Note: The bitrate in ZegoMixerOutput has a higher priority than that in ZegoMixerVideoConfig.
String h264StreamID = ""; // Enter the h264StreamID.
String h265StreamID = ""; // Enter the h265StreamID.
int h264Bitrate = 2244; // Enter H.264 bitrate, this bitrate is recommended when the resolution and frame rate is (720p, 15fps).
int h265Bitrate = 1795; // Enter H.265 bitrate, this bitrate is recommended when the resolution and frame rate is (720p, 15fps).
ArrayList<ZegoMixerOutput> outputList = new ArrayList<>();

ZegoMixerOutput outputH264 = new ZegoMixerOutput(h264StreamID);
ZegoMixerOutputVideoConfig outputH264VideoConfig = new ZegoMixerOutputVideoConfig(ZegoVideoCodecID.getZegoVideoCodecID(0), h264Bitrate);
outputH264.setVideoConfig(outputH264VideoConfig);
outputList.add(outputH264);
ZegoMixerOutput outputH265 = new ZegoMixerOutput(h265StreamID);
ZegoMixerOutputVideoConfig outputH265VideoConfig = new ZegoMixerOutputVideoConfig(ZegoVideoCodecID.getZegoVideoCodecID(3), h265Bitrate);
outputH265.setVideoConfig(outputH265VideoConfig);
outputList.add(outputH265);
task.setOutputList(outputList);

// Start the stream mixing. 
engine.startMixerTask(task, new IZegoMixerStartCallback() {

    @Override
    public void onMixerStartResult(int errorCode, JSONObject var2) {
        // Callback for the stream mixing task.
    }
});
// The developer sends out a notification to the App Server side that new streams published.

Audience side

1. Receive the notification from the App Server side that new streams are published.
2. Call the isVideoDecoderSupported method to test whether the H.265 decoding is supported on the device.
   • If supported, call the startPlayingStream to play the H.265 mixed stream from the CDN.
   • If it is not supported, call the startPlayingStream to play the H.264 mixed stream from the CDN.

// Receive the notification from the App Server side that new stream published.

boolean h265DecoderSupport = engine.isVideoDecoderSupported(ZegoVideoCodecID.getZegoVideoCodecID(3));

String h264StreamID = ""; // h264StreamID
String h265StreamID = ""; // h265StreamID
View view = playView; //  Play the view that needs to be rendered.
ZegoCanvas playCanvas = new ZegoCanvas(view);

if (h265DecoderSupport) {
    //Supports the H.265 decoding.
    engine.startPlayingStream(h265StreamID, playCanvas);
}
else {
    //  Does't support the H.265 decoding.
    engine.startPlayingStream(h264StreamID, playCanvas);
}

Use the CDN for transcoding after mixed streams

In this case, after the ZEGO Real-Time Audio and Video Cloud received the streams published by the host and the guest host, with the stream mixing service, it will only output a single H.265 mixed stream, and then this stream will be published to the CDN for transcoding.

For the audience, they can choose to play the H.265 or H.264 stream from the CDN according to the H.265 video encoding ability of their devices.

Host side

1. Call the constructor ZegoMixerTask method to create a stream mixing task object, and call the instance method to set the input and output parameters.

2. Set the setInputList property of the ZegoMixerTask object to set up the input stream list of stream mixing task (supports H.264 and H.265 formats), 9 channels of streams are allowed by default and set the stream layout manually.

3. Set the setOutputList property of the ZegoMixerTask object to set the output stream list of the stream mixing task. The stream mixing service outputs an H.265 mixed stream (that is, the target mixed outputs are in H.265 format).

4. Call the startMixerTask method to start the stream mixing task.

5. Developers send out a notification to the App Server side that new stream published.

// Call the startMixerTask to start the stream mixing task.
String taskID = ""; // Enter the taskID.
ZegoMixerTask task = new ZegoMixerTask(taskID);

// Set up the videoConfig.
ZegoMixerVideoConfig videoConfig = new ZegoMixerVideoConfig(720, 1280, 15, 1500);
task.setVideoConfig(videoConfig);

task.setAudioConfig(new ZegoMixerAudioConfig());

// Note: the input stream of the stream mixing task supports the H.264 and H.265. You need to set up the stream layout and stream input list manually.
ArrayList<ZegoMixerInput> inputList = new ArrayList<>();
task.setInputList(inputList);

String publishCdnUrl = ""; // Enter the CDN URL.
int h265Bitrate = 1795; // Enter the H.265 bitrate, this bitrate is recommended when the resolution and frame rate is (720p, 15fps).
// Note: To use the CDN for transcoding, you need to enter CDN URL to the target.
ArrayList<ZegoMixerOutput> outputList = new ArrayList<>();

ZegoMixerOutput outputH265 = new ZegoMixerOutput(publishCdnUrl);
ZegoMixerOutputVideoConfig outputH265VideoConfig = new ZegoMixerOutputVideoConfig(ZegoVideoCodecID.getZegoVideoCodecID(3), h265Bitrate);
outputH265.setVideoConfig(outputH265VideoConfig);
outputList.add(outputH265);
task.setOutputList(outputList);

// Start the stream mixing task.
engine.startMixerTask(task, new IZegoMixerStartCallback() {

    @Override
    public void onMixerStartResult(int errorCode, JSONObject var2) {
        // Callback for stream mixing task.
    }
});

// The developer sends out a notification to the App Server side that new stream published.

Audience side

1. Receive the notification from the App Server side that new streams are published.
2. Call the isVideoDecoderSupported method to test whether the H.265 decoding is supported on the device.
   • If supported, call the startPlayingStream to play the H.265 mixed stream from the CDN.
   • If it is not supported, call the startPlayingStream to play the H.264 mixed stream from the CDN.

// Received a notification from the App Server side that new stream published. 

boolean h265DecoderSupport = engine.isVideoDecoderSupported(ZegoVideoCodecID.getZegoVideoCodecID(3));

String playStreamID = "";
View view = playView; // Play the view that needs to be rendered.
ZegoCanvas playCanvas = new ZegoCanvas(view);

if (h265DecoderSupport) {
    // Supports the H.265 decoding.
    String h265Url = ""; //  Enter the H265 url.

    // Note: ZegoCDNConfig and ZegoPlayerConfig have other configurable items.
    ZegoCDNConfig cdnConfig = new ZegoCDNConfig();
    // The CDN URL for playing H.265 streams.
    cdnConfig.url = h265Url;
    ZegoPlayerConfig playerConfig = new ZegoPlayerConfig();
    playerConfig.cdnConfig = cdnConfig;
    engine.startPlayingStream(playStreamID, playCanvas, playerConfig);
}
else {
    // Does't support H.265 decoding.
    String h264Url = ""; // Enter the H264url.

    // Note: ZegoCDNConfig and ZegoPlayerConfig have other configurable items.
    ZegoCDNConfig cdnConfig = new ZegoCDNConfig();
    // The CDN URL for playing the H.264 streams.
    cdnConfig.url = h264Url;
    ZegoPlayerConfig playerConfig = new ZegoPlayerConfig();
    playerConfig.cdnConfig = cdnConfig;
    engine.startPlayingStream(playStreamID, playCanvas, playerConfig);
}

Implement a single host live streaming

Single host live streaming via CDN

This use case has the following two features:

• The host forwards the streams that are published to the ZEGO Real-Time Audio and Video Cloud to the CDN for transcoding, and the audience can decide to play H.265 mixed stream or H.264 mixed stream based on the H.265 decoding ability of the devices.

• The guest host can play the streams directly from the ZEGO Real-Time Audio and Video Cloud when the host publishes his stream to the Cloud for interactive communication, but this also requires the device the guest host using to support the H.265 decoding ability.

Host side

1. After the engine is created, call the enableHardwareEncoder method to enable the hardware encoding first (if modified after the stream publishing, it takes effect when the next stream is published), CPU usage can be improved after this is enabled.
2. Call the isVideoEncoderSupported to test whether the specified video encoding format is supported on the host side.

3. Before stream publishing, call the setVideoConfig method and set the video encoding format by setting the codecID property (if this is modified after the stream publishing, it takes effects after the next stream is published). If set the encoding format to H.265: a. Call the enableH265EncodeFallback method to enable the feature that adaptive degrade from H.265 encoding to H.264 encoding (enabled by default). For example, when H.265 encoding is not supported or failed to encode with H.265, with this is enabled, SDK uses the H.264 encoding for stream publishing. But with this is disabled, the stream publishing will fail.

   b. Call the addPublishCdnUrl method to add the CDN URL for forwarding the streams from ZEGO Real-Time Audio and Video Cloud to the CDN.

   c. Call the startPublishingStream method to start the stream publishing. When the stream is published successfully, the audience in the room can listen for and receive the related stream status through the callback onRoomStreamUpdate.

   d. The developer sends out a notification to the App Server with the encoding format of the stream, so that the stream playing side can process accordingly based on the different encoding formats of the stream.

// For mobile devices, to use the H.265 encoding, hard encoding needs to be enabled first. 
engine.enableHardwareEncoder(true);

// Test whether the H.265 encoding is supported.
boolean h265EncoderSupport = engine.isVideoEncoderSupported(ZegoVideoCodecID.getZegoVideoCodecID(3));

ZegoVideoConfig videoConfig = new ZegoVideoConfig();
if (h265EncoderSupport) {
    // Supports the H.265 encoding.
    videoConfig.codecID = ZegoVideoCodecID.getZegoVideoCodecID(3);
} else {
    // Does't support the H.265 encoding.
    videoConfig.codecID = ZegoVideoCodecID.getZegoVideoCodecID(0);
}
engine.setVideoConfig(videoConfig);

if (h265EncoderSupport) {
    //  Throught the [enableH265EncodeFallback] to choose whether to enable the feature that adaptive degradates from H.265 encoding to H.264 encoding when failed to encode using H.265.
    engine.enableH265EncodeFallback(true);
}

String publishStreamID = ""; // Enter the streamID.
String publishCdnUrl = ""; // Enter the CdnUrl.

// Add the URL for CDN forwarding. 
engine.addPublishCdnUrl(publishCdnUrl, publishStreamID, new IZegoPublisherUpdateCdnUrlCallback() {
    @Override
    public void onPublisherUpdateCdnUrlResult(int i) {
        // Check whether the CDN URL is added successfully.
    }
});
engine.startPublishingStream(publishStreamID);

// Send out a notification to the App Server side that the encoding format of the stream you published, so that the stream playing side can process accordingly based on the different encoding format. 

Audience side

1. Receive notifications that new stream published through the callback onRoomStreamUpdate after the host published the stream.
2. Get the encoding format of the stream from the App Server side.

• If the stream is in H.264 format, call the startPlayingStream to play the stream directly from the CDN.
• If the stream is in H.265 format, call the isVideoDecoderSupported method to test the H.265 decoding ability of the device first.
  • If supported, call the startPlayingStream method to play the H.265 stream from the CDN.
  • If it is not supported, call the startPlayingStream method to play the H.264 stream from the CDN.

// Receives the notification that new stream published through the callback onRoomStreamUpdate.
@Override
public void onRoomStreamUpdate(String roomID, ZegoUpdateType updateType, ArrayList<ZegoStream> streamList, JSONObject extendedData) {
    super.onRoomStreamUpdate(roomID, updateType, streamList, extendedData);

    int videoCodecID = 0; // Get the stream encoding format from the App Server side.
    String playStreamID = ""; // Enter the streamID.
    View view = playView; // Play the view that needs to be rendered.
    ZegoCanvas playCanvas = new ZegoCanvas(view);

    if (videoCodecID == ZegoVideoCodecID.getZegoVideoCodecID(3).value()) {
        // Encoding format is H.265.
        boolean h265DecoderSupport = engine.isVideoDecoderSupported(ZegoVideoCodecID.getZegoVideoCodecID(3));

        // No streams will be played if the H.265 decoding is not supported.
        if (h265DecoderSupport) {
            // Supports the H.265 decoding.
            String h265Url = ""; // Enter the H265 Url.

            // Note: ZegoCDNConfig and ZegoPlayerConfig have other configurable items.
            ZegoCDNConfig cdnConfig = new ZegoCDNConfig();
            // The CDN URL for playing the H.265 streams.
            cdnConfig.url = h265Url;
            ZegoPlayerConfig playerConfig = new ZegoPlayerConfig();
            playerConfig.cdnConfig = cdnConfig;
            engine.startPlayingStream(playStreamID, playCanvas, playerConfig);
        } else {
            // Does't support the H.265 decoding.
            String h264Url = ""; // Enter the H265 Url.

            // Note: ZegoCDNConfig and ZegoPlayerConfig have other configurable items.
            ZegoCDNConfig cdnConfig = new ZegoCDNConfig();
            // The CDN URL for playing the H.264 streams.
            cdnConfig.url = h264Url;
            ZegoPlayerConfig playerConfig = new ZegoPlayerConfig();
            playerConfig.cdnConfig = cdnConfig;
            engine.startPlayingStream(playStreamID, playCanvas, playerConfig);
        }
    }
    else if (videoCodecID == ZegoVideoCodecID.getZegoVideoCodecID(0).value()) {
        // Decoding format is H.264.
        engine.startPlayingStream(playStreamID, playCanvas);
    }
}

Guest host side

1. Receive a notification that new stream published through the onRoomStreamUpdate callback after the host published streams.
2. Get the encoding format of the stream from the App Server side.

• If the stream is in H.264 format, the guest host can call the startPlayingStream method to play the stream from ZEGO Real-Time Audio and Video Cloud directly.
• If the stream is in H.265 format, the guest host needs to call the isVideoDecoderSupported method to test the H.265 decoding ability first.
  • If supported, call the startPlayingStream method to play the H.265 stream from the ZEGO Real-Time Audio and Video Cloud.
  • If it is not supported, a prompt shows and indicates that this stream can't be played.

// Receives notifications that new stream published through the callback onRoomStreamUpdate.
@Override
public void onRoomStreamUpdate(String roomID, ZegoUpdateType updateType, ArrayList<ZegoStream> streamList, JSONObject extendedData) {
    super.onRoomStreamUpdate(roomID, updateType, streamList, extendedData);

    int videoCodecID = 0; // Get the stream encoding format from the App Server side. 
    String playStreamID = ""; // Enter the streamID.
    View view = playView; // Play the view that needs to be rendered.
    ZegoCanvas playCanvas = new ZegoCanvas(view);

    if (videoCodecID == ZegoVideoCodecID.getZegoVideoCodecID(3).value()) {
        // The encoding format is H.265.
        boolean h265DecoderSupport = engine.isVideoDecoderSupported(ZegoVideoCodecID.getZegoVideoCodecID(3));

        // If H.265 decoding is not supported, then stream won't be played.
        if (h265DecoderSupport) {
            // Supports the H.265 decoding.
            engine.startPlayingStream(playStreamID, playCanvas);
        }
    }
    else if (videoCodecID == ZegoVideoCodecID.getZegoVideoCodecID(0).value()) {
        // Decoding format is H.264.
        engine.startPlayingStream(playStreamID, playCanvas);
    }
}

Recording

If the H.265 encoding/decoding feature is used for On-Premises Recording, Cloud Recording, the generation of recorded files will be affected as follows:

When recoding the stream publishing (using the H.265 for video encoding), the adaptive degradation may be triggered. In this case, the SDK sends outa notification that the video encoding format changed through the callback onPublisherVideoEncoderChanged.

In this case, to prevent file damage, the SDK ends and saves the current recording task, and starts a new recording task automatically. And generates a new file and storage path of the new recording task to prevent from overwriting the original recorded file. The new file name will have a timestamp for identification.

For example, the original path of the recording file is /user/data/mediarecord.mp4. And the new path will change to: /user/data/mediarecord_1626880634948.mp4, and the 1626880634948 is the timestamp.

FAQ

1. How to deal with the error when using the H.265 for decoding?

Call the enableH265EncodeFallback method to enable the adaptive degradation feature (enabled by default). The SDK automatically uses the H.264 decoding when the failed to decode using H.265. If you are recording at the same time, the recording file will be doubled.

2. Does H.265 automatically degrade from hardware decoding to software decoding?

Yes. If the hardware encoding fails is found during decoding, it adaptive degrades to soft decoding.

3. What's the billing plan of the H.265?

It's free to only enable the H.265 feature. But it starts to charge if you output H.265 mixed streams, which is higher on the price than H.264. For details, contact our sales.

4. Is there any change in charging when I want to input H.265 stream and output H.264 mixed stream?

There are no changes. The pricing is based on the output H.264 mixed streams.

5. What are the hardware requirements for H.265 decoding?

All models currently on the market support H.265 decoding. And for outdated or low-end models before 2013 may have framerate fluctuations.