02/10/2022

Fibas Tech

Only Good Technology

Linux hardware video encoding on Amlogic A311D2 processor

Linux hardware video encoding on Amlogic A311D2 processor

I’ve spent a bit more time with Ubuntu 22.04 on Khadas VIM4 Amogic A311D2 SBC, and while the performance is generally good features like 3D graphics acceleration and hardware video decoding are missing. But I was pleased to see a Linux hardware video encoding section in the Wiki, as it’s not something we often see supported early on. So I’ve given it a try…

First, we need to make a video in NV12 pixel format that’s commonly outputted from cameras. I downloaded a 45-second 1080p H.264 sample video from Linaro, and converted it with ffmpeg:

I did this on my laptop. As a raw video, it’s pretty big with 3.3GB of storage used for a 45-second video:

Now let’s try to encode the video to H.264 on Khadas VIM4 board using aml_enc_test hardware video encoding sample:

The output explains the parameters used. There are some error messages, but the video can be played back with ffplay on my computer without issues.

Linux hardware video encoding on Amlogic A311D2 processor

We can also see that encoding took place in 26 seconds, which is faster than real-time since the video is 45 seconds long.

Let’s try the same with H.265 encoding:

That’s surprising but H.265 video encoding is quite faster than H.264 video encoding. Let’s try H.264 encoding again:

Ah. It’s now taking less than 9 seconds. The first time it’s reading the data from the eMMC flash it is slow, but since the file is 3.3GB, it can fit into the cache so the second time there’s no bottleneck from storage.

amlogic a311d2 h265 hardware video encoding sample

Nevertheless, dump.h265 file could also play fine on my computer so the conversion was successful.

Amlogic A311D2 specifications say “H.265 & H.264 at 4Kp50” video encoding is supported. So let’s create a 45-second 4Kp50 video and convert it to NV12 YUV format. Oops, the size of the raw video is 27GB, and it won’t fit into the board’s eMMC flash… Let’s cut that to 30 seconds (about 18GB)…

Now we can encode the video to H.264:

Two minutes to encode a 30 seconds video! That does not cut it, so let’s run the sample again:

It’s even slower… I really think the storage is the bottleneck here because the required read speed for that file would be over 600 MB/s for real-time encoding. The system would typically encode video from the camera stream, not from the eMMC flash. I should have run iozone before:

The sequential read speed is about 178MB/s. I have a MINIX USB Hub with a 480GB SSD that I had tested at 400MB/s. Not quite what we need, but we should see an improvement.

Khadas VIM4 USB-C SSD

Sadly, the drive was not mounted, and even no recognized at all even with tools like fdisk and GParted. When double-checking Khadas VIM4 specifications, I realized the USB Type-C port was a USB 2.0 OTG interface that should recognize the drive, but only support 480 Mbps, so it’s a lost cause anyway…  The only way to achieve over 600MB/s would be to use a USB 3.0 NVMe SSD, but I don’t have any.

So instead, I’ll make a 5-second 4Kp50 video that’s about 2.9GB in size.

First run using H.265:

Second run:

One last try with H.264:

Not quite real-time, but it’s getting closer, and that means 4Kp30 should be feasible. That’s the result with a 5-second 4Kp30 NV12 video encoded with H.264:

Less than four seconds. So real-time 4Kp30 H.264 hardware video encoding is definitely working on Amlogic A311D2 processor.

Amlogic A311D2 4Kp30 hardware video encoding

It’s playing fine on my PC too.

It’s also possible to encode NV12 YUV images into JPEG, but it won’t work with khadas user:

But no problem with sudo:

Probably just a simple permission issue. it was performed the task in 44ms, and I can open dump.jpg (a screenshot) without issues.

jpeg hardware encoding Amlogic A311D2

If I use ffmpeg to convert the NV12 file to jpeg, presumably with software encoding, it takes just under 200ms:

aml_enc_test and jpeg_enc_test are nice little utilities to test hardware video/image encoding in Linux on Amlogic A311D2, but the source code would be nice in order to integrate this into an application. But it does not appear to be public at this time, so I’d assume it’s part of Amlogic SDK. I’ll ask Khadas for the source code, or the method to get it.