视频解码分为软解和硬解两种，简单的来说纯粹依靠CPU来解码的称为软解，而使用GPU/VPU解码的称为硬解。常规地直接用CPU解码会极大地消耗CPU的运算能力，硬件解码是通过显卡的视频加速功能对高清视频进行解码,能够将CPU从繁重的视频解码运算中释放出来。
RIoTBoard使用的芯片是i.mx6 solo，芯片中包含一个VPU （video processing unit），可以进行硬件编解码加速，心血来潮，折腾一下视频硬件解码加速。
NXP官方提供了一个libimxvpuapi lib封装了vpu的使用方法，libimxvpuapi 又依赖libimxdmabuffer，所以只要编译出libimxvpuapi和libimxdmabuffer，运行libimxvpuapi 的testcase 就可以实现硬件编解码。
libimxdmabuffer代码下载地址：
git clone https://github.com/Freescale/libimxdmabuffer.git
git 仓库里的readme有简单的说明，支持那些平台、如何编译等等，可以进行参考。
编译前需要配置toolchain以及imx-linux-headers-path，imx-linux-headers-path指向BSP中的linux-imx/include/即可。
./waf configure CC=arm-none-linux-gnueabi-gcc --imx-linux-headers-path=/home/zhy/code/mypc/linux/linux-imx/include/ --prefix=/home/zhy/code/rIoTboard/libimxdmabuffer
配置的过程中出现了linux/mxcfb.h not found的error:
Setting top to : /home/zhy/code/mypc/gstreamer/third_lib/libimxdmabuffer Setting out to : /home/zhy/code/mypc/gstreamer/third_lib/libimxdmabuffer/build Checking for 'gcc' (C compiler) : arm-none-linux-gnueabi-gcc Checking for compiler switch -O2 : yes Checking for compiler switch -DPIC : yes Checking for compiler switch -fPIC : yes Checking for compiler switch -pedantic : yes Checking for compiler switch -std=gnu99 : yes Checking for compiler switch -Wall : yes Checking for compiler switch -Wextra : yes Checking for header linux/mxcfb.h : not found Could not find linux/mxcfb.h in path "/home/zhy/code/mypc/linux/linux-imx/include/" specified by --imx-linux-headers-path
查看log发现是这个头文件中uint未定义。
../../../../../../linux/linux-imx/include/linux/mxcfb.h:113:2: error: unknown type name 'uint'
解决方法把uint更改unsigned int 即可。
unsigned int flags;
重新配置，这次就可以配置成功，会输出如下log：
checking for linux/fb.h and the IPU header linux/ipu.h : yes Checking for library g2d : not found Checking for header g2d.h : not found G2D not found (library found: 0 header found: 0); disabling G2D allocator checking for linux/pxp_device.h : no linux/pxp_device.h was not found in i.MX linux headers path; disabling PxP allocator libimxdmabuffer version 1.0.1 'configure' finished successfully (0.662s)
配置好后执行编译命令进行编译：
./waf
编译成功后显示如下log：
[5/8] Compiling libimxdmabuffer.pc.in [6/8] Linking build/libimxdmabuffer.so [7/8] Linking build/test-alloc [8/8] Symlinking build/libimxdmabuffer.so Waf: Leaving directory `/home/zhy/code/mypc/gstreamer/third_lib/libimxdmabuffer/build' 'build' finished successfully (0.422s)
执行install命令，install目录为prefix指定的路径：
./waf install
测试一下能不能链接到limxdmabuffer这个lib:
arm-none-linux-gnueabi-gcc -L/home/zhy/code/rIoTboard/libimxdmabuffer/lib -limxdmabuffer
输出一下log就代表可以链接到，编译OK。
/usr/local/complie_tool/fsl-linaro-toolchain/bin/../arm-fsl-linux-gnueabi/multi-libs/default/usr/lib/crt1.o: In function `_start': init.c:(.text+0x34): undefined reference to `main' collect2: ld returned 1 exit status
最后将编译好的lib 打包一下，方便以后使用，这一步不是必须的。
tar czvf libimxdmabuffer.tar.gz libimxdmabuffer/2 编译libimxvpuapi
编译好libimxdmabuffer就可以编译libimxvpuapi，首先下载代码下载：
git clone https://github.com/Freescale/libimxvpuapi.git
libimxvpuapi编译依赖libimxdmabuffer和libvpu这两个lib ,libimxdmabuffer已经编译好了，在libimxvpuapi目录新建一个third_lib文件夹，把libimxdmabuffer编译好的so和头文件放进取。至于libvpu的头文件和so,可以到开发板的里面去copy，需要copy ipu.h、vpu_io.h、vpu_lib.h这三个头文件以及libvpu.so、libvpu.so.4这两个so，同样把这写放到third_lib 下面。最后在third_lib/lib/pkgconfig目录下放上libimxdmabuffer.pc和 vpu_lib.pc这两个pkg-config文件，最终third_lib下面的目录结构：
├── include
│ ├── imxdmabuffer
│ │ ├── imxdmabuffer_config.h
│ │ ├── imxdmabuffer.h
│ │ ├── imxdmabuffer_ipu_allocator.h
│ │ └── imxdmabuffer_physaddr.h
│ ├── ipu.h
│ ├── vpu_io.h
│ └── vpu_lib.h
└── lib
├── libimxdmabuffer.so -> libimxdmabuffer.so.1.0.1
├── libimxdmabuffer.so.1 -> libimxdmabuffer.so.1.0.1
├── libimxdmabuffer.so.1.0.1
├── libvpu.a
├── libvpu.so
├── libvpu.so.4
└── pkgconfig
├── libimxdmabuffer.pc
└── vpu_lib.pc
其中libimxdmabuffer.pc中内容，这里面的路径根据自己的目录可以进行修改：
prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
exec_prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
libdir=${prefix}/lib
includedir=${prefix}/include
libimxdmabuffer_ion=0
Name: libimxdmabuffer
Description: library for allocating and managing physically contiguous memory ("DMA memory" or "DMA buffers") on i.MX devices
Version: 1.0.1
Libs: -L${libdir} -limxdmabuffer
Cflags: -I${includedir}
vpu_lib.pc中内容：
prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
exec_prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
libdir=${prefix}/lib
includedir=${prefix}/include
Name: vpu
Description: libvpu
Version: 1.0.1
Libs: -L${libdir} -lvpu
Cflags: -I${includedir}
准备工作完成后进行编译配置：
./waf configure CC=arm-none-linux-gnueabi-gcc --sysroot-path=/home/zhy/code/mypc/linux/linux-imx/include/ --prefix=/home/zhy/code/rIoTboard/libimxvpuapi --imx-platform=imx6 --imx-headers=/home/zhy/code/mypc/linux/linux-imx/include/
配置的时候显示找不到libimxdmabuffer 这个lib:
Checking if this combination works : yes Checking for program 'pkg-config' : /usr/bin/pkg-config Checking for 'libimxdmabuffer >= 0.9.0' : not found The configuration failed (complete log in /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build/config.log)
查看log 发现是没有找到libimxdmabuffer的pkg-config文件，
---------------------------------------- Checking for program 'pkg-config' /usr/bin/pkg-config find program=['pkg-config'] paths=['/usr/local/sbin', '/usr/local/bin', '/usr/sbin', '/usr/bin', '/sbin', '/bin', '/usr/games', '/usr/local/games', '/snap/bin', '/usr/local/complie_tool/fsl-linaro-toolchain/bin'] var='PKGCONFIG' -> ['/usr/bin/pkg-config'] ---------------------------------------- Checking for 'libimxdmabuffer >= 0.9.0' ['/usr/bin/pkg-config', '--cflags', '--libs', 'libimxdmabuffer', '>=', '0.9.0'] err: Package libimxdmabuffer was not found in the pkg-config search path. Perhaps you should add the directory containing `libimxdmabuffer.pc' to the PKG_CONFIG_PATH environment variable No package 'libimxdmabuffer' found
通过PKG_CONFIG_PATH关键字增加pkg-config文件的路径即可。
./waf configure CC=arm-none-linux-gnueabi-gcc PKG_CONFIG_PATH=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib/lib/pkgconfig --imx-platform=imx6 --sysroot-path=/home/zhy/code/mypc/linux/linux-imx/include/ --prefix=/home/zhy/code/rIoTboard/libimxvpuapi --imx-headers=/home/zhy/code/mypc/linux/linux-imx/include/
再次配置，有以下log印出则代表配置成功：
Setting top to : /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi Setting out to : /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build Checking for 'gcc' (C compiler) : arm-none-linux-gnueabi-gcc Checking for compiler switch -Wextra : yes Checking for compiler switch -Wall : yes Checking for compiler switch -pedantic : yes Checking for compiler switch -fPIC : yes Checking for compiler switch -DPIC : yes Checking for compiler switch -O2 : yes Checking for compiler switch -std=gnu99 : yes Now checking the combination of CFLAGS -Wextra -Wall -pedantic -fPIC -DPIC -O2 -std=gnu99 Checking if this combination works : yes Checking for compiler switch -std=c99 : yes Now checking the combination of CFLAGS -Wextra -Wall -pedantic -fPIC -DPIC -O2 -std=c99 Checking if this combination works : yes Checking for program 'pkg-config' : /usr/bin/pkg-config Checking for 'libimxdmabuffer >= 0.9.0' : yes Using "/home/zhy/code/mypc/linux/linux-imx/include" as sysroot path Checking for library vpu : yes checking if ENC_ENABLE_SOF_STUFF exists : no i.MX linux headers path: /home/zhy/code/mypc/linux/linux-imx/include/ checking for the IPU header linux/ipu.h : yes libimxvpuapi version 2.2.0 'configure' finished successfully (0.692s)
执行以下命令进行编译
./waf
编译时出现了’IPU_PIX_FMT_NV16’ undeclared的error。
../imxvpuapi2/imxvpuapi2_imx6_coda_ipu.c: In function 'get_ipu_pixel_format': ../imxvpuapi2/imxvpuapi2_imx6_coda_ipu.c:33:76: error: 'IPU_PIX_FMT_NV16' undeclared (first use in this function)
ipu.h里面没有定义这个format，目前不需要NV12这个color format，在get_ipu_pixel_format函数中注释掉IPU_PIX_FMT_NV16 这个color_format即可。
static uint32_t get_ipu_pixel_format(ImxVpuApiColorFormat color_format)
{
switch (color_format)
{
case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV420_8BIT: return IPU_PIX_FMT_YUV420P;
case IMX_VPU_API_COLOR_FORMAT_SEMI_PLANAR_YUV420_8BIT: return IPU_PIX_FMT_NV12;
case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV411_8BIT: return IPU_PIX_FMT_YUV410P;
case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV422_HORIZONTAL_8BIT: return IPU_PIX_FMT_YUV422P;
//case IMX_VPU_API_COLOR_FORMAT_SEMI_PLANAR_YUV422_HORIZONTAL_8BIT: return IPU_PIX_FMT_NV16;
case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV444_8BIT: return IPU_PIX_FMT_YUV444P;
case IMX_VPU_API_COLOR_FORMAT_YUV400_8BIT: return IPU_PIX_FMT_GREY;
default: return 0;
}
}
修改后编译成功，会有以下log印出，并在build 目录会生成libimxvpuapi2.so。
[14/19] Linking build/libimxvpuapi2.so [15/19] Linking build/example/decode-example [16/19] Linking build/example/encode-example [17/19] Linking build/example/jpeg-dec-example [18/19] Linking build/example/jpeg-enc-example [19/19] Symlinking build/libimxvpuapi2.so Waf: Leaving directory `/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build' 'build' finished successfully (1.970s)
测试一下能否动态链接到libimxvpuapi：
arm-none-linux-gnueabi-gcc -L/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build -limxvpuapi2
有以下log印出则代表链接OK:
zhy@zhy-ThinkPad-E480:~/code/mypc/gstreamer/third_lib/libimxvpuapi/build$ arm-none-linux-gnueabi-gcc -L/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build -limxvpuapi2 /usr/local/complie_tool/fsl-linaro-toolchain/bin/../lib/gcc/arm-fsl-linux-gnueabi/4.6.2/../../../../arm-fsl-linux-gnueabi/bin/ld: warning: libimxdmabuffer.so.1, needed by /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build/libimxvpuapi2.so, not found (try using -rpath or -rpath-link) /usr/local/complie_tool/fsl-linaro-toolchain/bin/../arm-fsl-linux-gnueabi/multi-libs/default/usr/lib/crt1.o: In function `_start': init.c:(.text+0x34): undefined reference to `main'
将编译好的lib进行打包，以便后续使用，这一部分为非必要步骤。
mkdir libimxvpuapi mkdir -p libimxvpuapi/lib libimxvpuapi/include/imxvpuapi2 libimxvpuapi/pkgconfig cp -rf build/*.so.* libimxvpuapi/lib/ cp -rf imxvpuapi2/*.h libimxvpuapi/include/imxvpuapi2 cp -rf build/libimxvpuapi2.pc libimxvpuapi/lib/pkgconfig
然后在更改一下libimxvpuapi2.pc ，将libdir和includedir改成可根据prefix动态配置。
prefix=/home/zhy/code/rIoTboard/libimxvpuapi
exec_prefix=/home/zhy/code/rIoTboard/libimxvpuapi
libdir=${prefix}/lib
includedir=${prefix}/include
最后打包lib:
tar czvf libimxvpuapi.tar.gz libimxvpuapi/3 开发板上测试使用vpu解码
libimxvpuapi编译好后在build/example 下面后生成可执行文件，将example整个文件夹复制到开放板上，同时还要复制libimxvpuapi和libimxdmabuffer编译好的so到开放板的/lib/目录下，下载一个h264码流，用mediainfo看一下信息：
把这个码流也复制到开发板上libimxvpuapi/example目录，在example执行以测试使用vpu解码h264。
./decode-example -i ./ds.h264 -o out.yuv
用手机拍了一下执行结果：
把解码后的out.yuv复制到PC 上，再用mediainfo查看，已经解码成yuv数据了，解码前190kb，解码后18.7M。
用VLC也可以播放out.yuv：
