MTKHAL算法集成之单帧算法

和你一起终身学习，这里是程序员Android

经典好文推荐，通过阅读本文，您将收获以下知识点:

算法概览一、算法集成前的准备二、为算法选择feature三、将算法对应的feature添加到scenario配置表四、挂载算法五、自定义metadata六、APP调用算法七、遇到的问题及解决方法八、结语

算法概览

为了给用户提供更好的成像效果，现在的手机都会接入一些第三方的图像处理算法。MTK平台的HAL3也在P2这一层提供接入的plugin。按图像处理算法需要的帧数和摄像头数量，大体可以分为三类：

单帧算法：常见的单帧算法有：美颜算法(瘦脸、磨皮、大眼)、广角镜头畸变校正算法、附加表情算法、单摄背景虚化算法(伪双摄算法)等等，仅需单帧图像输入的算法都属于单帧算法。一般情况下，输入一帧图像，算法处理完输出一帧处理后的图像。

多帧算法：常见的多帧算法有：MFNR（多帧降噪）、HDR（高动态范围）等等，需要连续多帧图像输入的算法都属于多帧算法。一般情况下，输入连续多帧图像，算法处理完输出一帧处理后的图像。

双摄算法：最常见的双摄算法是双摄景深算法或者叫双摄背景虚化算法，除此之外，也有彩色+黑白用于增强夜拍效果的双摄算法。单帧算法和多帧算法仅需要获取一个摄像头的图像。而双摄算法需要获取主、辅两个摄像头的图像，并且一般还会要求主、辅摄像头同步。分别获取主、辅摄像头的两帧同步图像，处理后输出一帧主摄图像，用户也仅能看到主摄图像。

根据这个大体上的分类，MTKHAL算法集成系列文章共三篇：

MTKHAL算法集成之单帧算法

MTKHAL算法集成之多帧算法

MTKHAL算法集成之双摄算法

本文是其中的第一篇。这个系列文章均基于Android9.0，MT平台，HAL版本是HAL3。

一、算法集成前的准备

在开展集成工作之前，首先要对算法有一个基本的评估，并且对于集成也应有一定的要求。

1.1算法要求及评估

处理效果好，不能比竞品差，超过竞品更佳。（这条和camera调试的主观效果一样，主观性较强，往往一厢情愿，具体看项目要求吧）

各个场景及压力测试下效果稳定。

处理后照片无色差、锐度和饱和度无损失，或者损失在可接受范围。

达到可接受的分辨率，最好可达到摄像头的最大分辨率。

处理时间越快越好，不超过竞品时间、不超过项目和产品的目标时间。

无内存泄露，占用内存少。

提供必要的集成说明文档，包括算法类型、输入及输出图像要求、输入参数要求等等。

注意：如果有条件，处理时间、内存占用、分辨率等等可量化的指标可要求算法提供方给出具体的参考数据，以便集成完后测试验证。

1.2算法集成要求

编译时可根据项目控制是否集成算法。

运行时可以用参数控制是否启用算法。

集成算法库正常运行、压力测试下效果稳定、无内存泄露。

1.3算法集成的步骤

(1).根据算法选择feature类型，如果与MTK提供的feature不能对号入座，则需要添加自定义feature。(2).将算法对应的feature类型添加到scenario配置表。(3).根据算法选择plugin类型，编写CPP文件实现plugin，挂载算法。(4).如果算法不能复用Android和MTK提供的metadata，则还需要为算法配置自定义的metadata以便APP控制是否启用算法。

首先，我准备了一个libwatermark.so，它仅仅实现了一个添加水印的功能，用它来模拟第三方的单帧算法库。如果想了解添加水印的实现代码，可以参考我另外一篇文章：Android实现图片加水印或logo。接下来，我们就按照集成步骤，逐步详细讲解。

二、为算法选择feature2.1MTK提供的feature

MTK在mtk_feature_type.h和customer_feature_type.h已经提供了一些feature。vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/mtk/mtk_feature_type.h：

NO_FEATURE_NORMAL=0ULL,//MTK(bit0-31)MTK_FEATURE_MFNR=1ULL0,MTK_FEATURE_HDR=1ULL1,MTK_FEATURE_REMOSAIC=1ULL2,MTK_FEATURE_ABF=1ULL3,MTK_FEATURE_NR=1ULL4,MTK_FEATURE_FB=1ULL5,MTK_FEATURE_CZ=1ULL6,MTK_FEATURE_DRE=1ULL7,MTK_FEATURE_DEPTH=1ULL8,MTK_FEATURE_BOKEH=1ULL9,MTK_FEATURE_VSDOF=(MTK_FEATURE_DEPTH

MTK_FEATURE_BOKEH),MTK_FEATURE_FSC=1ULL10,MTK_FEATURE_3DNR=1ULL11,MTK_FEATURE_EIS=1ULL12,MTK_FEATURE_AINR=1ULL13,MTK_FEATURE_DUAL_YUV=1ULL14,MTK_FEATURE_DUAL_HWDEPTH=1ULL15,MTK_FEATURE_AIS=1ULL16,MTK_FEATURE_HFG=1ULL17,MTK_FEATURE_DCE=1ULL18,

vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h：

//ThirdParty(bit32-63)TP_FEATURE_HDR=1ULL32,TP_FEATURE_MFNR=1ULL33,TP_FEATURE_EIS=1ULL34,TP_FEATURE_FB=1ULL35,TP_FEATURE_FILTER=1ULL36,TP_FEATURE_DEPTH=1ULL37,TP_FEATURE_BOKEH=1ULL38,TP_FEATURE_VSDOF=(TP_FEATURE_DEPTH

TP_FEATURE_BOKEH),TP_FEATURE_FUSION=1ULL39,TP_FEATURE_HDR_DC=1ULL40,//usedbyDualCamTP_FEATURE_DUAL_YUV=1ULL41,TP_FEATURE_DUAL_HWDEPTH=1ULL42,TP_FEATURE_PUREBOKEH=1ULL43,TP_FEATURE_RAW_HDR=1ULL44,TP_FEATURE_RELIGHTING=1ULL45,

MTK提供的这些feature可以满足绝大多数算法的集成，在可以对号入座的情况下，我们直接使用已有feature即可。如果不能够满足我们的要求，可以参考下节内容添加新的feature。

2.2添加自定义feature

本来单帧算法对应的feature可以选择MTK提供的MTK_FEATURE_FB和TP_FEATURE_FB，但是为了讲解如何添加新feature，我们选择添加一个自定义feature：TP_FEATURE_WATERMARK。

vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h：

diff--gita/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.hb/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.holdmodenewmodeindexa41fdf5..17bc35eea8---a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h+++b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h

-59,6+59,7

enumeFeatureIndexCustomer{TP_FEATURE_PUREBOKEH=1ULL43,TP_FEATURE_RAW_HDR=1ULL44,TP_FEATURE_RELIGHTING=1ULL45,+TP_FEATURE_WATERMARK=1ULL46,//TODO:reserveforcustomerfeatureindex(bit32-63)};

vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp：

diff--gita/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cppb/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cppoldmodenewmodeindexe32f80a..b01c7---a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp+++b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp

-,6+,7

constchar*FeatID2Name(FeatureID_Tfid)caseFID_FUSION_3RD_PARTY:return"fusion_3rd_party";caseFID_PUREBOKEH_3RD_PARTY:return"purebokeh_3rd_party";caseFID_RELIGHTING_3RD_PARTY:return"relighting_3rd_party";+caseFID_WATERMARK_3RD_PARTY:return"watermark_3rd_party";default:return"unknown";};

vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp：

diff--gita/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cppb/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cppindex8bbba02..daaccf---a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp+++b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp

-,7+,8

MBOOLYUVNode::onInit()featId=FID_FB_3RD_PARTY;elseif(rProperty.mFeaturesTP_FEATURE_RELIGHTING)featId=FID_RELIGHTING_3RD_PARTY;-+elseif(rProperty.mFeaturesTP_FEATURE_WATERMARK)+featId=FID_WATERMARK_3RD_PARTY;if(featId!=NULL_FEATURE){MY_LOGD_IF(mLogLevel,"%sfindsplugin:%s,priority:%d",

vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h：

diff--gita/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.hb/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.holdmodenewmodeindex2f1ad8a..ab47aae---a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h+++b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h

-,6+,7

enumCaptureFeatureFeatureID{FID_FUSION_3RD_PARTY,FID_PUREBOKEH_3RD_PARTY,FID_RELIGHTING_3RD_PARTY,+FID_WATERMARK_3RD_PARTY,NUM_OF_FEATURE,NULL_FEATURE=0xFF,};

vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp：

diff--gita/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cppb/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cppoldmodenewmodeindexcc1dcfd..cbc30---a/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp+++b/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp

-,6+,9

MBOOLCaptureProcessor::onEnque(constspP2FrameRequestpP2Frame)pCapRequest-addFeature(FID_HFG);if(featureMTK_FEATURE_DCE)pCapRequest-addFeature(FID_DCE);+if(featureTP_FEATURE_WATERMARK)+pCapRequest-addFeature(FID_WATERMARK_3RD_PARTY);+}}三、将算法对应的feature添加到scenario配置表

在我们打开camera进行预览和拍照的时候，MTKHAL3会执行vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/policy/FeatureSettingPolicy.cpp的代码，会分别调用vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/scenario_mgr.cpp的get_streaming_scenario函数和get_capture_scenario函数。它们会读取一个scenario的feature配置表，遍历所有的feature，决定哪些feature会被执行。这个配置表中有许多的scenario，一个scenario可能对应多个feature。因此添加自定义feature后，还需将自定义的feature添加到配置表中。MTKfeature对应的配置表是gMtkScenarioFeaturesMaps，customerfeature对应的配置表是gCustomerScenarioFeaturesMaps。

vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp：

diff--gita/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cppb/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cppoldmodenewmodeindexf8de..fe---a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp+++b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp

-93,30+93,30

usingnamespaceNSCam::v3::pipeline::policy::scenariomgr;//#definefeature

转载请注明：http://www.zmax-alibaba.com/qgda/138024.html