[{"id":27868,"web_url":"https://patchwork.libcamera.org/comment/27868/","msgid":"<20230924213403.GD6390@pendragon.ideasonboard.com>","date":"2023-09-24T21:34:03","subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Mattijs,\n\nThank you for the patch.\n\nOn Fri, Sep 15, 2023 at 09:57:30AM +0200, Mattijs Korpershoek via libcamera-devel wrote:\n> On am62x platforms, the receiver driver (j721e-csi2rx) only\n> supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.\n> \n> The receiver and the sensor (ov5640) hardware are both capable of\n> YUV420, however:\n> \n> * we are not aware of OV5640 being tested with YUV420 formats on any\n> vendor tree.\n\nAre you talking about packed YUV 4:2:0 here, or planar YUV 4:2:0 ?\nSensors only output packed formats (there could be exceptions, but they\nwould be very rare), it is the DMA engine on the host size that would\nconvert them to planar formats. That would be the DMA engine handled by\nthe k3-udma driver. It's fairly complicated, and I don't know if it\nwould hahev the ability to output planar formats. Tomi (on CC) may have\nmore information.\n\n> * NV12 has different line lines (even lines are twice as long as odd\n\ns/lines/lengths/\n\n> lines) Different line-sized DMA transfers have not been tested on\n\ns/)/)./\n\n> the TI CSI-RX SHIM IP.\n\nAs mentioned just above, the sensor will not output planar formats, so\nthe issue is about the \n\n> On the other hand, the graphics allocator (gralloc) cannot allocate\n> YUV422 buffers directly. It mainly allocated NV12 buffers when\n> userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED.\n\nThe DSS seems to support YUYV, is there a way to fix the TI gralloc to\nsupport it ?\n\n> Because of the above, we need a pixel conversion from YUYV to NV12,\n> which is handled in the yuv processor via libyuv.\n> \n> Signed-off-by: Mattijs Korpershoek \n> ---\n> src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++---------\n> 1 file changed, 68 insertions(+), 23 deletions(-)\n> \n> diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\n> index 734bb85b7351..b680b833dafb 100644\n> --- a/src/android/yuv/post_processor_yuv.cpp\n> +++ b/src/android/yuv/post_processor_yuv.cpp\n> @@ -7,6 +7,9 @@\n> \n> #include \"post_processor_yuv.h\"\n> \n> +#include \n> +\n> +#include \n> #include \n> \n> #include \n> @@ -22,14 +25,42 @@ using namespace libcamera;\n> \n> LOG_DEFINE_CATEGORY(YUV)\n> \n> +namespace {\n> +\n> +/**\n> + * \\var supportedConversions\n> + * \\brief list of supported output pixel formats for an input pixel format\n> + */\n> +const std::map> supportedConversions = {\n> +\t{ formats::YUYV, { formats::NV12 } },\n> +};\n> +\n> +} /* namespace */\n> +\n> int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> \t\t\t\tconst StreamConfiguration &outCfg)\n> {\n> \tif (inCfg.pixelFormat != outCfg.pixelFormat) {\n> -\t\tLOG(YUV, Error) << \"Pixel format conversion is not supported\"\n> -\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> -\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> -\t\treturn -EINVAL;\n> +\t\tconst auto it = supportedConversions.find(inCfg.pixelFormat);\n> +\t\tif (it == supportedConversions.end()) {\n> +\t\t\tLOG(YUV, Error) << \"Unsupported source format \" << inCfg.pixelFormat;\n\n\t\t\tLOG(YUV, Error)\n\t\t\t\t<< \"Unsupported source format \"\n\t\t\t\t<< inCfg.pixelFormat;\n\n> +\t\t\treturn -EINVAL;\n> +\t\t}\n> +\n> +\t\tstd::vector outFormats = it->second;\n\nNo need for a copy.\n\n> +\t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat);\n\n\t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(),\n\t\t\t\t\t outCfg.pixelFormat);\n\n> +\t\tif (match == outFormats.end()) {\n> +\t\t\tLOG(YUV, Error) << \"Requested pixel format conversion is not supported\"\n> +\t\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> +\t\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n\n\t\t\tLOG(YUV, Error)\n\t\t\t\t<< \"Requested pixel format conversion is not supported\"\n\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n\n> +\t\t\treturn -EINVAL;\n> +\t\t}\n> +\t} else {\n> +\t\tif (inCfg.pixelFormat != formats::NV12) {\n> +\t\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> +\t\t\t\t\t<< \" (only NV12 is supported for scaling)\";\n\n\t\t\tLOG(YUV, Error)\n\t\t\t\t<< \"Unsupported format \" << inCfg.pixelFormat\n\t\t\t\t<< \" (only NV12 is supported for scaling)\";\n\n> +\t\t\treturn -EINVAL;\n> +\t\t}\n> \t}\n> \n> \tif (inCfg.size < outCfg.size) {\n> @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> \t\treturn -EINVAL;\n> \t}\n> \n> -\tif (inCfg.pixelFormat != formats::NV12) {\n> -\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> -\t\t\t\t<< \" (only NV12 is supported)\";\n> -\t\treturn -EINVAL;\n> -\t}\n> -\n> \tcalculateLengths(inCfg, outCfg);\n> \treturn 0;\n> }\n> @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf\n> \t\treturn;\n> \t}\n> \n> -\tint ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> -\t\t\t\t sourceStride_[0],\n> -\t\t\t\t sourceMapped.planes()[1].data(),\n> -\t\t\t\t sourceStride_[1],\n> -\t\t\t\t sourceSize_.width, sourceSize_.height,\n> -\t\t\t\t destination->plane(0).data(),\n> -\t\t\t\t destinationStride_[0],\n> -\t\t\t\t destination->plane(1).data(),\n> -\t\t\t\t destinationStride_[1],\n> -\t\t\t\t destinationSize_.width,\n> -\t\t\t\t destinationSize_.height,\n> -\t\t\t\t libyuv::FilterMode::kFilterBilinear);\n> +\tint ret = 0;\n> +\tswitch (sourceFormat_) {\n> +\tcase formats::NV12:\n> +\t\tret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> +\t\t\t\t\tsourceStride_[0],\n> +\t\t\t\t\tsourceMapped.planes()[1].data(),\n> +\t\t\t\t\tsourceStride_[1],\n> +\t\t\t\t\tsourceSize_.width, sourceSize_.height,\n> +\t\t\t\t\tdestination->plane(0).data(),\n> +\t\t\t\t\tdestinationStride_[0],\n> +\t\t\t\t\tdestination->plane(1).data(),\n> +\t\t\t\t\tdestinationStride_[1],\n> +\t\t\t\t\tdestinationSize_.width,\n> +\t\t\t\t\tdestinationSize_.height,\n> +\t\t\t\t\tlibyuv::FilterMode::kFilterBilinear);\n> +\t\tbreak;\n> +\tcase formats::YUYV:\n> +\t\tret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(),\n> +\t\t\t\t\t sourceStride_[0],\n> +\t\t\t\t\t destination->plane(0).data(),\n> +\t\t\t\t\t destinationStride_[0],\n> +\t\t\t\t\t destination->plane(1).data(),\n> +\t\t\t\t\t destinationStride_[1],\n> +\t\t\t\t\t destinationSize_.width,\n> +\t\t\t\t\t destinationSize_.height);\n> +\t\tbreak;\n> +\tdefault:\n> +\t\tLOG(YUV, Error) << \"Unsupported source format \" << sourceFormat_;\n> +\t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> +\t\tbreak;\n\nThis can't happen, can it ?\n\n> +\t}\n> +\n> \tif (ret) {\n> -\t\tLOG(YUV, Error) << \"Failed NV12 scaling: \" << ret;\n> +\t\tLOG(YUV, Error) << \"Libyuv operation failure: \" << ret;\n\ns/Libyuv/libyuv/\n\nAlthough I would write \"YUV post-processing failure: \".\n\n> \t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> \t\treturn;\n> \t}\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5F42FC326B\n\tfor ;\n\tSun, 24 Sep 2023 21:33:54 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B03F962944;\n\tSun, 24 Sep 2023 23:33:53 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 473D062916\n\tfor ;\n\tSun, 24 Sep 2023 23:33:52 +0200 (CEST)","from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi\n\t[213.243.189.158])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 3DD386BE;\n\tSun, 24 Sep 2023 23:32:12 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1695591233;\n\tbh=2uFwbc0kUT3ltvgdUTFtPrD5fYM3Yh/5UmnE9n7jiLU=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=hI40UJ6EXNn868jPicjCVhlgGYj9a640Z5Ko2DRdSMic9psWjqxX1czPU138Ceqkb\n\tln2YgVare156H4WFWMJrb3JaCKOgIrmV2RQoUq4OqmFHEc1h4kh7P8bbvqPSn1jrgR\n\tTwi8g0cgUKz1CqVQ3C+sFUytqHk5HGc368o6ZhOKGGkrJS6wv+iz5fxsaK2UGHBjff\n\tIOhM/tHhMn0nVTVIjF01OtPTkeMwU2bT57jvAQzAGyxw5DQl0hai1gGSLHzL2nUow1\n\tJJaG0i2iqUiVz5z0Cgs2pcu5A/0NHohJLfzJL94OTi2W6zcE1SE22m6JWzDr4W3Tcf\n\trXQ+ixrYipCfw==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1695591132;\n\tbh=2uFwbc0kUT3ltvgdUTFtPrD5fYM3Yh/5UmnE9n7jiLU=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=f1aykJmAlk2k+Q81bc71qIDGQyPgJSbYlUxw92bt3k11Fv+HwfIcxFU/SEVIfb+Zj\n\tw/ZshV0DipTeSqHyrFaJKfpqLcOnKkxeZ0X/RZfmYN/BSG48h2nyqPqoiRqNS6McHX\n\tsFIsovNQpDrdX880oTpgYbG/Asw+uoe7NRHbxsak="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"f1aykJmA\"; dkim-atps=neutral","Date":"Mon, 25 Sep 2023 00:34:03 +0300","To":"Mattijs Korpershoek ","Message-ID":"<20230924213403.GD6390@pendragon.ideasonboard.com>","References":"<20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com>\n\t<20230915-libyuv-convert-v1-6-1e5bcf68adac@baylibre.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20230915-libyuv-convert-v1-6-1e5bcf68adac@baylibre.com>","Subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org,\n\tGuillaume La Roque ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":27872,"web_url":"https://patchwork.libcamera.org/comment/27872/","msgid":"","date":"2023-09-25T10:52:40","subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","submitter":{"id":109,"url":"https://patchwork.libcamera.org/api/people/109/","name":"Tomi Valkeinen","email":"tomi.valkeinen@ideasonboard.com"},"content":"On 25/09/2023 00:34, Laurent Pinchart wrote:\n> Hi Mattijs,\n> \n> Thank you for the patch.\n> \n> On Fri, Sep 15, 2023 at 09:57:30AM +0200, Mattijs Korpershoek via libcamera-devel wrote:\n>> On am62x platforms, the receiver driver (j721e-csi2rx) only\n>> supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.\n>>\n>> The receiver and the sensor (ov5640) hardware are both capable of\n>> YUV420, however:\n>>\n>> * we are not aware of OV5640 being tested with YUV420 formats on any\n>> vendor tree.\n> \n> Are you talking about packed YUV 4:2:0 here, or planar YUV 4:2:0 ?\n> Sensors only output packed formats (there could be exceptions, but they\n> would be very rare), it is the DMA engine on the host size that would\n> convert them to planar formats. That would be the DMA engine handled by\n> the k3-udma driver. It's fairly complicated, and I don't know if it\n> would hahev the ability to output planar formats. Tomi (on CC) may have\n> more information.\n\nNo, I don't know if the CSI-2 RX or the DMA engine support packed to \nplanar conversions. Maybe Jai knows.\n\nIf the point is to show the frames on the screen, it sounds to me that \nthe YUV422 formats would work the best.\n\n Tomi\n\n>> * NV12 has different line lines (even lines are twice as long as odd\n> \n> s/lines/lengths/\n> \n>> lines) Different line-sized DMA transfers have not been tested on\n> \n> s/)/)./\n> \n>> the TI CSI-RX SHIM IP.\n> \n> As mentioned just above, the sensor will not output planar formats, so\n> the issue is about the\n> \n>> On the other hand, the graphics allocator (gralloc) cannot allocate\n>> YUV422 buffers directly. It mainly allocated NV12 buffers when\n>> userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED.\n> \n> The DSS seems to support YUYV, is there a way to fix the TI gralloc to\n> support it ?\n> \n>> Because of the above, we need a pixel conversion from YUYV to NV12,\n>> which is handled in the yuv processor via libyuv.\n>>\n>> Signed-off-by: Mattijs Korpershoek \n>> ---\n>> src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++---------\n>> 1 file changed, 68 insertions(+), 23 deletions(-)\n>>\n>> diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\n>> index 734bb85b7351..b680b833dafb 100644\n>> --- a/src/android/yuv/post_processor_yuv.cpp\n>> +++ b/src/android/yuv/post_processor_yuv.cpp\n>> @@ -7,6 +7,9 @@\n>> \n>> #include \"post_processor_yuv.h\"\n>> \n>> +#include \n>> +\n>> +#include \n>> #include \n>> \n>> #include \n>> @@ -22,14 +25,42 @@ using namespace libcamera;\n>> \n>> LOG_DEFINE_CATEGORY(YUV)\n>> \n>> +namespace {\n>> +\n>> +/**\n>> + * \\var supportedConversions\n>> + * \\brief list of supported output pixel formats for an input pixel format\n>> + */\n>> +const std::map> supportedConversions = {\n>> +\t{ formats::YUYV, { formats::NV12 } },\n>> +};\n>> +\n>> +} /* namespace */\n>> +\n>> int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n>> \t\t\t\tconst StreamConfiguration &outCfg)\n>> {\n>> \tif (inCfg.pixelFormat != outCfg.pixelFormat) {\n>> -\t\tLOG(YUV, Error) << \"Pixel format conversion is not supported\"\n>> -\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n>> -\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n>> -\t\treturn -EINVAL;\n>> +\t\tconst auto it = supportedConversions.find(inCfg.pixelFormat);\n>> +\t\tif (it == supportedConversions.end()) {\n>> +\t\t\tLOG(YUV, Error) << \"Unsupported source format \" << inCfg.pixelFormat;\n> \n> \t\t\tLOG(YUV, Error)\n> \t\t\t\t<< \"Unsupported source format \"\n> \t\t\t\t<< inCfg.pixelFormat;\n> \n>> +\t\t\treturn -EINVAL;\n>> +\t\t}\n>> +\n>> +\t\tstd::vector outFormats = it->second;\n> \n> No need for a copy.\n> \n>> +\t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat);\n> \n> \t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(),\n> \t\t\t\t\t outCfg.pixelFormat);\n> \n>> +\t\tif (match == outFormats.end()) {\n>> +\t\t\tLOG(YUV, Error) << \"Requested pixel format conversion is not supported\"\n>> +\t\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n>> +\t\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> \n> \t\t\tLOG(YUV, Error)\n> \t\t\t\t<< \"Requested pixel format conversion is not supported\"\n> \t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> \t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> \n>> +\t\t\treturn -EINVAL;\n>> +\t\t}\n>> +\t} else {\n>> +\t\tif (inCfg.pixelFormat != formats::NV12) {\n>> +\t\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n>> +\t\t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> \n> \t\t\tLOG(YUV, Error)\n> \t\t\t\t<< \"Unsupported format \" << inCfg.pixelFormat\n> \t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> \n>> +\t\t\treturn -EINVAL;\n>> +\t\t}\n>> \t}\n>> \n>> \tif (inCfg.size < outCfg.size) {\n>> @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n>> \t\treturn -EINVAL;\n>> \t}\n>> \n>> -\tif (inCfg.pixelFormat != formats::NV12) {\n>> -\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n>> -\t\t\t\t<< \" (only NV12 is supported)\";\n>> -\t\treturn -EINVAL;\n>> -\t}\n>> -\n>> \tcalculateLengths(inCfg, outCfg);\n>> \treturn 0;\n>> }\n>> @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf\n>> \t\treturn;\n>> \t}\n>> \n>> -\tint ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n>> -\t\t\t\t sourceStride_[0],\n>> -\t\t\t\t sourceMapped.planes()[1].data(),\n>> -\t\t\t\t sourceStride_[1],\n>> -\t\t\t\t sourceSize_.width, sourceSize_.height,\n>> -\t\t\t\t destination->plane(0).data(),\n>> -\t\t\t\t destinationStride_[0],\n>> -\t\t\t\t destination->plane(1).data(),\n>> -\t\t\t\t destinationStride_[1],\n>> -\t\t\t\t destinationSize_.width,\n>> -\t\t\t\t destinationSize_.height,\n>> -\t\t\t\t libyuv::FilterMode::kFilterBilinear);\n>> +\tint ret = 0;\n>> +\tswitch (sourceFormat_) {\n>> +\tcase formats::NV12:\n>> +\t\tret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n>> +\t\t\t\t\tsourceStride_[0],\n>> +\t\t\t\t\tsourceMapped.planes()[1].data(),\n>> +\t\t\t\t\tsourceStride_[1],\n>> +\t\t\t\t\tsourceSize_.width, sourceSize_.height,\n>> +\t\t\t\t\tdestination->plane(0).data(),\n>> +\t\t\t\t\tdestinationStride_[0],\n>> +\t\t\t\t\tdestination->plane(1).data(),\n>> +\t\t\t\t\tdestinationStride_[1],\n>> +\t\t\t\t\tdestinationSize_.width,\n>> +\t\t\t\t\tdestinationSize_.height,\n>> +\t\t\t\t\tlibyuv::FilterMode::kFilterBilinear);\n>> +\t\tbreak;\n>> +\tcase formats::YUYV:\n>> +\t\tret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(),\n>> +\t\t\t\t\t sourceStride_[0],\n>> +\t\t\t\t\t destination->plane(0).data(),\n>> +\t\t\t\t\t destinationStride_[0],\n>> +\t\t\t\t\t destination->plane(1).data(),\n>> +\t\t\t\t\t destinationStride_[1],\n>> +\t\t\t\t\t destinationSize_.width,\n>> +\t\t\t\t\t destinationSize_.height);\n>> +\t\tbreak;\n>> +\tdefault:\n>> +\t\tLOG(YUV, Error) << \"Unsupported source format \" << sourceFormat_;\n>> +\t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n>> +\t\tbreak;\n> \n> This can't happen, can it ?\n> \n>> +\t}\n>> +\n>> \tif (ret) {\n>> -\t\tLOG(YUV, Error) << \"Failed NV12 scaling: \" << ret;\n>> +\t\tLOG(YUV, Error) << \"Libyuv operation failure: \" << ret;\n> \n> s/Libyuv/libyuv/\n> \n> Although I would write \"YUV post-processing failure: \".\n> \n>> \t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n>> \t\treturn;\n>> \t}\n>>\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4B717C326B\n\tfor ;\n\tMon, 25 Sep 2023 10:52:47 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9D7D362944;\n\tMon, 25 Sep 2023 12:52:46 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B7C35628FA\n\tfor ;\n\tMon, 25 Sep 2023 12:52:44 +0200 (CEST)","from [192.168.88.20] (91-157-153-81.elisa-laajakaista.fi\n\t[91.157.153.81])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 525516EF;\n\tMon, 25 Sep 2023 12:51:04 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1695639166;\n\tbh=DXKSFulGPnbXTwmM2jxWrsI9h4Sm3EoZKt6wOUIdu0k=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=TeYcla+3vEH/CUAug3UfoeT4d5c8ZkGwZf0fHyxoSFpfsMznkYPiVXgPwiiHfyht1\n\tAbX8atyXACsiYU7sC26ac4K8bEX9ywMmrUc2XZ54we+qzFMmYeMzlP5oLBJxdmBvDm\n\t3jjUZ8vn9YSe2r1t+RmdHXn+3Hckm28YQzGZDF+E1fU2WVucyT45sJHcWpZQNjRpUd\n\t9XSuZnJMuztY6bkHmb3uoxH85+5u+WQ6dSSjMBBd5u/F8+fLWsEAGCJPsMGx1hwoQ3\n\tAVHLZdWBF06eTBMXIoE29XeykGFb5HSEZ5n6DLJCmi18b59TKKt7WVk7GWRpBmb/oo\n\tEF9B9hfYNL7VQ==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1695639064;\n\tbh=DXKSFulGPnbXTwmM2jxWrsI9h4Sm3EoZKt6wOUIdu0k=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=X0XBP15VfPpQMrEiK4hrMw1AHTcvlaRhKrSXI87fuKvpLvoH4/8Le2nAnUg6YM6A6\n\t7TrX0IPie+akLcB1Nus6B/cwAeb7+e++pY5sY+/rBOHC5HFzaY/KMpHg6Rhtng/Xek\n\t52wg9vjIWgYXfyGFFCHtulgFriyf2biHJmGFSU88="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"X0XBP15V\"; dkim-atps=neutral","Message-ID":"","Date":"Mon, 25 Sep 2023 13:52:40 +0300","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101\n\tThunderbird/102.15.1","Content-Language":"en-US","To":"Laurent Pinchart ,\n\tMattijs Korpershoek ","References":"<20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com>\n\t<20230915-libyuv-convert-v1-6-1e5bcf68adac@baylibre.com>\n\t<20230924213403.GD6390@pendragon.ideasonboard.com>","In-Reply-To":"<20230924213403.GD6390@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Tomi Valkeinen via libcamera-devel\n\t","Reply-To":"Tomi Valkeinen ","Cc":"libcamera-devel@lists.libcamera.org,\n\tGuillaume La Roque ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":27873,"web_url":"https://patchwork.libcamera.org/comment/27873/","msgid":"<20230925105653.GA375@pendragon.ideasonboard.com>","date":"2023-09-25T10:56:53","subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Mon, Sep 25, 2023 at 01:52:40PM +0300, Tomi Valkeinen wrote:\n> On 25/09/2023 00:34, Laurent Pinchart wrote:\n> > Hi Mattijs,\n> > \n> > Thank you for the patch.\n> > \n> > On Fri, Sep 15, 2023 at 09:57:30AM +0200, Mattijs Korpershoek via libcamera-devel wrote:\n> >> On am62x platforms, the receiver driver (j721e-csi2rx) only\n> >> supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.\n> >>\n> >> The receiver and the sensor (ov5640) hardware are both capable of\n> >> YUV420, however:\n> >>\n> >> * we are not aware of OV5640 being tested with YUV420 formats on any\n> >> vendor tree.\n> > \n> > Are you talking about packed YUV 4:2:0 here, or planar YUV 4:2:0 ?\n> > Sensors only output packed formats (there could be exceptions, but they\n> > would be very rare), it is the DMA engine on the host size that would\n> > convert them to planar formats. That would be the DMA engine handled by\n> > the k3-udma driver. It's fairly complicated, and I don't know if it\n> > would hahev the ability to output planar formats. Tomi (on CC) may have\n> > more information.\n> \n> No, I don't know if the CSI-2 RX or the DMA engine support packed to \n> planar conversions. Maybe Jai knows.\n> \n> If the point is to show the frames on the screen, it sounds to me that \n> the YUV422 formats would work the best.\n\nDo you mean packed YUV 4:2:2 or (a.k.a. YUYV & co), semi-planar YUV\n4:2:2 (a.k.a. NV16 and NV61) or planar YUV 4:2:2 ? -)\n\n> >> * NV12 has different line lines (even lines are twice as long as odd\n> > \n> > s/lines/lengths/\n> > \n> >> lines) Different line-sized DMA transfers have not been tested on\n> > \n> > s/)/)./\n> > \n> >> the TI CSI-RX SHIM IP.\n> > \n> > As mentioned just above, the sensor will not output planar formats, so\n> > the issue is about the\n> > \n> >> On the other hand, the graphics allocator (gralloc) cannot allocate\n> >> YUV422 buffers directly. It mainly allocated NV12 buffers when\n> >> userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED.\n> > \n> > The DSS seems to support YUYV, is there a way to fix the TI gralloc to\n> > support it ?\n> > \n> >> Because of the above, we need a pixel conversion from YUYV to NV12,\n> >> which is handled in the yuv processor via libyuv.\n> >>\n> >> Signed-off-by: Mattijs Korpershoek \n> >> ---\n> >> src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++---------\n> >> 1 file changed, 68 insertions(+), 23 deletions(-)\n> >>\n> >> diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\n> >> index 734bb85b7351..b680b833dafb 100644\n> >> --- a/src/android/yuv/post_processor_yuv.cpp\n> >> +++ b/src/android/yuv/post_processor_yuv.cpp\n> >> @@ -7,6 +7,9 @@\n> >> \n> >> #include \"post_processor_yuv.h\"\n> >> \n> >> +#include \n> >> +\n> >> +#include \n> >> #include \n> >> \n> >> #include \n> >> @@ -22,14 +25,42 @@ using namespace libcamera;\n> >> \n> >> LOG_DEFINE_CATEGORY(YUV)\n> >> \n> >> +namespace {\n> >> +\n> >> +/**\n> >> + * \\var supportedConversions\n> >> + * \\brief list of supported output pixel formats for an input pixel format\n> >> + */\n> >> +const std::map> supportedConversions = {\n> >> +\t{ formats::YUYV, { formats::NV12 } },\n> >> +};\n> >> +\n> >> +} /* namespace */\n> >> +\n> >> int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> >> \t\t\t\tconst StreamConfiguration &outCfg)\n> >> {\n> >> \tif (inCfg.pixelFormat != outCfg.pixelFormat) {\n> >> -\t\tLOG(YUV, Error) << \"Pixel format conversion is not supported\"\n> >> -\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> >> -\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> >> -\t\treturn -EINVAL;\n> >> +\t\tconst auto it = supportedConversions.find(inCfg.pixelFormat);\n> >> +\t\tif (it == supportedConversions.end()) {\n> >> +\t\t\tLOG(YUV, Error) << \"Unsupported source format \" << inCfg.pixelFormat;\n> > \n> > \t\t\tLOG(YUV, Error)\n> > \t\t\t\t<< \"Unsupported source format \"\n> > \t\t\t\t<< inCfg.pixelFormat;\n> > \n> >> +\t\t\treturn -EINVAL;\n> >> +\t\t}\n> >> +\n> >> +\t\tstd::vector outFormats = it->second;\n> > \n> > No need for a copy.\n> > \n> >> +\t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat);\n> > \n> > \t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(),\n> > \t\t\t\t\t outCfg.pixelFormat);\n> > \n> >> +\t\tif (match == outFormats.end()) {\n> >> +\t\t\tLOG(YUV, Error) << \"Requested pixel format conversion is not supported\"\n> >> +\t\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> >> +\t\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > \n> > \t\t\tLOG(YUV, Error)\n> > \t\t\t\t<< \"Requested pixel format conversion is not supported\"\n> > \t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > \t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > \n> >> +\t\t\treturn -EINVAL;\n> >> +\t\t}\n> >> +\t} else {\n> >> +\t\tif (inCfg.pixelFormat != formats::NV12) {\n> >> +\t\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> >> +\t\t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > \n> > \t\t\tLOG(YUV, Error)\n> > \t\t\t\t<< \"Unsupported format \" << inCfg.pixelFormat\n> > \t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > \n> >> +\t\t\treturn -EINVAL;\n> >> +\t\t}\n> >> \t}\n> >> \n> >> \tif (inCfg.size < outCfg.size) {\n> >> @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> >> \t\treturn -EINVAL;\n> >> \t}\n> >> \n> >> -\tif (inCfg.pixelFormat != formats::NV12) {\n> >> -\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> >> -\t\t\t\t<< \" (only NV12 is supported)\";\n> >> -\t\treturn -EINVAL;\n> >> -\t}\n> >> -\n> >> \tcalculateLengths(inCfg, outCfg);\n> >> \treturn 0;\n> >> }\n> >> @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf\n> >> \t\treturn;\n> >> \t}\n> >> \n> >> -\tint ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> >> -\t\t\t\t sourceStride_[0],\n> >> -\t\t\t\t sourceMapped.planes()[1].data(),\n> >> -\t\t\t\t sourceStride_[1],\n> >> -\t\t\t\t sourceSize_.width, sourceSize_.height,\n> >> -\t\t\t\t destination->plane(0).data(),\n> >> -\t\t\t\t destinationStride_[0],\n> >> -\t\t\t\t destination->plane(1).data(),\n> >> -\t\t\t\t destinationStride_[1],\n> >> -\t\t\t\t destinationSize_.width,\n> >> -\t\t\t\t destinationSize_.height,\n> >> -\t\t\t\t libyuv::FilterMode::kFilterBilinear);\n> >> +\tint ret = 0;\n> >> +\tswitch (sourceFormat_) {\n> >> +\tcase formats::NV12:\n> >> +\t\tret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> >> +\t\t\t\t\tsourceStride_[0],\n> >> +\t\t\t\t\tsourceMapped.planes()[1].data(),\n> >> +\t\t\t\t\tsourceStride_[1],\n> >> +\t\t\t\t\tsourceSize_.width, sourceSize_.height,\n> >> +\t\t\t\t\tdestination->plane(0).data(),\n> >> +\t\t\t\t\tdestinationStride_[0],\n> >> +\t\t\t\t\tdestination->plane(1).data(),\n> >> +\t\t\t\t\tdestinationStride_[1],\n> >> +\t\t\t\t\tdestinationSize_.width,\n> >> +\t\t\t\t\tdestinationSize_.height,\n> >> +\t\t\t\t\tlibyuv::FilterMode::kFilterBilinear);\n> >> +\t\tbreak;\n> >> +\tcase formats::YUYV:\n> >> +\t\tret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(),\n> >> +\t\t\t\t\t sourceStride_[0],\n> >> +\t\t\t\t\t destination->plane(0).data(),\n> >> +\t\t\t\t\t destinationStride_[0],\n> >> +\t\t\t\t\t destination->plane(1).data(),\n> >> +\t\t\t\t\t destinationStride_[1],\n> >> +\t\t\t\t\t destinationSize_.width,\n> >> +\t\t\t\t\t destinationSize_.height);\n> >> +\t\tbreak;\n> >> +\tdefault:\n> >> +\t\tLOG(YUV, Error) << \"Unsupported source format \" << sourceFormat_;\n> >> +\t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> >> +\t\tbreak;\n> > \n> > This can't happen, can it ?\n> > \n> >> +\t}\n> >> +\n> >> \tif (ret) {\n> >> -\t\tLOG(YUV, Error) << \"Failed NV12 scaling: \" << ret;\n> >> +\t\tLOG(YUV, Error) << \"Libyuv operation failure: \" << ret;\n> > \n> > s/Libyuv/libyuv/\n> > \n> > Although I would write \"YUV post-processing failure: \".\n> > \n> >> \t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> >> \t\treturn;\n> >> \t}\n> >>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id AB5C5BD808\n\tfor ;\n\tMon, 25 Sep 2023 10:56:46 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EA2E162945;\n\tMon, 25 Sep 2023 12:56:45 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3399062931\n\tfor ;\n\tMon, 25 Sep 2023 12:56:44 +0200 (CEST)","from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi\n\t[213.243.189.158])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 86A3375A;\n\tMon, 25 Sep 2023 12:55:03 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1695639406;\n\tbh=ori1zslkWJO0NrLryoLouypbDvD3Q4yP1GkTuHRSBSs=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=r9qr1UP+djCQ647tHSpWWr6/fOl4ys62JBGn9lDf8nswaTfK5rirHYm1pHAFAJFG9\n\tBnXuLZQqCT8NcTpWVlZDG0P/LE34iRU4fMHd2HFQT9BN+vSRWpFxfsLF42sCcgEaiN\n\tdjX6x1uEsJ6O3CmD6vS7y7ARN5NGHFJyBKeQMaGrKT0tJQIOL1DkE3Lq8B/Tv+UJvR\n\tH1w2m9f5+npXCGgYfHcF6kcRowfVcuvezBaBmmo4p6cDKsuvkC5E4EGhAMxyxhyuNq\n\t6a5exyACBVJSktXNJhWJ3xVsZCAMz5XRWYvV+c01kAIEoAW3cIPqyIUlddB3NosJB+\n\titdByQOD9RBsg==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1695639303;\n\tbh=ori1zslkWJO0NrLryoLouypbDvD3Q4yP1GkTuHRSBSs=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Ldo0xIlASyli0Z9ncC7nvfD8OPwDvmwB7usm1zbEmU3TbNp3Sde3nfSpk3OTIqlJp\n\taiB2eGYs9JrvTOcBT2ItUFkNXTJjM7sVAjadhQuS2kdNVWrexTaiRp15PMLyDybCjs\n\tU/p7vnWRWPap/rOB+78ZfKLVZrc8ZV4YS6jj2HJw="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"Ldo0xIlA\"; dkim-atps=neutral","Date":"Mon, 25 Sep 2023 13:56:53 +0300","To":"Tomi Valkeinen ","Message-ID":"<20230925105653.GA375@pendragon.ideasonboard.com>","References":"<20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com>\n\t<20230915-libyuv-convert-v1-6-1e5bcf68adac@baylibre.com>\n\t<20230924213403.GD6390@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org,\n\tGuillaume La Roque ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":27874,"web_url":"https://patchwork.libcamera.org/comment/27874/","msgid":"","date":"2023-09-25T10:59:30","subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","submitter":{"id":109,"url":"https://patchwork.libcamera.org/api/people/109/","name":"Tomi Valkeinen","email":"tomi.valkeinen@ideasonboard.com"},"content":"On 25/09/2023 13:56, Laurent Pinchart wrote:\n> On Mon, Sep 25, 2023 at 01:52:40PM +0300, Tomi Valkeinen wrote:\n>> On 25/09/2023 00:34, Laurent Pinchart wrote:\n>>> Hi Mattijs,\n>>>\n>>> Thank you for the patch.\n>>>\n>>> On Fri, Sep 15, 2023 at 09:57:30AM +0200, Mattijs Korpershoek via libcamera-devel wrote:\n>>>> On am62x platforms, the receiver driver (j721e-csi2rx) only\n>>>> supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.\n>>>>\n>>>> The receiver and the sensor (ov5640) hardware are both capable of\n>>>> YUV420, however:\n>>>>\n>>>> * we are not aware of OV5640 being tested with YUV420 formats on any\n>>>> vendor tree.\n>>>\n>>> Are you talking about packed YUV 4:2:0 here, or planar YUV 4:2:0 ?\n>>> Sensors only output packed formats (there could be exceptions, but they\n>>> would be very rare), it is the DMA engine on the host size that would\n>>> convert them to planar formats. That would be the DMA engine handled by\n>>> the k3-udma driver. It's fairly complicated, and I don't know if it\n>>> would hahev the ability to output planar formats. Tomi (on CC) may have\n>>> more information.\n>>\n>> No, I don't know if the CSI-2 RX or the DMA engine support packed to\n>> planar conversions. Maybe Jai knows.\n>>\n>> If the point is to show the frames on the screen, it sounds to me that\n>> the YUV422 formats would work the best.\n> \n> Do you mean packed YUV 4:2:2 or (a.k.a. YUYV & co), semi-planar YUV\n> 4:2:2 (a.k.a. NV16 and NV61) or planar YUV 4:2:2 ? -)\n\nPacked. That's the only option the caneras will will send, isn't it? But \nI could rephrase: any format that is supported both by the DSS and the \nsensor =).\n\n Tomi\n\n>>>> * NV12 has different line lines (even lines are twice as long as odd\n>>>\n>>> s/lines/lengths/\n>>>\n>>>> lines) Different line-sized DMA transfers have not been tested on\n>>>\n>>> s/)/)./\n>>>\n>>>> the TI CSI-RX SHIM IP.\n>>>\n>>> As mentioned just above, the sensor will not output planar formats, so\n>>> the issue is about the\n>>>\n>>>> On the other hand, the graphics allocator (gralloc) cannot allocate\n>>>> YUV422 buffers directly. It mainly allocated NV12 buffers when\n>>>> userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED.\n>>>\n>>> The DSS seems to support YUYV, is there a way to fix the TI gralloc to\n>>> support it ?\n>>>\n>>>> Because of the above, we need a pixel conversion from YUYV to NV12,\n>>>> which is handled in the yuv processor via libyuv.\n>>>>\n>>>> Signed-off-by: Mattijs Korpershoek \n>>>> ---\n>>>> src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++---------\n>>>> 1 file changed, 68 insertions(+), 23 deletions(-)\n>>>>\n>>>> diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\n>>>> index 734bb85b7351..b680b833dafb 100644\n>>>> --- a/src/android/yuv/post_processor_yuv.cpp\n>>>> +++ b/src/android/yuv/post_processor_yuv.cpp\n>>>> @@ -7,6 +7,9 @@\n>>>> \n>>>> #include \"post_processor_yuv.h\"\n>>>> \n>>>> +#include \n>>>> +\n>>>> +#include \n>>>> #include \n>>>> \n>>>> #include \n>>>> @@ -22,14 +25,42 @@ using namespace libcamera;\n>>>> \n>>>> LOG_DEFINE_CATEGORY(YUV)\n>>>> \n>>>> +namespace {\n>>>> +\n>>>> +/**\n>>>> + * \\var supportedConversions\n>>>> + * \\brief list of supported output pixel formats for an input pixel format\n>>>> + */\n>>>> +const std::map> supportedConversions = {\n>>>> +\t{ formats::YUYV, { formats::NV12 } },\n>>>> +};\n>>>> +\n>>>> +} /* namespace */\n>>>> +\n>>>> int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n>>>> \t\t\t\tconst StreamConfiguration &outCfg)\n>>>> {\n>>>> \tif (inCfg.pixelFormat != outCfg.pixelFormat) {\n>>>> -\t\tLOG(YUV, Error) << \"Pixel format conversion is not supported\"\n>>>> -\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n>>>> -\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n>>>> -\t\treturn -EINVAL;\n>>>> +\t\tconst auto it = supportedConversions.find(inCfg.pixelFormat);\n>>>> +\t\tif (it == supportedConversions.end()) {\n>>>> +\t\t\tLOG(YUV, Error) << \"Unsupported source format \" << inCfg.pixelFormat;\n>>>\n>>> \t\t\tLOG(YUV, Error)\n>>> \t\t\t\t<< \"Unsupported source format \"\n>>> \t\t\t\t<< inCfg.pixelFormat;\n>>>\n>>>> +\t\t\treturn -EINVAL;\n>>>> +\t\t}\n>>>> +\n>>>> +\t\tstd::vector outFormats = it->second;\n>>>\n>>> No need for a copy.\n>>>\n>>>> +\t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat);\n>>>\n>>> \t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(),\n>>> \t\t\t\t\t outCfg.pixelFormat);\n>>>\n>>>> +\t\tif (match == outFormats.end()) {\n>>>> +\t\t\tLOG(YUV, Error) << \"Requested pixel format conversion is not supported\"\n>>>> +\t\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n>>>> +\t\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n>>>\n>>> \t\t\tLOG(YUV, Error)\n>>> \t\t\t\t<< \"Requested pixel format conversion is not supported\"\n>>> \t\t\t\t<< \" (from \" << inCfg.pixelFormat\n>>> \t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n>>>\n>>>> +\t\t\treturn -EINVAL;\n>>>> +\t\t}\n>>>> +\t} else {\n>>>> +\t\tif (inCfg.pixelFormat != formats::NV12) {\n>>>> +\t\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n>>>> +\t\t\t\t\t<< \" (only NV12 is supported for scaling)\";\n>>>\n>>> \t\t\tLOG(YUV, Error)\n>>> \t\t\t\t<< \"Unsupported format \" << inCfg.pixelFormat\n>>> \t\t\t\t<< \" (only NV12 is supported for scaling)\";\n>>>\n>>>> +\t\t\treturn -EINVAL;\n>>>> +\t\t}\n>>>> \t}\n>>>> \n>>>> \tif (inCfg.size < outCfg.size) {\n>>>> @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n>>>> \t\treturn -EINVAL;\n>>>> \t}\n>>>> \n>>>> -\tif (inCfg.pixelFormat != formats::NV12) {\n>>>> -\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n>>>> -\t\t\t\t<< \" (only NV12 is supported)\";\n>>>> -\t\treturn -EINVAL;\n>>>> -\t}\n>>>> -\n>>>> \tcalculateLengths(inCfg, outCfg);\n>>>> \treturn 0;\n>>>> }\n>>>> @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf\n>>>> \t\treturn;\n>>>> \t}\n>>>> \n>>>> -\tint ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n>>>> -\t\t\t\t sourceStride_[0],\n>>>> -\t\t\t\t sourceMapped.planes()[1].data(),\n>>>> -\t\t\t\t sourceStride_[1],\n>>>> -\t\t\t\t sourceSize_.width, sourceSize_.height,\n>>>> -\t\t\t\t destination->plane(0).data(),\n>>>> -\t\t\t\t destinationStride_[0],\n>>>> -\t\t\t\t destination->plane(1).data(),\n>>>> -\t\t\t\t destinationStride_[1],\n>>>> -\t\t\t\t destinationSize_.width,\n>>>> -\t\t\t\t destinationSize_.height,\n>>>> -\t\t\t\t libyuv::FilterMode::kFilterBilinear);\n>>>> +\tint ret = 0;\n>>>> +\tswitch (sourceFormat_) {\n>>>> +\tcase formats::NV12:\n>>>> +\t\tret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n>>>> +\t\t\t\t\tsourceStride_[0],\n>>>> +\t\t\t\t\tsourceMapped.planes()[1].data(),\n>>>> +\t\t\t\t\tsourceStride_[1],\n>>>> +\t\t\t\t\tsourceSize_.width, sourceSize_.height,\n>>>> +\t\t\t\t\tdestination->plane(0).data(),\n>>>> +\t\t\t\t\tdestinationStride_[0],\n>>>> +\t\t\t\t\tdestination->plane(1).data(),\n>>>> +\t\t\t\t\tdestinationStride_[1],\n>>>> +\t\t\t\t\tdestinationSize_.width,\n>>>> +\t\t\t\t\tdestinationSize_.height,\n>>>> +\t\t\t\t\tlibyuv::FilterMode::kFilterBilinear);\n>>>> +\t\tbreak;\n>>>> +\tcase formats::YUYV:\n>>>> +\t\tret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(),\n>>>> +\t\t\t\t\t sourceStride_[0],\n>>>> +\t\t\t\t\t destination->plane(0).data(),\n>>>> +\t\t\t\t\t destinationStride_[0],\n>>>> +\t\t\t\t\t destination->plane(1).data(),\n>>>> +\t\t\t\t\t destinationStride_[1],\n>>>> +\t\t\t\t\t destinationSize_.width,\n>>>> +\t\t\t\t\t destinationSize_.height);\n>>>> +\t\tbreak;\n>>>> +\tdefault:\n>>>> +\t\tLOG(YUV, Error) << \"Unsupported source format \" << sourceFormat_;\n>>>> +\t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n>>>> +\t\tbreak;\n>>>\n>>> This can't happen, can it ?\n>>>\n>>>> +\t}\n>>>> +\n>>>> \tif (ret) {\n>>>> -\t\tLOG(YUV, Error) << \"Failed NV12 scaling: \" << ret;\n>>>> +\t\tLOG(YUV, Error) << \"Libyuv operation failure: \" << ret;\n>>>\n>>> s/Libyuv/libyuv/\n>>>\n>>> Although I would write \"YUV post-processing failure: \".\n>>>\n>>>> \t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n>>>> \t\treturn;\n>>>> \t}\n>>>>\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B96C8C326B\n\tfor ;\n\tMon, 25 Sep 2023 10:59:35 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 307BA62944;\n\tMon, 25 Sep 2023 12:59:35 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B41DB628FA\n\tfor ;\n\tMon, 25 Sep 2023 12:59:33 +0200 (CEST)","from [192.168.88.20] (91-157-153-81.elisa-laajakaista.fi\n\t[91.157.153.81])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 3F7A46EF;\n\tMon, 25 Sep 2023 12:57:53 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1695639575;\n\tbh=7SSyNgQ1jseUTCpqipySz9Ja1EJ04X3fBSlBZ8SGrK8=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=HdNn98lC/DVsDTpmlf1CYOZpApQIDiJPmoVKdX2tXrFOtROXdIcxThfuYxv1lNi9f\n\t4CNC6/U2+P3mVBcdpYviegMh8GZMBiKB/+HLeYo1v4Gasn3PS059CTfDvuAcpnQcWE\n\t2oUCAWtgLTAMrQk+NrjGeGzUdmvhD6rncDPZNi28YuRggi3rm7bNBV7Bz8Pmj4WClT\n\th6XU8Bewi+VQhid17Iu5zeK25AiDfwNLRH9+wU1G7P4fdRQ+WjllYIxulER91GQHL7\n\t5feG596Wl0+jyNKaOcRWf4LmZzTXDsiJUkzFE6HmeukhTCgF54ooy8/ArhW/pDtiQs\n\tN7jneeKwTWxtg==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1695639473;\n\tbh=7SSyNgQ1jseUTCpqipySz9Ja1EJ04X3fBSlBZ8SGrK8=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=Xc46B0svYXtV0VPzNFLfuRP7826HOc1y045MJDLyTBAo8qw00/7jAG6z9/wSNdP3/\n\tPRyAgaFWqeZw/VszvD96n3VBSsjLP+M8Bq484RDPvocMsM82sh8xHjGpBaxQ4RcmjC\n\tbN+4MhDTeUCudxHI6HU7eriI4LjZGiuPd/xiqwqE="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"Xc46B0sv\"; dkim-atps=neutral","Message-ID":"","Date":"Mon, 25 Sep 2023 13:59:30 +0300","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101\n\tThunderbird/102.15.1","Content-Language":"en-US","To":"Laurent Pinchart ","References":"<20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com>\n\t<20230915-libyuv-convert-v1-6-1e5bcf68adac@baylibre.com>\n\t<20230924213403.GD6390@pendragon.ideasonboard.com>\n\t\n\t<20230925105653.GA375@pendragon.ideasonboard.com>","In-Reply-To":"<20230925105653.GA375@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Tomi Valkeinen via libcamera-devel\n\t","Reply-To":"Tomi Valkeinen ","Cc":"libcamera-devel@lists.libcamera.org,\n\tGuillaume La Roque ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":27875,"web_url":"https://patchwork.libcamera.org/comment/27875/","msgid":"","date":"2023-09-25T11:47:59","subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","submitter":{"id":170,"url":"https://patchwork.libcamera.org/api/people/170/","name":"Jai Luthra","email":"j-luthra@ti.com"},"content":"Hi,\n\nOn Sep 25, 2023 at 13:52:40 +0300, Tomi Valkeinen wrote:\n> On 25/09/2023 00:34, Laurent Pinchart wrote:\n> > Hi Mattijs,\n> > \n> > Thank you for the patch.\n> > \n> > On Fri, Sep 15, 2023 at 09:57:30AM +0200, Mattijs Korpershoek via libcamera-devel wrote:\n> > > On am62x platforms, the receiver driver (j721e-csi2rx) only\n> > > supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.\n> > > \n> > > The receiver and the sensor (ov5640) hardware are both capable of\n> > > YUV420, however:\n> > > \n> > > * we are not aware of OV5640 being tested with YUV420 formats on any\n> > > vendor tree.\n> > \n> > Are you talking about packed YUV 4:2:0 here, or planar YUV 4:2:0 ?\n> > Sensors only output packed formats (there could be exceptions, but they\n> > would be very rare), it is the DMA engine on the host size that would\n> > convert them to planar formats. That would be the DMA engine handled by\n> > the k3-udma driver. It's fairly complicated, and I don't know if it\n> > would hahev the ability to output planar formats. Tomi (on CC) may have\n> > more information.\n> \n> No, I don't know if the CSI-2 RX or the DMA engine support packed to planar\n> conversions. Maybe Jai knows.\n> \n> If the point is to show the frames on the screen, it sounds to me that the\n> YUV422 formats would work the best.\n\nUnfortunately the DMA hardware does not support pixel-level interleaving \nthat would be required to convert packed YUV422/420 formats to planar or \nsemi-planar formats.\n\nBut yes the display supports packed UYVY formats, we even use it for \ncamera to display pipelines on linux. The limitation on android is from \nthe gralloc implementation.\n\n> \n> Tomi\n> \n> > > * NV12 has different line lines (even lines are twice as long as odd\n> > \n> > s/lines/lengths/\n> > \n> > > lines) Different line-sized DMA transfers have not been tested on\n> > \n> > s/)/)./\n> > \n> > > the TI CSI-RX SHIM IP.\n> > \n> > As mentioned just above, the sensor will not output planar formats, so\n> > the issue is about the\n> > \n> > > On the other hand, the graphics allocator (gralloc) cannot allocate\n> > > YUV422 buffers directly. It mainly allocated NV12 buffers when\n> > > userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED.\n> > \n> > The DSS seems to support YUYV, is there a way to fix the TI gralloc to\n> > support it ?\n> > \n> > > Because of the above, we need a pixel conversion from YUYV to NV12,\n> > > which is handled in the yuv processor via libyuv.\n> > > \n> > > Signed-off-by: Mattijs Korpershoek \n> > > ---\n> > > src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++---------\n> > > 1 file changed, 68 insertions(+), 23 deletions(-)\n> > > \n> > > diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\n> > > index 734bb85b7351..b680b833dafb 100644\n> > > --- a/src/android/yuv/post_processor_yuv.cpp\n> > > +++ b/src/android/yuv/post_processor_yuv.cpp\n> > > @@ -7,6 +7,9 @@\n> > > #include \"post_processor_yuv.h\"\n> > > +#include \n> > > +\n> > > +#include \n> > > #include \n> > > #include \n> > > @@ -22,14 +25,42 @@ using namespace libcamera;\n> > > LOG_DEFINE_CATEGORY(YUV)\n> > > +namespace {\n> > > +\n> > > +/**\n> > > + * \\var supportedConversions\n> > > + * \\brief list of supported output pixel formats for an input pixel format\n> > > + */\n> > > +const std::map> supportedConversions = {\n> > > +\t{ formats::YUYV, { formats::NV12 } },\n> > > +};\n> > > +\n> > > +} /* namespace */\n> > > +\n> > > int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> > > \t\t\t\tconst StreamConfiguration &outCfg)\n> > > {\n> > > \tif (inCfg.pixelFormat != outCfg.pixelFormat) {\n> > > -\t\tLOG(YUV, Error) << \"Pixel format conversion is not supported\"\n> > > -\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > -\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > -\t\treturn -EINVAL;\n> > > +\t\tconst auto it = supportedConversions.find(inCfg.pixelFormat);\n> > > +\t\tif (it == supportedConversions.end()) {\n> > > +\t\t\tLOG(YUV, Error) << \"Unsupported source format \" << inCfg.pixelFormat;\n> > \n> > \t\t\tLOG(YUV, Error)\n> > \t\t\t\t<< \"Unsupported source format \"\n> > \t\t\t\t<< inCfg.pixelFormat;\n> > \n> > > +\t\t\treturn -EINVAL;\n> > > +\t\t}\n> > > +\n> > > +\t\tstd::vector outFormats = it->second;\n> > \n> > No need for a copy.\n> > \n> > > +\t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat);\n> > \n> > \t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(),\n> > \t\t\t\t\t outCfg.pixelFormat);\n> > \n> > > +\t\tif (match == outFormats.end()) {\n> > > +\t\t\tLOG(YUV, Error) << \"Requested pixel format conversion is not supported\"\n> > > +\t\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > +\t\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > \n> > \t\t\tLOG(YUV, Error)\n> > \t\t\t\t<< \"Requested pixel format conversion is not supported\"\n> > \t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > \t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > \n> > > +\t\t\treturn -EINVAL;\n> > > +\t\t}\n> > > +\t} else {\n> > > +\t\tif (inCfg.pixelFormat != formats::NV12) {\n> > > +\t\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> > > +\t\t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > \n> > \t\t\tLOG(YUV, Error)\n> > \t\t\t\t<< \"Unsupported format \" << inCfg.pixelFormat\n> > \t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > \n> > > +\t\t\treturn -EINVAL;\n> > > +\t\t}\n> > > \t}\n> > > \tif (inCfg.size < outCfg.size) {\n> > > @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> > > \t\treturn -EINVAL;\n> > > \t}\n> > > -\tif (inCfg.pixelFormat != formats::NV12) {\n> > > -\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> > > -\t\t\t\t<< \" (only NV12 is supported)\";\n> > > -\t\treturn -EINVAL;\n> > > -\t}\n> > > -\n> > > \tcalculateLengths(inCfg, outCfg);\n> > > \treturn 0;\n> > > }\n> > > @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf\n> > > \t\treturn;\n> > > \t}\n> > > -\tint ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> > > -\t\t\t\t sourceStride_[0],\n> > > -\t\t\t\t sourceMapped.planes()[1].data(),\n> > > -\t\t\t\t sourceStride_[1],\n> > > -\t\t\t\t sourceSize_.width, sourceSize_.height,\n> > > -\t\t\t\t destination->plane(0).data(),\n> > > -\t\t\t\t destinationStride_[0],\n> > > -\t\t\t\t destination->plane(1).data(),\n> > > -\t\t\t\t destinationStride_[1],\n> > > -\t\t\t\t destinationSize_.width,\n> > > -\t\t\t\t destinationSize_.height,\n> > > -\t\t\t\t libyuv::FilterMode::kFilterBilinear);\n> > > +\tint ret = 0;\n> > > +\tswitch (sourceFormat_) {\n> > > +\tcase formats::NV12:\n> > > +\t\tret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> > > +\t\t\t\t\tsourceStride_[0],\n> > > +\t\t\t\t\tsourceMapped.planes()[1].data(),\n> > > +\t\t\t\t\tsourceStride_[1],\n> > > +\t\t\t\t\tsourceSize_.width, sourceSize_.height,\n> > > +\t\t\t\t\tdestination->plane(0).data(),\n> > > +\t\t\t\t\tdestinationStride_[0],\n> > > +\t\t\t\t\tdestination->plane(1).data(),\n> > > +\t\t\t\t\tdestinationStride_[1],\n> > > +\t\t\t\t\tdestinationSize_.width,\n> > > +\t\t\t\t\tdestinationSize_.height,\n> > > +\t\t\t\t\tlibyuv::FilterMode::kFilterBilinear);\n> > > +\t\tbreak;\n> > > +\tcase formats::YUYV:\n> > > +\t\tret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(),\n> > > +\t\t\t\t\t sourceStride_[0],\n> > > +\t\t\t\t\t destination->plane(0).data(),\n> > > +\t\t\t\t\t destinationStride_[0],\n> > > +\t\t\t\t\t destination->plane(1).data(),\n> > > +\t\t\t\t\t destinationStride_[1],\n> > > +\t\t\t\t\t destinationSize_.width,\n> > > +\t\t\t\t\t destinationSize_.height);\n> > > +\t\tbreak;\n> > > +\tdefault:\n> > > +\t\tLOG(YUV, Error) << \"Unsupported source format \" << sourceFormat_;\n> > > +\t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> > > +\t\tbreak;\n> > \n> > This can't happen, can it ?\n> > \n> > > +\t}\n> > > +\n> > > \tif (ret) {\n> > > -\t\tLOG(YUV, Error) << \"Failed NV12 scaling: \" << ret;\n> > > +\t\tLOG(YUV, Error) << \"Libyuv operation failure: \" << ret;\n> > \n> > s/Libyuv/libyuv/\n> > \n> > Although I would write \"YUV post-processing failure: \".\n> > \n> > > \t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> > > \t\treturn;\n> > > \t}\n> > > \n> > \n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 92305BD808\n\tfor ;\n\tMon, 25 Sep 2023 11:48:14 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id F1E2C62931;\n\tMon, 25 Sep 2023 13:48:13 +0200 (CEST)","from fllv0015.ext.ti.com (fllv0015.ext.ti.com [198.47.19.141])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 60688628FA\n\tfor ;\n\tMon, 25 Sep 2023 13:48:12 +0200 (CEST)","from lelv0266.itg.ti.com ([10.180.67.225])\n\tby fllv0015.ext.ti.com (8.15.2/8.15.2) with ESMTP id 38PBm6Nc086430; \n\tMon, 25 Sep 2023 06:48:06 -0500","from DLEE100.ent.ti.com (dlee100.ent.ti.com [157.170.170.30])\n\tby lelv0266.itg.ti.com (8.15.2/8.15.2) with ESMTPS id 38PBm6m7124831\n\t(version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL);\n\tMon, 25 Sep 2023 06:48:06 -0500","from DLEE105.ent.ti.com (157.170.170.35) by DLEE100.ent.ti.com\n\t(157.170.170.30) with Microsoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23;\n\tMon, 25 Sep 2023 06:48:06 -0500","from fllv0039.itg.ti.com (10.64.41.19) by DLEE105.ent.ti.com\n\t(157.170.170.35) with Microsoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23\n\tvia Frontend Transport; Mon, 25 Sep 2023 06:48:06 -0500","from localhost (ileaxei01-snat.itg.ti.com [10.180.69.5])\n\tby fllv0039.itg.ti.com (8.15.2/8.15.2) with ESMTP id 38PBm4KO020655; \n\tMon, 25 Sep 2023 06:48:04 -0500"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1695642494;\n\tbh=fULb9VdfBxvM4RG0n2wUkReydwBPRK+k03F8G7skyoU=;\n\th=Date:To:In-Reply-To:Subject:List-Id:List-Unsubscribe:List-Archive:\n\tList-Post:List-Help:List-Subscribe:From:Reply-To:Cc:From;\n\tb=gJbsX1CIBPRDNP6VjVBRXi2rblFz17SkjAZ11Rd1kLiwwArB3pcds9/UVotQLRa6M\n\tnl2vW23xN2Uehz9M5IxdouGT89/hLXVFvP7wKKu+pORusbU6HEuXB8wgRTC1lsCQBH\n\twh0gsryB0DBoC7hiuI71Hb2rMJ1vE0Lv1O9RPGhfB1uwTUSnht429uIT3hQOuN8cM1\n\taZCxzTKXmQtX9jdNHZ5I6rITjMAhOyWxbDdL3k3nWVmEsOBbJYsCbbuwzUNC61464w\n\td/0CBctYny+Ef8+HWn7v5Y5haAhBJwDSohK8ff3hEz3MY5dUdSL2LQM9NM4X3M+pQX\n\tFN9dvttZI7xGg==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com;\n\ts=ti-com-17Q1; t=1695642486;\n\tbh=+S+UU6bQ0yRF9GAKYisQe+b8hBep7FXVzMy2Hn8q0Ho=;\n\th=Date:From:To:CC:Subject:In-Reply-To;\n\tb=kchQT55uLt7FQTTGk5iXWU7FsbsLsbgwkY544LfevP3N/tFsfzYKCXP1B4XUz09XI\n\tpBnGQUJ/nN2q28J0nZ6c0KFV46/Ydzv6x54x+nlZvKO5s5lu4tHZAUkkxVnkEeSgUo\n\tEdQRP/q1Iybo1DC5NwlsHc/abqlH2qITToB1D7OU="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ti.com header.i=@ti.com\n\theader.b=\"kchQT55u\"; dkim-atps=neutral","Date":"Mon, 25 Sep 2023 17:17:59 +0530","To":"Tomi Valkeinen , Laurent Pinchart\n\t, Mattijs Korpershoek\n\t","Message-ID":"","MIME-Version":"1.0","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Disposition":"inline","In-Reply-To":"","X-EXCLAIMER-MD-CONFIG":"e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180","Subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jai Luthra via libcamera-devel ","Reply-To":"Jai Luthra ","Cc":"libcamera-devel@lists.libcamera.org,\n\tGuillaume La Roque ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":27876,"web_url":"https://patchwork.libcamera.org/comment/27876/","msgid":"<20230925115548.GD8583@pendragon.ideasonboard.com>","date":"2023-09-25T11:55:48","subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Jai,\n\nOn Mon, Sep 25, 2023 at 05:17:59PM +0530, Jai Luthra wrote:\n> On Sep 25, 2023 at 13:52:40 +0300, Tomi Valkeinen wrote:\n> > On 25/09/2023 00:34, Laurent Pinchart wrote:\n> > > On Fri, Sep 15, 2023 at 09:57:30AM +0200, Mattijs Korpershoek via libcamera-devel wrote:\n> > > > On am62x platforms, the receiver driver (j721e-csi2rx) only\n> > > > supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.\n> > > > \n> > > > The receiver and the sensor (ov5640) hardware are both capable of\n> > > > YUV420, however:\n> > > > \n> > > > * we are not aware of OV5640 being tested with YUV420 formats on any\n> > > > vendor tree.\n> > > \n> > > Are you talking about packed YUV 4:2:0 here, or planar YUV 4:2:0 ?\n> > > Sensors only output packed formats (there could be exceptions, but they\n> > > would be very rare), it is the DMA engine on the host size that would\n> > > convert them to planar formats. That would be the DMA engine handled by\n> > > the k3-udma driver. It's fairly complicated, and I don't know if it\n> > > would hahev the ability to output planar formats. Tomi (on CC) may have\n> > > more information.\n> > \n> > No, I don't know if the CSI-2 RX or the DMA engine support packed to planar\n> > conversions. Maybe Jai knows.\n> > \n> > If the point is to show the frames on the screen, it sounds to me that the\n> > YUV422 formats would work the best.\n> \n> Unfortunately the DMA hardware does not support pixel-level interleaving \n> that would be required to convert packed YUV422/420 formats to planar or \n> semi-planar formats.\n> \n> But yes the display supports packed UYVY formats, we even use it for \n> camera to display pipelines on linux. The limitation on android is from \n> the gralloc implementation.\n\nThank you for the confirmation. Now for the annoying question: Could\nthat be fixed in gralloc ?\n\n> > > > * NV12 has different line lines (even lines are twice as long as odd\n> > > \n> > > s/lines/lengths/\n> > > \n> > > > lines) Different line-sized DMA transfers have not been tested on\n> > > \n> > > s/)/)./\n> > > \n> > > > the TI CSI-RX SHIM IP.\n> > > \n> > > As mentioned just above, the sensor will not output planar formats, so\n> > > the issue is about the\n> > > \n> > > > On the other hand, the graphics allocator (gralloc) cannot allocate\n> > > > YUV422 buffers directly. It mainly allocated NV12 buffers when\n> > > > userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED.\n> > > \n> > > The DSS seems to support YUYV, is there a way to fix the TI gralloc to\n> > > support it ?\n> > > \n> > > > Because of the above, we need a pixel conversion from YUYV to NV12,\n> > > > which is handled in the yuv processor via libyuv.\n> > > > \n> > > > Signed-off-by: Mattijs Korpershoek \n> > > > ---\n> > > > src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++---------\n> > > > 1 file changed, 68 insertions(+), 23 deletions(-)\n> > > > \n> > > > diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\n> > > > index 734bb85b7351..b680b833dafb 100644\n> > > > --- a/src/android/yuv/post_processor_yuv.cpp\n> > > > +++ b/src/android/yuv/post_processor_yuv.cpp\n> > > > @@ -7,6 +7,9 @@\n> > > > #include \"post_processor_yuv.h\"\n> > > > +#include \n> > > > +\n> > > > +#include \n> > > > #include \n> > > > #include \n> > > > @@ -22,14 +25,42 @@ using namespace libcamera;\n> > > > LOG_DEFINE_CATEGORY(YUV)\n> > > > +namespace {\n> > > > +\n> > > > +/**\n> > > > + * \\var supportedConversions\n> > > > + * \\brief list of supported output pixel formats for an input pixel format\n> > > > + */\n> > > > +const std::map> supportedConversions = {\n> > > > +\t{ formats::YUYV, { formats::NV12 } },\n> > > > +};\n> > > > +\n> > > > +} /* namespace */\n> > > > +\n> > > > int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> > > > \t\t\t\tconst StreamConfiguration &outCfg)\n> > > > {\n> > > > \tif (inCfg.pixelFormat != outCfg.pixelFormat) {\n> > > > -\t\tLOG(YUV, Error) << \"Pixel format conversion is not supported\"\n> > > > -\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > > -\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > > -\t\treturn -EINVAL;\n> > > > +\t\tconst auto it = supportedConversions.find(inCfg.pixelFormat);\n> > > > +\t\tif (it == supportedConversions.end()) {\n> > > > +\t\t\tLOG(YUV, Error) << \"Unsupported source format \" << inCfg.pixelFormat;\n> > > \n> > > \t\t\tLOG(YUV, Error)\n> > > \t\t\t\t<< \"Unsupported source format \"\n> > > \t\t\t\t<< inCfg.pixelFormat;\n> > > \n> > > > +\t\t\treturn -EINVAL;\n> > > > +\t\t}\n> > > > +\n> > > > +\t\tstd::vector outFormats = it->second;\n> > > \n> > > No need for a copy.\n> > > \n> > > > +\t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat);\n> > > \n> > > \t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(),\n> > > \t\t\t\t\t outCfg.pixelFormat);\n> > > \n> > > > +\t\tif (match == outFormats.end()) {\n> > > > +\t\t\tLOG(YUV, Error) << \"Requested pixel format conversion is not supported\"\n> > > > +\t\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > > +\t\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > \n> > > \t\t\tLOG(YUV, Error)\n> > > \t\t\t\t<< \"Requested pixel format conversion is not supported\"\n> > > \t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > \t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > \n> > > > +\t\t\treturn -EINVAL;\n> > > > +\t\t}\n> > > > +\t} else {\n> > > > +\t\tif (inCfg.pixelFormat != formats::NV12) {\n> > > > +\t\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> > > > +\t\t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > > \n> > > \t\t\tLOG(YUV, Error)\n> > > \t\t\t\t<< \"Unsupported format \" << inCfg.pixelFormat\n> > > \t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > > \n> > > > +\t\t\treturn -EINVAL;\n> > > > +\t\t}\n> > > > \t}\n> > > > \tif (inCfg.size < outCfg.size) {\n> > > > @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> > > > \t\treturn -EINVAL;\n> > > > \t}\n> > > > -\tif (inCfg.pixelFormat != formats::NV12) {\n> > > > -\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> > > > -\t\t\t\t<< \" (only NV12 is supported)\";\n> > > > -\t\treturn -EINVAL;\n> > > > -\t}\n> > > > -\n> > > > \tcalculateLengths(inCfg, outCfg);\n> > > > \treturn 0;\n> > > > }\n> > > > @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf\n> > > > \t\treturn;\n> > > > \t}\n> > > > -\tint ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> > > > -\t\t\t\t sourceStride_[0],\n> > > > -\t\t\t\t sourceMapped.planes()[1].data(),\n> > > > -\t\t\t\t sourceStride_[1],\n> > > > -\t\t\t\t sourceSize_.width, sourceSize_.height,\n> > > > -\t\t\t\t destination->plane(0).data(),\n> > > > -\t\t\t\t destinationStride_[0],\n> > > > -\t\t\t\t destination->plane(1).data(),\n> > > > -\t\t\t\t destinationStride_[1],\n> > > > -\t\t\t\t destinationSize_.width,\n> > > > -\t\t\t\t destinationSize_.height,\n> > > > -\t\t\t\t libyuv::FilterMode::kFilterBilinear);\n> > > > +\tint ret = 0;\n> > > > +\tswitch (sourceFormat_) {\n> > > > +\tcase formats::NV12:\n> > > > +\t\tret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> > > > +\t\t\t\t\tsourceStride_[0],\n> > > > +\t\t\t\t\tsourceMapped.planes()[1].data(),\n> > > > +\t\t\t\t\tsourceStride_[1],\n> > > > +\t\t\t\t\tsourceSize_.width, sourceSize_.height,\n> > > > +\t\t\t\t\tdestination->plane(0).data(),\n> > > > +\t\t\t\t\tdestinationStride_[0],\n> > > > +\t\t\t\t\tdestination->plane(1).data(),\n> > > > +\t\t\t\t\tdestinationStride_[1],\n> > > > +\t\t\t\t\tdestinationSize_.width,\n> > > > +\t\t\t\t\tdestinationSize_.height,\n> > > > +\t\t\t\t\tlibyuv::FilterMode::kFilterBilinear);\n> > > > +\t\tbreak;\n> > > > +\tcase formats::YUYV:\n> > > > +\t\tret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(),\n> > > > +\t\t\t\t\t sourceStride_[0],\n> > > > +\t\t\t\t\t destination->plane(0).data(),\n> > > > +\t\t\t\t\t destinationStride_[0],\n> > > > +\t\t\t\t\t destination->plane(1).data(),\n> > > > +\t\t\t\t\t destinationStride_[1],\n> > > > +\t\t\t\t\t destinationSize_.width,\n> > > > +\t\t\t\t\t destinationSize_.height);\n> > > > +\t\tbreak;\n> > > > +\tdefault:\n> > > > +\t\tLOG(YUV, Error) << \"Unsupported source format \" << sourceFormat_;\n> > > > +\t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> > > > +\t\tbreak;\n> > > \n> > > This can't happen, can it ?\n> > > \n> > > > +\t}\n> > > > +\n> > > > \tif (ret) {\n> > > > -\t\tLOG(YUV, Error) << \"Failed NV12 scaling: \" << ret;\n> > > > +\t\tLOG(YUV, Error) << \"Libyuv operation failure: \" << ret;\n> > > \n> > > s/Libyuv/libyuv/\n> > > \n> > > Although I would write \"YUV post-processing failure: \".\n> > > \n> > > > \t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> > > > \t\treturn;\n> > > > \t}","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9D318C326B\n\tfor ;\n\tMon, 25 Sep 2023 11:55:39 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1161962944;\n\tMon, 25 Sep 2023 13:55:39 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id AC704628FA\n\tfor ;\n\tMon, 25 Sep 2023 13:55:37 +0200 (CEST)","from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi\n\t[213.243.189.158])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 4106DDB7;\n\tMon, 25 Sep 2023 13:53:57 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1695642939;\n\tbh=8prJ5IwJFliY4A+pi5XqUYM/bq9QenQvkfSgfDwfYwE=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=hBL4P+IQVGHxPVtoz08Qy76blLOcPyP2avhzOsAGANzaeNLWnu9hhUIZmQ230+5xa\n\tD8hJ5A2eqb3iyZLt4vp87mkI1Z2K6sljQ/Vjlx9spvfwsMtyrKkqPn8gr704Qzu2P2\n\tAdu+nJ6TwAOBKcix+h2EzeHMctuCStvb/HKHM0K60td8Kdn/LS5PYKZo43lY6/AqW4\n\tD/61HunfYZ0VNfqHA9lDhumWBx7RPiNCCKmsP81P4zbRXV05MuQV8r1rqTTiDvge1c\n\tTVKXkPGhhlf9K2STn4LsyKf7hPEGI+5acXBhdceaBUexu2KHVtYs+LIeLPuSC3lvEk\n\t6fVwzsJ0grFjA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1695642837;\n\tbh=8prJ5IwJFliY4A+pi5XqUYM/bq9QenQvkfSgfDwfYwE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=LB0bRubH5jtUA6VdX+XtajTMwhPRRpAyIoAYZYYfioMwkHwHj7xm+/+lZ18HeKnN0\n\tKtJzKxgQDzOJGlBYBQ1tN9lQjeb/2ZnwWtUiCv7AB2mIGF/z1WbrX/rHgHvmpOXEyk\n\t9a/gnUBsgQ03u/si3gNDQpHWEUrpA5I6DhdGp/rM="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"LB0bRubH\"; dkim-atps=neutral","Date":"Mon, 25 Sep 2023 14:55:48 +0300","To":"Jai Luthra ","Message-ID":"<20230925115548.GD8583@pendragon.ideasonboard.com>","References":"\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org,\n\tGuillaume La Roque ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":27878,"web_url":"https://patchwork.libcamera.org/comment/27878/","msgid":"","date":"2023-09-25T12:23:03","subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","submitter":{"id":170,"url":"https://patchwork.libcamera.org/api/people/170/","name":"Jai Luthra","email":"j-luthra@ti.com"},"content":"Hi Laurent, Mattijs,\n\nOn Sep 25, 2023 at 14:55:48 +0300, Laurent Pinchart wrote:\n> Hi Jai,\n> \n> On Mon, Sep 25, 2023 at 05:17:59PM +0530, Jai Luthra wrote:\n> > On Sep 25, 2023 at 13:52:40 +0300, Tomi Valkeinen wrote:\n> > > On 25/09/2023 00:34, Laurent Pinchart wrote:\n> > > > On Fri, Sep 15, 2023 at 09:57:30AM +0200, Mattijs Korpershoek via libcamera-devel wrote:\n> > > > > On am62x platforms, the receiver driver (j721e-csi2rx) only\n> > > > > supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.\n> > > > > \n> > > > > The receiver and the sensor (ov5640) hardware are both capable of\n> > > > > YUV420, however:\n> > > > > \n> > > > > * we are not aware of OV5640 being tested with YUV420 formats on any\n> > > > > vendor tree.\n> > > > \n> > > > Are you talking about packed YUV 4:2:0 here, or planar YUV 4:2:0 ?\n> > > > Sensors only output packed formats (there could be exceptions, but they\n> > > > would be very rare), it is the DMA engine on the host size that would\n> > > > convert them to planar formats. That would be the DMA engine handled by\n> > > > the k3-udma driver. It's fairly complicated, and I don't know if it\n> > > > would hahev the ability to output planar formats. Tomi (on CC) may have\n> > > > more information.\n> > > \n> > > No, I don't know if the CSI-2 RX or the DMA engine support packed to planar\n> > > conversions. Maybe Jai knows.\n> > > \n> > > If the point is to show the frames on the screen, it sounds to me that the\n> > > YUV422 formats would work the best.\n> > \n> > Unfortunately the DMA hardware does not support pixel-level interleaving \n> > that would be required to convert packed YUV422/420 formats to planar or \n> > semi-planar formats.\n> > \n> > But yes the display supports packed UYVY formats, we even use it for \n> > camera to display pipelines on linux. The limitation on android is from \n> > the gralloc implementation.\n> \n> Thank you for the confirmation. Now for the annoying question: Could\n> that be fixed in gralloc ?\n> \n\nFixing gralloc could be possible (we may have to ask the GPU vendor I am \nnot sure). But IIRC when we were discussing the options the concern was \nthat just fixing the gralloc would not be enough, as there are other \nplaces in Android framework where things are hardcoded to require NV12.\n\nI don't understand Android/HAL stack enough, maybe Mattijs can share \nwhat else was needed other than changes to gralloc.\n\n> > > > > * NV12 has different line lines (even lines are twice as long as odd\n> > > > \n> > > > s/lines/lengths/\n> > > > \n> > > > > lines) Different line-sized DMA transfers have not been tested on\n> > > > \n> > > > s/)/)./\n> > > > \n> > > > > the TI CSI-RX SHIM IP.\n> > > > \n> > > > As mentioned just above, the sensor will not output planar formats, so\n> > > > the issue is about the\n> > > > \n> > > > > On the other hand, the graphics allocator (gralloc) cannot allocate\n> > > > > YUV422 buffers directly. It mainly allocated NV12 buffers when\n> > > > > userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED.\n> > > > \n> > > > The DSS seems to support YUYV, is there a way to fix the TI gralloc to\n> > > > support it ?\n> > > > \n> > > > > Because of the above, we need a pixel conversion from YUYV to NV12,\n> > > > > which is handled in the yuv processor via libyuv.\n> > > > > \n> > > > > Signed-off-by: Mattijs Korpershoek \n> > > > > ---\n> > > > > src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++---------\n> > > > > 1 file changed, 68 insertions(+), 23 deletions(-)\n> > > > > \n> > > > > diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\n> > > > > index 734bb85b7351..b680b833dafb 100644\n> > > > > --- a/src/android/yuv/post_processor_yuv.cpp\n> > > > > +++ b/src/android/yuv/post_processor_yuv.cpp\n> > > > > @@ -7,6 +7,9 @@\n> > > > > #include \"post_processor_yuv.h\"\n> > > > > +#include \n> > > > > +\n> > > > > +#include \n> > > > > #include \n> > > > > #include \n> > > > > @@ -22,14 +25,42 @@ using namespace libcamera;\n> > > > > LOG_DEFINE_CATEGORY(YUV)\n> > > > > +namespace {\n> > > > > +\n> > > > > +/**\n> > > > > + * \\var supportedConversions\n> > > > > + * \\brief list of supported output pixel formats for an input pixel format\n> > > > > + */\n> > > > > +const std::map> supportedConversions = {\n> > > > > +\t{ formats::YUYV, { formats::NV12 } },\n> > > > > +};\n> > > > > +\n> > > > > +} /* namespace */\n> > > > > +\n> > > > > int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> > > > > \t\t\t\tconst StreamConfiguration &outCfg)\n> > > > > {\n> > > > > \tif (inCfg.pixelFormat != outCfg.pixelFormat) {\n> > > > > -\t\tLOG(YUV, Error) << \"Pixel format conversion is not supported\"\n> > > > > -\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > > > -\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > > > -\t\treturn -EINVAL;\n> > > > > +\t\tconst auto it = supportedConversions.find(inCfg.pixelFormat);\n> > > > > +\t\tif (it == supportedConversions.end()) {\n> > > > > +\t\t\tLOG(YUV, Error) << \"Unsupported source format \" << inCfg.pixelFormat;\n> > > > \n> > > > \t\t\tLOG(YUV, Error)\n> > > > \t\t\t\t<< \"Unsupported source format \"\n> > > > \t\t\t\t<< inCfg.pixelFormat;\n> > > > \n> > > > > +\t\t\treturn -EINVAL;\n> > > > > +\t\t}\n> > > > > +\n> > > > > +\t\tstd::vector outFormats = it->second;\n> > > > \n> > > > No need for a copy.\n> > > > \n> > > > > +\t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat);\n> > > > \n> > > > \t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(),\n> > > > \t\t\t\t\t outCfg.pixelFormat);\n> > > > \n> > > > > +\t\tif (match == outFormats.end()) {\n> > > > > +\t\t\tLOG(YUV, Error) << \"Requested pixel format conversion is not supported\"\n> > > > > +\t\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > > > +\t\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > > \n> > > > \t\t\tLOG(YUV, Error)\n> > > > \t\t\t\t<< \"Requested pixel format conversion is not supported\"\n> > > > \t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > > \t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > > \n> > > > > +\t\t\treturn -EINVAL;\n> > > > > +\t\t}\n> > > > > +\t} else {\n> > > > > +\t\tif (inCfg.pixelFormat != formats::NV12) {\n> > > > > +\t\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> > > > > +\t\t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > > > \n> > > > \t\t\tLOG(YUV, Error)\n> > > > \t\t\t\t<< \"Unsupported format \" << inCfg.pixelFormat\n> > > > \t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > > > \n> > > > > +\t\t\treturn -EINVAL;\n> > > > > +\t\t}\n> > > > > \t}\n> > > > > \tif (inCfg.size < outCfg.size) {\n> > > > > @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> > > > > \t\treturn -EINVAL;\n> > > > > \t}\n> > > > > -\tif (inCfg.pixelFormat != formats::NV12) {\n> > > > > -\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> > > > > -\t\t\t\t<< \" (only NV12 is supported)\";\n> > > > > -\t\treturn -EINVAL;\n> > > > > -\t}\n> > > > > -\n> > > > > \tcalculateLengths(inCfg, outCfg);\n> > > > > \treturn 0;\n> > > > > }\n> > > > > @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf\n> > > > > \t\treturn;\n> > > > > \t}\n> > > > > -\tint ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> > > > > -\t\t\t\t sourceStride_[0],\n> > > > > -\t\t\t\t sourceMapped.planes()[1].data(),\n> > > > > -\t\t\t\t sourceStride_[1],\n> > > > > -\t\t\t\t sourceSize_.width, sourceSize_.height,\n> > > > > -\t\t\t\t destination->plane(0).data(),\n> > > > > -\t\t\t\t destinationStride_[0],\n> > > > > -\t\t\t\t destination->plane(1).data(),\n> > > > > -\t\t\t\t destinationStride_[1],\n> > > > > -\t\t\t\t destinationSize_.width,\n> > > > > -\t\t\t\t destinationSize_.height,\n> > > > > -\t\t\t\t libyuv::FilterMode::kFilterBilinear);\n> > > > > +\tint ret = 0;\n> > > > > +\tswitch (sourceFormat_) {\n> > > > > +\tcase formats::NV12:\n> > > > > +\t\tret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> > > > > +\t\t\t\t\tsourceStride_[0],\n> > > > > +\t\t\t\t\tsourceMapped.planes()[1].data(),\n> > > > > +\t\t\t\t\tsourceStride_[1],\n> > > > > +\t\t\t\t\tsourceSize_.width, sourceSize_.height,\n> > > > > +\t\t\t\t\tdestination->plane(0).data(),\n> > > > > +\t\t\t\t\tdestinationStride_[0],\n> > > > > +\t\t\t\t\tdestination->plane(1).data(),\n> > > > > +\t\t\t\t\tdestinationStride_[1],\n> > > > > +\t\t\t\t\tdestinationSize_.width,\n> > > > > +\t\t\t\t\tdestinationSize_.height,\n> > > > > +\t\t\t\t\tlibyuv::FilterMode::kFilterBilinear);\n> > > > > +\t\tbreak;\n> > > > > +\tcase formats::YUYV:\n> > > > > +\t\tret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(),\n> > > > > +\t\t\t\t\t sourceStride_[0],\n> > > > > +\t\t\t\t\t destination->plane(0).data(),\n> > > > > +\t\t\t\t\t destinationStride_[0],\n> > > > > +\t\t\t\t\t destination->plane(1).data(),\n> > > > > +\t\t\t\t\t destinationStride_[1],\n> > > > > +\t\t\t\t\t destinationSize_.width,\n> > > > > +\t\t\t\t\t destinationSize_.height);\n> > > > > +\t\tbreak;\n> > > > > +\tdefault:\n> > > > > +\t\tLOG(YUV, Error) << \"Unsupported source format \" << sourceFormat_;\n> > > > > +\t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> > > > > +\t\tbreak;\n> > > > \n> > > > This can't happen, can it ?\n> > > > \n> > > > > +\t}\n> > > > > +\n> > > > > \tif (ret) {\n> > > > > -\t\tLOG(YUV, Error) << \"Failed NV12 scaling: \" << ret;\n> > > > > +\t\tLOG(YUV, Error) << \"Libyuv operation failure: \" << ret;\n> > > > \n> > > > s/Libyuv/libyuv/\n> > > > \n> > > > Although I would write \"YUV post-processing failure: \".\n> > > > \n> > > > > \t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> > > > > \t\treturn;\n> > > > > \t}\n> \n> -- \n> Regards,\n> \n> Laurent Pinchart","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 86B79C326B\n\tfor ;\n\tMon, 25 Sep 2023 12:23:19 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id F3D3F628FA;\n\tMon, 25 Sep 2023 14:23:18 +0200 (CEST)","from lelv0142.ext.ti.com (lelv0142.ext.ti.com [198.47.23.249])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id AE5DD628FA\n\tfor ;\n\tMon, 25 Sep 2023 14:23:16 +0200 (CEST)","from lelv0265.itg.ti.com ([10.180.67.224])\n\tby lelv0142.ext.ti.com (8.15.2/8.15.2) with ESMTP id 38PCNAhS060050; \n\tMon, 25 Sep 2023 07:23:10 -0500","from DLEE104.ent.ti.com (dlee104.ent.ti.com [157.170.170.34])\n\tby lelv0265.itg.ti.com (8.15.2/8.15.2) with ESMTPS id 38PCNAZ2012355\n\t(version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL);\n\tMon, 25 Sep 2023 07:23:10 -0500","from DLEE107.ent.ti.com (157.170.170.37) by DLEE104.ent.ti.com\n\t(157.170.170.34) with Microsoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23;\n\tMon, 25 Sep 2023 07:23:09 -0500","from lelv0326.itg.ti.com (10.180.67.84) by DLEE107.ent.ti.com\n\t(157.170.170.37) with Microsoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23\n\tvia Frontend Transport; Mon, 25 Sep 2023 07:23:09 -0500","from localhost (ileaxei01-snat.itg.ti.com [10.180.69.5])\n\tby lelv0326.itg.ti.com (8.15.2/8.15.2) with ESMTP id 38PCN8vK027615; \n\tMon, 25 Sep 2023 07:23:08 -0500"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1695644599;\n\tbh=E59iqQVAJxGvk7MDHJ21sUNFbbVLsvLwpHehyCJda8M=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=H20kY1uzubC4Njjbge0LqbNedrJTP/4Lu7Sm5R98hwJC/MmeNqzJ4p2JFy2ybusDZ\n\thV57W8pdzV9lbaxjjHLoLRw4iriyN9m1hpHSbBTmUsSBog4Ug/hv5UshGrYcTWvW2l\n\tqV4IX5ldSADrFFJ4NriaHhhRS3u2myxkTxa8n87SOqk1tdxh9GuM+irHlDgcU43H1Q\n\t2UJl/bZ2V0LTNlQPhgUjPjcmmkLKgkUTCv2uZtYZXkU2TBv8I4zpAU/Pkz0mRMMXri\n\tlSxPwTQwSwd0BklzbHNfiBXFoY5kM7RHvVNaroRdfRfDZziyQx03o4OhBQXZpeo9t2\n\tFcggAYR8yWB7w==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com;\n\ts=ti-com-17Q1; t=1695644590;\n\tbh=1KhtLTxei/zNBPD98iyo+79jq7IGvGB5ec86XwqLrfw=;\n\th=Date:From:To:CC:Subject:References:In-Reply-To;\n\tb=qIxN+KoQ4//++KpABrBUjyOEZdIg+AticObtJu6u/8NARIF/Y/PDaYcImduXLNZKY\n\tiQcZ1IABo76QznLj0ZShGHTmM+vgoQFOWj5dBih9YTk/UfVDf/i0UsrBJd/nKtwk9y\n\tTlfmI6r6bjQxrLUk4wE+IU5L+EBo0M6mFdJtVGzg="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ti.com header.i=@ti.com\n\theader.b=\"qIxN+KoQ\"; dkim-atps=neutral","Date":"Mon, 25 Sep 2023 17:53:03 +0530","To":"Laurent Pinchart , Mattijs Korpershoek\n\t","Message-ID":"","References":"\n\t\n\t<20230925115548.GD8583@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Disposition":"inline","In-Reply-To":"<20230925115548.GD8583@pendragon.ideasonboard.com>","X-EXCLAIMER-MD-CONFIG":"e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180","Subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jai Luthra via libcamera-devel ","Reply-To":"Jai Luthra ","Cc":"libcamera-devel@lists.libcamera.org,\n\tGuillaume La Roque ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":27879,"web_url":"https://patchwork.libcamera.org/comment/27879/","msgid":"<20230925124331.GD375@pendragon.ideasonboard.com>","date":"2023-09-25T12:43:31","subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Mon, Sep 25, 2023 at 05:53:03PM +0530, Jai Luthra wrote:\n> On Sep 25, 2023 at 14:55:48 +0300, Laurent Pinchart wrote:\n> > On Mon, Sep 25, 2023 at 05:17:59PM +0530, Jai Luthra wrote:\n> > > On Sep 25, 2023 at 13:52:40 +0300, Tomi Valkeinen wrote:\n> > > > On 25/09/2023 00:34, Laurent Pinchart wrote:\n> > > > > On Fri, Sep 15, 2023 at 09:57:30AM +0200, Mattijs Korpershoek via libcamera-devel wrote:\n> > > > > > On am62x platforms, the receiver driver (j721e-csi2rx) only\n> > > > > > supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY.\n> > > > > > \n> > > > > > The receiver and the sensor (ov5640) hardware are both capable of\n> > > > > > YUV420, however:\n> > > > > > \n> > > > > > * we are not aware of OV5640 being tested with YUV420 formats on any\n> > > > > > vendor tree.\n> > > > > \n> > > > > Are you talking about packed YUV 4:2:0 here, or planar YUV 4:2:0 ?\n> > > > > Sensors only output packed formats (there could be exceptions, but they\n> > > > > would be very rare), it is the DMA engine on the host size that would\n> > > > > convert them to planar formats. That would be the DMA engine handled by\n> > > > > the k3-udma driver. It's fairly complicated, and I don't know if it\n> > > > > would hahev the ability to output planar formats. Tomi (on CC) may have\n> > > > > more information.\n> > > > \n> > > > No, I don't know if the CSI-2 RX or the DMA engine support packed to planar\n> > > > conversions. Maybe Jai knows.\n> > > > \n> > > > If the point is to show the frames on the screen, it sounds to me that the\n> > > > YUV422 formats would work the best.\n> > > \n> > > Unfortunately the DMA hardware does not support pixel-level interleaving \n> > > that would be required to convert packed YUV422/420 formats to planar or \n> > > semi-planar formats.\n> > > \n> > > But yes the display supports packed UYVY formats, we even use it for \n> > > camera to display pipelines on linux. The limitation on android is from \n> > > the gralloc implementation.\n> > \n> > Thank you for the confirmation. Now for the annoying question: Could\n> > that be fixed in gralloc ?\n> \n> Fixing gralloc could be possible (we may have to ask the GPU vendor I am \n> not sure). But IIRC when we were discussing the options the concern was \n> that just fixing the gralloc would not be enough, as there are other \n> places in Android framework where things are hardcoded to require NV12.\n\nDo you know where those places are ? My understanding is that the\nmapping from HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED to an actual pixel\nformat isn't hardcoded in Android, but is left for the platform to\ndefine. This means, in practice, that all HALs need to agree on this.\nThere is no runtime API to query the information, it gets hardcoded in\nthe HALs.\n\n> I don't understand Android/HAL stack enough, maybe Mattijs can share \n> what else was needed other than changes to gralloc.\n> \n> > > > > > * NV12 has different line lines (even lines are twice as long as odd\n> > > > > \n> > > > > s/lines/lengths/\n> > > > > \n> > > > > > lines) Different line-sized DMA transfers have not been tested on\n> > > > > \n> > > > > s/)/)./\n> > > > > \n> > > > > > the TI CSI-RX SHIM IP.\n> > > > > \n> > > > > As mentioned just above, the sensor will not output planar formats, so\n> > > > > the issue is about the\n> > > > > \n> > > > > > On the other hand, the graphics allocator (gralloc) cannot allocate\n> > > > > > YUV422 buffers directly. It mainly allocated NV12 buffers when\n> > > > > > userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED.\n> > > > > \n> > > > > The DSS seems to support YUYV, is there a way to fix the TI gralloc to\n> > > > > support it ?\n> > > > > \n> > > > > > Because of the above, we need a pixel conversion from YUYV to NV12,\n> > > > > > which is handled in the yuv processor via libyuv.\n> > > > > > \n> > > > > > Signed-off-by: Mattijs Korpershoek \n> > > > > > ---\n> > > > > > src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++---------\n> > > > > > 1 file changed, 68 insertions(+), 23 deletions(-)\n> > > > > > \n> > > > > > diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\n> > > > > > index 734bb85b7351..b680b833dafb 100644\n> > > > > > --- a/src/android/yuv/post_processor_yuv.cpp\n> > > > > > +++ b/src/android/yuv/post_processor_yuv.cpp\n> > > > > > @@ -7,6 +7,9 @@\n> > > > > > #include \"post_processor_yuv.h\"\n> > > > > > +#include \n> > > > > > +\n> > > > > > +#include \n> > > > > > #include \n> > > > > > #include \n> > > > > > @@ -22,14 +25,42 @@ using namespace libcamera;\n> > > > > > LOG_DEFINE_CATEGORY(YUV)\n> > > > > > +namespace {\n> > > > > > +\n> > > > > > +/**\n> > > > > > + * \\var supportedConversions\n> > > > > > + * \\brief list of supported output pixel formats for an input pixel format\n> > > > > > + */\n> > > > > > +const std::map> supportedConversions = {\n> > > > > > +\t{ formats::YUYV, { formats::NV12 } },\n> > > > > > +};\n> > > > > > +\n> > > > > > +} /* namespace */\n> > > > > > +\n> > > > > > int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> > > > > > \t\t\t\tconst StreamConfiguration &outCfg)\n> > > > > > {\n> > > > > > \tif (inCfg.pixelFormat != outCfg.pixelFormat) {\n> > > > > > -\t\tLOG(YUV, Error) << \"Pixel format conversion is not supported\"\n> > > > > > -\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > > > > -\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > > > > -\t\treturn -EINVAL;\n> > > > > > +\t\tconst auto it = supportedConversions.find(inCfg.pixelFormat);\n> > > > > > +\t\tif (it == supportedConversions.end()) {\n> > > > > > +\t\t\tLOG(YUV, Error) << \"Unsupported source format \" << inCfg.pixelFormat;\n> > > > > \n> > > > > \t\t\tLOG(YUV, Error)\n> > > > > \t\t\t\t<< \"Unsupported source format \"\n> > > > > \t\t\t\t<< inCfg.pixelFormat;\n> > > > > \n> > > > > > +\t\t\treturn -EINVAL;\n> > > > > > +\t\t}\n> > > > > > +\n> > > > > > +\t\tstd::vector outFormats = it->second;\n> > > > > \n> > > > > No need for a copy.\n> > > > > \n> > > > > > +\t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat);\n> > > > > \n> > > > > \t\tconst auto &match = std::find(outFormats.begin(), outFormats.end(),\n> > > > > \t\t\t\t\t outCfg.pixelFormat);\n> > > > > \n> > > > > > +\t\tif (match == outFormats.end()) {\n> > > > > > +\t\t\tLOG(YUV, Error) << \"Requested pixel format conversion is not supported\"\n> > > > > > +\t\t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > > > > +\t\t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > > > \n> > > > > \t\t\tLOG(YUV, Error)\n> > > > > \t\t\t\t<< \"Requested pixel format conversion is not supported\"\n> > > > > \t\t\t\t<< \" (from \" << inCfg.pixelFormat\n> > > > > \t\t\t\t<< \" to \" << outCfg.pixelFormat << \")\";\n> > > > > \n> > > > > > +\t\t\treturn -EINVAL;\n> > > > > > +\t\t}\n> > > > > > +\t} else {\n> > > > > > +\t\tif (inCfg.pixelFormat != formats::NV12) {\n> > > > > > +\t\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> > > > > > +\t\t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > > > > \n> > > > > \t\t\tLOG(YUV, Error)\n> > > > > \t\t\t\t<< \"Unsupported format \" << inCfg.pixelFormat\n> > > > > \t\t\t\t<< \" (only NV12 is supported for scaling)\";\n> > > > > \n> > > > > > +\t\t\treturn -EINVAL;\n> > > > > > +\t\t}\n> > > > > > \t}\n> > > > > > \tif (inCfg.size < outCfg.size) {\n> > > > > > @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> > > > > > \t\treturn -EINVAL;\n> > > > > > \t}\n> > > > > > -\tif (inCfg.pixelFormat != formats::NV12) {\n> > > > > > -\t\tLOG(YUV, Error) << \"Unsupported format \" << inCfg.pixelFormat\n> > > > > > -\t\t\t\t<< \" (only NV12 is supported)\";\n> > > > > > -\t\treturn -EINVAL;\n> > > > > > -\t}\n> > > > > > -\n> > > > > > \tcalculateLengths(inCfg, outCfg);\n> > > > > > \treturn 0;\n> > > > > > }\n> > > > > > @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf\n> > > > > > \t\treturn;\n> > > > > > \t}\n> > > > > > -\tint ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> > > > > > -\t\t\t\t sourceStride_[0],\n> > > > > > -\t\t\t\t sourceMapped.planes()[1].data(),\n> > > > > > -\t\t\t\t sourceStride_[1],\n> > > > > > -\t\t\t\t sourceSize_.width, sourceSize_.height,\n> > > > > > -\t\t\t\t destination->plane(0).data(),\n> > > > > > -\t\t\t\t destinationStride_[0],\n> > > > > > -\t\t\t\t destination->plane(1).data(),\n> > > > > > -\t\t\t\t destinationStride_[1],\n> > > > > > -\t\t\t\t destinationSize_.width,\n> > > > > > -\t\t\t\t destinationSize_.height,\n> > > > > > -\t\t\t\t libyuv::FilterMode::kFilterBilinear);\n> > > > > > +\tint ret = 0;\n> > > > > > +\tswitch (sourceFormat_) {\n> > > > > > +\tcase formats::NV12:\n> > > > > > +\t\tret = libyuv::NV12Scale(sourceMapped.planes()[0].data(),\n> > > > > > +\t\t\t\t\tsourceStride_[0],\n> > > > > > +\t\t\t\t\tsourceMapped.planes()[1].data(),\n> > > > > > +\t\t\t\t\tsourceStride_[1],\n> > > > > > +\t\t\t\t\tsourceSize_.width, sourceSize_.height,\n> > > > > > +\t\t\t\t\tdestination->plane(0).data(),\n> > > > > > +\t\t\t\t\tdestinationStride_[0],\n> > > > > > +\t\t\t\t\tdestination->plane(1).data(),\n> > > > > > +\t\t\t\t\tdestinationStride_[1],\n> > > > > > +\t\t\t\t\tdestinationSize_.width,\n> > > > > > +\t\t\t\t\tdestinationSize_.height,\n> > > > > > +\t\t\t\t\tlibyuv::FilterMode::kFilterBilinear);\n> > > > > > +\t\tbreak;\n> > > > > > +\tcase formats::YUYV:\n> > > > > > +\t\tret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(),\n> > > > > > +\t\t\t\t\t sourceStride_[0],\n> > > > > > +\t\t\t\t\t destination->plane(0).data(),\n> > > > > > +\t\t\t\t\t destinationStride_[0],\n> > > > > > +\t\t\t\t\t destination->plane(1).data(),\n> > > > > > +\t\t\t\t\t destinationStride_[1],\n> > > > > > +\t\t\t\t\t destinationSize_.width,\n> > > > > > +\t\t\t\t\t destinationSize_.height);\n> > > > > > +\t\tbreak;\n> > > > > > +\tdefault:\n> > > > > > +\t\tLOG(YUV, Error) << \"Unsupported source format \" << sourceFormat_;\n> > > > > > +\t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> > > > > > +\t\tbreak;\n> > > > > \n> > > > > This can't happen, can it ?\n> > > > > \n> > > > > > +\t}\n> > > > > > +\n> > > > > > \tif (ret) {\n> > > > > > -\t\tLOG(YUV, Error) << \"Failed NV12 scaling: \" << ret;\n> > > > > > +\t\tLOG(YUV, Error) << \"Libyuv operation failure: \" << ret;\n> > > > > \n> > > > > s/Libyuv/libyuv/\n> > > > > \n> > > > > Although I would write \"YUV post-processing failure: \".\n> > > > > \n> > > > > > \t\tprocessComplete.emit(streamBuffer, PostProcessor::Status::Error);\n> > > > > > \t\treturn;\n> > > > > > \t}","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8D347C32B0\n\tfor ;\n\tMon, 25 Sep 2023 12:43:22 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DC45C62944;\n\tMon, 25 Sep 2023 14:43:21 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 99494628FA\n\tfor ;\n\tMon, 25 Sep 2023 14:43:20 +0200 (CEST)","from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi\n\t[213.243.189.158])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2773ADB7;\n\tMon, 25 Sep 2023 14:41:40 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1695645801;\n\tbh=Dmxqz1HrWGiy24Bq9u0DpW/jDZr+hZ3n/Kkc2T2s8Bc=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=QmY6dke67NK7GlUZxw4E1GlWfTDxlY9hi+0N1rCo64Rj5AsuEmkodkH/Hu3Wir+nS\n\ttyxpYWUan8SUi4QQj8bNBO5euGHpx673ccjFus4XCfCT4zfroyKVDwbQqF+ehS2mgE\n\txeaQkq6xyGEpMN6BaUPRPYBXmlzvwy0o/wfyHFtGFhrnM/Ain5P+iqfdAJZA5YJ80v\n\tctWz6tB/PrZPIkDxXn9fRmcqJ/T7krwbSyVtQmSYc355AcVHrqQPFM7QVj44mWOEbs\n\tnf+0AXPeN0QdbWyC1RrH7FmcwNfJGBoUsrUoIMpNSf/7SuyPC7w4nk2aJV1pL8mMiF\n\tkSA0Qyz0IRMfw==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1695645700;\n\tbh=Dmxqz1HrWGiy24Bq9u0DpW/jDZr+hZ3n/Kkc2T2s8Bc=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=dCTqsbxqdTcc5xiJirPnZkWboejar6aML/k6jCyPrUmLVP2s2JaXW0uQy6BdLsYVl\n\tu3oPSQibGKPzzdgWyWYWjZ6c7hT+dVJeUFtdk0uVSLiqevuYGq67prpGajcMoqoh1k\n\tyAEZhPOWwgPAMMvLekgyw5eiUpg+WxbxKRDC4liw="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"dCTqsbxq\"; dkim-atps=neutral","Date":"Mon, 25 Sep 2023 15:43:31 +0300","To":"Jai Luthra ","Message-ID":"<20230925124331.GD375@pendragon.ideasonboard.com>","References":"\n\t\n\t<20230925115548.GD8583@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV ->\n\tNV12 conversion","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org,\n\tGuillaume La Roque ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]