[{"id":15336,"web_url":"https://patchwork.libcamera.org/comment/15336/","msgid":"","date":"2021-02-28T18:42:17","subject":"Re: [libcamera-devel] [PATCH 08/12] android: post_processor: Use\n\tCameraBuffer API","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Jacopo,\n\nThank you for the patch.\n\nOn Fri, Feb 26, 2021 at 02:29:28PM +0100, Jacopo Mondi wrote:\n> Use the newly introduced CameraBuffer class as the type for the\n> destination buffer in the PostProcessor class hierarchy in place of the\n> libcamera::MappedFrameBuffer one and use its API to retrieve the length\n> and the location of the CameraBuffer plane allocated for JPEG\n> post-processing.\n> \n> Remove all the assumption on the underlying memory storage and only go\n> through the CameraBuffer API when dealing with memory buffers. To do so\n> rework the Encoder interface to use a raw pointer and an explicit size\n> to remove access to the Span maps that serve as memory storage\n> for the current implementation but might not be ideal for other memory\n> backend.\n> \n> Now that the whole PostProcessor hierarchy has been converted to use\n> the CameraBuffer API remove libcamera::MappedBuffer as base class\n> of the CameraBuffer interface and only reply on its interface.\n> \n> Signed-off-by: Jacopo Mondi \n> ---\n> src/android/camera_buffer.h | 2 +-\n> src/android/jpeg/encoder.h | 4 +++-\n> src/android/jpeg/encoder_libjpeg.cpp | 19 ++++++++--------\n> src/android/jpeg/encoder_libjpeg.h | 4 ++--\n> src/android/jpeg/post_processor_jpeg.cpp | 28 ++++++++++--------------\n> src/android/jpeg/post_processor_jpeg.h | 2 +-\n> src/android/post_processor.h | 4 +++-\n> src/android/yuv/post_processor_yuv.cpp | 20 ++++++++---------\n> src/android/yuv/post_processor_yuv.h | 4 ++--\n> 9 files changed, 42 insertions(+), 45 deletions(-)\n> \n> diff --git a/src/android/camera_buffer.h b/src/android/camera_buffer.h\n> index 2a91e6a3c9c1..b251e4514864 100644\n> --- a/src/android/camera_buffer.h\n> +++ b/src/android/camera_buffer.h\n> @@ -11,7 +11,7 @@\n> \n> #include \n> \n> -class CameraBuffer : public libcamera::MappedBuffer\n> +class CameraBuffer\n> {\n> public:\n> \tCameraBuffer(buffer_handle_t camera3Buffer, int flags);\n> diff --git a/src/android/jpeg/encoder.h b/src/android/jpeg/encoder.h\n> index 8d449369869f..a3c47c09a06e 100644\n> --- a/src/android/jpeg/encoder.h\n> +++ b/src/android/jpeg/encoder.h\n> @@ -11,6 +11,8 @@\n> #include \n> #include \n> \n> +#include \"../camera_buffer.h\"\n\nThis doesn't seem to be needed.\n\n> +\n> class Encoder\n> {\n> public:\n> @@ -18,7 +20,7 @@ public:\n> \n> \tvirtual int configure(const libcamera::StreamConfiguration &cfg) = 0;\n> \tvirtual int encode(const libcamera::FrameBuffer &source,\n> -\t\t\t libcamera::Span destination,\n> +\t\t\t uint8_t *destination, size_t destinationSize,\n\nWhenever we need to pass a pointer and a size, a span is the right tool.\nIt's its core purpose, grouping pointer and size together. This will not\nsupport multi-planar buffers, but neither will a pointer and a size, so\nthe interface will need to be reworked later.\n\n> \t\t\t libcamera::Span exifData,\n> \t\t\t unsigned int quality) = 0;\n> };\n> diff --git a/src/android/jpeg/encoder_libjpeg.cpp b/src/android/jpeg/encoder_libjpeg.cpp\n> index f006e1d1999a..a965620703df 100644\n> --- a/src/android/jpeg/encoder_libjpeg.cpp\n> +++ b/src/android/jpeg/encoder_libjpeg.cpp\n> @@ -177,8 +177,9 @@ void EncoderLibJpeg::compressNV(Span frame)\n> \t}\n> }\n> \n> -int EncoderLibJpeg::encode(const FrameBuffer &source, Span dest,\n> -\t\t\t Span exifData, unsigned int quality)\n> +int EncoderLibJpeg::encode(const FrameBuffer &source, uint8_t *destination,\n> +\t\t\t size_t destinationSize, Span exifData,\n> +\t\t\t unsigned int quality)\n> {\n> \tMappedFrameBuffer frame(&source, PROT_READ);\n> \tif (!frame.isValid()) {\n> @@ -187,15 +188,13 @@ int EncoderLibJpeg::encode(const FrameBuffer &source, Span dest,\n> \t\treturn frame.error();\n> \t}\n> \n> -\treturn encode(frame.maps()[0], dest, exifData, quality);\n> +\treturn encode(frame.maps()[0], destination, destinationSize, exifData, quality);\n> }\n> \n> -int EncoderLibJpeg::encode(Span src, Span dest,\n> -\t\t\t Span exifData, unsigned int quality)\n> +int EncoderLibJpeg::encode(Span src, uint8_t *destination,\n> +\t\t\t size_t destinationSize, Span exifData,\n> +\t\t\t unsigned int quality)\n> {\n> -\tunsigned char *destination = dest.data();\n> -\tunsigned long size = dest.size();\n> -\n> \tjpeg_set_quality(&compress_, quality, TRUE);\n> \n> \t/*\n> @@ -206,7 +205,7 @@ int EncoderLibJpeg::encode(Span src, Span dest,\n> \t * \\todo Implement our own custom memory destination to prevent\n> \t * reallocation and prefer failure with correct reporting.\n> \t */\n> -\tjpeg_mem_dest(&compress_, &destination, &size);\n> +\tjpeg_mem_dest(&compress_, &destination, &destinationSize);\n> \n> \tjpeg_start_compress(&compress_, TRUE);\n> \n> @@ -226,5 +225,5 @@ int EncoderLibJpeg::encode(Span src, Span dest,\n> \n> \tjpeg_finish_compress(&compress_);\n> \n> -\treturn size;\n> +\treturn destinationSize;\n> }\n> diff --git a/src/android/jpeg/encoder_libjpeg.h b/src/android/jpeg/encoder_libjpeg.h\n> index 838da7728382..bda5c16089df 100644\n> --- a/src/android/jpeg/encoder_libjpeg.h\n> +++ b/src/android/jpeg/encoder_libjpeg.h\n> @@ -22,11 +22,11 @@ public:\n> \n> \tint configure(const libcamera::StreamConfiguration &cfg) override;\n> \tint encode(const libcamera::FrameBuffer &source,\n> -\t\t libcamera::Span destination,\n> +\t\t uint8_t *destination, size_t destinationSize,\n> \t\t libcamera::Span exifData,\n> \t\t unsigned int quality) override;\n> \tint encode(libcamera::Span source,\n> -\t\t libcamera::Span destination,\n> +\t\t uint8_t *destination, size_t destinationSize,\n> \t\t libcamera::Span exifData,\n> \t\t unsigned int quality);\n> \n> diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\n> index ab5b63844067..d6eeb962e81d 100644\n> --- a/src/android/jpeg/post_processor_jpeg.cpp\n> +++ b/src/android/jpeg/post_processor_jpeg.cpp\n> @@ -73,7 +73,9 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n> \t\tthumbnail->resize(rawThumbnail.size());\n> \n> \t\tint jpeg_size = thumbnailEncoder_.encode(rawThumbnail,\n> -\t\t\t\t\t\t\t *thumbnail, {}, quality);\n> +\t\t\t\t\t\t\t thumbnail->data(),\n> +\t\t\t\t\t\t\t thumbnail->size(),\n> +\t\t\t\t\t\t\t {}, quality);\n> \t\tthumbnail->resize(jpeg_size);\n> \n> \t\tLOG(JPEG, Debug)\n> @@ -83,13 +85,15 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n> }\n> \n> int PostProcessorJpeg::process(const FrameBuffer &source,\n> -\t\t\t libcamera::MappedBuffer *destination,\n> +\t\t\t CameraBuffer *destination,\n> \t\t\t const CameraMetadata &requestMetadata,\n> \t\t\t CameraMetadata *resultMetadata)\n> {\n> \tif (!encoder_)\n> \t\treturn 0;\n> \n> +\tASSERT(destination->numPlanes() == 1);\n> +\n> \tcamera_metadata_ro_entry_t entry;\n> \tint ret;\n> \n> @@ -172,27 +176,17 @@ int PostProcessorJpeg::process(const FrameBuffer &source,\n> \tconst uint8_t quality = ret ? *entry.data.u8 : 95;\n> \tresultMetadata->addEntry(ANDROID_JPEG_QUALITY, &quality, 1);\n> \n> -\tint jpeg_size = encoder_->encode(source, destination->maps()[0],\n> +\tint jpeg_size = encoder_->encode(source, destination->plane(0),\n> +\t\t\t\t\t destination->planeSize(0),\n\nThe CameraBuffer::plane() function should return a span, and I'd then\ndrop CameraBuffer::planeSize().\n\n> \t\t\t\t\t exif.data(), quality);\n> \tif (jpeg_size < 0) {\n> \t\tLOG(JPEG, Error) << \"Failed to encode stream image\";\n> \t\treturn jpeg_size;\n> \t}\n> \n> -\t/*\n> -\t * Fill in the JPEG blob header.\n> -\t *\n> -\t * The mapped size of the buffer is being returned as\n> -\t * substantially larger than the requested JPEG_MAX_SIZE\n> -\t * (which is referenced from maxJpegBufferSize_). Utilise\n> -\t * this static size to ensure the correct offset of the blob is\n> -\t * determined.\n> -\t *\n> -\t * \\todo Investigate if the buffer size mismatch is an issue or\n> -\t * expected behaviour.\n> -\t */\n> -\tuint8_t *resultPtr = destination->maps()[0].data() +\n> -\t\t\t cameraDevice_->maxJpegBufferSize() -\n> +\t/* Fill in the JPEG blob header. */\n> +\tuint8_t *resultPtr = destination->plane(0) +\n> +\t\t\t destination->planeSize(0) -\n> \t\t\t sizeof(struct camera3_jpeg_blob);\n> \tauto *blob = reinterpret_cast(resultPtr);\n> \tblob->jpeg_blob_id = CAMERA3_JPEG_BLOB_ID;\n> diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h\n> index 7689de73c664..5d2d4ab224b1 100644\n> --- a/src/android/jpeg/post_processor_jpeg.h\n> +++ b/src/android/jpeg/post_processor_jpeg.h\n> @@ -25,7 +25,7 @@ public:\n> \tint configure(const libcamera::StreamConfiguration &incfg,\n> \t\t const libcamera::StreamConfiguration &outcfg) override;\n> \tint process(const libcamera::FrameBuffer &source,\n> -\t\t libcamera::MappedBuffer *destination,\n> +\t\t CameraBuffer *destination,\n> \t\t const CameraMetadata &requestMetadata,\n> \t\t CameraMetadata *resultMetadata) override;\n> \n> diff --git a/src/android/post_processor.h b/src/android/post_processor.h\n> index ac40d3414892..4944078b490c 100644\n> --- a/src/android/post_processor.h\n> +++ b/src/android/post_processor.h\n> @@ -12,6 +12,8 @@\n> \n> #include \n> \n> +#include \"camera_buffer.h\"\n> +\n\nMaybe a forward declaration of CameraBuffer would be enough here ?\n\nThe rest of the patch looks good to me. With these issues addressed,\n\nReviewed-by: Laurent Pinchart \n\n> class CameraMetadata;\n> \n> class PostProcessor\n> @@ -22,7 +24,7 @@ public:\n> \tvirtual int configure(const libcamera::StreamConfiguration &inCfg,\n> \t\t\t const libcamera::StreamConfiguration &outCfg) = 0;\n> \tvirtual int process(const libcamera::FrameBuffer &source,\n> -\t\t\t libcamera::MappedBuffer *destination,\n> +\t\t\t CameraBuffer *destination,\n> \t\t\t const CameraMetadata &requestMetadata,\n> \t\t\t CameraMetadata *resultMetadata) = 0;\n> };\n> diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp\n> index 1318349ad66b..f1487185a95a 100644\n> --- a/src/android/yuv/post_processor_yuv.cpp\n> +++ b/src/android/yuv/post_processor_yuv.cpp\n> @@ -48,7 +48,7 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg,\n> }\n> \n> int PostProcessorYuv::process(const FrameBuffer &source,\n> -\t\t\t libcamera::MappedBuffer *destination,\n> +\t\t\t CameraBuffer *destination,\n> \t\t\t [[maybe_unused]] const CameraMetadata &requestMetadata,\n> \t\t\t [[maybe_unused]] CameraMetadata *metadata)\n> {\n> @@ -66,9 +66,9 @@ int PostProcessorYuv::process(const FrameBuffer &source,\n> \t\t\t\t sourceMapped.maps()[1].data(),\n> \t\t\t\t sourceStride_[1],\n> \t\t\t\t sourceSize_.width, sourceSize_.height,\n> -\t\t\t\t destination->maps()[0].data(),\n> +\t\t\t\t destination->plane(0),\n> \t\t\t\t destinationStride_[0],\n> -\t\t\t\t destination->maps()[1].data(),\n> +\t\t\t\t destination->plane(1),\n> \t\t\t\t destinationStride_[1],\n> \t\t\t\t destinationSize_.width,\n> \t\t\t\t destinationSize_.height,\n> @@ -82,16 +82,16 @@ int PostProcessorYuv::process(const FrameBuffer &source,\n> }\n> \n> bool PostProcessorYuv::isValidBuffers(const FrameBuffer &source,\n> -\t\t\t\t const libcamera::MappedBuffer &destination) const\n> +\t\t\t\t const CameraBuffer &destination) const\n> {\n> \tif (source.planes().size() != 2) {\n> \t\tLOG(YUV, Error) << \"Invalid number of source planes: \"\n> \t\t\t\t<< source.planes().size();\n> \t\treturn false;\n> \t}\n> -\tif (destination.maps().size() != 2) {\n> +\tif (destination.numPlanes() != 2) {\n> \t\tLOG(YUV, Error) << \"Invalid number of destination planes: \"\n> -\t\t\t\t<< destination.maps().size();\n> +\t\t\t\t<< destination.numPlanes();\n> \t\treturn false;\n> \t}\n> \n> @@ -106,12 +106,12 @@ bool PostProcessorYuv::isValidBuffers(const FrameBuffer &source,\n> \t\t\t<< sourceLength_[1] << \"}\";\n> \t\treturn false;\n> \t}\n> -\tif (destination.maps()[0].size() < destinationLength_[0] ||\n> -\t destination.maps()[1].size() < destinationLength_[1]) {\n> +\tif (destination.planeSize(0) < destinationLength_[0] ||\n> +\t destination.planeSize(1) < destinationLength_[1]) {\n> \t\tLOG(YUV, Error)\n> \t\t\t<< \"The destination planes lengths are too small, actual size: {\"\n> -\t\t\t<< destination.maps()[0].size() << \", \"\n> -\t\t\t<< destination.maps()[1].size()\n> +\t\t\t<< destination.planeSize(0) << \", \"\n> +\t\t\t<< destination.planeSize(1)\n> \t\t\t<< \"}, expected size: {\"\n> \t\t\t<< sourceLength_[0] << \", \"\n> \t\t\t<< sourceLength_[1] << \"}\";\n> diff --git a/src/android/yuv/post_processor_yuv.h b/src/android/yuv/post_processor_yuv.h\n> index c58b4cf790fc..f8b1ba23fa6c 100644\n> --- a/src/android/yuv/post_processor_yuv.h\n> +++ b/src/android/yuv/post_processor_yuv.h\n> @@ -21,13 +21,13 @@ public:\n> \tint configure(const libcamera::StreamConfiguration &incfg,\n> \t\t const libcamera::StreamConfiguration &outcfg) override;\n> \tint process(const libcamera::FrameBuffer &source,\n> -\t\t libcamera::MappedBuffer *destination,\n> +\t\t CameraBuffer *destination,\n> \t\t const CameraMetadata &requestMetadata,\n> \t\t CameraMetadata *metadata) override;\n> \n> private:\n> \tbool isValidBuffers(const libcamera::FrameBuffer &source,\n> -\t\t\t const libcamera::MappedBuffer &destination) const;\n> +\t\t\t const CameraBuffer &destination) const;\n> \tvoid calculateLengths(const libcamera::StreamConfiguration &inCfg,\n> \t\t\t const libcamera::StreamConfiguration &outCfg);\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4C054BD1F1\n\tfor ;\n\tSun, 28 Feb 2021 18:42:49 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C39D368A7A;\n\tSun, 28 Feb 2021 19:42:48 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 98A4A68A45\n\tfor ;\n\tSun, 28 Feb 2021 19:42:46 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 080A980F;\n\tSun, 28 Feb 2021 19:42:45 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"aVITDTrC\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1614537766;\n\tbh=fdH/UufLPGb9LRjs8ycpsP5+FxRNJdX9j1JO8HQzzWM=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=aVITDTrCYQSTzYCWq3cTznYWcbjwAD2xLLbGSBaltfTr4fbGEMfIvp4oL9nIHVTks\n\tkgNYKjhhvohFyyeYxPlAnt1lEhEAc8xjKrKBweLPdR+6gdmn56gH23gBkXjru7/scU\n\tV3FFjidixbWf9QKqqnbojY3QMD4O7UA7SVGG+HLU=","Date":"Sun, 28 Feb 2021 20:42:17 +0200","From":"Laurent Pinchart ","To":"Jacopo Mondi ","Message-ID":"","References":"<20210226132932.165484-1-jacopo@jmondi.org>\n\t<20210226132932.165484-9-jacopo@jmondi.org>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210226132932.165484-9-jacopo@jmondi.org>","Subject":"Re: [libcamera-devel] [PATCH 08/12] android: post_processor: Use\n\tCameraBuffer API","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]