[{"id":13145,"web_url":"https://patchwork.libcamera.org/comment/13145/","msgid":"<6c025653-12be-077f-d9b0-35aa5e248bb7@ideasonboard.com>","date":"2020-10-09T13:33:54","subject":"Re: [libcamera-devel] [RFC PATCH 3/3] android: jpeg: Add a basic\n\tNV12 image thumbnailer","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Umang,\n\nOn 08/10/2020 15:10, Umang Jain wrote:\n> Add a basic image thumbnailer for NV12 frames being captured.\n> It shall generate a thumbnail image to be embedded as a part of\n> EXIF metadata of the frame. The output of the thumbnail will still\n> be NV12.\n> \n> Signed-off-by: Umang Jain <email@uajain.com>\n> ---\n>  src/android/jpeg/post_processor_jpeg.cpp |  11 ++\n>  src/android/jpeg/thumbnailer.cpp         | 134 +++++++++++++++++++++++\n>  src/android/jpeg/thumbnailer.h           |  34 ++++++\n>  src/android/meson.build                  |   1 +\n>  4 files changed, 180 insertions(+)\n>  create mode 100644 src/android/jpeg/thumbnailer.cpp\n>  create mode 100644 src/android/jpeg/thumbnailer.h\n> \n> diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\n> index eeb4e95..9076d04 100644\n> --- a/src/android/jpeg/post_processor_jpeg.cpp\n> +++ b/src/android/jpeg/post_processor_jpeg.cpp\n> @@ -8,6 +8,7 @@\n>  #include \"post_processor_jpeg.h\"\n>  \n>  #include \"exif.h\"\n> +#include \"thumbnailer.h\"\n>  \n>  #include \"../camera_device.h\"\n>  \n> @@ -286,6 +287,16 @@ int PostProcessorJpeg::encode(const FrameBuffer *source,\n>  \tLOG(JPEG, Debug) << \"JPEG Encode Starting:\" << compress_.image_width\n>  \t\t\t << \"x\" << compress_.image_height;\n>  \n> +\tThumbnailer th;\n> +\tlibcamera::Span<uint8_t> thumbnail;\n> +\tth.configure(Size (compress_.image_width, compress_.image_height),\n> +\t\t     pixelFormatInfo_->format);\n\n\nYou could move the Thumbnailer into the PostProcessorJpeg class as a\nprivate, and call th.configure during PostProcessorJpeg::configure().\n\nPotentially even allocate the destination thumbnail buffer there too?\n\n> +\tth.scaleBuffer(source, thumbnail);\n> +\t/*\n> +\t * \\todo: Compress the thumbnail again encode() and set it in the\n> +\t * respective EXIF field.\n> +\t */\n> +\n>  \tif (nv_)\n>  \t\tcompressNV(&frame);\n>  \telse\n> diff --git a/src/android/jpeg/thumbnailer.cpp b/src/android/jpeg/thumbnailer.cpp\n> new file mode 100644\n> index 0000000..d01b4af\n> --- /dev/null\n> +++ b/src/android/jpeg/thumbnailer.cpp\n> @@ -0,0 +1,134 @@\n> +/* SPDX-License-Identifier: GPL-2.0-or-later */\n\nLGPL-2.1...\n\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * thumbnailer.cpp - Basic image thumbnailer from NV12\n> + */\n> +\n> +#include \"thumbnailer.h\"\n> +\n> +#include <libcamera/formats.h>\n> +\n> +#include \"libcamera/internal/file.h\"\n> +#include \"libcamera/internal/log.h\"\n> +\n> +using namespace libcamera;\n> +\n> +LOG_DEFINE_CATEGORY(Thumbnailer)\n> +\n> +Thumbnailer::Thumbnailer()\n> +\t: validConfiguration_(false)\n> +{\n> +}\n> +\n> +void Thumbnailer::configure(const Size &sourceSize, PixelFormat pixelFormat)\n> +{\n> +\tsourceSize_ = sourceSize;\n> +\tpixelFormat_ = pixelFormat;\n> +\n> +\tif (pixelFormat_ != formats::NV12) {\n> +\t\tLOG (Thumbnailer, Error) << \"Failed to configure: Pixel Format \"\n> +\t\t\t\t    << pixelFormat_.toString() << \" unsupported.\";\n> +\t\treturn;\n> +\t}\n> +\n> +\tvalidConfiguration_ = true;\n> +}\n> +\n> +static std::string datetime()\n> +{\n> +\ttime_t rawtime;\n> +\tstruct tm *timeinfo;\n> +\tchar buffer[80];\n> +\tstatic unsigned int milliseconds = 0;\n> +\n> +\ttime(&rawtime);\n> +\ttimeinfo = localtime(&rawtime);\n> +\n> +\tstrftime(buffer, 80, \"%d-%m-%Y.%H-%M-%S.\", timeinfo);\n> +\n> +\t/* milliseconds is just a fast hack to ensure unique filenames */\n> +\treturn std::string(buffer) + std::to_string(milliseconds++);\n> +}\n> +\n> +/*\n> + * The Exif specification recommends the width of the thumbnail to be a\n> + * mutiple of 16 (section 4.8.1). Hence, compute the corresponding height\n\ns/mutiple/multiple/\n\n> + * keeping the aspect ratio same as of the source.\n> + */\n> +Size Thumbnailer::computeThumbnailSize()\n> +{\n> +\tunsigned int targetHeight;\n> +\tunsigned int targetWidth = 160;\n> +\n> +\ttargetHeight = targetWidth * sourceSize_.height / sourceSize_.width;\n> +\n> +\tif (targetHeight & 1)\n> +\t\ttargetHeight++;\n> +\n> +\treturn Size(targetWidth, targetHeight);\n> +}\n> +\n> +int Thumbnailer::scaleBuffer(const FrameBuffer *source, Span<uint8_t> &dest)\n> +{\n> +\tMappedFrameBuffer frame(source, PROT_READ);\n> +\tif (!frame.isValid()) {\n> +\t\tLOG(Thumbnailer, Error) << \"Failed to map FrameBuffer : \"\n> +\t\t\t\t        << strerror(frame.error());\n> +\t\treturn frame.error();\n> +\t}\n> +\n> +\tif (!validConfiguration_) {\n> +\t\tLOG(Thumbnailer, Error) << \"config is unconfigured or invalid.\";\n> +\t\treturn -1;\n> +\t}\n> +\n> +\ttargetSize_ = computeThumbnailSize();\n> +\n> +\tconst unsigned int sw = sourceSize_.width;\n> +\tconst unsigned int sh = sourceSize_.height;\n> +\tconst unsigned int tw = targetSize_.width;\n> +\tconst unsigned int th = targetSize_.height;\n> +\n> +\t/* Image scaling block implementing nearest-neighbour algorithm. */\n> +\tunsigned char *src = static_cast<unsigned char *>(frame.maps()[0].data());\n> +\tunsigned char *src_c = src + sh * sw;\n> +\tunsigned int cb_pos = 0;\n> +\tunsigned int cr_pos = 1;\n> +\tunsigned char *src_cb, *src_cr;\n> +\n> +\tsize_t dstSize = (th * tw) + ((th/2) * tw);\n\nHow about move this to a helper function:\n\nint Thumbnailer::outputSize()\n{\n\tif (!validConfiguration_)\n\t\treturn -1\n\t\n\treturn targetSize_.height * targetSize_.width * 3 / 2\n}\n\n> +\tunsigned char *destination = static_cast<unsigned char *>(malloc(dstSize));\n\nAnd allocate this in the caller, passing in the buffer as a const Span\nreference or such.\n\n> +\tunsigned char *dst = destination;\n> +\tunsigned char *dst_c = destination + th * tw;\n> +\n> +\tfor (unsigned int y = 0; y < th; y+=2) {\n> +\t\tunsigned int sourceY = (sh*y + th/2) / th;\n> +\n> +\t\tsrc_cb = src_c + (sourceY/2) * sw + cb_pos;\n> +\t\tsrc_cr = src_c + (sourceY/2) * sw + cr_pos;\n> +\n> +\t\tfor (unsigned int x = 0; x < tw; x+=2) {\n> +\t\t\tunsigned int sourceX = (sw*x + tw/2) / tw;\n> +\n> +\t\t\tdst[y     * tw + x]     = src[sw * sourceY     + sourceX];\n> +\t\t\tdst[(y+1) * tw + x]     = src[sw * (sourceY+1) + sourceX];\n> +\t\t\tdst[y     * tw + (x+1)] = src[sw * sourceY     + (sourceX+1)];\n> +\t\t\tdst[(y+1) * tw + (x+1)] = src[sw * (sourceY+1) + (sourceX+1)];\n> +\n> +\t\t\tdst_c[(y/2) * tw + x + cb_pos] = src_cb[(sourceX/2) * 2];\n> +\t\t\tdst_c[(y/2) * tw + x + cr_pos] = src_cr[(sourceX/2) * 2];\n> +\t\t}\n> +\t}\n> +\n> +\t/* Helper code: Write the output pixels to a file so we can inspect */\n> +\tFile file(\"/tmp/\" + datetime() + \".raw\");\n> +\tint32_t ret = file.open(File::WriteOnly);\n> +\tret = file.write({ destination, dstSize });\n> +\tLOG(Thumbnailer, Info) << \"Wrote \" << ret << \" bytes: \" << targetSize_.width << \"x\" << targetSize_.height;\n> +\n\nAnd of course that will be removed for a non-rfc...\n\n> +\t/* Write scaled pixels to dest */\n> +\tdest = { destination, dstSize };\n\nAnd this wouldnt' be needed if the dest is passed in.\n\n> +\n> +\treturn 0;\n> +}\n> diff --git a/src/android/jpeg/thumbnailer.h b/src/android/jpeg/thumbnailer.h\n> new file mode 100644\n> index 0000000..af3a194\n> --- /dev/null\n> +++ b/src/android/jpeg/thumbnailer.h\n> @@ -0,0 +1,34 @@\n> +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * thumbnailer.h - Basic image thumbnailer from NV12\n> + */\n> +#ifndef __ANDROID_JPEG_THUMBNAILER_H__\n> +#define __ANDROID_JPEG_THUMBNAILER_H__\n> +\n> +#include <libcamera/geometry.h>\n> +\n> +#include \"libcamera/internal/buffer.h\"\n> +#include \"libcamera/internal/formats.h\"\n> +\n> +class Thumbnailer\n> +{\n> +public:\n> +\tThumbnailer();\n> +\n> +\tvoid configure(const libcamera::Size &sourceSize,\n> +\t\t       libcamera::PixelFormat pixelFormat);\n> +\tint scaleBuffer(const libcamera::FrameBuffer *source, libcamera::Span<uint8_t> &dest);\n> +\n> +private:\n> +\tlibcamera::Size computeThumbnailSize();\n> +\n> +\tlibcamera::PixelFormat pixelFormat_;\n> +\tlibcamera::Size sourceSize_;\n> +\tlibcamera::Size targetSize_;\n> +\n> +\tbool validConfiguration_;\n> +};\n> +\n> +#endif /* __ANDROID_JPEG_THUMBNAILER_H__ */\n> diff --git a/src/android/meson.build b/src/android/meson.build\n> index 02b3b47..854005e 100644\n> --- a/src/android/meson.build\n> +++ b/src/android/meson.build\n> @@ -23,6 +23,7 @@ android_hal_sources = files([\n>      'camera_stream.cpp',\n>      'jpeg/exif.cpp',\n>      'jpeg/post_processor_jpeg.cpp',\n> +    'jpeg/thumbnailer.cpp',\n>  ])\n>  \n>  android_camera_metadata_sources = files([\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4D5D7BEEE0\n\tfor <parsemail@patchwork.libcamera.org>;\n\tFri,  9 Oct 2020 13:33:59 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D26FF60725;\n\tFri,  9 Oct 2020 15:33:58 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2122960358\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri,  9 Oct 2020 15:33:58 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 7DE25539;\n\tFri,  9 Oct 2020 15:33:57 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Ql+areTV\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1602250437;\n\tbh=wx1vGX8rlPtrxPsWsJWDE9c7igaBQfYtekbwIprzNj0=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=Ql+areTV3Gy0/RVQIlTR+6JY/XbSKPZcxnb1P5tb1CZBgJH61J4Y8wcAvtU4Qek4S\n\t4dLrTnjCC4sVaYggop2sofbzdssYTRy8DplbeunM9RVGGfrU6792AEup76+DrwVvW5\n\t2a+Hsm6J8lcfhVmCHtRfnaMhNJjqEYu7T10FUluM=","To":"Umang Jain <email@uajain.com>, libcamera-devel@lists.libcamera.org","References":"<20201008141038.83425-1-email@uajain.com>\n\t<20201008141038.83425-4-email@uajain.com>","From":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<6c025653-12be-077f-d9b0-35aa5e248bb7@ideasonboard.com>","Date":"Fri, 9 Oct 2020 14:33:54 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20201008141038.83425-4-email@uajain.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [RFC PATCH 3/3] android: jpeg: Add a basic\n\tNV12 image thumbnailer","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":13152,"web_url":"https://patchwork.libcamera.org/comment/13152/","msgid":"<20201009223116.GH25040@pendragon.ideasonboard.com>","date":"2020-10-09T22:31:16","subject":"Re: [libcamera-devel] [RFC PATCH 3/3] android: jpeg: Add a basic\n\tNV12 image thumbnailer","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hello,\n\nOn Fri, Oct 09, 2020 at 02:33:54PM +0100, Kieran Bingham wrote:\n> On 08/10/2020 15:10, Umang Jain wrote:\n> > Add a basic image thumbnailer for NV12 frames being captured.\n> > It shall generate a thumbnail image to be embedded as a part of\n> > EXIF metadata of the frame. The output of the thumbnail will still\n> > be NV12.\n> > \n> > Signed-off-by: Umang Jain <email@uajain.com>\n> > ---\n> >  src/android/jpeg/post_processor_jpeg.cpp |  11 ++\n> >  src/android/jpeg/thumbnailer.cpp         | 134 +++++++++++++++++++++++\n> >  src/android/jpeg/thumbnailer.h           |  34 ++++++\n> >  src/android/meson.build                  |   1 +\n> >  4 files changed, 180 insertions(+)\n> >  create mode 100644 src/android/jpeg/thumbnailer.cpp\n> >  create mode 100644 src/android/jpeg/thumbnailer.h\n> > \n> > diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\n> > index eeb4e95..9076d04 100644\n> > --- a/src/android/jpeg/post_processor_jpeg.cpp\n> > +++ b/src/android/jpeg/post_processor_jpeg.cpp\n> > @@ -8,6 +8,7 @@\n> >  #include \"post_processor_jpeg.h\"\n> >  \n> >  #include \"exif.h\"\n> > +#include \"thumbnailer.h\"\n> >  \n> >  #include \"../camera_device.h\"\n> >  \n> > @@ -286,6 +287,16 @@ int PostProcessorJpeg::encode(const FrameBuffer *source,\n> >  \tLOG(JPEG, Debug) << \"JPEG Encode Starting:\" << compress_.image_width\n> >  \t\t\t << \"x\" << compress_.image_height;\n> >  \n> > +\tThumbnailer th;\n> > +\tlibcamera::Span<uint8_t> thumbnail;\n> > +\tth.configure(Size (compress_.image_width, compress_.image_height),\n> > +\t\t     pixelFormatInfo_->format);\n> \n> You could move the Thumbnailer into the PostProcessorJpeg class as a\n> private, and call th.configure during PostProcessorJpeg::configure().\n> \n> Potentially even allocate the destination thumbnail buffer there too?\n\nGiven that there's not much to configure (it's just about storing the\nsize internally), how about passing the target size to the\nThumbnailer::scaleBuffer() function ? configure() is useful for\ncomponents that are configured once (potentially with expensive\noperations at configure time) and run multiple times, but here I don't\nthink that's needed.\n\n> > +\tth.scaleBuffer(source, thumbnail);\n> > +\t/*\n> > +\t * \\todo: Compress the thumbnail again encode() and set it in the\n> > +\t * respective EXIF field.\n> > +\t */\n> > +\n> >  \tif (nv_)\n> >  \t\tcompressNV(&frame);\n> >  \telse\n> > diff --git a/src/android/jpeg/thumbnailer.cpp b/src/android/jpeg/thumbnailer.cpp\n> > new file mode 100644\n> > index 0000000..d01b4af\n> > --- /dev/null\n> > +++ b/src/android/jpeg/thumbnailer.cpp\n> > @@ -0,0 +1,134 @@\n> > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> \n> LGPL-2.1...\n> \n> > +/*\n> > + * Copyright (C) 2020, Google Inc.\n> > + *\n> > + * thumbnailer.cpp - Basic image thumbnailer from NV12\n> > + */\n> > +\n> > +#include \"thumbnailer.h\"\n> > +\n> > +#include <libcamera/formats.h>\n> > +\n> > +#include \"libcamera/internal/file.h\"\n> > +#include \"libcamera/internal/log.h\"\n> > +\n> > +using namespace libcamera;\n> > +\n> > +LOG_DEFINE_CATEGORY(Thumbnailer)\n> > +\n> > +Thumbnailer::Thumbnailer()\n> > +\t: validConfiguration_(false)\n> > +{\n> > +}\n> > +\n> > +void Thumbnailer::configure(const Size &sourceSize, PixelFormat pixelFormat)\n> > +{\n> > +\tsourceSize_ = sourceSize;\n> > +\tpixelFormat_ = pixelFormat;\n> > +\n> > +\tif (pixelFormat_ != formats::NV12) {\n> > +\t\tLOG (Thumbnailer, Error) << \"Failed to configure: Pixel Format \"\n> > +\t\t\t\t    << pixelFormat_.toString() << \" unsupported.\";\n> > +\t\treturn;\n> > +\t}\n> > +\n> > +\tvalidConfiguration_ = true;\n> > +}\n> > +\n> > +static std::string datetime()\n> > +{\n> > +\ttime_t rawtime;\n> > +\tstruct tm *timeinfo;\n> > +\tchar buffer[80];\n> > +\tstatic unsigned int milliseconds = 0;\n> > +\n> > +\ttime(&rawtime);\n> > +\ttimeinfo = localtime(&rawtime);\n> > +\n> > +\tstrftime(buffer, 80, \"%d-%m-%Y.%H-%M-%S.\", timeinfo);\n> > +\n> > +\t/* milliseconds is just a fast hack to ensure unique filenames */\n> > +\treturn std::string(buffer) + std::to_string(milliseconds++);\n> > +}\n> > +\n> > +/*\n> > + * The Exif specification recommends the width of the thumbnail to be a\n> > + * mutiple of 16 (section 4.8.1). Hence, compute the corresponding height\n> \n> s/mutiple/multiple/\n> \n> > + * keeping the aspect ratio same as of the source.\n> > + */\n> > +Size Thumbnailer::computeThumbnailSize()\n> > +{\n> > +\tunsigned int targetHeight;\n> > +\tunsigned int targetWidth = 160;\n> > +\n> > +\ttargetHeight = targetWidth * sourceSize_.height / sourceSize_.width;\n> > +\n> > +\tif (targetHeight & 1)\n> > +\t\ttargetHeight++;\n> > +\n> > +\treturn Size(targetWidth, targetHeight);\n> > +}\n> > +\n> > +int Thumbnailer::scaleBuffer(const FrameBuffer *source, Span<uint8_t> &dest)\n> > +{\n> > +\tMappedFrameBuffer frame(source, PROT_READ);\n> > +\tif (!frame.isValid()) {\n> > +\t\tLOG(Thumbnailer, Error) << \"Failed to map FrameBuffer : \"\n> > +\t\t\t\t        << strerror(frame.error());\n> > +\t\treturn frame.error();\n> > +\t}\n> > +\n> > +\tif (!validConfiguration_) {\n> > +\t\tLOG(Thumbnailer, Error) << \"config is unconfigured or invalid.\";\n> > +\t\treturn -1;\n> > +\t}\n> > +\n> > +\ttargetSize_ = computeThumbnailSize();\n> > +\n> > +\tconst unsigned int sw = sourceSize_.width;\n> > +\tconst unsigned int sh = sourceSize_.height;\n> > +\tconst unsigned int tw = targetSize_.width;\n> > +\tconst unsigned int th = targetSize_.height;\n> > +\n> > +\t/* Image scaling block implementing nearest-neighbour algorithm. */\n> > +\tunsigned char *src = static_cast<unsigned char *>(frame.maps()[0].data());\n> > +\tunsigned char *src_c = src + sh * sw;\n> > +\tunsigned int cb_pos = 0;\n> > +\tunsigned int cr_pos = 1;\n> > +\tunsigned char *src_cb, *src_cr;\n> > +\n> > +\tsize_t dstSize = (th * tw) + ((th/2) * tw);\n> \n> How about move this to a helper function:\n> \n> int Thumbnailer::outputSize()\n> {\n> \tif (!validConfiguration_)\n> \t\treturn -1\n> \t\n> \treturn targetSize_.height * targetSize_.width * 3 / 2\n> }\n\nIt's only done once though, not sure it's worth it.\n\n> > +\tunsigned char *destination = static_cast<unsigned char *>(malloc(dstSize));\n> \n> And allocate this in the caller, passing in the buffer as a const Span\n> reference or such.\n\nWouldn't that make it more complex for the caller, that would need to\nretrieve the size, allocate the buffer and then call scaleBuffer() ? How\nabout returning a std::vector from this function instead ? That would\nmake buffer ownership clear, simplify the API for the caller, and with\ncopy elision there will be no copy.\n\nNow that I've written this, I've read\nhttps://en.cppreference.com/w/cpp/language/copy_elision, and it seems\ncopy elision is only mandatory for the compiler to implement when the\noperand to the return statement is a prvalue. In practice I expect the\ncopy to always be eluded (I've tested it with recent g++ and clang++\nwith -O0 and -Os), but if we want a strong guarantee, we would need to\npass the vector as an argument.\n\n> > +\tunsigned char *dst = destination;\n> > +\tunsigned char *dst_c = destination + th * tw;\n> > +\n> > +\tfor (unsigned int y = 0; y < th; y+=2) {\n> > +\t\tunsigned int sourceY = (sh*y + th/2) / th;\n> > +\n> > +\t\tsrc_cb = src_c + (sourceY/2) * sw + cb_pos;\n> > +\t\tsrc_cr = src_c + (sourceY/2) * sw + cr_pos;\n> > +\n> > +\t\tfor (unsigned int x = 0; x < tw; x+=2) {\n> > +\t\t\tunsigned int sourceX = (sw*x + tw/2) / tw;\n> > +\n> > +\t\t\tdst[y     * tw + x]     = src[sw * sourceY     + sourceX];\n> > +\t\t\tdst[(y+1) * tw + x]     = src[sw * (sourceY+1) + sourceX];\n> > +\t\t\tdst[y     * tw + (x+1)] = src[sw * sourceY     + (sourceX+1)];\n> > +\t\t\tdst[(y+1) * tw + (x+1)] = src[sw * (sourceY+1) + (sourceX+1)];\n> > +\n> > +\t\t\tdst_c[(y/2) * tw + x + cb_pos] = src_cb[(sourceX/2) * 2];\n> > +\t\t\tdst_c[(y/2) * tw + x + cr_pos] = src_cr[(sourceX/2) * 2];\n> > +\t\t}\n> > +\t}\n> > +\n> > +\t/* Helper code: Write the output pixels to a file so we can inspect */\n> > +\tFile file(\"/tmp/\" + datetime() + \".raw\");\n> > +\tint32_t ret = file.open(File::WriteOnly);\n> > +\tret = file.write({ destination, dstSize });\n> > +\tLOG(Thumbnailer, Info) << \"Wrote \" << ret << \" bytes: \" << targetSize_.width << \"x\" << targetSize_.height;\n> > +\n> \n> And of course that will be removed for a non-rfc...\n> \n> > +\t/* Write scaled pixels to dest */\n> > +\tdest = { destination, dstSize };\n> \n> And this wouldnt' be needed if the dest is passed in.\n> \n> > +\n> > +\treturn 0;\n> > +}\n> > diff --git a/src/android/jpeg/thumbnailer.h b/src/android/jpeg/thumbnailer.h\n> > new file mode 100644\n> > index 0000000..af3a194\n> > --- /dev/null\n> > +++ b/src/android/jpeg/thumbnailer.h\n> > @@ -0,0 +1,34 @@\n> > +/* SPDX-License-Identifier: GPL-2.0-or-later */\n> > +/*\n> > + * Copyright (C) 2020, Google Inc.\n> > + *\n> > + * thumbnailer.h - Basic image thumbnailer from NV12\n> > + */\n> > +#ifndef __ANDROID_JPEG_THUMBNAILER_H__\n> > +#define __ANDROID_JPEG_THUMBNAILER_H__\n> > +\n> > +#include <libcamera/geometry.h>\n> > +\n> > +#include \"libcamera/internal/buffer.h\"\n> > +#include \"libcamera/internal/formats.h\"\n> > +\n> > +class Thumbnailer\n> > +{\n> > +public:\n> > +\tThumbnailer();\n> > +\n> > +\tvoid configure(const libcamera::Size &sourceSize,\n> > +\t\t       libcamera::PixelFormat pixelFormat);\n> > +\tint scaleBuffer(const libcamera::FrameBuffer *source, libcamera::Span<uint8_t> &dest);\n> > +\n> > +private:\n> > +\tlibcamera::Size computeThumbnailSize();\n> > +\n> > +\tlibcamera::PixelFormat pixelFormat_;\n> > +\tlibcamera::Size sourceSize_;\n> > +\tlibcamera::Size targetSize_;\n> > +\n> > +\tbool validConfiguration_;\n> > +};\n> > +\n> > +#endif /* __ANDROID_JPEG_THUMBNAILER_H__ */\n> > diff --git a/src/android/meson.build b/src/android/meson.build\n> > index 02b3b47..854005e 100644\n> > --- a/src/android/meson.build\n> > +++ b/src/android/meson.build\n> > @@ -23,6 +23,7 @@ android_hal_sources = files([\n> >      'camera_stream.cpp',\n> >      'jpeg/exif.cpp',\n> >      'jpeg/post_processor_jpeg.cpp',\n> > +    'jpeg/thumbnailer.cpp',\n> >  ])\n> >  \n> >  android_camera_metadata_sources = files([","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1BF56BEEDF\n\tfor <parsemail@patchwork.libcamera.org>;\n\tFri,  9 Oct 2020 22:32:03 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8AB5C60725;\n\tSat, 10 Oct 2020 00:32:02 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 089F760359\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tSat, 10 Oct 2020 00:32:01 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 78BA3528;\n\tSat, 10 Oct 2020 00:32:00 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"wMsryKVk\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1602282720;\n\tbh=L2UoyvcPKysOG5XKrIqd5mwdHsR+pZO4fE+ug6QGXp0=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=wMsryKVkQnJZnbIrN3Epv70FKMTZqzOWvurqsxa5dolElcO61plN11hlOn+WHD/pt\n\tf5g/7daVF/iNHK9fQ+Gnvb1EmulxRLwqzL70qHTgRoPwJjohlJjEDAtcC9icVgqex9\n\tgL3wlqe11cSyMMbG47HS3XGxEErC0LfjRrcoKsYk=","Date":"Sat, 10 Oct 2020 01:31:16 +0300","From":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","To":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Message-ID":"<20201009223116.GH25040@pendragon.ideasonboard.com>","References":"<20201008141038.83425-1-email@uajain.com>\n\t<20201008141038.83425-4-email@uajain.com>\n\t<6c025653-12be-077f-d9b0-35aa5e248bb7@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<6c025653-12be-077f-d9b0-35aa5e248bb7@ideasonboard.com>","Subject":"Re: [libcamera-devel] [RFC PATCH 3/3] android: jpeg: Add a basic\n\tNV12 image thumbnailer","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]