{"id":10984,"url":"https://patchwork.libcamera.org/api/patches/10984/?format=json","web_url":"https://patchwork.libcamera.org/patch/10984/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210125071444.26252-7-paul.elder@ideasonboard.com>","date":"2021-01-25T07:14:42","name":"[libcamera-devel,v4,6/8] android: jpeg: Configure thumbnailer based on request metadata","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"e057fd18073f478cd738c6dd96e1a8e8dcebfe8b","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/?format=json","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"delegate":{"id":17,"url":"https://patchwork.libcamera.org/api/users/17/?format=json","username":"epaul","first_name":"Paul","last_name":"Elder","email":"paul.elder@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/patch/10984/mbox/","series":[{"id":1603,"url":"https://patchwork.libcamera.org/api/series/1603/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1603","date":"2021-01-25T07:14:36","name":"Fill in android result metadata and EXIF tags","version":4,"mbox":"https://patchwork.libcamera.org/series/1603/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/10984/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/10984/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E02C6C0F2B\n\tfor ;\n\tMon, 25 Jan 2021 07:15:10 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AEC7B682AD;\n\tMon, 25 Jan 2021 08:15:10 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D59C2682A8\n\tfor ;\n\tMon, 25 Jan 2021 08:15:08 +0100 (CET)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2DD63331;\n\tMon, 25 Jan 2021 08:15:06 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Yjj1BDv4\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1611558908;\n\tbh=EmyEaGJZaCkGrmvy62xLt/GNUdeZlpLPPagqqWda/ro=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=Yjj1BDv4NWYm7H1eVzwN286b2/K+uInxomw/Gpsh/2WwLRD8+y8+1zjCOULGaKO5f\n\tixST0XVXdyTx/BhWBpxaOl2s/lfQr1/9tzYPI1kUmDhZj9is939Je4CtnN/5F94eZ/\n\tkC1/kHHL/W/8lk07OBuume0/e5kfCQnS5xy0aCBM=","From":"Paul Elder ","To":"libcamera-devel@lists.libcamera.org","Date":"Mon, 25 Jan 2021 16:14:42 +0900","Message-Id":"<20210125071444.26252-7-paul.elder@ideasonboard.com>","X-Mailer":"git-send-email 2.27.0","In-Reply-To":"<20210125071444.26252-1-paul.elder@ideasonboard.com>","References":"<20210125071444.26252-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v4 6/8] android: jpeg: Configure\n\tthumbnailer based on request metadata","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Configure the thumbnailer based on the thumbnail parameters given by the\nandroid request metadata. Only the thumbnail encoder needs to be\nconfigured, and since it is only used at post-processing time, move the\nconfiguration out of the post-processor constructor and into the\nprocessing step.\n\nAlso set the following android result metadata tags:\n- ANDROID_JPEG_THUMBNAIL_SIZE\n- ANDROID_JPEG_THUMBNAIL_QUALITY\n\nSigned-off-by: Paul Elder \nReviewed-by: Laurent Pinchart \n\n---\nChanges in v4:\n- cosmetic changes\n\nNew in v3\n- split from \"android: Set result metadata and EXIF fields based on\n request\"\n---\n src/android/camera_device.cpp | 6 ++--\n src/android/jpeg/post_processor_jpeg.cpp | 43 +++++++++++++++++-------\n src/android/jpeg/post_processor_jpeg.h | 1 +\n src/android/jpeg/thumbnailer.cpp | 25 ++------------\n src/android/jpeg/thumbnailer.h | 6 ++--\n 5 files changed, 40 insertions(+), 41 deletions(-)","diff":"diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\nindex 2cd3c8a1..3068f89f 100644\n--- a/src/android/camera_device.cpp\n+++ b/src/android/camera_device.cpp\n@@ -1915,7 +1915,7 @@ CameraDevice::getResultMetadata(Camera3RequestDescriptor *descriptor,\n \n \t/*\n \t * \\todo Keep this in sync with the actual number of entries.\n-\t * Currently: 38 entries, 147 bytes\n+\t * Currently: 40 entries, 156 bytes\n \t *\n \t * Reserve more space for the JPEG metadata set by the post-processor.\n \t * Currently: ANDROID_JPEG_SIZE (int32_t), ANDROID_JPEG_QUALITY (byte),\n@@ -1923,11 +1923,13 @@ CameraDevice::getResultMetadata(Camera3RequestDescriptor *descriptor,\n \t * ANDROID_JPEG_GPS_PROCESSING_METHOD (byte x 32) = 32 bytes\n \t * ANDROID_JPEG_GPS_TIMESTAMP (int64) = 8 bytes\n \t * ANDROID_JPEG_ORIENTATION (int32_t) = 3 entries, 9 bytes.\n+\t * ANDROID_JPEG_THUMBNAIL_QUALITY (byte) = 1 byte\n+\t * ANDROID_JPEG_THUMBNAIL_SIZE (int32 x 2) = 8 bytes\n \t * ANDROID_LENS_APERTURE (float) = 4 bytes\n \t * ANDROID_LENS_FOCAL_LENGTH (float) = 4 bytes\n \t */\n \tstd::unique_ptr resultMetadata =\n-\t\tstd::make_unique(38, 147);\n+\t\tstd::make_unique(40, 156);\n \tif (!resultMetadata->isValid()) {\n \t\tLOG(HAL, Error) << \"Failed to allocate static metadata\";\n \t\treturn nullptr;\ndiff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\nindex 10c71a47..d35fe361 100644\n--- a/src/android/jpeg/post_processor_jpeg.cpp\n+++ b/src/android/jpeg/post_processor_jpeg.cpp\n@@ -43,12 +43,6 @@ int PostProcessorJpeg::configure(const StreamConfiguration &inCfg,\n \tstreamSize_ = outCfg.size;\n \n \tthumbnailer_.configure(inCfg.size, inCfg.pixelFormat);\n-\tStreamConfiguration thCfg = inCfg;\n-\tthCfg.size = thumbnailer_.size();\n-\tif (thumbnailEncoder_.configure(thCfg) != 0) {\n-\t\tLOG(JPEG, Error) << \"Failed to configure thumbnail encoder\";\n-\t\treturn -EINVAL;\n-\t}\n \n \tencoder_ = std::make_unique();\n \n@@ -56,14 +50,20 @@ int PostProcessorJpeg::configure(const StreamConfiguration &inCfg,\n }\n \n void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n+\t\t\t\t\t const Size &targetSize,\n \t\t\t\t\t std::vector *thumbnail)\n {\n \t/* Stores the raw scaled-down thumbnail bytes. */\n \tstd::vector rawThumbnail;\n \n-\tthumbnailer_.createThumbnail(source, &rawThumbnail);\n+\tthumbnailer_.createThumbnail(source, targetSize, &rawThumbnail);\n \n-\tif (!rawThumbnail.empty()) {\n+\tStreamConfiguration thCfg;\n+\tthCfg.size = targetSize;\n+\tthCfg.pixelFormat = thumbnailer_.pixelFormat();\n+\tint ret = thumbnailEncoder_.configure(thCfg);\n+\n+\tif (!rawThumbnail.empty() && !ret) {\n \t\t/*\n \t\t * \\todo Avoid value-initialization of all elements of the\n \t\t * vector.\n@@ -127,6 +127,28 @@ int PostProcessorJpeg::process(const FrameBuffer &source,\n \t\t\t\t\t entry.data.i64, 1);\n \t}\n \n+\tret = requestMetadata.getEntry(ANDROID_JPEG_THUMBNAIL_SIZE, &entry);\n+\tif (ret) {\n+\t\tconst int32_t *data = entry.data.i32;\n+\t\tSize thumbnailSize = { static_cast(data[0]),\n+\t\t\t\t static_cast(data[1]) };\n+\n+\t\tif (thumbnailSize != Size(0, 0)) {\n+\t\t\tstd::vector thumbnail;\n+\t\t\tgenerateThumbnail(source, thumbnailSize, &thumbnail);\n+\t\t\tif (!thumbnail.empty())\n+\t\t\t\texif.setThumbnail(thumbnail, Exif::Compression::JPEG);\n+\t\t}\n+\n+\t\tresultMetadata->addEntry(ANDROID_JPEG_THUMBNAIL_SIZE, data, 2);\n+\n+\t\t/* \\todo Use this quality as a parameter to the encoder */\n+\t\tret = requestMetadata.getEntry(ANDROID_JPEG_THUMBNAIL_QUALITY, &entry);\n+\t\tif (ret)\n+\t\t\tresultMetadata->addEntry(ANDROID_JPEG_THUMBNAIL_QUALITY,\n+\t\t\t\t\t\t entry.data.u8, 1);\n+\t}\n+\n \tret = requestMetadata.getEntry(ANDROID_JPEG_GPS_COORDINATES, &entry);\n \tif (ret) {\n \t\texif.setGPSLocation(entry.data.d);\n@@ -142,11 +164,6 @@ int PostProcessorJpeg::process(const FrameBuffer &source,\n \t\t\t\t\t entry.data.u8, entry.count);\n \t}\n \n-\tstd::vector thumbnail;\n-\tgenerateThumbnail(source, &thumbnail);\n-\tif (!thumbnail.empty())\n-\t\texif.setThumbnail(thumbnail, Exif::Compression::JPEG);\n-\n \tif (exif.generate() != 0)\n \t\tLOG(JPEG, Error) << \"Failed to generate valid EXIF data\";\n \ndiff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h\nindex d721d1b9..660b79b4 100644\n--- a/src/android/jpeg/post_processor_jpeg.h\n+++ b/src/android/jpeg/post_processor_jpeg.h\n@@ -31,6 +31,7 @@ public:\n \n private:\n \tvoid generateThumbnail(const libcamera::FrameBuffer &source,\n+\t\t\t const libcamera::Size &targetSize,\n \t\t\t std::vector *thumbnail);\n \n \tCameraDevice *const cameraDevice_;\ndiff --git a/src/android/jpeg/thumbnailer.cpp b/src/android/jpeg/thumbnailer.cpp\nindex 5374538a..f709d343 100644\n--- a/src/android/jpeg/thumbnailer.cpp\n+++ b/src/android/jpeg/thumbnailer.cpp\n@@ -32,30 +32,11 @@ void Thumbnailer::configure(const Size &sourceSize, PixelFormat pixelFormat)\n \t\treturn;\n \t}\n \n-\ttargetSize_ = computeThumbnailSize();\n-\n \tvalid_ = true;\n }\n \n-/*\n- * The Exif specification recommends the width of the thumbnail to be a\n- * multiple of 16 (section 4.8.1). Hence, compute the corresponding height\n- * keeping the aspect ratio same as of the source.\n- */\n-Size Thumbnailer::computeThumbnailSize() const\n-{\n-\tunsigned int targetHeight;\n-\tconstexpr unsigned int kTargetWidth = 160;\n-\n-\ttargetHeight = kTargetWidth * sourceSize_.height / sourceSize_.width;\n-\n-\tif (targetHeight & 1)\n-\t\ttargetHeight++;\n-\n-\treturn Size(kTargetWidth, targetHeight);\n-}\n-\n void Thumbnailer::createThumbnail(const FrameBuffer &source,\n+\t\t\t\t const Size &targetSize,\n \t\t\t\t std::vector *destination)\n {\n \tMappedFrameBuffer frame(&source, PROT_READ);\n@@ -73,8 +54,8 @@ void Thumbnailer::createThumbnail(const FrameBuffer &source,\n \n \tconst unsigned int sw = sourceSize_.width;\n \tconst unsigned int sh = sourceSize_.height;\n-\tconst unsigned int tw = targetSize_.width;\n-\tconst unsigned int th = targetSize_.height;\n+\tconst unsigned int tw = targetSize.width;\n+\tconst unsigned int th = targetSize.height;\n \n \tASSERT(tw % 2 == 0 && th % 2 == 0);\n \ndiff --git a/src/android/jpeg/thumbnailer.h b/src/android/jpeg/thumbnailer.h\nindex 98f11833..4e9226c3 100644\n--- a/src/android/jpeg/thumbnailer.h\n+++ b/src/android/jpeg/thumbnailer.h\n@@ -20,15 +20,13 @@ public:\n \tvoid configure(const libcamera::Size &sourceSize,\n \t\t libcamera::PixelFormat pixelFormat);\n \tvoid createThumbnail(const libcamera::FrameBuffer &source,\n+\t\t\t const libcamera::Size &targetSize,\n \t\t\t std::vector *dest);\n-\tconst libcamera::Size &size() const { return targetSize_; }\n+\tconst libcamera::PixelFormat &pixelFormat() const { return pixelFormat_; }\n \n private:\n-\tlibcamera::Size computeThumbnailSize() const;\n-\n \tlibcamera::PixelFormat pixelFormat_;\n \tlibcamera::Size sourceSize_;\n-\tlibcamera::Size targetSize_;\n \n \tbool valid_;\n };\n","prefixes":["libcamera-devel","v4","6/8"]}