From patchwork Tue Jan 26 10:28:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 11002 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EBC66BD808 for ; Tue, 26 Jan 2021 10:28:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B6F5868307; Tue, 26 Jan 2021 11:28:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="QI3MaVck"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7913D68304 for ; Tue, 26 Jan 2021 11:28:49 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BC5408B5; Tue, 26 Jan 2021 11:28:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1611656929; bh=qKle4AOpgVVwmY+HVZGZ6IEANLOhdnbSAgUW+FyN6RI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QI3MaVckMIJ6uqlzNRy4uDLavkVgg15TUp+Lh2ywdmOBLey3PoA0RiP0skDD23rLw +48znfy/hErDLLOstUfKNRKmezIAmBJzZHlr/NdrlLCz2qArpSzETItIgkgYE4c+6j yPuSwldZrwmJ8jVDAN9a7eG0JrLEYgeSn1Uj6k40= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Tue, 26 Jan 2021 19:28:23 +0900 Message-Id: <20210126102825.147026-7-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210126102825.147026-1-paul.elder@ideasonboard.com> References: <20210126102825.147026-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 6/8] android: jpeg: Configure thumbnailer based on request metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Configure the thumbnailer based on the thumbnail parameters given by the android request metadata. Only the thumbnail encoder needs to be configured, and since it is only used at post-processing time, move the configuration out of the post-processor constructor and into the processing step. Also set the following android result metadata tags: - ANDROID_JPEG_THUMBNAIL_SIZE - ANDROID_JPEG_THUMBNAIL_QUALITY Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- Changes in v5: - add metadata entries to availableResultKeys and add the necessary bytes to the static metadata pack size Changes in v4: - cosmetic changes New in v3 - split from "android: Set result metadata and EXIF fields based on request" --- src/android/camera_device.cpp | 8 +++-- src/android/jpeg/post_processor_jpeg.cpp | 43 +++++++++++++++++------- src/android/jpeg/post_processor_jpeg.h | 1 + src/android/jpeg/thumbnailer.cpp | 25 ++------------ src/android/jpeg/thumbnailer.h | 6 ++-- 5 files changed, 41 insertions(+), 42 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 096c2463..f0a24514 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1918,7 +1918,7 @@ CameraDevice::getResultMetadata(Camera3RequestDescriptor *descriptor, /* * \todo Keep this in sync with the actual number of entries. - * Currently: 38 entries, 147 bytes + * Currently: 40 entries, 156 bytes * * Reserve more space for the JPEG metadata set by the post-processor. * Currently: ANDROID_JPEG_SIZE (int32_t), ANDROID_JPEG_QUALITY (byte), @@ -1926,10 +1926,12 @@ CameraDevice::getResultMetadata(Camera3RequestDescriptor *descriptor, * ANDROID_JPEG_GPS_PROCESSING_METHOD (byte x 32) = 32 bytes * ANDROID_JPEG_GPS_TIMESTAMP (int64) = 8 bytes * ANDROID_JPEG_ORIENTATION (int32_t) = 4 bytes - * Total bytes for JPEG metadata: 73 + * ANDROID_JPEG_THUMBNAIL_QUALITY (byte) = 1 byte + * ANDROID_JPEG_THUMBNAIL_SIZE (int32 x 2) = 8 bytes + * Total bytes for JPEG metadata: 82 */ std::unique_ptr resultMetadata = - std::make_unique(38, 147); + std::make_unique(40, 156); if (!resultMetadata->isValid()) { LOG(HAL, Error) << "Failed to allocate static metadata"; return nullptr; diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index 7e49e9d9..e990ba04 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -44,12 +44,6 @@ int PostProcessorJpeg::configure(const StreamConfiguration &inCfg, streamSize_ = outCfg.size; thumbnailer_.configure(inCfg.size, inCfg.pixelFormat); - StreamConfiguration thCfg = inCfg; - thCfg.size = thumbnailer_.size(); - if (thumbnailEncoder_.configure(thCfg) != 0) { - LOG(JPEG, Error) << "Failed to configure thumbnail encoder"; - return -EINVAL; - } encoder_ = std::make_unique(); @@ -57,14 +51,20 @@ int PostProcessorJpeg::configure(const StreamConfiguration &inCfg, } void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source, + const Size &targetSize, std::vector *thumbnail) { /* Stores the raw scaled-down thumbnail bytes. */ std::vector rawThumbnail; - thumbnailer_.createThumbnail(source, &rawThumbnail); + thumbnailer_.createThumbnail(source, targetSize, &rawThumbnail); - if (!rawThumbnail.empty()) { + StreamConfiguration thCfg; + thCfg.size = targetSize; + thCfg.pixelFormat = thumbnailer_.pixelFormat(); + int ret = thumbnailEncoder_.configure(thCfg); + + if (!rawThumbnail.empty() && !ret) { /* * \todo Avoid value-initialization of all elements of the * vector. @@ -129,6 +129,28 @@ int PostProcessorJpeg::process(const FrameBuffer &source, entry.data.i64, 1); } + ret = requestMetadata.getEntry(ANDROID_JPEG_THUMBNAIL_SIZE, &entry); + if (ret) { + const int32_t *data = entry.data.i32; + Size thumbnailSize = { static_cast(data[0]), + static_cast(data[1]) }; + + if (thumbnailSize != Size(0, 0)) { + std::vector thumbnail; + generateThumbnail(source, thumbnailSize, &thumbnail); + if (!thumbnail.empty()) + exif.setThumbnail(thumbnail, Exif::Compression::JPEG); + } + + resultMetadata->addEntry(ANDROID_JPEG_THUMBNAIL_SIZE, data, 2); + + /* \todo Use this quality as a parameter to the encoder */ + ret = requestMetadata.getEntry(ANDROID_JPEG_THUMBNAIL_QUALITY, &entry); + if (ret) + resultMetadata->addEntry(ANDROID_JPEG_THUMBNAIL_QUALITY, + entry.data.u8, 1); + } + ret = requestMetadata.getEntry(ANDROID_JPEG_GPS_COORDINATES, &entry); if (ret) { exif.setGPSLocation(entry.data.d); @@ -144,11 +166,6 @@ int PostProcessorJpeg::process(const FrameBuffer &source, entry.data.u8, entry.count); } - std::vector thumbnail; - generateThumbnail(source, &thumbnail); - if (!thumbnail.empty()) - exif.setThumbnail(thumbnail, Exif::Compression::JPEG); - if (exif.generate() != 0) LOG(JPEG, Error) << "Failed to generate valid EXIF data"; diff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h index d721d1b9..660b79b4 100644 --- a/src/android/jpeg/post_processor_jpeg.h +++ b/src/android/jpeg/post_processor_jpeg.h @@ -31,6 +31,7 @@ public: private: void generateThumbnail(const libcamera::FrameBuffer &source, + const libcamera::Size &targetSize, std::vector *thumbnail); CameraDevice *const cameraDevice_; diff --git a/src/android/jpeg/thumbnailer.cpp b/src/android/jpeg/thumbnailer.cpp index 5374538a..f709d343 100644 --- a/src/android/jpeg/thumbnailer.cpp +++ b/src/android/jpeg/thumbnailer.cpp @@ -32,30 +32,11 @@ void Thumbnailer::configure(const Size &sourceSize, PixelFormat pixelFormat) return; } - targetSize_ = computeThumbnailSize(); - valid_ = true; } -/* - * The Exif specification recommends the width of the thumbnail to be a - * multiple of 16 (section 4.8.1). Hence, compute the corresponding height - * keeping the aspect ratio same as of the source. - */ -Size Thumbnailer::computeThumbnailSize() const -{ - unsigned int targetHeight; - constexpr unsigned int kTargetWidth = 160; - - targetHeight = kTargetWidth * sourceSize_.height / sourceSize_.width; - - if (targetHeight & 1) - targetHeight++; - - return Size(kTargetWidth, targetHeight); -} - void Thumbnailer::createThumbnail(const FrameBuffer &source, + const Size &targetSize, std::vector *destination) { MappedFrameBuffer frame(&source, PROT_READ); @@ -73,8 +54,8 @@ void Thumbnailer::createThumbnail(const FrameBuffer &source, const unsigned int sw = sourceSize_.width; const unsigned int sh = sourceSize_.height; - const unsigned int tw = targetSize_.width; - const unsigned int th = targetSize_.height; + const unsigned int tw = targetSize.width; + const unsigned int th = targetSize.height; ASSERT(tw % 2 == 0 && th % 2 == 0); diff --git a/src/android/jpeg/thumbnailer.h b/src/android/jpeg/thumbnailer.h index 98f11833..4e9226c3 100644 --- a/src/android/jpeg/thumbnailer.h +++ b/src/android/jpeg/thumbnailer.h @@ -20,15 +20,13 @@ public: void configure(const libcamera::Size &sourceSize, libcamera::PixelFormat pixelFormat); void createThumbnail(const libcamera::FrameBuffer &source, + const libcamera::Size &targetSize, std::vector *dest); - const libcamera::Size &size() const { return targetSize_; } + const libcamera::PixelFormat &pixelFormat() const { return pixelFormat_; } private: - libcamera::Size computeThumbnailSize() const; - libcamera::PixelFormat pixelFormat_; libcamera::Size sourceSize_; - libcamera::Size targetSize_; bool valid_; };