{"id":16806,"url":"https://patchwork.libcamera.org/api/1.1/patches/16806/?format=json","web_url":"https://patchwork.libcamera.org/patch/16806/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220726182500.425182-6-hanlinchen@chromium.org>","date":"2022-07-26T18:24:59","name":"[libcamera-devel,5/6] android: Add JpegExifMetadata to store tags setting into Exif","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"cc99a5f752c632c760e1a4756a7fa09f30d1145d","submitter":{"id":98,"url":"https://patchwork.libcamera.org/api/1.1/people/98/?format=json","name":"Hanlin Chen","email":"hanlinchen@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/16806/mbox/","series":[{"id":3329,"url":"https://patchwork.libcamera.org/api/1.1/series/3329/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3329","date":"2022-07-26T18:24:54","name":"Implement Android Partial Result Featrue","version":1,"mbox":"https://patchwork.libcamera.org/series/3329/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/16806/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/16806/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C6A54BE173\n\tfor ;\n\tTue, 26 Jul 2022 18:25:38 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8309D63321;\n\tTue, 26 Jul 2022 20:25:38 +0200 (CEST)","from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com\n\t[IPv6:2607:f8b0:4864:20::1029])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id F1B836331C\n\tfor ;\n\tTue, 26 Jul 2022 20:25:36 +0200 (CEST)","by mail-pj1-x1029.google.com with SMTP id\n\t15-20020a17090a098f00b001f305b453feso293405pjo.1\n\tfor ;\n\tTue, 26 Jul 2022 11:25:36 -0700 (PDT)","from localhost ([2401:fa00:1:17:739:6574:4032:5911])\n\tby smtp.gmail.com with UTF8SMTPSA id\n\tw23-20020a627b17000000b0051bc5f4df1csm12039134pfc.154.2022.07.26.11.25.34\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tTue, 26 Jul 2022 11:25:35 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1658859938;\n\tbh=SGvs+XAYBHG0g8CkbMXJzzZoDY1auSkliWcYVb7uQng=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=j0i/nIrooU3OnAgnNMgujBbQpHw7k6KgsZhWtCgJJm654hxQpeHkN24s2o4IMqsLG\n\tfPKHmrYWb42sp0k3Gd9RsHC3L4Y5k36+ebGllek0GES/yCp1x1iZhSFeCKK5dobEdT\n\tcytIMEUkfe2+HpPceoc/5dXHbHZHZND4SScaG5Jc1O62w0rM2D99/F4ZaUnSDs6B81\n\tHJRWKxatSCo4dergl1P/iEMD8C2K+GqQkUj9EqSS6Ksmqq9zGjWepVYqfc1+9eO2Ml\n\tcPpDpKggs+t94FUShVldjYXWLnJIjYpjNMgSbbzA9lG7iiqh9JfhkLQmncyPE19o8m\n\t390ismP+uXxAQ==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=e1btxfjXcpjZDCkIbAtMdRbWtTDz46Fdh+JYTpzSH1s=;\n\tb=E5UHaHix21pWM+b7u6P7TDvLpYguoP9Lz5o8fxZJsuFIe9B42CxLRPrfvWhHfjnoRr\n\tpG8Fx6df/3bdX43IPWrCaLSvBUBdpgxSFsIXhxElbEpSGMWDXWQJM8Y2j4liQxlVG3yQ\n\tzC7tEJq6dPkDYO+9fFCjAfScYQQ3hEKJYYB/k="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=chromium.org\n\theader.i=@chromium.org header.b=\"E5UHaHix\"; \n\tdkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=e1btxfjXcpjZDCkIbAtMdRbWtTDz46Fdh+JYTpzSH1s=;\n\tb=A1IYF/VjNfjE9Q0HK4PXG+EVzG4J3JPDFtM/K3vibSEvHou0EnutfjZjbwPkjzr3Jx\n\tNHp8f30kUD8YPWDXITSxvtQLzi92oAKpu/d6D/0eQ4DOr1CRT/AK0iD8Lv2qkEJS3kAU\n\t0h3PTC4/tPoMkTSTBBSLV0BdCQZ36qwFBpI9+5rPZN9wdHRS8H4J0bCvh5TmnldZhI9E\n\t09xeibmssJHR9UVuyGXlAYrbpYfjGAeRySpdRskiILVY0DP44HufJkce70VJKyvjfu6q\n\tTABE105AoE+IIJVRMptfFhdOgoILxWrWsJWG3gjkrJ9vFhD9KrwVKn5xDzYxJc025B3z\n\tUIhQ==","X-Gm-Message-State":"AJIora8fNeTkLlb8LY3X/cmTaD4WO1VGUHPYK3vVV7QKRa+rBZvUh8KE\n\ttqRH14ICI69rLCCWmpDIhx1hgCMksl/Mfw==","X-Google-Smtp-Source":"AGRyM1s8oEy3zrWIxOO88fzd9Z5XgwhHQPsbC0euHo2sqiTAUrqvsqY418DzmRTSB3DoondqDhcp6A==","X-Received":"by 2002:a17:90b:3e83:b0:1f0:4233:b222 with SMTP id\n\trj3-20020a17090b3e8300b001f04233b222mr470136pjb.218.1658859935385; \n\tTue, 26 Jul 2022 11:25:35 -0700 (PDT)","To":"libcamera-devel@lists.libcamera.org","Date":"Wed, 27 Jul 2022 02:24:59 +0800","Message-Id":"<20220726182500.425182-6-hanlinchen@chromium.org>","X-Mailer":"git-send-email 2.37.1.359.gd136c6c3e2-goog","In-Reply-To":"<20220726182500.425182-1-hanlinchen@chromium.org>","References":"<20220726182500.425182-1-hanlinchen@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH 5/6] android: Add JpegExifMetadata to\n\tstore tags setting into Exif","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Han-Lin Chen via libcamera-devel ","Reply-To":"Han-Lin Chen ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"With partial result, some metadata needed to adding into Exif may be sent\nback to framework earlier before Jpeg post-processing. Add a type\nJpegExifMetadata associated with StreamBuffer to store the values, so Jpeg\npost-processing doesn't need to reference to current metadata for the them.\n\nSigned-off-by: Han-Lin Chen \n---\n src/android/camera_device.cpp | 27 ++++++++++++++++++++++++\n src/android/camera_device.h | 2 ++\n src/android/camera_request.h | 6 ++++++\n src/android/camera_stream.h | 4 ++++\n src/android/jpeg/post_processor_jpeg.cpp | 13 ++++++------\n 5 files changed, 46 insertions(+), 6 deletions(-)","diff":"diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\nindex 0248f146..a14d5de9 100644\n--- a/src/android/camera_device.cpp\n+++ b/src/android/camera_device.cpp\n@@ -1221,6 +1221,10 @@ void CameraDevice::requestComplete(Request *request)\n \t\tCameraStream *stream = iter->first;\n \t\tStreamBuffer *buffer = iter->second;\n \n+\t\tif (stream->isJpegStream()) {\n+\t\t\tgenerateJpegExifMetadata(descriptor, buffer);\n+\t\t}\n+\n \t\tFrameBuffer *src = request->findBuffer(stream->stream());\n \t\tif (!src) {\n \t\t\tLOG(HAL, Error) << \"Failed to find a source stream buffer\";\n@@ -1410,6 +1414,29 @@ void CameraDevice::notifyError(uint32_t frameNumber, camera3_stream_t *stream,\n \tcallbacks_->notify(callbacks_, ¬ify);\n }\n \n+/*\n+ * Set jpeg metadata used to generate EXIF in the JPEG post processing.\n+ */\n+void CameraDevice::generateJpegExifMetadata(Camera3RequestDescriptor *request,\n+\t\t\t\t\t StreamBuffer *buffer) const\n+{\n+\tconst ControlList &metadata = request->request_->metadata();\n+\tauto &jpegExifMetadata = buffer->jpegExifMetadata;\n+\tjpegExifMetadata.emplace(StreamBuffer::JpegExifMetadata());\n+\n+\tjpegExifMetadata->sensorExposureTime = 0;\n+\tif (metadata.contains(controls::ExposureTime)) {\n+\t\tjpegExifMetadata->sensorExposureTime = metadata.get(controls::ExposureTime) * 1000ULL;\n+\t}\n+\n+\t/*\n+\t * todo: Android Sensitivity = analog gain X digital gain only on sensor.\n+\t * Digital gain on ISP shouldn't be included. Calculate sensitivity\n+\t * accordingingly when we can differentiate the source of digital gains.\n+\t */\n+\tjpegExifMetadata->sensorSensitivityISO = 100;\n+}\n+\n /*\n * Produce a set of fixed result metadata.\n */\ndiff --git a/src/android/camera_device.h b/src/android/camera_device.h\nindex 9eb7221b..7b279895 100644\n--- a/src/android/camera_device.h\n+++ b/src/android/camera_device.h\n@@ -97,6 +97,8 @@ private:\n \t\tLIBCAMERA_TSA_EXCLUDES(descriptorsMutex_);\n \tvoid sendCaptureResults() LIBCAMERA_TSA_REQUIRES(descriptorsMutex_);\n \tvoid setBufferStatus(StreamBuffer &buffer, StreamBuffer::Status status);\n+\tvoid generateJpegExifMetadata(Camera3RequestDescriptor *request,\n+\t\t\t\t StreamBuffer *buffer) const;\n \tstd::unique_ptr getResultMetadata(\n \t\tconst Camera3RequestDescriptor &descriptor) const;\n \ndiff --git a/src/android/camera_request.h b/src/android/camera_request.h\nindex 5aa4eea8..f91de955 100644\n--- a/src/android/camera_request.h\n+++ b/src/android/camera_request.h\n@@ -42,6 +42,11 @@ public:\n \tStreamBuffer(StreamBuffer &&);\n \tStreamBuffer &operator=(StreamBuffer &&);\n \n+\tstruct JpegExifMetadata {\n+\t\tint64_t sensorExposureTime;\n+\t\tint32_t sensorSensitivityISO;\n+\t};\n+\n \tCameraStream *stream;\n \tbuffer_handle_t *camera3Buffer;\n \tstd::unique_ptr frameBuffer;\n@@ -50,6 +55,7 @@ public:\n \tlibcamera::FrameBuffer *internalBuffer = nullptr;\n \tconst libcamera::FrameBuffer *srcBuffer = nullptr;\n \tstd::unique_ptr dstBuffer;\n+\tstd::optional jpegExifMetadata;\n \tCamera3RequestDescriptor *request;\n \n private:\ndiff --git a/src/android/camera_stream.h b/src/android/camera_stream.h\nindex c3b2e325..fc50d412 100644\n--- a/src/android/camera_stream.h\n+++ b/src/android/camera_stream.h\n@@ -125,6 +125,10 @@ public:\n \tconst libcamera::StreamConfiguration &configuration() const;\n \tlibcamera::Stream *stream() const;\n \tCameraStream *sourceStream() const { return sourceStream_; }\n+\tbool isJpegStream() const\n+\t{\n+\t\treturn camera3Stream_->format == HAL_PIXEL_FORMAT_BLOB;\n+\t}\n \n \tint configure();\n \tint process(StreamBuffer *streamBuffer);\ndiff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\nindex b9ba38ce..91fd329c 100644\n--- a/src/android/jpeg/post_processor_jpeg.cpp\n+++ b/src/android/jpeg/post_processor_jpeg.cpp\n@@ -104,8 +104,11 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer)\n \n \tconst FrameBuffer &source = *streamBuffer->srcBuffer;\n \tCameraBuffer *destination = streamBuffer->dstBuffer.get();\n+\tconst std::optional &jpegExifMetadata =\n+\t\tstreamBuffer->jpegExifMetadata;\n \n \tASSERT(destination->numPlanes() == 1);\n+\tASSERT(jpegExifMetadata.has_value());\n \n \tconst CameraMetadata &requestMetadata = streamBuffer->request->settings_;\n \tCameraMetadata *resultMetadata = streamBuffer->request->resultMetadata_.get();\n@@ -131,15 +134,13 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer)\n \t */\n \texif.setTimestamp(std::time(nullptr), 0ms);\n \n-\tret = resultMetadata->getEntry(ANDROID_SENSOR_EXPOSURE_TIME, &entry);\n-\texif.setExposureTime(ret ? *entry.data.i64 : 0);\n+\texif.setExposureTime(jpegExifMetadata->sensorExposureTime);\n+\texif.setISO(jpegExifMetadata->sensorSensitivityISO);\n+\n \tret = requestMetadata.getEntry(ANDROID_LENS_APERTURE, &entry);\n \tif (ret)\n \t\texif.setAperture(*entry.data.f);\n \n-\tret = resultMetadata->getEntry(ANDROID_SENSOR_SENSITIVITY, &entry);\n-\texif.setISO(ret ? *entry.data.i32 : 100);\n-\n \texif.setFlash(Exif::Flash::FlashNotPresent);\n \texif.setWhiteBalance(Exif::WhiteBalance::Auto);\n \n@@ -152,6 +153,7 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer)\n \t\t\t\t\t *entry.data.i64);\n \t}\n \n+\tstd::vector thumbnail;\n \tret = requestMetadata.getEntry(ANDROID_JPEG_THUMBNAIL_SIZE, &entry);\n \tif (ret) {\n \t\tconst int32_t *data = entry.data.i32;\n@@ -163,7 +165,6 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer)\n \t\tresultMetadata->addEntry(ANDROID_JPEG_THUMBNAIL_QUALITY, quality);\n \n \t\tif (thumbnailSize != Size(0, 0)) {\n-\t\t\tstd::vector thumbnail;\n \t\t\tgenerateThumbnail(source, thumbnailSize, quality, &thumbnail);\n \t\t\tif (!thumbnail.empty())\n \t\t\t\texif.setThumbnail(thumbnail, Exif::Compression::JPEG);\n","prefixes":["libcamera-devel","5/6"]}