From patchwork Wed Nov 27 09:25:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 22119 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3853EC3213 for ; Wed, 27 Nov 2024 09:27:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A11EC660B9; Wed, 27 Nov 2024 10:27:00 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="cnuCQ81b"; dkim-atps=neutral Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DD997660C2 for ; Wed, 27 Nov 2024 10:26:51 +0100 (CET) Received: by mail-pg1-x52b.google.com with SMTP id 41be03b00d2f7-7e6cbf6cd1dso4453571a12.3 for ; Wed, 27 Nov 2024 01:26:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1732699610; x=1733304410; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AoZO/hfgntIi/tXlmTlmoFRfHC/LykvzY6f4ocf5hXU=; b=cnuCQ81bvlVoz5eoK2Q42jr9WzIELJUokfbYRkXOqm04Ezz7ucLqfIaZG3OrpJTRXB 5pcNBLBf1oqbGRXe6dE98In4EkeebumMq260BqyFZmogajr9y2qs1sEMx3T5j73rQwcb lraqpFsMAwP0vUBQJEiRHfB48jI2YXlf2Pu1M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732699610; x=1733304410; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AoZO/hfgntIi/tXlmTlmoFRfHC/LykvzY6f4ocf5hXU=; b=gs2nJbvk/HSyp41uXnQa6bblZm1Q0WwyZ1ozBR9qOGgGczewEUHFj5NCYo4L+lQI5x WEUyR2kIqqzekBvPgdl8pCjXs365AlI/wLQ5c5itCnfureOS081/LfTzxLemZk6mOQkQ 3BVbeD750t3p+xOznzM9OhDrhLRSKpipWfXbaq18kLADCMxdZGj1qoKbQpkM5D5iiHVG Mbx/OaK31PHcm1xS0Tn5Vm8jdeTELd3Q/3hp5mHWb9q4XrezqJl4AiHx5jYFnEff7WlI OqPgruvyC5wvV0oisFKuPrvR6g11PCwysjP2RQr4CcZs10nNOvger8Gwgp5m6JxgRlbK zz6A== X-Gm-Message-State: AOJu0YzixCRTB5uxvpgW0DIPvIHDHsPsAxwelPFeQhAz+9M1gw6Y7+TA /PmN99UTKRcX9FoTlkU5hU3Nv0SHfNUdsciFTvkvioGQ4EAuyqR/KFgXi3OfpejSK5gGvt2dudY = X-Gm-Gg: ASbGnctbefb6Y4XCeV8T1C6Xq+FQ/fkUickn5rhFFRiZl7AE3Kmst+cv7QqsMwOr5DS rzJmRynEnRgFoDizs2jMmmLwVsc+fn5M1ArniBRDeHeEbr8TWGya2w1Pc8Wzvm1PZuU7643sUtr 9znYmDhhqMAUzjpPNI9vySlz/5tNJt/0KZ7i8zMmY/wQqKf3N/RtCjv3E6SGIomXS9QFd4ID9Mw 4lLLlcsW971YabYAKZpn+QaMDYuqj5Sm1FaiOHlBXdyKqTJ3Dyty2Q2iBnfXuH07BgxTsAwkzzj Ge/WXBGs/Rmxg1VGIjyKzgsA+YFijYoGOugniKb+jHGOnvEjZtkFQA== X-Google-Smtp-Source: AGHT+IHv6ezMs3I6FehAsRDLUMqTqh8pWgFh4NDxShAem9+bDFVkJY8laV14HZD4vuUotwKdgYYWJA== X-Received: by 2002:a05:6a20:2583:b0:1db:dae1:22e5 with SMTP id adf61e73a8af0-1e0e0b2c74emr5364897637.20.1732699609722; Wed, 27 Nov 2024 01:26:49 -0800 (PST) Received: from chenghaoyang-low.c.googlers.com.com (27.247.221.35.bc.googleusercontent.com. [35.221.247.27]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fbcbfc41f9sm8693027a12.8.2024.11.27.01.26.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Nov 2024 01:26:49 -0800 (PST) From: Harvey Yang To: libcamera-devel@lists.libcamera.org Cc: Han-Lin Chen , Harvey Yang Subject: [PATCH v2 8/9] android: Add JpegExifMetadata to store tags setting into Exif Date: Wed, 27 Nov 2024 09:25:58 +0000 Message-ID: <20241127092632.3145984-9-chenghaoyang@chromium.org> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog In-Reply-To: <20241127092632.3145984-1-chenghaoyang@chromium.org> References: <20241127092632.3145984-1-chenghaoyang@chromium.org> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Han-Lin Chen With partial result, some metadata, which needs to be added into Exif, may be sent back to framework earlier before Jpeg post-processing. Add a type JpegExifMetadata associated with StreamBuffer to store the values, so Jpeg post-processing doesn't need to reference to current metadata. Signed-off-by: Han-Lin Chen Co-developed-by: Harvey Yang Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 26 ++++++++++++++++++++++++ src/android/camera_device.h | 2 ++ src/android/camera_request.h | 6 ++++++ src/android/camera_stream.h | 4 ++++ src/android/jpeg/post_processor_jpeg.cpp | 12 ++++++----- 5 files changed, 45 insertions(+), 5 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 9fd851bc8..e085e18b2 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1250,6 +1250,10 @@ void CameraDevice::requestComplete(Request *request) CameraStream *stream = iter->first; StreamBuffer *buffer = iter->second; + if (stream->isJpegStream()) { + generateJpegExifMetadata(descriptor, buffer); + } + FrameBuffer *src = request->findBuffer(stream->stream()); if (!src) { LOG(HAL, Error) << "Failed to find a source stream buffer"; @@ -1443,6 +1447,28 @@ void CameraDevice::notifyError(uint32_t frameNumber, camera3_stream_t *stream, callbacks_->notify(callbacks_, ¬ify); } +/* + * Set jpeg metadata used to generate EXIF in the JPEG post processing. + */ +void CameraDevice::generateJpegExifMetadata(Camera3RequestDescriptor *request, + StreamBuffer *buffer) const +{ + const ControlList &metadata = request->request_->metadata(); + auto &jpegExifMetadata = buffer->jpegExifMetadata; + jpegExifMetadata.emplace(StreamBuffer::JpegExifMetadata()); + + const int64_t exposureTime = metadata.get(controls::ExposureTime).value_or(0); + jpegExifMetadata->sensorExposureTime = exposureTime; + + /* + * todo: Android Sensitivity should only include analog gain X digital + * gain from sensor. Digital gain on ISP shouldn't be included. + * Calculate sensitivity accordingly when we can differentiate + * the source of digital gains. + */ + jpegExifMetadata->sensorSensitivityISO = 100; +} + /* * Produce a set of fixed result metadata. */ diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 815a695d1..3c46ff918 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -102,6 +102,8 @@ private: void sendCaptureResult(Camera3RequestDescriptor *request) const; void setBufferStatus(StreamBuffer &buffer, StreamBuffer::Status status); + void generateJpegExifMetadata(Camera3RequestDescriptor *request, + StreamBuffer *buffer) const; std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; diff --git a/src/android/camera_request.h b/src/android/camera_request.h index bd75d4595..bd87b36fd 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -44,6 +44,11 @@ public: StreamBuffer(StreamBuffer &&); StreamBuffer &operator=(StreamBuffer &&); + struct JpegExifMetadata { + int64_t sensorExposureTime; + int32_t sensorSensitivityISO; + }; + CameraStream *stream; buffer_handle_t *camera3Buffer; std::unique_ptr frameBuffer; @@ -51,6 +56,7 @@ public: Status status = Status::Success; const libcamera::FrameBuffer *srcBuffer = nullptr; std::unique_ptr dstBuffer; + std::optional jpegExifMetadata; Camera3RequestDescriptor *request; private: diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 30f64f690..47cd7ab85 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -125,6 +125,10 @@ public: const libcamera::StreamConfiguration &configuration() const; libcamera::Stream *stream() const; CameraStream *sourceStream() const { return sourceStream_; } + bool isJpegStream() const + { + return camera3Stream_->format == HAL_PIXEL_FORMAT_BLOB; + } int configure(); int process(StreamBuffer *streamBuffer); diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index f5a90785d..48782b574 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -112,8 +112,11 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) const FrameBuffer &source = *streamBuffer->srcBuffer; CameraBuffer *destination = streamBuffer->dstBuffer.get(); + const std::optional &jpegExifMetadata = + streamBuffer->jpegExifMetadata; ASSERT(destination->numPlanes() == 1); + ASSERT(jpegExifMetadata.has_value()); const CameraMetadata &requestMetadata = streamBuffer->request->settings_; CameraMetadata *resultMetadata = streamBuffer->request->resultMetadata_.get(); @@ -139,15 +142,14 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) */ exif.setTimestamp(std::time(nullptr), 0ms); - ret = resultMetadata->getEntry(ANDROID_SENSOR_EXPOSURE_TIME, &entry); - exif.setExposureTime(ret ? *entry.data.i64 : 0); + /* Exif requires nsec for exposure time */ + exif.setExposureTime(jpegExifMetadata->sensorExposureTime * 1000); + exif.setISO(jpegExifMetadata->sensorSensitivityISO); + ret = requestMetadata.getEntry(ANDROID_LENS_APERTURE, &entry); if (ret) exif.setAperture(*entry.data.f); - ret = resultMetadata->getEntry(ANDROID_SENSOR_SENSITIVITY, &entry); - exif.setISO(ret ? *entry.data.i32 : 100); - exif.setFlash(Exif::Flash::FlashNotPresent); exif.setWhiteBalance(Exif::WhiteBalance::Auto);