From patchwork Tue Jul 26 18:24:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanlin Chen X-Patchwork-Id: 16806 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C6A54BE173 for ; Tue, 26 Jul 2022 18:25:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8309D63321; Tue, 26 Jul 2022 20:25:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658859938; bh=SGvs+XAYBHG0g8CkbMXJzzZoDY1auSkliWcYVb7uQng=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=j0i/nIrooU3OnAgnNMgujBbQpHw7k6KgsZhWtCgJJm654hxQpeHkN24s2o4IMqsLG fPKHmrYWb42sp0k3Gd9RsHC3L4Y5k36+ebGllek0GES/yCp1x1iZhSFeCKK5dobEdT cytIMEUkfe2+HpPceoc/5dXHbHZHZND4SScaG5Jc1O62w0rM2D99/F4ZaUnSDs6B81 HJRWKxatSCo4dergl1P/iEMD8C2K+GqQkUj9EqSS6Ksmqq9zGjWepVYqfc1+9eO2Ml cPpDpKggs+t94FUShVldjYXWLnJIjYpjNMgSbbzA9lG7iiqh9JfhkLQmncyPE19o8m 390ismP+uXxAQ== Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F1B836331C for ; Tue, 26 Jul 2022 20:25:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="E5UHaHix"; dkim-atps=neutral Received: by mail-pj1-x1029.google.com with SMTP id 15-20020a17090a098f00b001f305b453feso293405pjo.1 for ; Tue, 26 Jul 2022 11:25:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e1btxfjXcpjZDCkIbAtMdRbWtTDz46Fdh+JYTpzSH1s=; b=E5UHaHix21pWM+b7u6P7TDvLpYguoP9Lz5o8fxZJsuFIe9B42CxLRPrfvWhHfjnoRr pG8Fx6df/3bdX43IPWrCaLSvBUBdpgxSFsIXhxElbEpSGMWDXWQJM8Y2j4liQxlVG3yQ zC7tEJq6dPkDYO+9fFCjAfScYQQ3hEKJYYB/k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e1btxfjXcpjZDCkIbAtMdRbWtTDz46Fdh+JYTpzSH1s=; b=A1IYF/VjNfjE9Q0HK4PXG+EVzG4J3JPDFtM/K3vibSEvHou0EnutfjZjbwPkjzr3Jx NHp8f30kUD8YPWDXITSxvtQLzi92oAKpu/d6D/0eQ4DOr1CRT/AK0iD8Lv2qkEJS3kAU 0h3PTC4/tPoMkTSTBBSLV0BdCQZ36qwFBpI9+5rPZN9wdHRS8H4J0bCvh5TmnldZhI9E 09xeibmssJHR9UVuyGXlAYrbpYfjGAeRySpdRskiILVY0DP44HufJkce70VJKyvjfu6q TABE105AoE+IIJVRMptfFhdOgoILxWrWsJWG3gjkrJ9vFhD9KrwVKn5xDzYxJc025B3z UIhQ== X-Gm-Message-State: AJIora8fNeTkLlb8LY3X/cmTaD4WO1VGUHPYK3vVV7QKRa+rBZvUh8KE tqRH14ICI69rLCCWmpDIhx1hgCMksl/Mfw== X-Google-Smtp-Source: AGRyM1s8oEy3zrWIxOO88fzd9Z5XgwhHQPsbC0euHo2sqiTAUrqvsqY418DzmRTSB3DoondqDhcp6A== X-Received: by 2002:a17:90b:3e83:b0:1f0:4233:b222 with SMTP id rj3-20020a17090b3e8300b001f04233b222mr470136pjb.218.1658859935385; Tue, 26 Jul 2022 11:25:35 -0700 (PDT) Received: from localhost ([2401:fa00:1:17:739:6574:4032:5911]) by smtp.gmail.com with UTF8SMTPSA id w23-20020a627b17000000b0051bc5f4df1csm12039134pfc.154.2022.07.26.11.25.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 26 Jul 2022 11:25:35 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 27 Jul 2022 02:24:59 +0800 Message-Id: <20220726182500.425182-6-hanlinchen@chromium.org> X-Mailer: git-send-email 2.37.1.359.gd136c6c3e2-goog In-Reply-To: <20220726182500.425182-1-hanlinchen@chromium.org> References: <20220726182500.425182-1-hanlinchen@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/6] android: Add JpegExifMetadata to store tags setting into Exif X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Han-Lin Chen via libcamera-devel From: Hanlin Chen Reply-To: Han-Lin Chen Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With partial result, some metadata needed to adding into Exif may be sent back to framework earlier before Jpeg post-processing. Add a type JpegExifMetadata associated with StreamBuffer to store the values, so Jpeg post-processing doesn't need to reference to current metadata for the them. Signed-off-by: Han-Lin Chen --- src/android/camera_device.cpp | 27 ++++++++++++++++++++++++ src/android/camera_device.h | 2 ++ src/android/camera_request.h | 6 ++++++ src/android/camera_stream.h | 4 ++++ src/android/jpeg/post_processor_jpeg.cpp | 13 ++++++------ 5 files changed, 46 insertions(+), 6 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 0248f146..a14d5de9 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1221,6 +1221,10 @@ void CameraDevice::requestComplete(Request *request) CameraStream *stream = iter->first; StreamBuffer *buffer = iter->second; + if (stream->isJpegStream()) { + generateJpegExifMetadata(descriptor, buffer); + } + FrameBuffer *src = request->findBuffer(stream->stream()); if (!src) { LOG(HAL, Error) << "Failed to find a source stream buffer"; @@ -1410,6 +1414,29 @@ void CameraDevice::notifyError(uint32_t frameNumber, camera3_stream_t *stream, callbacks_->notify(callbacks_, ¬ify); } +/* + * Set jpeg metadata used to generate EXIF in the JPEG post processing. + */ +void CameraDevice::generateJpegExifMetadata(Camera3RequestDescriptor *request, + StreamBuffer *buffer) const +{ + const ControlList &metadata = request->request_->metadata(); + auto &jpegExifMetadata = buffer->jpegExifMetadata; + jpegExifMetadata.emplace(StreamBuffer::JpegExifMetadata()); + + jpegExifMetadata->sensorExposureTime = 0; + if (metadata.contains(controls::ExposureTime)) { + jpegExifMetadata->sensorExposureTime = metadata.get(controls::ExposureTime) * 1000ULL; + } + + /* + * todo: Android Sensitivity = analog gain X digital gain only on sensor. + * Digital gain on ISP shouldn't be included. Calculate sensitivity + * accordingingly when we can differentiate the source of digital gains. + */ + jpegExifMetadata->sensorSensitivityISO = 100; +} + /* * Produce a set of fixed result metadata. */ diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 9eb7221b..7b279895 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -97,6 +97,8 @@ private: LIBCAMERA_TSA_EXCLUDES(descriptorsMutex_); void sendCaptureResults() LIBCAMERA_TSA_REQUIRES(descriptorsMutex_); void setBufferStatus(StreamBuffer &buffer, StreamBuffer::Status status); + void generateJpegExifMetadata(Camera3RequestDescriptor *request, + StreamBuffer *buffer) const; std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 5aa4eea8..f91de955 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -42,6 +42,11 @@ public: StreamBuffer(StreamBuffer &&); StreamBuffer &operator=(StreamBuffer &&); + struct JpegExifMetadata { + int64_t sensorExposureTime; + int32_t sensorSensitivityISO; + }; + CameraStream *stream; buffer_handle_t *camera3Buffer; std::unique_ptr frameBuffer; @@ -50,6 +55,7 @@ public: libcamera::FrameBuffer *internalBuffer = nullptr; const libcamera::FrameBuffer *srcBuffer = nullptr; std::unique_ptr dstBuffer; + std::optional jpegExifMetadata; Camera3RequestDescriptor *request; private: diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index c3b2e325..fc50d412 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -125,6 +125,10 @@ public: const libcamera::StreamConfiguration &configuration() const; libcamera::Stream *stream() const; CameraStream *sourceStream() const { return sourceStream_; } + bool isJpegStream() const + { + return camera3Stream_->format == HAL_PIXEL_FORMAT_BLOB; + } int configure(); int process(StreamBuffer *streamBuffer); diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index b9ba38ce..91fd329c 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -104,8 +104,11 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) const FrameBuffer &source = *streamBuffer->srcBuffer; CameraBuffer *destination = streamBuffer->dstBuffer.get(); + const std::optional &jpegExifMetadata = + streamBuffer->jpegExifMetadata; ASSERT(destination->numPlanes() == 1); + ASSERT(jpegExifMetadata.has_value()); const CameraMetadata &requestMetadata = streamBuffer->request->settings_; CameraMetadata *resultMetadata = streamBuffer->request->resultMetadata_.get(); @@ -131,15 +134,13 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) */ exif.setTimestamp(std::time(nullptr), 0ms); - ret = resultMetadata->getEntry(ANDROID_SENSOR_EXPOSURE_TIME, &entry); - exif.setExposureTime(ret ? *entry.data.i64 : 0); + exif.setExposureTime(jpegExifMetadata->sensorExposureTime); + exif.setISO(jpegExifMetadata->sensorSensitivityISO); + ret = requestMetadata.getEntry(ANDROID_LENS_APERTURE, &entry); if (ret) exif.setAperture(*entry.data.f); - ret = resultMetadata->getEntry(ANDROID_SENSOR_SENSITIVITY, &entry); - exif.setISO(ret ? *entry.data.i32 : 100); - exif.setFlash(Exif::Flash::FlashNotPresent); exif.setWhiteBalance(Exif::WhiteBalance::Auto); @@ -152,6 +153,7 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) *entry.data.i64); } + std::vector thumbnail; ret = requestMetadata.getEntry(ANDROID_JPEG_THUMBNAIL_SIZE, &entry); if (ret) { const int32_t *data = entry.data.i32; @@ -163,7 +165,6 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) resultMetadata->addEntry(ANDROID_JPEG_THUMBNAIL_QUALITY, quality); if (thumbnailSize != Size(0, 0)) { - std::vector thumbnail; generateThumbnail(source, thumbnailSize, quality, &thumbnail); if (!thumbnail.empty()) exif.setThumbnail(thumbnail, Exif::Compression::JPEG);