From patchwork Fri Aug 12 09:08:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanlin Chen X-Patchwork-Id: 17091 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 50F70C3272 for ; Fri, 12 Aug 2022 09:09:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 18F0D63337; Fri, 12 Aug 2022 11:09:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1660295340; bh=XusJXHRCLyfE1WSbCkM9zjB4yGYxgU0pXE2ifq9VIcQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=pBvJzsJGmNYSgdKbnQRDoeKprhqU3fQeV875ElZoUEW5qVmbPOTfVuatVL5PCaj0Z sKuzhDtKua7xayvDViZy72gHTBeB+eVhQSHU13+O+7CHiZGbztvnVJS5s/XvufL7sH zwEmjSKKzRsDNCDUi0XoYUx8DA7eEwYs5/cEJzgTVX1Nlg2BofD4vJOO3jB/fQE+GO XDPymn5azHSoii2tTWO6Sb3YM89OmAX73n+hRY9oXebMhMPsarcXQ3TjitKI1udngj rzt4/oKuVp9KD4xFWSmmkS3RG+YCIDDu669fNPrG3bfzeNWQaT5zH5b9qVksd+1znK 1s5EnXTcT7TXw== Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6C99763326 for ; Fri, 12 Aug 2022 11:08:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="lKRrpvwF"; dkim-atps=neutral Received: by mail-pg1-x532.google.com with SMTP id d71so304692pgc.13 for ; Fri, 12 Aug 2022 02:08:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=JfN4WG4RHWr9fhXxcCB7SBeaeQU7DHKgKdxX9Hwe+ZY=; b=lKRrpvwFJg154QPlJbJiPsvKsWMUh6S/cNozUmfQWzoVA1d2Um2yx6UKIKf+nU0pNq Q3iaAdjREeQVb3qn5bptkbvnupq44mxce+hCwsyZC8PI4AX4FIriVaPDZt1oJv7uB6S3 Md8n0IzD6sNdgbmo5lhYoEKX3ZSD+D8gLCRn0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=JfN4WG4RHWr9fhXxcCB7SBeaeQU7DHKgKdxX9Hwe+ZY=; b=MGbMYU+QOOjicGPbuJRCtjhN/4FJaATkbnBTtRVxBglFcT8+dI0ktsE87XkxZl/WUN Mb3ejSJtW0dXv1fy8JPdY25b2opGceavh1iVPpIsL0xrLV3QlHll8uf1JZfKJtfpambT /AJrirFMXSQOZVU3tiC7svSJ+yKPDfXswNgk04vXoNqFoPGucIIAuL6yEBV1qDClUApl BCo+w8FJ0q7xA7B0PgxoOlLEFmYOmWKtes90BAjpgBQyrb2D04vNPm8bUeR39jwZmwNt Bq14o9lT78oxC/1MyGjN0Kjhb9UsZ+xNHLNLzfcjk9dAiSSgH17FnkKITS/NCWpqBBUH qdfw== X-Gm-Message-State: ACgBeo1kZBGgLwhDbNQg+wqZYKr7bxtzOyS3MAugSOSsHdMX3TtR3Xw1 TCUKkVb+34UkmOxsF5gqbtcEYw4emleNtA== X-Google-Smtp-Source: AA6agR57M+Tn61CvEHN8SmE/j4WPxCzlXkE3/yNYrK32ZMN/9yoQ1jclKTFnBxfPJhzfkIDcbicHtA== X-Received: by 2002:a65:5941:0:b0:41d:a203:c043 with SMTP id g1-20020a655941000000b0041da203c043mr2315382pgu.483.1660295336747; Fri, 12 Aug 2022 02:08:56 -0700 (PDT) Received: from localhost ([2401:fa00:1:17:1705:d284:d114:2e24]) by smtp.gmail.com with UTF8SMTPSA id x2-20020a17090300c200b0016ee708350bsm1179295plc.14.2022.08.12.02.08.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 12 Aug 2022 02:08:56 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 12 Aug 2022 17:08:36 +0800 Message-Id: <20220812090838.1784703-6-hanlinchen@chromium.org> X-Mailer: git-send-email 2.37.1.595.g718a3a8f04-goog In-Reply-To: <20220812090838.1784703-1-hanlinchen@chromium.org> References: <20220812090838.1784703-1-hanlinchen@chromium.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/7] android: Add JpegExifMetadata to store tags setting into Exif X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Han-Lin Chen via libcamera-devel From: Hanlin Chen Reply-To: Han-Lin Chen Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With partial result, some metadata needed to adding into Exif may be sent back to framework earlier before Jpeg post-processing. Add a type JpegExifMetadata associated with StreamBuffer to store the values, so Jpeg post-processing doesn't need to reference to current metadata for the them. Signed-off-by: Han-Lin Chen --- src/android/camera_device.cpp | 27 ++++++++++++++++++++++++ src/android/camera_device.h | 2 ++ src/android/camera_request.h | 6 ++++++ src/android/camera_stream.h | 4 ++++ src/android/jpeg/post_processor_jpeg.cpp | 13 ++++++------ 5 files changed, 46 insertions(+), 6 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 0248f146..a14d5de9 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -1221,6 +1221,10 @@ void CameraDevice::requestComplete(Request *request) CameraStream *stream = iter->first; StreamBuffer *buffer = iter->second; + if (stream->isJpegStream()) { + generateJpegExifMetadata(descriptor, buffer); + } + FrameBuffer *src = request->findBuffer(stream->stream()); if (!src) { LOG(HAL, Error) << "Failed to find a source stream buffer"; @@ -1410,6 +1414,29 @@ void CameraDevice::notifyError(uint32_t frameNumber, camera3_stream_t *stream, callbacks_->notify(callbacks_, ¬ify); } +/* + * Set jpeg metadata used to generate EXIF in the JPEG post processing. + */ +void CameraDevice::generateJpegExifMetadata(Camera3RequestDescriptor *request, + StreamBuffer *buffer) const +{ + const ControlList &metadata = request->request_->metadata(); + auto &jpegExifMetadata = buffer->jpegExifMetadata; + jpegExifMetadata.emplace(StreamBuffer::JpegExifMetadata()); + + jpegExifMetadata->sensorExposureTime = 0; + if (metadata.contains(controls::ExposureTime)) { + jpegExifMetadata->sensorExposureTime = metadata.get(controls::ExposureTime) * 1000ULL; + } + + /* + * todo: Android Sensitivity = analog gain X digital gain only on sensor. + * Digital gain on ISP shouldn't be included. Calculate sensitivity + * accordingingly when we can differentiate the source of digital gains. + */ + jpegExifMetadata->sensorSensitivityISO = 100; +} + /* * Produce a set of fixed result metadata. */ diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 9eb7221b..7b279895 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -97,6 +97,8 @@ private: LIBCAMERA_TSA_EXCLUDES(descriptorsMutex_); void sendCaptureResults() LIBCAMERA_TSA_REQUIRES(descriptorsMutex_); void setBufferStatus(StreamBuffer &buffer, StreamBuffer::Status status); + void generateJpegExifMetadata(Camera3RequestDescriptor *request, + StreamBuffer *buffer) const; std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; diff --git a/src/android/camera_request.h b/src/android/camera_request.h index 5aa4eea8..f91de955 100644 --- a/src/android/camera_request.h +++ b/src/android/camera_request.h @@ -42,6 +42,11 @@ public: StreamBuffer(StreamBuffer &&); StreamBuffer &operator=(StreamBuffer &&); + struct JpegExifMetadata { + int64_t sensorExposureTime; + int32_t sensorSensitivityISO; + }; + CameraStream *stream; buffer_handle_t *camera3Buffer; std::unique_ptr frameBuffer; @@ -50,6 +55,7 @@ public: libcamera::FrameBuffer *internalBuffer = nullptr; const libcamera::FrameBuffer *srcBuffer = nullptr; std::unique_ptr dstBuffer; + std::optional jpegExifMetadata; Camera3RequestDescriptor *request; private: diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index c3b2e325..fc50d412 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -125,6 +125,10 @@ public: const libcamera::StreamConfiguration &configuration() const; libcamera::Stream *stream() const; CameraStream *sourceStream() const { return sourceStream_; } + bool isJpegStream() const + { + return camera3Stream_->format == HAL_PIXEL_FORMAT_BLOB; + } int configure(); int process(StreamBuffer *streamBuffer); diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp index b9ba38ce..7e3713f5 100644 --- a/src/android/jpeg/post_processor_jpeg.cpp +++ b/src/android/jpeg/post_processor_jpeg.cpp @@ -104,8 +104,11 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) const FrameBuffer &source = *streamBuffer->srcBuffer; CameraBuffer *destination = streamBuffer->dstBuffer.get(); + const std::optional &jpegExifMetadata = + streamBuffer->jpegExifMetadata; ASSERT(destination->numPlanes() == 1); + ASSERT(jpegExifMetadata.has_value()); const CameraMetadata &requestMetadata = streamBuffer->request->settings_; CameraMetadata *resultMetadata = streamBuffer->request->resultMetadata_.get(); @@ -131,15 +134,13 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) */ exif.setTimestamp(std::time(nullptr), 0ms); - ret = resultMetadata->getEntry(ANDROID_SENSOR_EXPOSURE_TIME, &entry); - exif.setExposureTime(ret ? *entry.data.i64 : 0); + exif.setExposureTime(jpegExifMetadata->sensorExposureTime * 1000); + exif.setISO(jpegExifMetadata->sensorSensitivityISO); + ret = requestMetadata.getEntry(ANDROID_LENS_APERTURE, &entry); if (ret) exif.setAperture(*entry.data.f); - ret = resultMetadata->getEntry(ANDROID_SENSOR_SENSITIVITY, &entry); - exif.setISO(ret ? *entry.data.i32 : 100); - exif.setFlash(Exif::Flash::FlashNotPresent); exif.setWhiteBalance(Exif::WhiteBalance::Auto); @@ -152,6 +153,7 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) *entry.data.i64); } + std::vector thumbnail; ret = requestMetadata.getEntry(ANDROID_JPEG_THUMBNAIL_SIZE, &entry); if (ret) { const int32_t *data = entry.data.i32; @@ -163,7 +165,6 @@ void PostProcessorJpeg::process(StreamBuffer *streamBuffer) resultMetadata->addEntry(ANDROID_JPEG_THUMBNAIL_QUALITY, quality); if (thumbnailSize != Size(0, 0)) { - std::vector thumbnail; generateThumbnail(source, thumbnailSize, quality, &thumbnail); if (!thumbnail.empty()) exif.setThumbnail(thumbnail, Exif::Compression::JPEG);