{"id":11001,"url":"https://patchwork.libcamera.org/api/patches/11001/?format=json","web_url":"https://patchwork.libcamera.org/patch/11001/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210126102825.147026-6-paul.elder@ideasonboard.com>","date":"2021-01-26T10:28:22","name":"[libcamera-devel,v5,5/8] android: Set result metadata and EXIF fields based on request metadata","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"3df2ba2716004f78049b2fbc1ff4e6433dd62618","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/?format=json","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/11001/mbox/","series":[{"id":1608,"url":"https://patchwork.libcamera.org/api/series/1608/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1608","date":"2021-01-26T10:28:17","name":"Fill in android result metadata and EXIF tags","version":5,"mbox":"https://patchwork.libcamera.org/series/1608/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/11001/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/11001/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5E91ABD808\n\tfor ;\n\tTue, 26 Jan 2021 10:28:49 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2940B68302;\n\tTue, 26 Jan 2021 11:28:49 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 554C4682FA\n\tfor ;\n\tTue, 26 Jan 2021 11:28:47 +0100 (CET)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 979C88B6;\n\tTue, 26 Jan 2021 11:28:45 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"UKE+elt9\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1611656927;\n\tbh=lg7ex7mwAtuevO/KM+lvXfeWk1ZxvXiGUaR4JeTbcS0=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=UKE+elt9gGK1NU9yMqLme3v05iLhrXVoYdA96fo6srWS9ibNNFMuA+EbnPWLCOIAx\n\timGvqTJh9ZT4bkzTspLVpVeOQzgZwLCu4+I4pSZr75CCRNNkzMRVRMGvPS3HnvRQ8Y\n\t76BLe7A2G8Bcf3fKS2GEbjK7dvn5qy1hZ+rxeJmE=","From":"Paul Elder ","To":"libcamera-devel@lists.libcamera.org","Date":"Tue, 26 Jan 2021 19:28:22 +0900","Message-Id":"<20210126102825.147026-6-paul.elder@ideasonboard.com>","X-Mailer":"git-send-email 2.27.0","In-Reply-To":"<20210126102825.147026-1-paul.elder@ideasonboard.com>","References":"<20210126102825.147026-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v5 5/8] android: Set result metadata and\n\tEXIF fields based on request metadata","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Set the following android result metadata:\n- ANDROID_LENS_FOCAL_LENGTH\n- ANDROID_LENS_APERTURE\n- ANDROID_JPEG_GPS_TIMESTAMP\n- ANDROID_JPEG_GPS_COORDINATES\n- ANDROID_JPEG_GPS_PROCESSING_METHOD\n- ANDROID_JPEG_SIZE\n- ANDROID_JPEG_QUALITY\n- ANDROID_JPEG_ORIENTATION\n\nAnd the following EXIF fields:\n- GPSDatestamp\n- GPSTimestamp\n- GPSLocation\n - GPSLatitudeRef\n - GPSLatitude\n - GPSLongitudeRef\n - GPSLongitude\n - GPSAltitudeRef\n - GPSAltitude\n- GPSProcessingMethod\n- FocalLength\n- ExposureTime\n- FNumber\n- ISO\n- Flash\n- WhiteBalance\n- SubsecTime\n- SubsecTimeOriginal\n- SubsecTimeDigitized\n\nBased on android request metadata.\n\nThis allows the following CTS tests to pass:\n- android.hardware.camera2.cts.StillCaptureTest#testFocalLengths\n- android.hardware.camera2.cts.StillCaptureTest#testJpegExif\n\nSigned-off-by: Paul Elder \nReviewed-by: Laurent Pinchart \n\n---\nChanges in v5:\n- fix jpegQuality type\n- drop todo about handling null request settings that's fixed in a later\n patch\n- add todo about getting exif information from\n libcamera::Request::metadata\n- fix JPEG metadata size allocation comment, add total\n- add metadata entries to availableResultKeys and add the necessary\n bytes to the static metadata pack size\n\nChanges in v4:\n- remove unused variable\n- move android JPEG thumbnail tags allocation to the thumbnail patch\n- move setting the timestamp with subsec to the patch that adds set\n subsec to timestamp (\"android: jpeg: exif: Add functions for\n setting various values\")\n- group setting GPS-related tags\n\nChanges in v3:\n- fix metadata entries and byte count\n- fix gps processing method string truncation\n- move thumbnail handling to a different patch\n\nChanges in v2:\n- break out processControls and thumbnailer configuration into\n different\n patches\n- fix android metadata entry number and size\n- other small changes\n---\n src/android/camera_device.cpp | 26 ++++++++--\n src/android/camera_stream.cpp | 7 ++-\n src/android/camera_stream.h | 4 +-\n src/android/jpeg/post_processor_jpeg.cpp | 63 +++++++++++++++++++-----\n src/android/jpeg/post_processor_jpeg.h | 3 +-\n src/android/post_processor.h | 3 +-\n 6 files changed, 86 insertions(+), 20 deletions(-)","diff":"diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\nindex 7929877c..096c2463 100644\n--- a/src/android/camera_device.cpp\n+++ b/src/android/camera_device.cpp\n@@ -708,7 +708,7 @@ std::tuple CameraDevice::calculateStaticMetadataSize()\n \t * Currently: 53 entries, 782 bytes of static metadata\n \t */\n \tuint32_t numEntries = 53;\n-\tuint32_t byteSize = 782;\n+\tuint32_t byteSize = 802;\n \n \t/*\n \t * Calculate space occupation in bytes for dynamically built metadata\n@@ -1253,6 +1253,8 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()\n \t\tANDROID_CONTROL_VIDEO_STABILIZATION_MODE,\n \t\tANDROID_FLASH_MODE,\n \t\tANDROID_FLASH_STATE,\n+\t\tANDROID_LENS_APERTURE,\n+\t\tANDROID_LENS_FOCAL_LENGTH,\n \t\tANDROID_LENS_STATE,\n \t\tANDROID_LENS_OPTICAL_STABILIZATION_MODE,\n \t\tANDROID_SENSOR_TIMESTAMP,\n@@ -1264,6 +1266,9 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()\n \t\tANDROID_NOISE_REDUCTION_MODE,\n \t\tANDROID_REQUEST_PIPELINE_DEPTH,\n \t\tANDROID_SCALER_CROP_REGION,\n+\t\tANDROID_JPEG_GPS_COORDINATES,\n+\t\tANDROID_JPEG_GPS_PROCESSING_METHOD,\n+\t\tANDROID_JPEG_GPS_TIMESTAMP,\n \t\tANDROID_JPEG_SIZE,\n \t\tANDROID_JPEG_QUALITY,\n \t\tANDROID_JPEG_ORIENTATION,\n@@ -1816,6 +1821,7 @@ void CameraDevice::requestComplete(Request *request)\n \t\t}\n \n \t\tint ret = cameraStream->process(*buffer, &mapped,\n+\t\t\t\t\t\tdescriptor->settings_,\n \t\t\t\t\t\tresultMetadata.get());\n \t\tif (ret) {\n \t\t\tstatus = CAMERA3_BUFFER_STATUS_ERROR;\n@@ -1912,14 +1918,18 @@ CameraDevice::getResultMetadata(Camera3RequestDescriptor *descriptor,\n \n \t/*\n \t * \\todo Keep this in sync with the actual number of entries.\n-\t * Currently: 33 entries, 75 bytes\n+\t * Currently: 38 entries, 147 bytes\n \t *\n \t * Reserve more space for the JPEG metadata set by the post-processor.\n \t * Currently: ANDROID_JPEG_SIZE (int32_t), ANDROID_JPEG_QUALITY (byte),\n-\t * ANDROID_JPEG_ORIENTATION (int32_t) = 3 entries, 9 bytes.\n+\t * ANDROID_JPEG_GPS_COORDINATES (double x 3) = 24 bytes\n+\t * ANDROID_JPEG_GPS_PROCESSING_METHOD (byte x 32) = 32 bytes\n+\t * ANDROID_JPEG_GPS_TIMESTAMP (int64) = 8 bytes\n+\t * ANDROID_JPEG_ORIENTATION (int32_t) = 4 bytes\n+\t * Total bytes for JPEG metadata: 73\n \t */\n \tstd::unique_ptr resultMetadata =\n-\t\tstd::make_unique(33, 75);\n+\t\tstd::make_unique(38, 147);\n \tif (!resultMetadata->isValid()) {\n \t\tLOG(HAL, Error) << \"Failed to allocate static metadata\";\n \t\treturn nullptr;\n@@ -1984,6 +1994,14 @@ CameraDevice::getResultMetadata(Camera3RequestDescriptor *descriptor,\n \tvalue = ANDROID_FLASH_STATE_UNAVAILABLE;\n \tresultMetadata->addEntry(ANDROID_FLASH_STATE, &value, 1);\n \n+\tcamera_metadata_ro_entry_t entry;\n+\tint ret = descriptor->settings_.getEntry(ANDROID_LENS_APERTURE, &entry);\n+\tif (ret)\n+\t\tresultMetadata->addEntry(ANDROID_LENS_APERTURE, entry.data.f, 1);\n+\n+\tfloat focal_length = 1.0;\n+\tresultMetadata->addEntry(ANDROID_LENS_FOCAL_LENGTH, &focal_length, 1);\n+\n \tvalue = ANDROID_LENS_STATE_STATIONARY;\n \tresultMetadata->addEntry(ANDROID_LENS_STATE, &value, 1);\n \ndiff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp\nindex dba351a4..4c8020e5 100644\n--- a/src/android/camera_stream.cpp\n+++ b/src/android/camera_stream.cpp\n@@ -96,12 +96,15 @@ int CameraStream::configure()\n }\n \n int CameraStream::process(const libcamera::FrameBuffer &source,\n-\t\t\t MappedCamera3Buffer *dest, CameraMetadata *metadata)\n+\t\t\t MappedCamera3Buffer *dest,\n+\t\t\t const CameraMetadata &requestMetadata,\n+\t\t\t CameraMetadata *resultMetadata)\n {\n \tif (!postProcessor_)\n \t\treturn 0;\n \n-\treturn postProcessor_->process(source, dest->maps()[0], metadata);\n+\treturn postProcessor_->process(source, dest->maps()[0],\n+\t\t\t\t requestMetadata, resultMetadata);\n }\n \n FrameBuffer *CameraStream::getBuffer()\ndiff --git a/src/android/camera_stream.h b/src/android/camera_stream.h\nindex cc9d5470..298ffbf6 100644\n--- a/src/android/camera_stream.h\n+++ b/src/android/camera_stream.h\n@@ -119,7 +119,9 @@ public:\n \n \tint configure();\n \tint process(const libcamera::FrameBuffer &source,\n-\t\t MappedCamera3Buffer *dest, CameraMetadata *metadata);\n+\t\t MappedCamera3Buffer *dest,\n+\t\t const CameraMetadata &requestMetadata,\n+\t\t CameraMetadata *resultMetadata);\n \tlibcamera::FrameBuffer *getBuffer();\n \tvoid putBuffer(libcamera::FrameBuffer *buffer);\n \ndiff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\nindex aa0f783d..7e49e9d9 100644\n--- a/src/android/jpeg/post_processor_jpeg.cpp\n+++ b/src/android/jpeg/post_processor_jpeg.cpp\n@@ -83,17 +83,26 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n \n int PostProcessorJpeg::process(const FrameBuffer &source,\n \t\t\t Span destination,\n-\t\t\t CameraMetadata *metadata)\n+\t\t\t const CameraMetadata &requestMetadata,\n+\t\t\t CameraMetadata *resultMetadata)\n {\n \tif (!encoder_)\n \t\treturn 0;\n \n+\tcamera_metadata_ro_entry_t entry;\n+\tint ret;\n+\n \t/* Set EXIF metadata for various tags. */\n \tExif exif;\n-\t/* \\todo Set Make and Model from external vendor tags. */\n-\texif.setMake(\"libcamera\");\n-\texif.setModel(\"cameraModel\");\n-\texif.setOrientation(cameraDevice_->orientation());\n+\texif.setMake(cameraDevice_->maker());\n+\texif.setModel(cameraDevice_->model());\n+\n+\tret = requestMetadata.getEntry(ANDROID_JPEG_ORIENTATION, &entry);\n+\n+\tconst uint32_t jpegOrientation = ret ? *entry.data.i32 : 0;\n+\tresultMetadata->addEntry(ANDROID_JPEG_ORIENTATION, &jpegOrientation, 1);\n+\texif.setOrientation(jpegOrientation);\n+\n \texif.setSize(streamSize_);\n \t/*\n \t * We set the frame's EXIF timestamp as the time of encode.\n@@ -102,6 +111,39 @@ int PostProcessorJpeg::process(const FrameBuffer &source,\n \t */\n \texif.setTimestamp(std::time(nullptr), 0ms);\n \n+\t/* \\todo Get this information from libcamera::Request::metadata */\n+\texif.setExposureTime(0);\n+\tret = requestMetadata.getEntry(ANDROID_LENS_APERTURE, &entry);\n+\tif (ret)\n+\t\texif.setAperture(*entry.data.f);\n+\texif.setISO(100);\n+\texif.setFlash(Exif::Flash::FlashNotPresent);\n+\texif.setWhiteBalance(Exif::WhiteBalance::Auto);\n+\n+\texif.setFocalLength(1.0);\n+\n+\tret = requestMetadata.getEntry(ANDROID_JPEG_GPS_TIMESTAMP, &entry);\n+\tif (ret) {\n+\t\texif.setGPSDateTimestamp(*entry.data.i64);\n+\t\tresultMetadata->addEntry(ANDROID_JPEG_GPS_TIMESTAMP,\n+\t\t\t\t\t entry.data.i64, 1);\n+\t}\n+\n+\tret = requestMetadata.getEntry(ANDROID_JPEG_GPS_COORDINATES, &entry);\n+\tif (ret) {\n+\t\texif.setGPSLocation(entry.data.d);\n+\t\tresultMetadata->addEntry(ANDROID_JPEG_GPS_COORDINATES,\n+\t\t\t\t\t entry.data.d, 3);\n+\t}\n+\n+\tret = requestMetadata.getEntry(ANDROID_JPEG_GPS_PROCESSING_METHOD, &entry);\n+\tif (ret) {\n+\t\tstd::string method(entry.data.u8, entry.data.u8 + entry.count);\n+\t\texif.setGPSMethod(method);\n+\t\tresultMetadata->addEntry(ANDROID_JPEG_GPS_PROCESSING_METHOD,\n+\t\t\t\t\t entry.data.u8, entry.count);\n+\t}\n+\n \tstd::vector thumbnail;\n \tgenerateThumbnail(source, &thumbnail);\n \tif (!thumbnail.empty())\n@@ -136,13 +178,12 @@ int PostProcessorJpeg::process(const FrameBuffer &source,\n \tblob->jpeg_size = jpeg_size;\n \n \t/* Update the JPEG result Metadata. */\n-\tmetadata->addEntry(ANDROID_JPEG_SIZE, &jpeg_size, 1);\n-\n-\tconst uint32_t jpeg_quality = 95;\n-\tmetadata->addEntry(ANDROID_JPEG_QUALITY, &jpeg_quality, 1);\n+\tresultMetadata->addEntry(ANDROID_JPEG_SIZE, &jpeg_size, 1);\n \n-\tconst uint32_t jpeg_orientation = 0;\n-\tmetadata->addEntry(ANDROID_JPEG_ORIENTATION, &jpeg_orientation, 1);\n+\t/* \\todo Configure JPEG encoder with this */\n+\tret = requestMetadata.getEntry(ANDROID_JPEG_QUALITY, &entry);\n+\tconst uint8_t jpegQuality = ret ? *entry.data.u8 : 95;\n+\tresultMetadata->addEntry(ANDROID_JPEG_QUALITY, &jpegQuality, 1);\n \n \treturn 0;\n }\ndiff --git a/src/android/jpeg/post_processor_jpeg.h b/src/android/jpeg/post_processor_jpeg.h\nindex 5afa831c..d721d1b9 100644\n--- a/src/android/jpeg/post_processor_jpeg.h\n+++ b/src/android/jpeg/post_processor_jpeg.h\n@@ -26,7 +26,8 @@ public:\n \t\t const libcamera::StreamConfiguration &outcfg) override;\n \tint process(const libcamera::FrameBuffer &source,\n \t\t libcamera::Span destination,\n-\t\t CameraMetadata *metadata) override;\n+\t\t const CameraMetadata &requestMetadata,\n+\t\t CameraMetadata *resultMetadata) override;\n \n private:\n \tvoid generateThumbnail(const libcamera::FrameBuffer &source,\ndiff --git a/src/android/post_processor.h b/src/android/post_processor.h\nindex e0f91880..bda93bb4 100644\n--- a/src/android/post_processor.h\n+++ b/src/android/post_processor.h\n@@ -22,7 +22,8 @@ public:\n \t\t\t const libcamera::StreamConfiguration &outCfg) = 0;\n \tvirtual int process(const libcamera::FrameBuffer &source,\n \t\t\t libcamera::Span destination,\n-\t\t\t CameraMetadata *metadata) = 0;\n+\t\t\t const CameraMetadata &requestMetadata,\n+\t\t\t CameraMetadata *resultMetadata) = 0;\n };\n \n #endif /* __ANDROID_POST_PROCESSOR_H__ */\n","prefixes":["libcamera-devel","v5","5/8"]}