{"id":17011,"url":"https://patchwork.libcamera.org/api/patches/17011/?format=json","web_url":"https://patchwork.libcamera.org/patch/17011/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220807121233.18353-3-rishikeshdonadkar@gmail.com>","date":"2022-08-07T12:12:33","name":"[libcamera-devel,v1,2/2] gstreamer: Add multiple colorimetry support","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"59e5a6a10778be6d82bd7fefc8029860c63c1ea3","submitter":{"id":118,"url":"https://patchwork.libcamera.org/api/people/118/?format=json","name":"Rishikesh Donadkar","email":"rishikeshdonadkar@gmail.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/17011/mbox/","series":[{"id":3382,"url":"https://patchwork.libcamera.org/api/series/3382/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3382","date":"2022-08-07T12:12:31","name":"Multiple colorimetry support for libcamerasrc.","version":1,"mbox":"https://patchwork.libcamera.org/series/3382/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/17011/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/17011/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9B684BE173\n\tfor ;\n\tSun, 7 Aug 2022 12:13:30 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6A52E63331;\n\tSun, 7 Aug 2022 14:13:30 +0200 (CEST)","from mail-pg1-x529.google.com (mail-pg1-x529.google.com\n\t[IPv6:2607:f8b0:4864:20::529])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7C2B463312\n\tfor ;\n\tSun, 7 Aug 2022 14:13:29 +0200 (CEST)","by mail-pg1-x529.google.com with SMTP id d7so6313556pgc.13\n\tfor ;\n\tSun, 07 Aug 2022 05:13:29 -0700 (PDT)","from localhost.localdomain ([49.36.99.220])\n\tby smtp.googlemail.com with ESMTPSA id\n\t78-20020a621651000000b0052d3ea78858sm6454883pfw.127.2022.08.07.05.13.25\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tSun, 07 Aug 2022 05:13:27 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1659874410;\n\tbh=T0jnohcfTr9lT2UfAQ99rG3DXIZHfb4WXxeFKBRj0hU=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=r2GFZgZd6d5e/FkCgwiYh3QeXhbDpgTo1xAMv3h4KMo2jOEuxQbjWFd/gN9Y/226+\n\tJGTTv5aX+7G5RaZrxCumdcozgXoiajR2EHvI42XPWeAQfF6WxI7iK0W7+R/74vIFS7\n\tmBk8fvM8+LqY9J7hbcIVMBKWLcqcIpEK2r7Nf5CG6KCKQy8j0JtRdXBwUKFOHeFlXf\n\tbVXD4PB9zhO4UAOPpiBlbRq32jbyDvuWYEPSPde30a4ga7s0mOzIPFD9A8XeWHPh5F\n\tU8A/tm1x9qXOcWa/48QDtKwx+DJu1SrCB5Nf+7z3gEP6kChMfQSK/y8veVBZLgGap7\n\tTxVKqjbLxuoXA==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc;\n\tbh=FMZ5QRsiDYfv8ikW+QSc7CgouWxwKhUVVOQl1aCcUE0=;\n\tb=PeGpa+XLBEbhfRjHzZC/kbQCxUscr4IHYLlcGujrzhtSAXAL81fO86HMvEX8uo3yml\n\t4IHiAdaukOfQtavkd/NQX75hzvo408XqPmXOEJaecN7c8Bwp0WNIQd+iwg4pgwL0lM2b\n\tmixirVgkh+fZ/i75kLNnyyWcOY8IJqxXlu3RnjsKT8VO/QamVFN3rjQsF3JAIP7j4wNm\n\tvxN8aNjzABb1L4HW8xwcC6Rgz3Z0Nsc642RZ0W61w42BHRypcyN8RwvEjYRRXU4ucrj6\n\thQLDFtHsLsNwLmMjpnsczeSaKtHT+C1OFDu/wlSNjRYParUC1wyg6Ghse9wUGx2ojQFi\n\tqWTQ=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"PeGpa+XL\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc;\n\tbh=FMZ5QRsiDYfv8ikW+QSc7CgouWxwKhUVVOQl1aCcUE0=;\n\tb=lh3fJO+Sk0Pthwl9pzwD5wCk+NifRaaOc238TJ7W9yb/YwVPR4Je12x3vUdAgbLqnp\n\t+6vCMQS/O20d9Hts266K4CQndJ5k7FFmJiO+1UqiEg6+Z0ZXcW5pHMPdstosAi/DYlxu\n\tw4mTfZyj6lHe4EQDV4tmjhSLeQP5kF2JkpUFG9gtQkQ/0ejv021k2DZLEWmNWbRZfg2j\n\t0FNOsCIKbP3oO1gJONSK52fkj4kV89IC1jru9WqHQ+uNNEdnzH2quSUo/wEIWZDJBJes\n\tmgfBZc4aD2mcbsUjL1bChVKl2Ehx2RoutzldxTevabZGXgzSpub7F6jdpbvrZ3hLP66Y\n\trLQQ==","X-Gm-Message-State":"ACgBeo3xTcS/fsCe9kIOa0kALbnLHksDvUsSojA+5XSkq70BOnJ6L4md\n\t6rfbRqWqWnf01ql/bJayyFCE4+ZQ3ZnkJJzt","X-Google-Smtp-Source":"AA6agR6RPhAZCz7yFp9J/13MbH/ujr2KJ1FUFhUby9htE3R52lDbKZOc3BTe3hNr/oyS2I0J8yslhQ==","X-Received":"by 2002:a65:604a:0:b0:401:9c5f:876b with SMTP id\n\ta10-20020a65604a000000b004019c5f876bmr11982660pgp.459.1659874407789; \n\tSun, 07 Aug 2022 05:13:27 -0700 (PDT)","To":"libcamera-devel@lists.libcamera.org","Date":"Sun, 7 Aug 2022 17:42:33 +0530","Message-Id":"<20220807121233.18353-3-rishikeshdonadkar@gmail.com>","X-Mailer":"git-send-email 2.25.1","In-Reply-To":"<20220807121233.18353-1-rishikeshdonadkar@gmail.com>","References":"<20220807121233.18353-1-rishikeshdonadkar@gmail.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v1 2/2] gstreamer: Add multiple\n\tcolorimetry support","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Rishikesh Donadkar via libcamera-devel\n\t","Reply-To":"Rishikesh Donadkar ","Cc":"Rishikesh Donadkar ,\n\tvedantparanjape160201@gmail.com","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This patch aims to add support for multiple colorimetry.\n\nCreate a fresh new caps which will contain a copy of the structure\nwith the best resolutions. Normalize the caps using gst_caps_normalize().\nThis will return caps where the colorimetry list is expanded.\nThe ncaps will contain as many structures as the number of\ncolorimetry specified in the gstreamer pipeline.\n\nIterate over each structure in the ncaps, retrieve the colorimetry string,\nconvert to colorspace and validate the camera configuration.\nRetrieve the colorspace after validation, convert to colorimetry and check\nif it is same as the colorimetry requested.\n\nIf none of the colorimetry requested is supported by the camera (ie not\nthe same after validation) then set the stream_cfg to the previous configuration\nthat was present before trying new colorimetry.\n---\n src/gstreamer/gstlibcamera-utils.cpp | 65 ++++++++++++++++++++++++++--\n src/gstreamer/gstlibcamera-utils.h | 4 +-\n src/gstreamer/gstlibcamerasrc.cpp | 3 +-\n 3 files changed, 66 insertions(+), 6 deletions(-)","diff":"diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp\nindex 24d8d035..06ab5bf9 100644\n--- a/src/gstreamer/gstlibcamera-utils.cpp\n+++ b/src/gstreamer/gstlibcamera-utils.cpp\n@@ -302,13 +302,47 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg\n \treturn caps;\n }\n \n+static void\n+configure_colorspace_from_caps(StreamConfiguration &stream_cfg,\n+\t\t\t GstStructure *s)\n+{\n+\tif (gst_structure_has_field(s, \"colorimetry\")) {\n+\t\tconst gchar *colorimetry_str = gst_structure_get_string(s, \"colorimetry\");\n+\t\tGstVideoColorimetry colorimetry;\n+\n+\t\tif (!gst_video_colorimetry_from_string(&colorimetry, colorimetry_str))\n+\t\t\tg_critical(\"Invalid colorimetry %s\", colorimetry_str);\n+\n+\t\tstream_cfg.colorSpace = colorspace_from_colorimetry(colorimetry);\n+\t\t/* Check if colorimetry had any identifiers which did not map */\n+\t\tif (colorimetry.primaries != GST_VIDEO_COLOR_PRIMARIES_UNKNOWN &&\n+\t\t stream_cfg.colorSpace == ColorSpace::Raw) {\n+\t\t\tGST_ERROR(\"One or more identifiers could not be mapped for %s colorimetry\",\n+\t\t\t\t colorimetry_str);\n+\t\t\tstream_cfg.colorSpace = std::nullopt;\n+\t\t}\n+\t}\n+}\n+\n+static gboolean\n+check_colorspace(const ColorSpace colorSpace, const gchar *colorimetry_old)\n+{\n+\tGstVideoColorimetry colorimetry = colorimetry_from_colorspace(colorSpace);\n+\tg_autofree gchar *colorimetry_new = gst_video_colorimetry_to_string(&colorimetry);\n+\tif (!g_strcmp0(colorimetry_old, colorimetry_new)) {\n+\t\treturn true;\n+\t}\n+\treturn false;\n+}\n+\n void\n-gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n+gst_libcamera_configure_stream_from_caps(CameraConfiguration &cam_cfg,\n+\t\t\t\t\t StreamConfiguration &stream_cfg,\n \t\t\t\t\t GstCaps *caps)\n {\n \tGstVideoFormat gst_format = pixel_format_to_gst_format(stream_cfg.pixelFormat);\n \tguint i;\n-\tgint best_fixed = -1, best_in_range = -1;\n+\tgint best_fixed = -1, best_in_range = -1, colorimetry_index = -1;\n \tGstStructure *s;\n \n \t/*\n@@ -354,10 +388,13 @@ gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n \t}\n \n \t/* Prefer reliable fixed value over ranges */\n-\tif (best_fixed >= 0)\n+\tif (best_fixed >= 0) {\n \t\ts = gst_caps_get_structure(caps, best_fixed);\n-\telse\n+\t\tcolorimetry_index = best_fixed;\n+\t} else {\n \t\ts = gst_caps_get_structure(caps, best_in_range);\n+\t\tcolorimetry_index = best_in_range;\n+\t}\n \n \tif (gst_structure_has_name(s, \"video/x-raw\")) {\n \t\tconst gchar *format = gst_video_format_to_string(gst_format);\n@@ -380,6 +417,26 @@ gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg,\n \tgst_structure_get_int(s, \"height\", &height);\n \tstream_cfg.size.width = width;\n \tstream_cfg.size.height = height;\n+\n+\t/* Create new caps, copy the structure with best resolutions\n+\t * and normalize the caps.\n+\t */\n+\tGstCaps *ncaps = gst_caps_copy_nth(caps, colorimetry_index);\n+\tncaps = gst_caps_normalize(ncaps);\n+\n+\t/* Configure Colorimetry */\n+\tStreamConfiguration dup_stream_cfg = stream_cfg;\n+\tfor (i = 0; i < gst_caps_get_size(ncaps); i++) {\n+\t\tGstStructure *ns = gst_caps_get_structure(ncaps, i);\n+\t\tconfigure_colorspace_from_caps(stream_cfg, ns);\n+\t\tg_autofree const gchar *colorimetry_old = gst_structure_get_string(ns, \"colorimetry\");\n+\t\tif (cam_cfg.validate() != CameraConfiguration::Invalid) {\n+\t\t\tif (check_colorspace(stream_cfg.colorSpace.value(), colorimetry_old))\n+\t\t\t\tbreak;\n+\t\t\telse\n+\t\t\t\tstream_cfg = dup_stream_cfg;\n+\t\t}\n+\t}\n }\n \n #if !GST_CHECK_VERSION(1, 17, 1)\ndiff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h\nindex 164189a2..90be6abe 100644\n--- a/src/gstreamer/gstlibcamera-utils.h\n+++ b/src/gstreamer/gstlibcamera-utils.h\n@@ -8,6 +8,7 @@\n \n #pragma once\n \n+#include \n #include \n #include \n \n@@ -16,7 +17,8 @@\n \n GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats);\n GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg);\n-void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg,\n+void gst_libcamera_configure_stream_from_caps(libcamera::CameraConfiguration &cam_cfg,\n+\t\t\t\t\t libcamera::StreamConfiguration &stream_cfg,\n \t\t\t\t\t GstCaps *caps);\n #if !GST_CHECK_VERSION(1, 17, 1)\n gboolean gst_task_resume(GstTask *task);\ndiff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp\nindex 16d70fea..3617170e 100644\n--- a/src/gstreamer/gstlibcamerasrc.cpp\n+++ b/src/gstreamer/gstlibcamerasrc.cpp\n@@ -492,6 +492,7 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread,\n \tfor (gsize i = 0; i < state->srcpads_.size(); i++) {\n \t\tGstPad *srcpad = state->srcpads_[i];\n \t\tStreamConfiguration &stream_cfg = state->config_->at(i);\n+\t\tCameraConfiguration &cam_cfg = *(state->config_);\n \n \t\t/* Retrieve the supported caps. */\n \t\tg_autoptr(GstCaps) filter = gst_libcamera_stream_formats_to_caps(stream_cfg.formats());\n@@ -503,7 +504,7 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread,\n \n \t\t/* Fixate caps and configure the stream. */\n \t\tcaps = gst_caps_make_writable(caps);\n-\t\tgst_libcamera_configure_stream_from_caps(stream_cfg, caps);\n+\t\tgst_libcamera_configure_stream_from_caps(cam_cfg, stream_cfg, caps);\n \t}\n \n \tif (flow_ret != GST_FLOW_OK)\n","prefixes":["libcamera-devel","v1","2/2"]}