From patchwork Sun Aug 7 12:12:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rishikesh Donadkar X-Patchwork-Id: 17011 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9B684BE173 for ; Sun, 7 Aug 2022 12:13:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6A52E63331; Sun, 7 Aug 2022 14:13:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659874410; bh=T0jnohcfTr9lT2UfAQ99rG3DXIZHfb4WXxeFKBRj0hU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=r2GFZgZd6d5e/FkCgwiYh3QeXhbDpgTo1xAMv3h4KMo2jOEuxQbjWFd/gN9Y/226+ JGTTv5aX+7G5RaZrxCumdcozgXoiajR2EHvI42XPWeAQfF6WxI7iK0W7+R/74vIFS7 mBk8fvM8+LqY9J7hbcIVMBKWLcqcIpEK2r7Nf5CG6KCKQy8j0JtRdXBwUKFOHeFlXf bVXD4PB9zhO4UAOPpiBlbRq32jbyDvuWYEPSPde30a4ga7s0mOzIPFD9A8XeWHPh5F U8A/tm1x9qXOcWa/48QDtKwx+DJu1SrCB5Nf+7z3gEP6kChMfQSK/y8veVBZLgGap7 TxVKqjbLxuoXA== Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7C2B463312 for ; Sun, 7 Aug 2022 14:13:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="PeGpa+XL"; dkim-atps=neutral Received: by mail-pg1-x529.google.com with SMTP id d7so6313556pgc.13 for ; Sun, 07 Aug 2022 05:13:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=FMZ5QRsiDYfv8ikW+QSc7CgouWxwKhUVVOQl1aCcUE0=; b=PeGpa+XLBEbhfRjHzZC/kbQCxUscr4IHYLlcGujrzhtSAXAL81fO86HMvEX8uo3yml 4IHiAdaukOfQtavkd/NQX75hzvo408XqPmXOEJaecN7c8Bwp0WNIQd+iwg4pgwL0lM2b mixirVgkh+fZ/i75kLNnyyWcOY8IJqxXlu3RnjsKT8VO/QamVFN3rjQsF3JAIP7j4wNm vxN8aNjzABb1L4HW8xwcC6Rgz3Z0Nsc642RZ0W61w42BHRypcyN8RwvEjYRRXU4ucrj6 hQLDFtHsLsNwLmMjpnsczeSaKtHT+C1OFDu/wlSNjRYParUC1wyg6Ghse9wUGx2ojQFi qWTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=FMZ5QRsiDYfv8ikW+QSc7CgouWxwKhUVVOQl1aCcUE0=; b=lh3fJO+Sk0Pthwl9pzwD5wCk+NifRaaOc238TJ7W9yb/YwVPR4Je12x3vUdAgbLqnp +6vCMQS/O20d9Hts266K4CQndJ5k7FFmJiO+1UqiEg6+Z0ZXcW5pHMPdstosAi/DYlxu w4mTfZyj6lHe4EQDV4tmjhSLeQP5kF2JkpUFG9gtQkQ/0ejv021k2DZLEWmNWbRZfg2j 0FNOsCIKbP3oO1gJONSK52fkj4kV89IC1jru9WqHQ+uNNEdnzH2quSUo/wEIWZDJBJes mgfBZc4aD2mcbsUjL1bChVKl2Ehx2RoutzldxTevabZGXgzSpub7F6jdpbvrZ3hLP66Y rLQQ== X-Gm-Message-State: ACgBeo3xTcS/fsCe9kIOa0kALbnLHksDvUsSojA+5XSkq70BOnJ6L4md 6rfbRqWqWnf01ql/bJayyFCE4+ZQ3ZnkJJzt X-Google-Smtp-Source: AA6agR6RPhAZCz7yFp9J/13MbH/ujr2KJ1FUFhUby9htE3R52lDbKZOc3BTe3hNr/oyS2I0J8yslhQ== X-Received: by 2002:a65:604a:0:b0:401:9c5f:876b with SMTP id a10-20020a65604a000000b004019c5f876bmr11982660pgp.459.1659874407789; Sun, 07 Aug 2022 05:13:27 -0700 (PDT) Received: from localhost.localdomain ([49.36.99.220]) by smtp.googlemail.com with ESMTPSA id 78-20020a621651000000b0052d3ea78858sm6454883pfw.127.2022.08.07.05.13.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 07 Aug 2022 05:13:27 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Sun, 7 Aug 2022 17:42:33 +0530 Message-Id: <20220807121233.18353-3-rishikeshdonadkar@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220807121233.18353-1-rishikeshdonadkar@gmail.com> References: <20220807121233.18353-1-rishikeshdonadkar@gmail.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/2] gstreamer: Add multiple colorimetry support X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Rishikesh Donadkar via libcamera-devel From: Rishikesh Donadkar Reply-To: Rishikesh Donadkar Cc: Rishikesh Donadkar , vedantparanjape160201@gmail.com Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch aims to add support for multiple colorimetry. Create a fresh new caps which will contain a copy of the structure with the best resolutions. Normalize the caps using gst_caps_normalize(). This will return caps where the colorimetry list is expanded. The ncaps will contain as many structures as the number of colorimetry specified in the gstreamer pipeline. Iterate over each structure in the ncaps, retrieve the colorimetry string, convert to colorspace and validate the camera configuration. Retrieve the colorspace after validation, convert to colorimetry and check if it is same as the colorimetry requested. If none of the colorimetry requested is supported by the camera (ie not the same after validation) then set the stream_cfg to the previous configuration that was present before trying new colorimetry. --- src/gstreamer/gstlibcamera-utils.cpp | 65 ++++++++++++++++++++++++++-- src/gstreamer/gstlibcamera-utils.h | 4 +- src/gstreamer/gstlibcamerasrc.cpp | 3 +- 3 files changed, 66 insertions(+), 6 deletions(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 24d8d035..06ab5bf9 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -302,13 +302,47 @@ gst_libcamera_stream_configuration_to_caps(const StreamConfiguration &stream_cfg return caps; } +static void +configure_colorspace_from_caps(StreamConfiguration &stream_cfg, + GstStructure *s) +{ + if (gst_structure_has_field(s, "colorimetry")) { + const gchar *colorimetry_str = gst_structure_get_string(s, "colorimetry"); + GstVideoColorimetry colorimetry; + + if (!gst_video_colorimetry_from_string(&colorimetry, colorimetry_str)) + g_critical("Invalid colorimetry %s", colorimetry_str); + + stream_cfg.colorSpace = colorspace_from_colorimetry(colorimetry); + /* Check if colorimetry had any identifiers which did not map */ + if (colorimetry.primaries != GST_VIDEO_COLOR_PRIMARIES_UNKNOWN && + stream_cfg.colorSpace == ColorSpace::Raw) { + GST_ERROR("One or more identifiers could not be mapped for %s colorimetry", + colorimetry_str); + stream_cfg.colorSpace = std::nullopt; + } + } +} + +static gboolean +check_colorspace(const ColorSpace colorSpace, const gchar *colorimetry_old) +{ + GstVideoColorimetry colorimetry = colorimetry_from_colorspace(colorSpace); + g_autofree gchar *colorimetry_new = gst_video_colorimetry_to_string(&colorimetry); + if (!g_strcmp0(colorimetry_old, colorimetry_new)) { + return true; + } + return false; +} + void -gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, +gst_libcamera_configure_stream_from_caps(CameraConfiguration &cam_cfg, + StreamConfiguration &stream_cfg, GstCaps *caps) { GstVideoFormat gst_format = pixel_format_to_gst_format(stream_cfg.pixelFormat); guint i; - gint best_fixed = -1, best_in_range = -1; + gint best_fixed = -1, best_in_range = -1, colorimetry_index = -1; GstStructure *s; /* @@ -354,10 +388,13 @@ gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, } /* Prefer reliable fixed value over ranges */ - if (best_fixed >= 0) + if (best_fixed >= 0) { s = gst_caps_get_structure(caps, best_fixed); - else + colorimetry_index = best_fixed; + } else { s = gst_caps_get_structure(caps, best_in_range); + colorimetry_index = best_in_range; + } if (gst_structure_has_name(s, "video/x-raw")) { const gchar *format = gst_video_format_to_string(gst_format); @@ -380,6 +417,26 @@ gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, gst_structure_get_int(s, "height", &height); stream_cfg.size.width = width; stream_cfg.size.height = height; + + /* Create new caps, copy the structure with best resolutions + * and normalize the caps. + */ + GstCaps *ncaps = gst_caps_copy_nth(caps, colorimetry_index); + ncaps = gst_caps_normalize(ncaps); + + /* Configure Colorimetry */ + StreamConfiguration dup_stream_cfg = stream_cfg; + for (i = 0; i < gst_caps_get_size(ncaps); i++) { + GstStructure *ns = gst_caps_get_structure(ncaps, i); + configure_colorspace_from_caps(stream_cfg, ns); + g_autofree const gchar *colorimetry_old = gst_structure_get_string(ns, "colorimetry"); + if (cam_cfg.validate() != CameraConfiguration::Invalid) { + if (check_colorspace(stream_cfg.colorSpace.value(), colorimetry_old)) + break; + else + stream_cfg = dup_stream_cfg; + } + } } #if !GST_CHECK_VERSION(1, 17, 1) diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 164189a2..90be6abe 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -8,6 +8,7 @@ #pragma once +#include #include #include @@ -16,7 +17,8 @@ GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &formats); GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg); -void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, +void gst_libcamera_configure_stream_from_caps(libcamera::CameraConfiguration &cam_cfg, + libcamera::StreamConfiguration &stream_cfg, GstCaps *caps); #if !GST_CHECK_VERSION(1, 17, 1) gboolean gst_task_resume(GstTask *task); diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 16d70fea..3617170e 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -492,6 +492,7 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread, for (gsize i = 0; i < state->srcpads_.size(); i++) { GstPad *srcpad = state->srcpads_[i]; StreamConfiguration &stream_cfg = state->config_->at(i); + CameraConfiguration &cam_cfg = *(state->config_); /* Retrieve the supported caps. */ g_autoptr(GstCaps) filter = gst_libcamera_stream_formats_to_caps(stream_cfg.formats()); @@ -503,7 +504,7 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread, /* Fixate caps and configure the stream. */ caps = gst_caps_make_writable(caps); - gst_libcamera_configure_stream_from_caps(stream_cfg, caps); + gst_libcamera_configure_stream_from_caps(cam_cfg, stream_cfg, caps); } if (flow_ret != GST_FLOW_OK)