From patchwork Wed Nov 2 13:56:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 17747 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2857DBD16B for ; Wed, 2 Nov 2022 13:56:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C101063080; Wed, 2 Nov 2022 14:56:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1667397391; bh=Pn7J48k/yYKWXnlUrI9OnY6j4RworjZkje1VK59bdfc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=zmidIl6Zm/k+yXWq8f75fFGfXo559sm1uPyBH2E4hq1rceKuhtWOxEgTp6lAMnSV+ u+j8bhKjFMyM4atzkd+oeDXOyqQgvaHG+ZtwUYsujy6awcPKt04jpYYLogPCYJFfrN PAWh69sUWVskL/Rn3OoyxaUISwvONloivqI/8Oxnav0U7kwcaSrSNeo7Vn3aI+JQ2i NqWKW/SMq2GYfCiziAt7vmXbCUhsqjXBVX7Na7jIDwMaXhm0+7wzeyPkt5ms4VjW+7 NEtMaf21FwFuhB95OfYxoiSx0PTBCQn8mYJjAt330w2klNyx+kWdFj7ir1Sa9iWiR9 1GD1u/8q5Zq9A== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 11C2A63078 for ; Wed, 2 Nov 2022 14:56:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XJa/G79P"; dkim-atps=neutral Received: from umang.jainideasonboard.com (unknown [103.251.226.107]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3265F1226; Wed, 2 Nov 2022 14:56:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1667397389; bh=Pn7J48k/yYKWXnlUrI9OnY6j4RworjZkje1VK59bdfc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XJa/G79PK6JnC/olrQg5gCvdL7/xZiix5t6kczEfSQ8+bc20WLFNDGHfy0sxsHyQr wBMtz1kq3WWFVFG/YcBSWx4HM2/wUcrXVPPyRAhxNTJq/SoHz3QyCmPnp/vjxsjk8o p0q5lr3/klZWWZ6FpVOLl/6o5NnKwqaat1/ku5NU= To: libcamera-devel@lists.libcamera.org Date: Wed, 2 Nov 2022 19:26:14 +0530 Message-Id: <20221102135614.657444-3-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221102135614.657444-1-umang.jain@ideasonboard.com> References: <20221102135614.657444-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 2/2] gstreamer: Provide framerate support for libcamerasrc X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Rishikesh Donadkar Control the framerate by passing the controls::FrameDurationLimits during Camera::start(). Framerate in gstreamer is expressed as GST_TYPE_FRACTION so we maximise on maintaining it as a fraction throughout and only do arithematic computations as and when required (to compute frame-duration and vice-versa). To weed out abritrary framerate as input, place the clamping via the controls::FrameDurationLimits provided after camera::configure() phase. This is handled by a helper function gst_libcamera_clamp_and_set_frameduration(). Set the bound checked framerate (done in the above mentioned helper) into the caps and pass the ControlList containing the frame-duration to Camera::start(ctrls). Signed-off-by: Rishikesh Donadkar Signed-off-by: Umang Jain Tested-by: Umang Jain --- src/gstreamer/gstlibcamera-utils.cpp | 78 ++++++++++++++++++++++++++++ src/gstreamer/gstlibcamera-utils.h | 6 +++ src/gstreamer/gstlibcamerasrc.cpp | 15 +++++- 3 files changed, 98 insertions(+), 1 deletion(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index 244a4a79..c14f72ec 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -8,6 +8,7 @@ #include "gstlibcamera-utils.h" +#include #include using namespace libcamera; @@ -407,6 +408,83 @@ gst_libcamera_configure_stream_from_caps(StreamConfiguration &stream_cfg, } } +void gst_libcamera_get_framerate_from_caps(GstCaps *caps, + GstStructure *container) +{ + GstStructure *s = gst_caps_get_structure(caps, 0); + /* + * Default to 30 fps. If the "framerate" fraction is invalid below, + * libcamerasrc will set 30fps as the framerate. + */ + gint fps_n = 30, fps_d = 1; + + if (gst_structure_has_field_typed(s, "framerate", GST_TYPE_FRACTION)) { + if (!gst_structure_get_fraction(s, "framerate", &fps_n, &fps_d)) + GST_WARNING("Invalid framerate in the caps"); + } + + gst_structure_set(container, "framerate", GST_TYPE_FRACTION, + fps_n, fps_d, nullptr); +} + +void gst_libcamera_clamp_and_set_frameduration(ControlList &initCtrls, + const ControlInfoMap &cam_ctrls, + GstStructure *container) +{ + gboolean in_bounds = false; + gint fps_caps_n, fps_caps_d; + + if (!gst_structure_has_field_typed(container, "framerate", GST_TYPE_FRACTION)) + return; + + auto iterFrameDuration = cam_ctrls.find(controls::FrameDurationLimits.id()); + if (iterFrameDuration == cam_ctrls.end()) { + GST_WARNING("FrameDurationLimits not found in camera controls."); + return; + } + + const GValue *framerate = gst_structure_get_value(container, "framerate"); + + fps_caps_n = gst_value_get_fraction_numerator(framerate); + fps_caps_d = gst_value_get_fraction_denominator(framerate); + + int64_t frame_duration = (fps_caps_d * 1000000.0) / fps_caps_n; + int64_t min_frame_duration = iterFrameDuration->second.min().get(); + int64_t max_frame_duration = iterFrameDuration->second.max().get(); + + if (frame_duration < min_frame_duration) { + frame_duration = min_frame_duration; + } else if (frame_duration > max_frame_duration) { + frame_duration = max_frame_duration; + } else { + in_bounds = true; + } + + if (!in_bounds) { + gint framerate_clamped = 1000000 / frame_duration; + + /* Update the framerate which then will be exposed in caps. */ + gst_structure_set(container, "framerate", GST_TYPE_FRACTION, + framerate_clamped, 1, nullptr); + } + + initCtrls.set(controls::FrameDurationLimits, + { frame_duration, frame_duration }); +} + +void gst_libcamera_framerate_to_caps(GstCaps *caps, const GValue *container) +{ + if (!GST_VALUE_HOLDS_FRACTION(container)) + return; + + GstStructure *s = gst_caps_get_structure(caps, 0); + gint fps_caps_n, fps_caps_d; + + fps_caps_n = gst_value_get_fraction_numerator(container); + fps_caps_d = gst_value_get_fraction_denominator(container); + gst_structure_set(s, "framerate", GST_TYPE_FRACTION, fps_caps_n, fps_caps_d, nullptr); +} + #if !GST_CHECK_VERSION(1, 17, 1) gboolean gst_task_resume(GstTask *task) diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 164189a2..3d217fcf 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -9,6 +9,7 @@ #pragma once #include +#include #include #include @@ -18,6 +19,11 @@ GstCaps *gst_libcamera_stream_formats_to_caps(const libcamera::StreamFormats &fo GstCaps *gst_libcamera_stream_configuration_to_caps(const libcamera::StreamConfiguration &stream_cfg); void gst_libcamera_configure_stream_from_caps(libcamera::StreamConfiguration &stream_cfg, GstCaps *caps); +void gst_libcamera_get_framerate_from_caps(GstCaps *caps, GstStructure *container); +void gst_libcamera_clamp_and_set_frameduration(libcamera::ControlList &controls, + const libcamera::ControlInfoMap &camera_controls, GstStructure *container); +void gst_libcamera_framerate_to_caps(GstCaps *caps, const GValue *container); + #if !GST_CHECK_VERSION(1, 17, 1) gboolean gst_task_resume(GstTask *task); #endif diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 60032236..955d6eac 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -131,6 +131,7 @@ struct GstLibcameraSrcState { std::queue> completedRequests_ LIBCAMERA_TSA_GUARDED_BY(lock_); + ControlList initControls_; guint group_id_; int queueRequest(); @@ -462,6 +463,8 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread, GstFlowReturn flow_ret = GST_FLOW_OK; gint ret; + GstStructure *container = gst_structure_new_empty("container"); + GST_DEBUG_OBJECT(self, "Streaming thread has started"); gint stream_id_num = 0; @@ -504,6 +507,7 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread, /* Fixate caps and configure the stream. */ caps = gst_caps_make_writable(caps); gst_libcamera_configure_stream_from_caps(stream_cfg, caps); + gst_libcamera_get_framerate_from_caps(caps, container); } if (flow_ret != GST_FLOW_OK) @@ -525,6 +529,10 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread, goto done; } + /* Check frame duration bounds within controls::FrameDurationLimits */ + gst_libcamera_clamp_and_set_frameduration(state->initControls_, + state->cam_->controls(), container); + /* * Regardless if it has been modified, create clean caps and push the * caps event. Downstream will decide if the caps are acceptable. @@ -532,8 +540,11 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread, for (gsize i = 0; i < state->srcpads_.size(); i++) { GstPad *srcpad = state->srcpads_[i]; const StreamConfiguration &stream_cfg = state->config_->at(i); + const GValue *framerate = gst_structure_get_value(container, "framerate"); g_autoptr(GstCaps) caps = gst_libcamera_stream_configuration_to_caps(stream_cfg); + gst_libcamera_framerate_to_caps(caps, framerate); + if (!gst_pad_push_event(srcpad, gst_event_new_caps(caps))) { flow_ret = GST_FLOW_NOT_NEGOTIATED; break; @@ -567,7 +578,7 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread, gst_flow_combiner_add_pad(self->flow_combiner, srcpad); } - ret = state->cam_->start(); + ret = state->cam_->start(&state->initControls_); if (ret) { GST_ELEMENT_ERROR(self, RESOURCE, SETTINGS, ("Failed to start the camera: %s", g_strerror(-ret)), @@ -577,6 +588,8 @@ gst_libcamera_src_task_enter(GstTask *task, [[maybe_unused]] GThread *thread, } done: + state->initControls_.clear(); + g_free(container); switch (flow_ret) { case GST_FLOW_NOT_NEGOTIATED: GST_ELEMENT_FLOW_ERROR(self, flow_ret);