From patchwork Thu Nov 30 15:43:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jaslo Ziska X-Patchwork-Id: 19259 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 058CCBDE6B for ; Thu, 30 Nov 2023 15:54:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B03E6629D2; Thu, 30 Nov 2023 16:54:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1701359683; bh=bK7cYUIfRBCf8bf2vdEPvCe1dbC4+QPJg91KvLeMeMs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=HAfcICaw4AV+Md7vbAo72iqYxuko+VWcEkpgIxoLUTVYmYMltWqWTgTIIg4E1RIWX jNzSyug8jWRDLY8xmIs9UYU3sxUfNr8xEPQkD65ZKFPdwnAZVVlO18X3HE06e6klcp urUjHgp/j/S8EafHuPxPHdG+vcP0ZzKyMbigdx6dau9s14qY6UIu7+7QFoflwNMDcN uEwV9Biww4iSFq27793NFzNbjtSzcb2+o8h/2AKPThqe64i4ioh/ymesiGU+i9wGPK HuE/+kzfGhZ852e+ZdpLkLRIVdugyv8jyQVI6hojdiGocdE9RtA8OxgrcT9J/gkQtS cw/0rgKi7JpNQ== Received: from mo4-p00-ob.smtp.rzone.de (mo4-p00-ob.smtp.rzone.de [81.169.146.160]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 34372629C2 for ; Thu, 30 Nov 2023 16:54:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=ziska.de header.i=@ziska.de header.b="p+11Xw4p"; dkim=permerror (0-bit key) header.d=ziska.de header.i=@ziska.de header.b="tXRj9ez9"; dkim-atps=neutral ARC-Seal: i=1; a=rsa-sha256; t=1701359682; cv=none; d=strato.com; s=strato-dkim-0002; b=eWJzw+qEOpV5dS0q64E3tc8xVjYRbUY6iNxaotOWolp5wmt0VP22jW15CJmgpY1K3l 5Ky5Md4QfoKjJzT5Si8/4/BUjn/JqXroEWyeRGC6MCAVpatf3XRtUIJC8b3Xgrj+G+wz 8BIOjXDUz1NZK6cRjD0iRh+oQKXUchBa7WUgDnBTmy47I/3ilG5hi7heewY6flC8ixE9 SBIJUm5kY8Y6M8M1S/C3Df/tbill3CKfxXhwKyw9vUG7ogwYhF6Mcs1hSRycePWICcEe LEQ0DTx1IL5bF2scrU3K4okdJHCdJRvQWhTERk0cjv/L3oNcDK3kx31SEwOG+tilo8lg mGrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; t=1701359682; s=strato-dkim-0002; d=strato.com; h=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=ahMdiRm4KDgm5BTPp4irOpCMRK1LsS/UNnMD4ZUT5sM=; b=pCSzdgUDcsJgOH96buAZOLre9NXWssYgCMvXx1Lt7poPQRHZw7eEGI3jd8EfCsz8FZ FpLpErBJ+FmVgjELKakqRV3QuG+XJ9zQmEbfN/6xxoUxL6Rd/OnsW6F2fxu6e7yYO+uk E3NxTllWD8IDvxOv+r3huZCO+MlYhqsXPm1Xk2HcVauT2Bk4deUSlGMv38Tazyrj5usQ z2IM67qwPLxF2b4J0XG2U1Kb+f6eYTyJvekrOjtQIlKhXa/yKgVTj6wHi4aPgWOhHZqr RPbUs21PEkje+Wj5I48e4ScQMN4xc/wTjj+dQ1H5FJAdh7SBpTXLyZCbXHhkLF5KFtUm jtAg== ARC-Authentication-Results: i=1; strato.com; arc=none; dkim=none X-RZG-CLASS-ID: mo00 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1701359682; s=strato-dkim-0002; d=ziska.de; h=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=ahMdiRm4KDgm5BTPp4irOpCMRK1LsS/UNnMD4ZUT5sM=; b=p+11Xw4pbRiJBnQcG52c2e+NZJ5xG5fxmvoOoBAANSG/KZ7B5+ijwq5t6w9+PB7t+k +SnqzqKysa1/wwvGsx19qo3ioEFcZmRSLyemBAJlWaj8WILqqN6hcirZrKubKr+2T9Lr NIkac4Wsx65oKhfjDEDipIkNYrumhk6ThsJEcOAFntSl+04h21atjeyEwgfWgUMZIM8f N49Y2KpW5n5kLBa13oKsbpHIiCY7YYxEBk3dVXF9SR4nDAe32fpqjdmpMCLWD4Aw/L5J vK9bEjKQzhWSuBWbjsHSNCWhrVYGNtr3mX1a5uaFMSe46W4+WgoW1umlWL+F7Jc1vEGn 1uSw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1701359682; s=strato-dkim-0003; d=ziska.de; h=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Cc:Date: From:Subject:Sender; bh=ahMdiRm4KDgm5BTPp4irOpCMRK1LsS/UNnMD4ZUT5sM=; b=tXRj9ez98ASxA+fMl+7vWG1fVzGaprdvyXPis0QQN+3BhyB1dYgMthVjvdIVCTI0bs C3G/YExiMI5aDngsGSCA== X-RZG-AUTH: ":Jm0XeU+IYfb0x77LHmrjN5Wlb7TBwusDqIM6Hizy8VdfzvKi4yoFC9cH04q6BfJa07bS3ov6I8QOM/V5x+uTrCUtBrk=" Received: from archlinux.fritz.box by smtp.strato.de (RZmta 49.9.7 AUTH) with ESMTPSA id n54b84zAUFsg0SD (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits)) (Client did not present a certificate); Thu, 30 Nov 2023 16:54:42 +0100 (CET) To: libcamera-devel@lists.libcamera.org Date: Thu, 30 Nov 2023 16:43:10 +0100 Message-ID: <20231130155323.13259-4-jaslo@ziska.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231130155323.13259-1-jaslo@ziska.de> References: <20231130155323.13259-1-jaslo@ziska.de> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/3] gstreamer: Implement renegotiation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jaslo Ziska via libcamera-devel From: Jaslo Ziska Reply-To: Jaslo Ziska Cc: Jaslo Ziska Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This commit implements renegotiation of the camera configuration and source pad caps. A renegotiation can happen when a downstream element decides to change caps or the pipeline is dynamically changed. To handle a renegotiation the GST_FLOW_NOT_NEGOTIATED return value has to be handled in GstLibcameraSrcState::processRequest. Otherwise the default would be to print an error and stop streaming. To archive this in a clean way the if statement is altered into a switch statement which now also has a case for GST_FLOW_NOT_NEGOTIATED. In the case of GST_FLOW_NOT_NEGOTIATED every source pad is checked for the reconfiguration flag with gst_pad_needs_reconfigure which does not clear this flag. If at least one pad requested a reconfiguration the function returns without an error and the renegotiation will happen later in the running task. If no pad requested a reconfiguration then the function will return with an error. In gst_libcamera_src_task_run the source pads are checked for the reconfigure flag by calling gst_pad_check_reconfigure and if one pad returns true and the caps are not sufficient anymore then the negotiation is triggered. It is fine to trigger the negotiation after only a single pad returns true for gst_pad_check_reconfigure because the reconfigure flags are cleared in the gst_libcamera_src_negotiate function. If any pad requested a reconfiguration the following will happen: 1. The camera is stopped because changing the configuration may not happen while running. 2. The completedRequests queue will be cleared by calling GstLibcameraSrcState::clearRequests because the completed buffers have the wrong configuration. 3. The new caps are negotiated by calling gst_libcamera_src_negotiate. When the negotiation fails streaming will stop. 4. The camera is started again. Signed-off-by: Jaslo Ziska Reviewed-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- src/gstreamer/gstlibcamerasrc.cpp | 72 ++++++++++++++++++++++++++----- 1 file changed, 61 insertions(+), 11 deletions(-) diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index d448a750..8bd32306 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -11,7 +11,6 @@ * - Implement GstElement::send_event * + Allowing application to use FLUSH/FLUSH_STOP * + Prevent the main thread from accessing streaming thread - * - Implement renegotiation (even if slow) * - Implement GstElement::request-new-pad (multi stream) * + Evaluate if a single streaming thread is fine * - Add application driven request (snapshot) @@ -302,18 +301,42 @@ int GstLibcameraSrcState::processRequest() srcpad, ret); } - if (ret != GST_FLOW_OK) { - if (ret == GST_FLOW_EOS) { - g_autoptr(GstEvent) eos = gst_event_new_eos(); - guint32 seqnum = gst_util_seqnum_next(); - gst_event_set_seqnum(eos, seqnum); - for (GstPad *srcpad : srcpads_) - gst_pad_push_event(srcpad, gst_event_ref(eos)); - } else if (ret != GST_FLOW_FLUSHING) { - GST_ELEMENT_FLOW_ERROR(src_, ret); + switch (ret) { + case GST_FLOW_OK: + break; + case GST_FLOW_NOT_NEGOTIATED: { + bool reconfigure = false; + for (GstPad *srcpad : srcpads_) { + if (gst_pad_needs_reconfigure(srcpad)) { + reconfigure = true; + break; + } } - return -EPIPE; + // if no pads needs a reconfiguration something went wrong + if (!reconfigure) + err = -EPIPE; + + break; + } + case GST_FLOW_EOS: { + g_autoptr(GstEvent) eos = gst_event_new_eos(); + guint32 seqnum = gst_util_seqnum_next(); + gst_event_set_seqnum(eos, seqnum); + for (GstPad *srcpad : srcpads_) + gst_pad_push_event(srcpad, gst_event_ref(eos)); + + err = -EPIPE; + break; + } + case GST_FLOW_FLUSHING: + err = -EPIPE; + break; + default: + GST_ELEMENT_FLOW_ERROR(src_, ret); + + err = -EPIPE; + break; } return err; @@ -463,6 +486,8 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self) G_CALLBACK(gst_task_resume), self->task); gst_libcamera_pad_set_pool(srcpad, pool); + + gst_pad_check_reconfigure(srcpad); // clear all reconfigure flags } return true; @@ -496,6 +521,31 @@ gst_libcamera_src_task_run(gpointer user_data) return; } + // check if a srcpad requested a renegotiation + bool reconfigure = false; + for (GstPad *srcpad : state->srcpads_) { + if (gst_pad_check_reconfigure(srcpad)) { + // check whether the caps even need changing + g_autoptr(GstCaps) caps = gst_pad_get_current_caps(srcpad); + if (!gst_pad_peer_query_accept_caps(srcpad, caps)) { + reconfigure = true; + break; + } + } + } + + if (reconfigure) { + state->cam_->stop(); + state->clearRequests(); + + if (!gst_libcamera_src_negotiate(self)) { + GST_ELEMENT_FLOW_ERROR(self, GST_FLOW_NOT_NEGOTIATED); + gst_task_stop(self->task); + } + + state->cam_->start(&state->initControls_); + } + /* * Create and queue one request. If no buffers are available the * function returns -ENOBUFS, which we ignore here as that's not a