{"id":19259,"url":"https://patchwork.libcamera.org/api/1.1/patches/19259/?format=json","web_url":"https://patchwork.libcamera.org/patch/19259/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20231130155323.13259-4-jaslo@ziska.de>","date":"2023-11-30T15:43:10","name":"[libcamera-devel,v2,3/3] gstreamer: Implement renegotiation","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"7c3a67047db78073d55233003041946010122b06","submitter":{"id":173,"url":"https://patchwork.libcamera.org/api/1.1/people/173/?format=json","name":"Jaslo Ziska","email":"jaslo@ziska.de"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19259/mbox/","series":[{"id":4093,"url":"https://patchwork.libcamera.org/api/1.1/series/4093/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4093","date":"2023-11-30T15:43:07","name":"gstreamer: Implement renegotiation","version":2,"mbox":"https://patchwork.libcamera.org/series/4093/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19259/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19259/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 058CCBDE6B\n\tfor ;\n\tThu, 30 Nov 2023 15:54:44 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B03E6629D2;\n\tThu, 30 Nov 2023 16:54:43 +0100 (CET)","from mo4-p00-ob.smtp.rzone.de (mo4-p00-ob.smtp.rzone.de\n\t[81.169.146.160])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 34372629C2\n\tfor ;\n\tThu, 30 Nov 2023 16:54:43 +0100 (CET)","from archlinux.fritz.box by smtp.strato.de (RZmta 49.9.7 AUTH)\n\twith ESMTPSA id n54b84zAUFsg0SD\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits))\n\t(Client did not present a certificate);\n\tThu, 30 Nov 2023 16:54:42 +0100 (CET)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1701359683;\n\tbh=bK7cYUIfRBCf8bf2vdEPvCe1dbC4+QPJg91KvLeMeMs=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=HAfcICaw4AV+Md7vbAo72iqYxuko+VWcEkpgIxoLUTVYmYMltWqWTgTIIg4E1RIWX\n\tjNzSyug8jWRDLY8xmIs9UYU3sxUfNr8xEPQkD65ZKFPdwnAZVVlO18X3HE06e6klcp\n\turUjHgp/j/S8EafHuPxPHdG+vcP0ZzKyMbigdx6dau9s14qY6UIu7+7QFoflwNMDcN\n\tuEwV9Biww4iSFq27793NFzNbjtSzcb2+o8h/2AKPThqe64i4ioh/ymesiGU+i9wGPK\n\tHuE/+kzfGhZ852e+ZdpLkLRIVdugyv8jyQVI6hojdiGocdE9RtA8OxgrcT9J/gkQtS\n\tcw/0rgKi7JpNQ==","v=1; a=rsa-sha256; c=relaxed/relaxed; t=1701359682;\n\ts=strato-dkim-0002; d=ziska.de;\n\th=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Cc:Date:\n\tFrom:Subject:Sender;\n\tbh=ahMdiRm4KDgm5BTPp4irOpCMRK1LsS/UNnMD4ZUT5sM=;\n\tb=p+11Xw4pbRiJBnQcG52c2e+NZJ5xG5fxmvoOoBAANSG/KZ7B5+ijwq5t6w9+PB7t+k\n\t+SnqzqKysa1/wwvGsx19qo3ioEFcZmRSLyemBAJlWaj8WILqqN6hcirZrKubKr+2T9Lr\n\tNIkac4Wsx65oKhfjDEDipIkNYrumhk6ThsJEcOAFntSl+04h21atjeyEwgfWgUMZIM8f\n\tN49Y2KpW5n5kLBa13oKsbpHIiCY7YYxEBk3dVXF9SR4nDAe32fpqjdmpMCLWD4Aw/L5J\n\tvK9bEjKQzhWSuBWbjsHSNCWhrVYGNtr3mX1a5uaFMSe46W4+WgoW1umlWL+F7Jc1vEGn\n\t1uSw==","v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1701359682;\n\ts=strato-dkim-0003; d=ziska.de;\n\th=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Cc:Date:\n\tFrom:Subject:Sender;\n\tbh=ahMdiRm4KDgm5BTPp4irOpCMRK1LsS/UNnMD4ZUT5sM=;\n\tb=tXRj9ez98ASxA+fMl+7vWG1fVzGaprdvyXPis0QQN+3BhyB1dYgMthVjvdIVCTI0bs\n\tC3G/YExiMI5aDngsGSCA=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=ziska.de header.i=@ziska.de\n\theader.b=\"p+11Xw4p\"; \n\tdkim=permerror (0-bit key) header.d=ziska.de header.i=@ziska.de\n\theader.b=\"tXRj9ez9\"; dkim-atps=neutral","ARC-Seal":"i=1; a=rsa-sha256; t=1701359682; cv=none;\n\td=strato.com; s=strato-dkim-0002;\n\tb=eWJzw+qEOpV5dS0q64E3tc8xVjYRbUY6iNxaotOWolp5wmt0VP22jW15CJmgpY1K3l\n\t5Ky5Md4QfoKjJzT5Si8/4/BUjn/JqXroEWyeRGC6MCAVpatf3XRtUIJC8b3Xgrj+G+wz\n\t8BIOjXDUz1NZK6cRjD0iRh+oQKXUchBa7WUgDnBTmy47I/3ilG5hi7heewY6flC8ixE9\n\tSBIJUm5kY8Y6M8M1S/C3Df/tbill3CKfxXhwKyw9vUG7ogwYhF6Mcs1hSRycePWICcEe\n\tLEQ0DTx1IL5bF2scrU3K4okdJHCdJRvQWhTERk0cjv/L3oNcDK3kx31SEwOG+tilo8lg\n\tmGrw==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; t=1701359682;\n\ts=strato-dkim-0002; d=strato.com;\n\th=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Cc:Date:\n\tFrom:Subject:Sender;\n\tbh=ahMdiRm4KDgm5BTPp4irOpCMRK1LsS/UNnMD4ZUT5sM=;\n\tb=pCSzdgUDcsJgOH96buAZOLre9NXWssYgCMvXx1Lt7poPQRHZw7eEGI3jd8EfCsz8FZ\n\tFpLpErBJ+FmVgjELKakqRV3QuG+XJ9zQmEbfN/6xxoUxL6Rd/OnsW6F2fxu6e7yYO+uk\n\tE3NxTllWD8IDvxOv+r3huZCO+MlYhqsXPm1Xk2HcVauT2Bk4deUSlGMv38Tazyrj5usQ\n\tz2IM67qwPLxF2b4J0XG2U1Kb+f6eYTyJvekrOjtQIlKhXa/yKgVTj6wHi4aPgWOhHZqr\n\tRPbUs21PEkje+Wj5I48e4ScQMN4xc/wTjj+dQ1H5FJAdh7SBpTXLyZCbXHhkLF5KFtUm\n\tjtAg==","ARC-Authentication-Results":"i=1; strato.com;\n arc=none;\n dkim=none","X-RZG-CLASS-ID":"mo00","X-RZG-AUTH":"\":Jm0XeU+IYfb0x77LHmrjN5Wlb7TBwusDqIM6Hizy8VdfzvKi4yoFC9cH04q6BfJa07bS3ov6I8QOM/V5x+uTrCUtBrk=\"","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 30 Nov 2023 16:43:10 +0100","Message-ID":"<20231130155323.13259-4-jaslo@ziska.de>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20231130155323.13259-1-jaslo@ziska.de>","References":"<20231130155323.13259-1-jaslo@ziska.de>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Content-Type":"text/plain; charset=\"us-ascii\"","Subject":"[libcamera-devel] [PATCH v2 3/3] gstreamer: Implement renegotiation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jaslo Ziska via libcamera-devel ","Reply-To":"Jaslo Ziska ","Cc":"Jaslo Ziska ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This commit implements renegotiation of the camera configuration and\nsource pad caps. A renegotiation can happen when a downstream element\ndecides to change caps or the pipeline is dynamically changed.\n\nTo handle a renegotiation the GST_FLOW_NOT_NEGOTIATED return value has\nto be handled in GstLibcameraSrcState::processRequest. Otherwise the\ndefault would be to print an error and stop streaming.\nTo archive this in a clean way the if statement is altered into a switch\nstatement which now also has a case for GST_FLOW_NOT_NEGOTIATED.\nIn the case of GST_FLOW_NOT_NEGOTIATED every source pad is checked for\nthe reconfiguration flag with gst_pad_needs_reconfigure which does not\nclear this flag. If at least one pad requested a reconfiguration the\nfunction returns without an error and the renegotiation will happen\nlater in the running task. If no pad requested a reconfiguration then\nthe function will return with an error.\n\nIn gst_libcamera_src_task_run the source pads are checked for the\nreconfigure flag by calling gst_pad_check_reconfigure and if one pad\nreturns true and the caps are not sufficient anymore then the\nnegotiation is triggered. It is fine to trigger the negotiation after\nonly a single pad returns true for gst_pad_check_reconfigure because the\nreconfigure flags are cleared in the gst_libcamera_src_negotiate\nfunction.\nIf any pad requested a reconfiguration the following will happen:\n1. The camera is stopped because changing the configuration may not\n happen while running.\n2. The completedRequests queue will be cleared by calling\n GstLibcameraSrcState::clearRequests because the completed buffers\n have the wrong configuration.\n3. The new caps are negotiated by calling gst_libcamera_src_negotiate.\n When the negotiation fails streaming will stop.\n4. The camera is started again.\n\nSigned-off-by: Jaslo Ziska \n---\n src/gstreamer/gstlibcamerasrc.cpp | 72 ++++++++++++++++++++++++++-----\n 1 file changed, 61 insertions(+), 11 deletions(-)","diff":"diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp\nindex d448a750..8bd32306 100644\n--- a/src/gstreamer/gstlibcamerasrc.cpp\n+++ b/src/gstreamer/gstlibcamerasrc.cpp\n@@ -11,7 +11,6 @@\n * - Implement GstElement::send_event\n * + Allowing application to use FLUSH/FLUSH_STOP\n * + Prevent the main thread from accessing streaming thread\n- * - Implement renegotiation (even if slow)\n * - Implement GstElement::request-new-pad (multi stream)\n * + Evaluate if a single streaming thread is fine\n * - Add application driven request (snapshot)\n@@ -302,18 +301,42 @@ int GstLibcameraSrcState::processRequest()\n \t\t\t\t\t\t\tsrcpad, ret);\n \t}\n \n-\tif (ret != GST_FLOW_OK) {\n-\t\tif (ret == GST_FLOW_EOS) {\n-\t\t\tg_autoptr(GstEvent) eos = gst_event_new_eos();\n-\t\t\tguint32 seqnum = gst_util_seqnum_next();\n-\t\t\tgst_event_set_seqnum(eos, seqnum);\n-\t\t\tfor (GstPad *srcpad : srcpads_)\n-\t\t\t\tgst_pad_push_event(srcpad, gst_event_ref(eos));\n-\t\t} else if (ret != GST_FLOW_FLUSHING) {\n-\t\t\tGST_ELEMENT_FLOW_ERROR(src_, ret);\n+\tswitch (ret) {\n+\tcase GST_FLOW_OK:\n+\t\tbreak;\n+\tcase GST_FLOW_NOT_NEGOTIATED: {\n+\t\tbool reconfigure = false;\n+\t\tfor (GstPad *srcpad : srcpads_) {\n+\t\t\tif (gst_pad_needs_reconfigure(srcpad)) {\n+\t\t\t\treconfigure = true;\n+\t\t\t\tbreak;\n+\t\t\t}\n \t\t}\n \n-\t\treturn -EPIPE;\n+\t\t// if no pads needs a reconfiguration something went wrong\n+\t\tif (!reconfigure)\n+\t\t\terr = -EPIPE;\n+\n+\t\tbreak;\n+\t}\n+\tcase GST_FLOW_EOS: {\n+\t\tg_autoptr(GstEvent) eos = gst_event_new_eos();\n+\t\tguint32 seqnum = gst_util_seqnum_next();\n+\t\tgst_event_set_seqnum(eos, seqnum);\n+\t\tfor (GstPad *srcpad : srcpads_)\n+\t\t\tgst_pad_push_event(srcpad, gst_event_ref(eos));\n+\n+\t\terr = -EPIPE;\n+\t\tbreak;\n+\t}\n+\tcase GST_FLOW_FLUSHING:\n+\t\terr = -EPIPE;\n+\t\tbreak;\n+\tdefault:\n+\t\tGST_ELEMENT_FLOW_ERROR(src_, ret);\n+\n+\t\terr = -EPIPE;\n+\t\tbreak;\n \t}\n \n \treturn err;\n@@ -463,6 +486,8 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self)\n \t\t\t\t\t G_CALLBACK(gst_task_resume), self->task);\n \n \t\tgst_libcamera_pad_set_pool(srcpad, pool);\n+\n+\t\tgst_pad_check_reconfigure(srcpad); // clear all reconfigure flags\n \t}\n \n \treturn true;\n@@ -496,6 +521,31 @@ gst_libcamera_src_task_run(gpointer user_data)\n \t\treturn;\n \t}\n \n+\t// check if a srcpad requested a renegotiation\n+\tbool reconfigure = false;\n+\tfor (GstPad *srcpad : state->srcpads_) {\n+\t\tif (gst_pad_check_reconfigure(srcpad)) {\n+\t\t\t// check whether the caps even need changing\n+\t\t\tg_autoptr(GstCaps) caps = gst_pad_get_current_caps(srcpad);\n+\t\t\tif (!gst_pad_peer_query_accept_caps(srcpad, caps)) {\n+\t\t\t\treconfigure = true;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\tif (reconfigure) {\n+\t\tstate->cam_->stop();\n+\t\tstate->clearRequests();\n+\n+\t\tif (!gst_libcamera_src_negotiate(self)) {\n+\t\t\tGST_ELEMENT_FLOW_ERROR(self, GST_FLOW_NOT_NEGOTIATED);\n+\t\t\tgst_task_stop(self->task);\n+\t\t}\n+\n+\t\tstate->cam_->start(&state->initControls_);\n+\t}\n+\n \t/*\n \t * Create and queue one request. If no buffers are available the\n \t * function returns -ENOBUFS, which we ignore here as that's not a\n","prefixes":["libcamera-devel","v2","3/3"]}