[{"id":32815,"web_url":"https://patchwork.libcamera.org/comment/32815/","msgid":"","date":"2024-12-17T13:41:43","subject":"Re: [PATCH] pipeline: simple: Use proper device for frame start\n\tevents","submitter":{"id":102,"url":"https://patchwork.libcamera.org/api/people/102/","name":"Hans de Goede","email":"hdegoede@redhat.com"},"content":"Hi Stanislaw,\n\nThank you for your patch.\n\nOn 17-Dec-24 11:52 AM, Stanislaw Gruszka wrote:\n> Currently we use frame start event from video capture device to\n> apply controls. But the capture device might not generate the events.\n> Usually CSI-2 receiver is proper device to subscribe for start\n> frame events.\n> \n> Without DelayedConntrols:applyControls() is possible that we can get\n> call to DelayedControls::get() with frame number that exceed number\n> of saved entries and get below assertion failure:\n> \n> ../src/libcamera/delayed_controls.cpp:227:\n> libcamera::ControlList libcamera::DelayedControls::get(uint32_t):\n> Assertion `info.type() != ControlTypeNone' failed\n> \n> Assertion failure can happen at the beginning of streaming when\n> ControlRingBuffer is not yet filled and there are errors on CSI-2.\n> \n> To fix, loop over devices in the pipeline (starting from the last one),\n> find one that emits start frame events and connect applyControls()\n> to it.\n> \n> Bug: https://bugs.libcamera.org/show_bug.cgi?id=241\n> Signed-off-by: Stanislaw Gruszka \n\nI noticed the issue with connecting to the framestart signal of\nthe /dev/video# capture node which cannot work myself too and I was\nabout to post this patch for this:\n\nhttps://github.com/jwrdegoede/libcamera/commit/5abbf43118e80bb4be6b893a6e5e28c65b59744a\n\nBut your solution is obviously better.\n\nNote that the simple-pipelinehandler already contains a workaroud for this\nin the form of setControls() directly applying the controls rather then\nonly pushing them into delayedCtrls_.\n\nThat workaround needs to be disabled in your patch when the frameStart signal\nis used successfully.\n\nSee my attached patch with suggested changes to your patch.\n\nI have successfully tested this with the suggested changes:\n\nTested-by: Hans de Goede \n\n> ---\n> src/libcamera/pipeline/simple/simple.cpp | 26 ++++++++++++++++++++++--\n> 1 file changed, 24 insertions(+), 2 deletions(-)\n> \n> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> index 8ac24e6e..52f3d520 100644\n> --- a/src/libcamera/pipeline/simple/simple.cpp\n> +++ b/src/libcamera/pipeline/simple/simple.cpp\n> @@ -382,6 +382,7 @@ private:\n> \tstd::map entities_;\n> \n> \tMediaDevice *converter_;\n> +\tV4L2Subdevice *eventEmitter_;\n> \tbool swIspEnabled_;\n> };\n> \n\nThe simplepipeline handler can register and potentially operate multiple\ncameras at the same time, so this needs to be part of SimpleCameraData.\n\n\n> @@ -1299,8 +1300,6 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n> \tdata->delayedCtrls_ =\n> \t\tstd::make_unique(data->sensor_->device(),\n> \t\t\t\t\t\t params);\n> -\tdata->video_->frameStart.connect(data->delayedCtrls_.get(),\n> -\t\t\t\t\t &DelayedControls::applyControls);\n> \n> \tStreamConfiguration inputCfg;\n> \tinputCfg.pixelFormat = pipeConfig->captureFormat;\n> @@ -1368,6 +1367,23 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL\n> \n> \tvideo->bufferReady.connect(data, &SimpleCameraData::imageBufferReady);\n> \n> +\t/*\n> +\t * Enable frame start event on last device in the pipeline\n> +\t * that provides the events.\n> +\t */\n> +\tfor (auto it = data->entities_.rbegin(); it != data->entities_.rend(); ++it) {\n> +\t\tV4L2Subdevice *sd = subdev(it->entity);\n> +\t\tif (!sd)\n> +\t\t\tcontinue;\n> +\t\tif (sd->setFrameStartEnabled(true) < 0)\n> +\t\t\tcontinue;\n> +\n\nIMHO it would be good to have a debug log here which entity is used for the frameStart\nsignal (see attached suggested changes).\n\n> +\t\tsd->frameStart.connect(data->delayedCtrls_.get(),\n> +\t\t\t\t &DelayedControls::applyControls);\n> +\t\teventEmitter_ = sd;\n> +\t\tbreak;\n> +\t}\n> +\n> \tret = video->streamOn();\n> \tif (ret < 0) {\n> \t\tstop(camera);\n> @@ -1400,6 +1416,12 @@ void SimplePipelineHandler::stopDevice(Camera *camera)\n> \tSimpleCameraData *data = cameraData(camera);\n> \tV4L2VideoDevice *video = data->video_;\n> \n> +\tif (eventEmitter_) {\n> +\t\teventEmitter_->setFrameStartEnabled(false);\n> +\t\teventEmitter_->frameStart.disconnect(data->delayedCtrls_.get(),\n> +\t\t\t\t\t\t &DelayedControls::applyControls);\n\nThis should also et eventEmitter_ to NULL, in case we fail to get it again\nwhen re-starting the stream.\n\n\n> +\t}\n> +\n> \tif (data->useConversion_) {\n> \t\tif (data->converter_)\n> \t\t\tdata->converter_->stop();\n\n\nA semi related issue is that atm data->delayedCtrls_->reset(); is not called\non pipeline start() so on a second start of the same camera there will be\nstale values in the delayedCtrls class and/or things will outright not\nwork because of the writeQueue internal counter being out of sync.\n\nRegards,\n\nHans","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 48767C32F6\n\tfor ;\n\tTue, 17 Dec 2024 13:41:55 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 3909567FCC;\n\tTue, 17 Dec 2024 14:41:54 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D123567F0D\n\tfor ;\n\tTue, 17 Dec 2024 14:41:52 +0100 (CET)","from mail-ed1-f70.google.com (mail-ed1-f70.google.com\n\t[209.85.208.70]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n\tus-mta-135-gNun_zm5PBO9AbW7xbcXRA-1; Tue, 17 Dec 2024 08:41:50 -0500","by mail-ed1-f70.google.com with SMTP id\n\t4fb4d7f45d1cf-5d121a79ad0so5596800a12.2\n\tfor ;\n\tTue, 17 Dec 2024 05:41:49 -0800 (PST)","from [10.87.1.157] ([145.15.244.207])\n\tby smtp.gmail.com with ESMTPSA id\n\t4fb4d7f45d1cf-5d652f351ddsm4474110a12.77.2024.12.17.05.41.44\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tTue, 17 Dec 2024 05:41:47 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"eKIUoIcy\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1734442911;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=RuMWkp7rxdU7WucLakHpWZCiCgstxixJByLu/x5axkE=;\n\tb=eKIUoIcy0v+cRi7wWVaVWYwFffzxSeb6dznaR3lM3gGqtkhLNeMGWNDe5yxQPdof/ST/5l\n\thJscvMH9tesV59B/OVjkq0eD1dGAtuYxnzV5i73Kn/X/LkrTkYTuPnSeMTJWvIU4d5FEgk\n\tMa2H7/63BZ0itvY1ddFIwzIoWNxqSHk=","X-MC-Unique":"gNun_zm5PBO9AbW7xbcXRA-1","X-Mimecast-MFC-AGG-ID":"gNun_zm5PBO9AbW7xbcXRA","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1734442909; x=1735047709;\n\th=in-reply-to:from:content-language:references:cc:to:subject\n\t:user-agent:mime-version:date:message-id:x-gm-message-state:from:to\n\t:cc:subject:date:message-id:reply-to;\n\tbh=p3y/rJpmAaxZOdE+MpJOB5mNi8VaOtTsYh8P3vRLjFE=;\n\tb=H+gA8acPJQnpnmIKT4uBx+KJKfLf7c+adMPU7LHLqkTHP9LZCBIR2FcJeV+qLG+jBR\n\tfk76KQhd95MFQ3WYnylbxZb7UFyhLG05h3O51L9RdTxeFClYBfPYpZ19cCFY2MyEunXB\n\tdppuiFDXJp5xx/d56T4CkghO9jdX2pQk3HMKX8kYlTWyoW0suOsuso16+5GYl5ySnU2N\n\t+ajgo12cjV4WPX/koE5ikAsAPw13La1YDCv2pqZpSF+20AcQdxfQ1fHGcJiYt1Tt62yW\n\tV3AReSgJBOn0UrsV+Ef5jk9FEy8dTNIP9454yKcGwBnes1A4lADmLSqSgzJXZLu78dFl\n\thuhw==","X-Forwarded-Encrypted":"i=1;\n\tAJvYcCW7vHXlb86/+KQ69OHj7uSni91CEYQm6sNUDByU1/TSwAiLTTrY3nUwtE2v7lJBsPsaHPAJdDg2zdkMYVKRUSQ=@lists.libcamera.org","X-Gm-Message-State":"AOJu0YzZh/w60DW1rl3zuPTNKIsHXwVg1aSkUaqFSZmRLjL+/9Rgb40g\n\tJWSsm4YyPr62i/HHyEvigYGwY/Hm+LsbAQnAarsITBGgpH1D9ggx0Mv9Qmefd5Dc/qhu0SvUlD6\n\tIrrzCReINPoAwvwRvTN8g9S0SNiDVA0LAtCgibetvkK12Uo3WjzeBtis7PWaLr1AgVHFhJPM=","X-Gm-Gg":"ASbGncsTGK2h9kALrqEzO9/lg65SqASmYbIJKwzdYwzPfffbe6uIr4dqKs3nedKHmp5\n\tUrbQd6M4LxNn7i4lqqpxSd8F3S5p/BltL4Wg11sfcR2ISkHQ1AEC+tTydhisnapCwjyS8hqKO3Y\n\tu8biOpCNs7K4Ke54kaShOy/f/Yz1FYcfP0aZZh0+apXX0FVQhnBLhH6poAmtTz0aIuP0WxuFhUJ\n\tY03jbNSqXfQdMmBjM3aYp/Km+TsYFWUuDHUG2oxjCAXX1ZxfUSkzB+zuRA=","X-Received":["by 2002:a05:6402:5303:b0:5d3:cff2:71a9 with SMTP id\n\t4fb4d7f45d1cf-5d7d410379cmr3393275a12.33.1734442908828; \n\tTue, 17 Dec 2024 05:41:48 -0800 (PST)","by 2002:a05:6402:5303:b0:5d3:cff2:71a9 with SMTP id\n\t4fb4d7f45d1cf-5d7d410379cmr3393251a12.33.1734442908348; \n\tTue, 17 Dec 2024 05:41:48 -0800 (PST)"],"X-Google-Smtp-Source":"AGHT+IGcUPSPmIjYI/nVW+fnk7bMRTZZ+w+nuYHH8pHQnwn9MxJwfvciKlBTbevNXVv/pXSWI4f5Hg==","Message-ID":"","Date":"Tue, 17 Dec 2024 14:41:43 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH] pipeline: simple: Use proper device for frame start\n\tevents","To":"Stanislaw Gruszka ,\n\tlibcamera-devel@lists.libcamera.org","Cc":"Milan Zamazal ,\n\tLaurent Pinchart ,\n\tKieran Bingham ,\n\tNaushir Patuck ,\n\tSakari Ailus ","References":"<20241217105206.357495-1-stanislaw.gruszka@linux.intel.com>","From":"Hans de Goede ","In-Reply-To":"<20241217105206.357495-1-stanislaw.gruszka@linux.intel.com>","X-Mimecast-Spam-Score":"0","X-Mimecast-MFC-PROC-ID":"uDZz7vaNrx_NR6ATIkL2oArocydV9k5DVKBmsHJHfgE_1734442909","X-Mimecast-Originator":"redhat.com","Content-Type":"multipart/mixed;\n\tboundary=\"------------p5S0ihinDhTEBBek5rX9e8FL\"","Content-Language":"en-US","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32816,"web_url":"https://patchwork.libcamera.org/comment/32816/","msgid":"","date":"2024-12-17T13:43:49","subject":"Re: [PATCH] pipeline: simple: Use proper device for frame start\n\tevents","submitter":{"id":102,"url":"https://patchwork.libcamera.org/api/people/102/","name":"Hans de Goede","email":"hdegoede@redhat.com"},"content":"Hi,\n\nOn 17-Dec-24 11:52 AM, Stanislaw Gruszka wrote:\n> Currently we use frame start event from video capture device to\n> apply controls. But the capture device might not generate the events.\n> Usually CSI-2 receiver is proper device to subscribe for start\n> frame events.\n> \n> Without DelayedConntrols:applyControls() is possible that we can get\n> call to DelayedControls::get() with frame number that exceed number\n> of saved entries and get below assertion failure:\n> \n> ../src/libcamera/delayed_controls.cpp:227:\n> libcamera::ControlList libcamera::DelayedControls::get(uint32_t):\n> Assertion `info.type() != ControlTypeNone' failed\n> \n> Assertion failure can happen at the beginning of streaming when\n> ControlRingBuffer is not yet filled and there are errors on CSI-2.\n> \n> To fix, loop over devices in the pipeline (starting from the last one),\n> find one that emits start frame events and connect applyControls()\n> to it.\n> \n> Bug: https://bugs.libcamera.org/show_bug.cgi?id=241\n> Signed-off-by: Stanislaw Gruszka \n\np.s.\n\nWith this patch in place we can also introduce a custom frameStart handler,\nrather then directly using delayedCtrls_->apply() as handler and also\nget a timestamp from the framestart handler fixing the todo comment\naround line 832 of simple.cpp.\n\nThis could/should be done in a follow-up patch to this one.\n\nRegards,\n\nHans\n\n\n> ---\n> src/libcamera/pipeline/simple/simple.cpp | 26 ++++++++++++++++++++++--\n> 1 file changed, 24 insertions(+), 2 deletions(-)\n> \n> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> index 8ac24e6e..52f3d520 100644\n> --- a/src/libcamera/pipeline/simple/simple.cpp\n> +++ b/src/libcamera/pipeline/simple/simple.cpp\n> @@ -382,6 +382,7 @@ private:\n> \tstd::map entities_;\n> \n> \tMediaDevice *converter_;\n> +\tV4L2Subdevice *eventEmitter_;\n> \tbool swIspEnabled_;\n> };\n> \n> @@ -1299,8 +1300,6 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n> \tdata->delayedCtrls_ =\n> \t\tstd::make_unique(data->sensor_->device(),\n> \t\t\t\t\t\t params);\n> -\tdata->video_->frameStart.connect(data->delayedCtrls_.get(),\n> -\t\t\t\t\t &DelayedControls::applyControls);\n> \n> \tStreamConfiguration inputCfg;\n> \tinputCfg.pixelFormat = pipeConfig->captureFormat;\n> @@ -1368,6 +1367,23 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL\n> \n> \tvideo->bufferReady.connect(data, &SimpleCameraData::imageBufferReady);\n> \n> +\t/*\n> +\t * Enable frame start event on last device in the pipeline\n> +\t * that provides the events.\n> +\t */\n> +\tfor (auto it = data->entities_.rbegin(); it != data->entities_.rend(); ++it) {\n> +\t\tV4L2Subdevice *sd = subdev(it->entity);\n> +\t\tif (!sd)\n> +\t\t\tcontinue;\n> +\t\tif (sd->setFrameStartEnabled(true) < 0)\n> +\t\t\tcontinue;\n> +\n> +\t\tsd->frameStart.connect(data->delayedCtrls_.get(),\n> +\t\t\t\t &DelayedControls::applyControls);\n> +\t\teventEmitter_ = sd;\n> +\t\tbreak;\n> +\t}\n> +\n> \tret = video->streamOn();\n> \tif (ret < 0) {\n> \t\tstop(camera);\n> @@ -1400,6 +1416,12 @@ void SimplePipelineHandler::stopDevice(Camera *camera)\n> \tSimpleCameraData *data = cameraData(camera);\n> \tV4L2VideoDevice *video = data->video_;\n> \n> +\tif (eventEmitter_) {\n> +\t\teventEmitter_->setFrameStartEnabled(false);\n> +\t\teventEmitter_->frameStart.disconnect(data->delayedCtrls_.get(),\n> +\t\t\t\t\t\t &DelayedControls::applyControls);\n> +\t}\n> +\n> \tif (data->useConversion_) {\n> \t\tif (data->converter_)\n> \t\t\tdata->converter_->stop();","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C2193BD1F1\n\tfor ;\n\tTue, 17 Dec 2024 13:43:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B9B9D67FCC;\n\tTue, 17 Dec 2024 14:43:56 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0993967F0D\n\tfor ;\n\tTue, 17 Dec 2024 14:43:54 +0100 (CET)","from mail-ed1-f70.google.com (mail-ed1-f70.google.com\n\t[209.85.208.70]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n\tus-mta-331-k3tZdu0GPrihsJjhMOtbdg-1; Tue, 17 Dec 2024 08:43:53 -0500","by mail-ed1-f70.google.com with SMTP id\n\t4fb4d7f45d1cf-5d3cef3ed56so4704481a12.1\n\tfor ;\n\tTue, 17 Dec 2024 05:43:52 -0800 (PST)","from [10.87.1.157] ([145.15.244.207])\n\tby smtp.gmail.com with ESMTPSA id\n\t4fb4d7f45d1cf-5d7cf1f0f56sm1375386a12.20.2024.12.17.05.43.49\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tTue, 17 Dec 2024 05:43:50 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"aR5GcKEn\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1734443034;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tcontent-transfer-encoding:content-transfer-encoding:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=mLFKTFI+rxXr5m+tGS5NQnKF2INL3rBNmhXusFFvILc=;\n\tb=aR5GcKEnNpkCdnrMgHAAetZXXOdX51qjfs/9xCcqRMKf1uSC5uhvAq8Vj+0SuNKNTOMPF2\n\t6PKIbEx0gI/CH6Eo6kP9Zujf9OBNWI5Qwmt9KXEp2PBAN+cq9uiU/6dM4LYOkwP1qm0kLN\n\tSwABuC01ak48EC1FueZaIbWZCfaYZYY=","X-MC-Unique":"k3tZdu0GPrihsJjhMOtbdg-1","X-Mimecast-MFC-AGG-ID":"k3tZdu0GPrihsJjhMOtbdg","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1734443032; x=1735047832;\n\th=content-transfer-encoding:in-reply-to:from:content-language\n\t:references:cc:to:subject:user-agent:mime-version:date:message-id\n\t:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;\n\tbh=mLFKTFI+rxXr5m+tGS5NQnKF2INL3rBNmhXusFFvILc=;\n\tb=ls1/g0c56HkL3K7l6n2Z0yQTpT7rlz6EH0Tjyb1r9/a6hPlN5gLy/pFNXpmhh9BEoS\n\tdhJ3fw5kTnHHaLdu0uQES6zYpIZVQlZq4PSfR4DiZ4tdCdFdqxdc5BVOMbybq0SwW+cG\n\t5doaEZVOXqhui6iMuc4tWyCD/PySQqPnWiFB8KL1HAyyiZGI998x9ZuETx2huBYoDGa+\n\tP/BeJEtwrigapC5FWMjvRVs1k4Dl3sVBX9BxH1T7Ck0zUB1XtZvFH8xmS7SgmzE3lR+I\n\thaF3/PLIeXgOuOUPSc0wp8FXs3qslDSe8PluhzVL1LshX2qdC8e95oT0/9HGi6LkFeej\n\t1X4g==","X-Forwarded-Encrypted":"i=1;\n\tAJvYcCVDHFQW0ENfbilpokR+78+8qeOqVvEmD12DhMcYkJQJNFr3yBuKi0mAzzlisQ0uJagxvmmlFefpITiXNppFXHk=@lists.libcamera.org","X-Gm-Message-State":"AOJu0Yxhi2l1inFLOpAxrDQLEwf0j/IQ20XQra+RGGPrHcVXE/X/r8MX\n\tm7yFBIxVzYbF+QfJkUZzupw/NPVB5TAo/Fq1j9C4kbZ/2EohkxpBE3F5XNQMPc4qXw0+XVGBcLJ\n\t7JByTd9HHslRXmkGM10gjvw3qMQ6/9Ge3NBcQ7qJpJ9yjQgivpPcfyTwgh2hd3MYVKpnerUk=","X-Gm-Gg":"ASbGncu2LkgPwLpr9Y8mLF0Pg6yFb0M2Qikm4HdOydmTrjOPWzCDYJvnu0SDv85MFn7\n\tFdYnY7g3nsOtPnGizcCg4fzMi/udzp2j99T8NGGtBP0IbA+kwXjsglSruGic/2KiUHLykWGQYq4\n\tvV5sCFUkcU6JajwcPLB5wyukEbMncQkvpp9Ctz/OYeA5GTyAeFbrcFmnAIXh4jFaWFtNTzD2ql7\n\t8FtAYACx8kav9TzhAtnAl/bXL0bzqao8G5bkSAGrGobjeGrojjPRZeZ","X-Received":["by 2002:a05:6402:3603:b0:5d0:f830:ef4e with SMTP id\n\t4fb4d7f45d1cf-5d63c3c1ca6mr15197341a12.34.1734443031587; \n\tTue, 17 Dec 2024 05:43:51 -0800 (PST)","by 2002:a05:6402:3603:b0:5d0:f830:ef4e with SMTP id\n\t4fb4d7f45d1cf-5d63c3c1ca6mr15197312a12.34.1734443031169; \n\tTue, 17 Dec 2024 05:43:51 -0800 (PST)"],"X-Google-Smtp-Source":"AGHT+IEkTn93GKrztx9BFS1oU640TitR4CwFDxdbZC2H+7Esbp2JCJMUV3pv8y39E64Co+qCtpEV4g==","Message-ID":"","Date":"Tue, 17 Dec 2024 14:43:49 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH] pipeline: simple: Use proper device for frame start\n\tevents","To":"Stanislaw Gruszka ,\n\tlibcamera-devel@lists.libcamera.org","Cc":"Milan Zamazal ,\n\tLaurent Pinchart ,\n\tKieran Bingham ,\n\tNaushir Patuck ,\n\tSakari Ailus ","References":"<20241217105206.357495-1-stanislaw.gruszka@linux.intel.com>","From":"Hans de Goede ","In-Reply-To":"<20241217105206.357495-1-stanislaw.gruszka@linux.intel.com>","X-Mimecast-Spam-Score":"0","X-Mimecast-MFC-PROC-ID":"gKXPLiziwlEzMyMhb5G7bvsKi4H0gn-aRP-9dRpEcPU_1734443032","X-Mimecast-Originator":"redhat.com","Content-Language":"en-US","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32872,"web_url":"https://patchwork.libcamera.org/comment/32872/","msgid":"","date":"2024-12-18T13:47:27","subject":"Re: [PATCH] pipeline: simple: Use proper device for frame start\n\tevents","submitter":{"id":211,"url":"https://patchwork.libcamera.org/api/people/211/","name":"Stanislaw Gruszka","email":"stanislaw.gruszka@linux.intel.com"},"content":"Hi Hans,\n\nOn Tue, Dec 17, 2024 at 02:41:43PM +0100, Hans de Goede wrote:\n> On 17-Dec-24 11:52 AM, Stanislaw Gruszka wrote:\n> > Currently we use frame start event from video capture device to\n> > apply controls. But the capture device might not generate the events.\n> > Usually CSI-2 receiver is proper device to subscribe for start\n> > frame events.\n> > \n> > Without DelayedConntrols:applyControls() is possible that we can get\n> > call to DelayedControls::get() with frame number that exceed number\n> > of saved entries and get below assertion failure:\n> > \n> > ../src/libcamera/delayed_controls.cpp:227:\n> > libcamera::ControlList libcamera::DelayedControls::get(uint32_t):\n> > Assertion `info.type() != ControlTypeNone' failed\n> > \n> > Assertion failure can happen at the beginning of streaming when\n> > ControlRingBuffer is not yet filled and there are errors on CSI-2.\n> > \n> > To fix, loop over devices in the pipeline (starting from the last one),\n> > find one that emits start frame events and connect applyControls()\n> > to it.\n> > \n> > Bug: https://bugs.libcamera.org/show_bug.cgi?id=241\n> > Signed-off-by: Stanislaw Gruszka \n> \n> I noticed the issue with connecting to the framestart signal of\n> the /dev/video# capture node which cannot work myself too and I was\n> about to post this patch for this:\n> \n> https://github.com/jwrdegoede/libcamera/commit/5abbf43118e80bb4be6b893a6e5e28c65b59744a\n> \n> But your solution is obviously better.\n> \n> Note that the simple-pipelinehandler already contains a workaroud for this\n> in the form of setControls() directly applying the controls rather then\n> only pushing them into delayedCtrls_.\n> \n> That workaround needs to be disabled in your patch when the frameStart signal\n> is used successfully.\n> \n> See my attached patch with suggested changes to your patch.\n\nAll good points, I'll integrate your changes into v2 and repost.\n\nThanks\nStanislaw","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2693FC3301\n\tfor ;\n\tWed, 18 Dec 2024 13:47:37 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EDDC96809F;\n\tWed, 18 Dec 2024 14:47:35 +0100 (CET)","from mgamail.intel.com (mgamail.intel.com [198.175.65.15])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id F008167F59\n\tfor ;\n\tWed, 18 Dec 2024 14:47:33 +0100 (CET)","from fmviesa008.fm.intel.com ([10.60.135.148])\n\tby orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n\t18 Dec 2024 05:47:33 -0800","from sgruszka-mobl.ger.corp.intel.com (HELO localhost)\n\t([10.246.8.237]) by fmviesa008-auth.fm.intel.com with\n\tESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Dec 2024 05:47:30 -0800"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=intel.com header.i=@intel.com\n\theader.b=\"LjYOYMTm\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n\tt=1734529655; x=1766065655;\n\th=date:from:to:cc:subject:message-id:references:\n\tmime-version:in-reply-to;\n\tbh=UhRjnogiLPEc1ARgPSuvNlcljzOhrQwRqK+izPyVw/c=;\n\tb=LjYOYMTmuQjxGyS2vwzbuD7/ckI+gVpu0mPekpEBM18ytmVXA/III98U\n\tR6CFh+cJsjdJCgeAEBrJMkhV6wwp6tlrXv0jMJcL1TNtkusBlgp7PdlKl\n\t15l08iHhbKSFnZkb3mZU6Fa36umCa0ROZ/VjmjyS0os7/AsXBjYLOlJL1\n\tbRwBgjqZuIDSSew3CS05vp25d6sPnDI7tkfqPpVR4wgsZ5w2UvyqJrRy4\n\tWc7KaxxXReerkUlChOFxZdX+DyUz0YrEzA2wFuzMcvVnJa5yFSPWhkOOy\n\tXJpazceop7nZ4KHzq9euYhf6wPNqlQo359jfGsjYQlBfpIZgFL86HpkOD g==;","X-CSE-ConnectionGUID":["AW8bJffZTAysGAM5IYpVhQ==","+Sm3bE3hSvqDWw33SLJPZQ=="],"X-CSE-MsgGUID":["/1LosngJS9SQPIyskJZQyQ==","1kfJoWEFR5yY9QpQr9ByVA=="],"X-IronPort-AV":["E=McAfee;i=\"6700,10204,11290\"; a=\"38679053\"","E=Sophos;i=\"6.12,244,1728975600\"; d=\"scan'208\";a=\"38679053\"","E=Sophos;i=\"6.12,244,1728975600\"; d=\"scan'208\";a=\"98099380\""],"X-ExtLoop1":"1","Date":"Wed, 18 Dec 2024 14:47:27 +0100","From":"Stanislaw Gruszka ","To":"Hans de Goede ","Cc":"libcamera-devel@lists.libcamera.org, Milan Zamazal ,\n\tLaurent Pinchart ,\n\tKieran Bingham ,\n\tNaushir Patuck ,\n\tSakari Ailus ","Subject":"Re: [PATCH] pipeline: simple: Use proper device for frame start\n\tevents","Message-ID":"","References":"<20241217105206.357495-1-stanislaw.gruszka@linux.intel.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32874,"web_url":"https://patchwork.libcamera.org/comment/32874/","msgid":"","date":"2024-12-18T14:11:16","subject":"Re: [PATCH] pipeline: simple: Use proper device for frame start\n\tevents","submitter":{"id":211,"url":"https://patchwork.libcamera.org/api/people/211/","name":"Stanislaw Gruszka","email":"stanislaw.gruszka@linux.intel.com"},"content":"Hi\n\nOn Tue, Dec 17, 2024 at 02:43:49PM +0100, Hans de Goede wrote:\n> On 17-Dec-24 11:52 AM, Stanislaw Gruszka wrote:\n> > Currently we use frame start event from video capture device to\n> > apply controls. But the capture device might not generate the events.\n> > Usually CSI-2 receiver is proper device to subscribe for start\n> > frame events.\n> > \n> > Without DelayedConntrols:applyControls() is possible that we can get\n> > call to DelayedControls::get() with frame number that exceed number\n> > of saved entries and get below assertion failure:\n> > \n> > ../src/libcamera/delayed_controls.cpp:227:\n> > libcamera::ControlList libcamera::DelayedControls::get(uint32_t):\n> > Assertion `info.type() != ControlTypeNone' failed\n> > \n> > Assertion failure can happen at the beginning of streaming when\n> > ControlRingBuffer is not yet filled and there are errors on CSI-2.\n> > \n> > To fix, loop over devices in the pipeline (starting from the last one),\n> > find one that emits start frame events and connect applyControls()\n> > to it.\n> > \n> > Bug: https://bugs.libcamera.org/show_bug.cgi?id=241\n> > Signed-off-by: Stanislaw Gruszka \n> \n> p.s.\n> \n> With this patch in place we can also introduce a custom frameStart handler,\n> rather then directly using delayedCtrls_->apply() as handler and also\n> get a timestamp from the framestart handler fixing the todo comment\n> around line 832 of simple.cpp.\n> \n> This could/should be done in a follow-up patch to this one.\n\nI'm going to do this. Will post the change after this one\nget applied.\n\nRegards\nStanislaw","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 82886C32FE\n\tfor ;\n\tWed, 18 Dec 2024 14:11:25 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 269AB6809F;\n\tWed, 18 Dec 2024 15:11:25 +0100 (CET)","from mgamail.intel.com (mgamail.intel.com [192.198.163.11])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4396267F59\n\tfor ;\n\tWed, 18 Dec 2024 15:11:22 +0100 (CET)","from fmviesa007.fm.intel.com ([10.60.135.147])\n\tby fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n\t18 Dec 2024 06:11:20 -0800","from sgruszka-mobl.ger.corp.intel.com (HELO localhost)\n\t([10.246.8.237]) by fmviesa007-auth.fm.intel.com with\n\tESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Dec 2024 06:11:18 -0800"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=intel.com header.i=@intel.com\n\theader.b=\"ikTJLk5F\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n\tt=1734531082; x=1766067082;\n\th=date:from:to:cc:subject:message-id:references:\n\tmime-version:in-reply-to;\n\tbh=DFf6jvNLdADD3UyY8CFcZy3meBC+8vUO/C7ahXcsjRc=;\n\tb=ikTJLk5FkJcRwcfWxxvNpnj9LCew73SuX0oUOq/P3bMtTNAOtblnU6hx\n\tQirvgc8tFO5gtAiyde8esG6m8KvJ/DWoRnIussrHK1tDpvdhsuklb4xKo\n\tOjlFlR9+q4UWMypJy3d5FiEvjZdHuvDVcY3xTrcjnrBx/XTORm6LXlfTp\n\tJ9JgHaEd+XADWhDf2CziZ+LB2rfCoKb1+awXHBpU/kJdIbLgsq47ON44C\n\tb8mSR/tq7TpdsfOTYeXzhT/GzmkehJJlVoL4XExglIiZ1mgiGdQJbCsvk\n\tEtGb17v5UGvGaSczeA0SHW+jVhOJzv/Z2nzTyEEs1tNziyDVl+QwMs+Fz A==;","X-CSE-ConnectionGUID":["t98C9PGmSoOX5ta0pO5zdA==","bTl06wwrQUm3E0F2AdBnCw=="],"X-CSE-MsgGUID":["/vN3utruT5CHjymn7B07xQ==","i8HFgQIiSQiFSSkt6kYnLA=="],"X-IronPort-AV":["E=McAfee;i=\"6700,10204,11290\"; a=\"45604701\"","E=Sophos;i=\"6.12,244,1728975600\"; d=\"scan'208\";a=\"45604701\"","E=Sophos;i=\"6.12,244,1728975600\"; d=\"scan'208\";a=\"97722340\""],"X-ExtLoop1":"1","Date":"Wed, 18 Dec 2024 15:11:16 +0100","From":"Stanislaw Gruszka ","To":"Hans de Goede ","Cc":"libcamera-devel@lists.libcamera.org, Milan Zamazal ,\n\tLaurent Pinchart ,\n\tKieran Bingham ,\n\tNaushir Patuck ,\n\tSakari Ailus ","Subject":"Re: [PATCH] pipeline: simple: Use proper device for frame start\n\tevents","Message-ID":"","References":"<20241217105206.357495-1-stanislaw.gruszka@linux.intel.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]