{"id":22811,"url":"https://patchwork.libcamera.org/api/patches/22811/?format=json","web_url":"https://patchwork.libcamera.org/patch/22811/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250218091951.13017-3-stanislaw.gruszka@linux.intel.com>","date":"2025-02-18T09:19:51","name":"[v3,2/2] pipeline: simple: Use proper device for frame start events","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"f42b4f050f03a3b42dcb75754cda6aa342b2be6b","submitter":{"id":211,"url":"https://patchwork.libcamera.org/api/people/211/?format=json","name":"Stanislaw Gruszka","email":"stanislaw.gruszka@linux.intel.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/22811/mbox/","series":[{"id":5008,"url":"https://patchwork.libcamera.org/api/series/5008/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5008","date":"2025-02-18T09:19:49","name":"libcamera: start frame events changes","version":3,"mbox":"https://patchwork.libcamera.org/series/5008/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/22811/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/22811/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 14BD6C3220\n\tfor ;\n\tTue, 18 Feb 2025 09:20:12 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id BE4D868690;\n\tTue, 18 Feb 2025 10:20:11 +0100 (CET)","from mgamail.intel.com (mgamail.intel.com [192.198.163.15])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 455296032B\n\tfor ;\n\tTue, 18 Feb 2025 10:20:05 +0100 (CET)","from fmviesa010.fm.intel.com ([10.60.135.150])\n\tby fmvoesa109.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n\t18 Feb 2025 01:20:05 -0800","from sgruszka-mobl.ger.corp.intel.com (HELO localhost)\n\t([10.246.8.237]) by fmviesa010-auth.fm.intel.com with\n\tESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2025 01:20:02 -0800"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=intel.com header.i=@intel.com\n\theader.b=\"gXtoOzvm\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=intel.com; i=@intel.com; q=dns/txt; s=Intel;\n\tt=1739870406; x=1771406406;\n\th=from:to:cc:subject:date:message-id:in-reply-to:\n\treferences:mime-version:content-transfer-encoding;\n\tbh=LLLmdQO58UaO6l69pODfORzXSSsUZaIhNVtuEthfTaU=;\n\tb=gXtoOzvm1WrxqspBQ1CdX9vgrh/Hs/FtJ7vQfGPEUN1zAsKrsk9q06TH\n\tQ8epzwSzTVZisI88QJCs10wNyhMbGveHNex0Y/NRaDB2zCVo5+gB/2PlL\n\tQW6PqkaPRi3KAd85rpXngPx/UpANx24vUz7ojgYjYc0WJH0SFY3rLs0lq\n\tfrn3RDEIdKEUO30km2JmNFFw9aaz+OhWynWND04+R25jFenc1ZiwXgj0E\n\tvU0HdDY0NlsE5Jaka5I0ZXVllqYeGzF5ZwJOMFEIK0LSt442Ro3DrORE3\n\tLMUn4E42V27XX8darJhtbzVLnFdEDfAFfGa17DXX4MSaQwgXZZIWASnyT Q==;","X-CSE-ConnectionGUID":["/oHU+ZRlQbevY/0J3XNS3Q==","bGbATp/OR6ivdFVtnEbUww=="],"X-CSE-MsgGUID":["tvUg6ALqTzWdbAZ02fQYOA==","lTFXM+EeTYG7/MODF/w3nw=="],"X-IronPort-AV":["E=McAfee;i=\"6700,10204,11348\"; a=\"40692058\"","E=Sophos;i=\"6.13,295,1732608000\"; d=\"scan'208\";a=\"40692058\"","E=Sophos;i=\"6.13,295,1732608000\"; d=\"scan'208\";a=\"114855872\""],"X-ExtLoop1":"1","From":"Stanislaw Gruszka ","To":"libcamera-devel@lists.libcamera.org","Cc":"Milan Zamazal ,\n\tLaurent Pinchart ,\n\tKieran Bingham ,\n\tNaushir Patuck ,\n\tSakari Ailus ,\n\tHans de Goede ","Subject":"[PATCH v3 2/2] pipeline: simple: Use proper device for frame start\n\tevents","Date":"Tue, 18 Feb 2025 10:19:51 +0100","Message-Id":"<20250218091951.13017-3-stanislaw.gruszka@linux.intel.com>","X-Mailer":"git-send-email 2.34.1","In-Reply-To":"<20250218091951.13017-1-stanislaw.gruszka@linux.intel.com>","References":"<20250218091951.13017-1-stanislaw.gruszka@linux.intel.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Currently we use frame start event from video capture device to\napply controls. But the capture device might not generate the events.\nUsually CSI-2 receiver is proper device to subscribe for start\nframe events.\n\nWithout DelayedConntrols:applyControls() is possible that we can get\ncall to DelayedControls::get() with frame number that exceed number\nof saved entries and get below assertion failure:\n\n../src/libcamera/delayed_controls.cpp:227:\nlibcamera::ControlList libcamera::DelayedControls::get(uint32_t):\nAssertion `info.type() != ControlTypeNone' failed\n\nAssertion failure can happen at the beginning of streaming when\nControlRingBuffer is not yet filled and there are errors on CSI-2.\n\nTo fix, loop over devices in the pipeline (starting from the last one),\nfind one that emits start frame events and connect applyControls()\nto it. Additionally remove direct call to sensor_->setControls() if\nthe emitter device was found.\n\nBug: https://bugs.libcamera.org/show_bug.cgi?id=241\nCo-developed-by: Hans de Goede \nSigned-off-by: Hans de Goede \nReviewed-by: Hans de Goede # v2\nTested-by: Hans de Goede # Lenovo X1 Yoga IPU6 + ov2740 # v2\nReviewed-by: Milan Zamazal # v2\nSigned-off-by: Stanislaw Gruszka \n---\n src/libcamera/pipeline/simple/simple.cpp | 40 +++++++++++++++++++++---\n 1 file changed, 36 insertions(+), 4 deletions(-)\n\n--\n2.43.0","diff":"diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\nindex 6e039bf3..ad2167dc 100644\n--- a/src/libcamera/pipeline/simple/simple.cpp\n+++ b/src/libcamera/pipeline/simple/simple.cpp\n@@ -277,6 +277,7 @@ public:\n \tstd::list entities_;\n \tstd::unique_ptr sensor_;\n \tV4L2VideoDevice *video_;\n+\tV4L2Subdevice *eventEmitter_;\n\n \tstd::vector configs_;\n \tstd::map> formats_;\n@@ -579,6 +580,19 @@ int SimpleCameraData::init()\n\n \tproperties_ = sensor_->properties();\n\n+\teventEmitter_ = nullptr;\n+\tfor (auto it = entities_.rbegin(); it != entities_.rend(); ++it) {\n+\t\tV4L2Subdevice *sd = pipe->subdev(it->entity);\n+\t\tif (!sd || !sd->supportsFrameStartEvent())\n+\t\t\tcontinue;\n+\n+\t\tLOG(SimplePipeline, Debug)\n+\t\t\t<< \"Using \" << it->entity->name() << \" frameStart signal\";\n+\n+\t\teventEmitter_ = sd;\n+\t\tbreak;\n+\t}\n+\n \treturn 0;\n }\n\n@@ -897,8 +911,11 @@ void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId)\n void SimpleCameraData::setSensorControls(const ControlList &sensorControls)\n {\n \tdelayedCtrls_->push(sensorControls);\n-\tControlList ctrls(sensorControls);\n-\tsensor_->setControls(&ctrls);\n+\t/* Directly apply controls now if there is no frameStart signal */\n+\tif (!eventEmitter_ || !eventEmitter_->frameStartEnabled()) {\n+\t\tControlList ctrls(sensorControls);\n+\t\tsensor_->setControls(&ctrls);\n+\t}\n }\n\n /* Retrieve all source pads connected to a sink pad through active routes. */\n@@ -1285,8 +1302,6 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)\n \tdata->delayedCtrls_ =\n \t\tstd::make_unique(data->sensor_->device(),\n \t\t\t\t\t\t params);\n-\tdata->video_->frameStart.connect(data->delayedCtrls_.get(),\n-\t\t\t\t\t &DelayedControls::applyControls);\n\n \tStreamConfiguration inputCfg;\n \tinputCfg.pixelFormat = pipeConfig->captureFormat;\n@@ -1325,6 +1340,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL\n {\n \tSimpleCameraData *data = cameraData(camera);\n \tV4L2VideoDevice *video = data->video_;\n+\tV4L2Subdevice *eventEmitter = data->eventEmitter_;\n \tint ret;\n\n \tconst MediaPad *pad = acquirePipeline(data);\n@@ -1354,6 +1370,15 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL\n\n \tvideo->bufferReady.connect(data, &SimpleCameraData::imageBufferReady);\n\n+\tif (eventEmitter) {\n+\t\tret = eventEmitter->setFrameStartEnabled(true);\n+\t\tif (ret == 0)\n+\t\t\teventEmitter->frameStart.connect(data->delayedCtrls_.get(),\n+\t\t\t\t\t\t\t &DelayedControls::applyControls);\n+\t}\n+\n+\tdata->delayedCtrls_->reset();\n+\n \tret = video->streamOn();\n \tif (ret < 0) {\n \t\tstop(camera);\n@@ -1385,6 +1410,13 @@ void SimplePipelineHandler::stopDevice(Camera *camera)\n {\n \tSimpleCameraData *data = cameraData(camera);\n \tV4L2VideoDevice *video = data->video_;\n+\tV4L2Subdevice *eventEmitter = data->eventEmitter_;\n+\n+\tif (eventEmitter) {\n+\t\teventEmitter->setFrameStartEnabled(false);\n+\t\teventEmitter->frameStart.disconnect(data->delayedCtrls_.get(),\n+\t\t\t\t\t\t &DelayedControls::applyControls);\n+\t}\n\n \tif (data->useConversion_) {\n \t\tif (data->converter_)\n","prefixes":["v3","2/2"]}