[{"id":35378,"web_url":"https://patchwork.libcamera.org/comment/35378/","msgid":"<20250813113417.GG6440@pendragon.ideasonboard.com>","date":"2025-08-13T11:34:17","subject":"Re: [RFC PATCH v2 2/2] libcamera: pipeline: virtual: Move image\n\tgeneration to separate thread","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Barnabás,\n\nThank you for the patch.\n\nOn Wed, Aug 13, 2025 at 12:25:01PM +0200, Barnabás Pőcze wrote:\n> Currently the virtual pipeline generates the images synchronously. This is not\n> ideal because it blocks the camera manager's internal thread, and because its\n> behaviour is different from other existing pipeline handlers, all of which\n> complete requests asynchronously.\n> \n> So move the image generation to a separate thread by deriving `VirtualCameraData`\n> from `Thread`, as well as `Object` and using the existing asynchronous signal\n> and method call mechanism.\n> \n> Signed-off-by: Barnabás Pőcze \n> ---\n> src/libcamera/pipeline/virtual/virtual.cpp | 87 ++++++++++++++--------\n> src/libcamera/pipeline/virtual/virtual.h | 9 ++-\n> 2 files changed, 66 insertions(+), 30 deletions(-)\n> \n> diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp\n> index 049ebcba5..48857491d 100644\n> --- a/src/libcamera/pipeline/virtual/virtual.cpp\n> +++ b/src/libcamera/pipeline/virtual/virtual.cpp\n> @@ -107,6 +107,14 @@ private:\n> \n> \tbool initFrameGenerator(Camera *camera);\n> \n> +\tvoid onBufferCompleted(FrameBuffer *buffer)\n\nWe don't normally prefix signal handlers with \"on\".\n\n> +\t{\n> +\t\tRequest *request = buffer->request();\n> +\n> +\t\tif (completeBuffer(request, buffer))\n> +\t\t\tcompleteRequest(request);\n> +\t}\n\nI wouldn't have made this an inline function, but no big deal.\n\n> +\n> \tDmaBufAllocator dmaBufAllocator_;\n> \n> \tbool resetCreated_ = false;\n> @@ -129,6 +137,38 @@ VirtualCameraData::VirtualCameraData(PipelineHandler *pipe,\n> \n> \t/* \\todo Support multiple streams and pass multi_stream_test */\n> \tstreamConfigs_.resize(kMaxStream);\n> +\n> +\tmoveToThread(this);\n> +}\n> +\n> +void VirtualCameraData::queueRequest(Request *request)\n\nThis function doesn't queue a request, but processes it. How about\nnaming it processRequest() ?\n\n> +{\n> +\tfor (auto const &[stream, buffer] : request->buffers()) {\n> +\t\tbool found = false;\n> +\t\t/* map buffer and fill test patterns */\n> +\t\tfor (auto &streamConfig : streamConfigs_) {\n> +\t\t\tif (stream == &streamConfig.stream) {\n> +\t\t\t\tFrameMetadata &fmd = buffer->_d()->metadata();\n> +\n> +\t\t\t\tfmd.status = FrameMetadata::Status::FrameSuccess;\n> +\t\t\t\tfmd.sequence = streamConfig.seq++;\n> +\t\t\t\tfmd.timestamp = currentTimestamp();\n> +\n> +\t\t\t\tfor (const auto [i, p] : utils::enumerate(buffer->planes()))\n> +\t\t\t\t\tfmd.planes()[i].bytesused = p.length;\n> +\n> +\t\t\t\tfound = true;\n> +\n> +\t\t\t\tif (streamConfig.frameGenerator->generateFrame(\n> +\t\t\t\t\t stream->configuration().size, buffer))\n> +\t\t\t\t\tfmd.status = FrameMetadata::Status::FrameError;\n> +\n> +\t\t\t\tbufferCompleted.emit(buffer);\n> +\t\t\t\tbreak;\n> +\t\t\t}\n> +\t\t}\n> +\t\tASSERT(found);\n> +\t}\n> }\n> \n> VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data)\n> @@ -291,11 +331,27 @@ int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera,\n> \tfor (auto &s : data->streamConfigs_)\n> \t\ts.seq = 0;\n> \n> +\tdata->bufferCompleted.connect(this, &PipelineHandlerVirtual::onBufferCompleted);\n> +\tdata->start();\n> +\n> \treturn 0;\n> }\n> \n> -void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera)\n> +void PipelineHandlerVirtual::stopDevice(Camera *camera)\n> {\n> +\tVirtualCameraData *data = cameraData(camera);\n> +\n> +\t/* Cancel pending work. */\n> +\tdata->exit();\n> +\tdata->wait();\n> +\tdata->removeMessages(data);\n> +\n> +\t/* Process pending `bufferCompleted` signals. */\n> +\tthread()->dispatchMessages(Message::Type::InvokeMessage, this);\n> +\tdata->bufferCompleted.disconnect(this);\n> +\n> +\twhile (!data->queuedRequests_.empty())\n> +\t\tcancelRequest(data->queuedRequests_.front());\n> }\n> \n> int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera,\n> @@ -304,35 +360,8 @@ int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera,\n> \tVirtualCameraData *data = cameraData(camera);\n> \tconst auto timestamp = currentTimestamp();\n> \n> -\tfor (auto const &[stream, buffer] : request->buffers()) {\n> -\t\tbool found = false;\n> -\t\t/* map buffer and fill test patterns */\n> -\t\tfor (auto &streamConfig : data->streamConfigs_) {\n> -\t\t\tif (stream == &streamConfig.stream) {\n> -\t\t\t\tFrameMetadata &fmd = buffer->_d()->metadata();\n> -\n> -\t\t\t\tfmd.status = FrameMetadata::Status::FrameSuccess;\n> -\t\t\t\tfmd.sequence = streamConfig.seq++;\n> -\t\t\t\tfmd.timestamp = timestamp;\n> -\n> -\t\t\t\tfor (const auto [i, p] : utils::enumerate(buffer->planes()))\n> -\t\t\t\t\tfmd.planes()[i].bytesused = p.length;\n> -\n> -\t\t\t\tfound = true;\n> -\n> -\t\t\t\tif (streamConfig.frameGenerator->generateFrame(\n> -\t\t\t\t\t stream->configuration().size, buffer))\n> -\t\t\t\t\tfmd.status = FrameMetadata::Status::FrameError;\n> -\n> -\t\t\t\tcompleteBuffer(request, buffer);\n> -\t\t\t\tbreak;\n> -\t\t\t}\n> -\t\t}\n> -\t\tASSERT(found);\n> -\t}\n> -\n> \trequest->metadata().set(controls::SensorTimestamp, timestamp);\n> -\tcompleteRequest(request);\n> +\tdata->invokeMethod(&VirtualCameraData::queueRequest, ConnectionTypeQueued, request);\n\nLine wrap.\n\n> \n> \treturn 0;\n> }\n> diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h\n> index 683cb82b4..2d83dfe54 100644\n> --- a/src/libcamera/pipeline/virtual/virtual.h\n> +++ b/src/libcamera/pipeline/virtual/virtual.h\n> @@ -11,6 +11,8 @@\n> #include \n> #include \n> \n> +#include \n> +#include \n\nMissing blank line.\n\nWith those minor issues addressed,\n\nReviewed-by: Laurent Pinchart \n\n> #include \n> #include \n> \n> @@ -25,7 +27,9 @@ namespace libcamera {\n> \n> using VirtualFrame = std::variant;\n> \n> -class VirtualCameraData : public Camera::Private\n> +class VirtualCameraData : public Camera::Private,\n> +\t\t\t public Thread,\n> +\t\t\t public Object\n> {\n> public:\n> \tconst static unsigned int kMaxStream = 3;\n> @@ -54,9 +58,12 @@ public:\n> \n> \t~VirtualCameraData() = default;\n> \n> +\tvoid queueRequest(Request *request);\n> +\n> \tConfiguration config_;\n> \n> \tstd::vector streamConfigs_;\n> +\tSignal bufferCompleted;\n> };\n> \n> } /* namespace libcamera */","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 408EFBDCC1\n\tfor ;\n\tWed, 13 Aug 2025 11:34:41 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 61EB461444;\n\tWed, 13 Aug 2025 13:34:40 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1BF2C61444\n\tfor ;\n\tWed, 13 Aug 2025 13:34:38 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 5BF282C5;\n\tWed, 13 Aug 2025 13:33:44 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"azWU5Ez3\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1755084824;\n\tbh=NHfz1goYZBacXdqZ5bzsbxUTvH1ZYjeFETBv0eRTVJM=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=azWU5Ez3dIQLWhn7Qhn8JUgxWvkezPEp8jcAD0PLEjPdkC/WP50VFbIqxYLrVGiSu\n\tsG/XRsD/E+gVi6Ctyqeu0Sml4Gs4XPSUyJcNkDJGwllg/95k61uyGVQ5s4qF8gMqSS\n\tvT1eHft3eP5sE1clzRfEJdn3HLRfNBGgtfpOoP6A=","Date":"Wed, 13 Aug 2025 14:34:17 +0300","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [RFC PATCH v2 2/2] libcamera: pipeline: virtual: Move image\n\tgeneration to separate thread","Message-ID":"<20250813113417.GG6440@pendragon.ideasonboard.com>","References":"<20250813102501.1645940-1-barnabas.pocze@ideasonboard.com>\n\t<20250813102501.1645940-3-barnabas.pocze@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20250813102501.1645940-3-barnabas.pocze@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":35384,"web_url":"https://patchwork.libcamera.org/comment/35384/","msgid":"<82a0153b-809c-4481-a740-b704de3035f2@ideasonboard.com>","date":"2025-08-13T13:00:21","subject":"Re: [RFC PATCH v2 2/2] libcamera: pipeline: virtual: Move image\n\tgeneration to separate thread","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"2025. 08. 13. 13:34 keltezéssel, Laurent Pinchart írta:\n> Hi Barnabás,\n> \n> Thank you for the patch.\n> \n> On Wed, Aug 13, 2025 at 12:25:01PM +0200, Barnabás Pőcze wrote:\n>> Currently the virtual pipeline generates the images synchronously. This is not\n>> ideal because it blocks the camera manager's internal thread, and because its\n>> behaviour is different from other existing pipeline handlers, all of which\n>> complete requests asynchronously.\n>>\n>> So move the image generation to a separate thread by deriving `VirtualCameraData`\n>> from `Thread`, as well as `Object` and using the existing asynchronous signal\n>> and method call mechanism.\n>>\n>> Signed-off-by: Barnabás Pőcze \n>> ---\n>> src/libcamera/pipeline/virtual/virtual.cpp | 87 ++++++++++++++--------\n>> src/libcamera/pipeline/virtual/virtual.h | 9 ++-\n>> 2 files changed, 66 insertions(+), 30 deletions(-)\n>>\n>> diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp\n>> index 049ebcba5..48857491d 100644\n>> --- a/src/libcamera/pipeline/virtual/virtual.cpp\n>> +++ b/src/libcamera/pipeline/virtual/virtual.cpp\n>> @@ -107,6 +107,14 @@ private:\n>> \n>> \tbool initFrameGenerator(Camera *camera);\n>> \n>> +\tvoid onBufferCompleted(FrameBuffer *buffer)\n> \n> We don't normally prefix signal handlers with \"on\".\n\nOK\n\n\n> \n>> +\t{\n>> +\t\tRequest *request = buffer->request();\n>> +\n>> +\t\tif (completeBuffer(request, buffer))\n>> +\t\t\tcompleteRequest(request);\n>> +\t}\n> \n> I wouldn't have made this an inline function, but no big deal.\n\nSo defining it out of line? I fear we're going from a short lambda\nto the longest possible version. :(\n\n\n> \n>> +\n>> \tDmaBufAllocator dmaBufAllocator_;\n>> \n>> \tbool resetCreated_ = false;\n>> @@ -129,6 +137,38 @@ VirtualCameraData::VirtualCameraData(PipelineHandler *pipe,\n>> \n>> \t/* \\todo Support multiple streams and pass multi_stream_test */\n>> \tstreamConfigs_.resize(kMaxStream);\n>> +\n>> +\tmoveToThread(this);\n>> +}\n>> +\n>> +void VirtualCameraData::queueRequest(Request *request)\n> \n> This function doesn't queue a request, but processes it. How about\n> naming it processRequest() ?\n\nOK\n\n\n> \n>> +{\n>> +\tfor (auto const &[stream, buffer] : request->buffers()) {\n>> +\t\tbool found = false;\n>> +\t\t/* map buffer and fill test patterns */\n>> +\t\tfor (auto &streamConfig : streamConfigs_) {\n>> +\t\t\tif (stream == &streamConfig.stream) {\n>> +\t\t\t\tFrameMetadata &fmd = buffer->_d()->metadata();\n>> +\n>> +\t\t\t\tfmd.status = FrameMetadata::Status::FrameSuccess;\n>> +\t\t\t\tfmd.sequence = streamConfig.seq++;\n>> +\t\t\t\tfmd.timestamp = currentTimestamp();\n>> +\n>> +\t\t\t\tfor (const auto [i, p] : utils::enumerate(buffer->planes()))\n>> +\t\t\t\t\tfmd.planes()[i].bytesused = p.length;\n>> +\n>> +\t\t\t\tfound = true;\n>> +\n>> +\t\t\t\tif (streamConfig.frameGenerator->generateFrame(\n>> +\t\t\t\t\t stream->configuration().size, buffer))\n>> +\t\t\t\t\tfmd.status = FrameMetadata::Status::FrameError;\n>> +\n>> +\t\t\t\tbufferCompleted.emit(buffer);\n>> +\t\t\t\tbreak;\n>> +\t\t\t}\n>> +\t\t}\n>> +\t\tASSERT(found);\n>> +\t}\n>> }\n>> \n>> VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data)\n>> @@ -291,11 +331,27 @@ int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera,\n>> \tfor (auto &s : data->streamConfigs_)\n>> \t\ts.seq = 0;\n>> \n>> +\tdata->bufferCompleted.connect(this, &PipelineHandlerVirtual::onBufferCompleted);\n>> +\tdata->start();\n>> +\n>> \treturn 0;\n>> }\n>> \n>> -void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera)\n>> +void PipelineHandlerVirtual::stopDevice(Camera *camera)\n>> {\n>> +\tVirtualCameraData *data = cameraData(camera);\n>> +\n>> +\t/* Cancel pending work. */\n>> +\tdata->exit();\n>> +\tdata->wait();\n>> +\tdata->removeMessages(data);\n>> +\n>> +\t/* Process pending `bufferCompleted` signals. */\n>> +\tthread()->dispatchMessages(Message::Type::InvokeMessage, this);\n>> +\tdata->bufferCompleted.disconnect(this);\n>> +\n>> +\twhile (!data->queuedRequests_.empty())\n>> +\t\tcancelRequest(data->queuedRequests_.front());\n>> }\n>> \n>> int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera,\n>> @@ -304,35 +360,8 @@ int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera,\n>> \tVirtualCameraData *data = cameraData(camera);\n>> \tconst auto timestamp = currentTimestamp();\n>> \n>> -\tfor (auto const &[stream, buffer] : request->buffers()) {\n>> -\t\tbool found = false;\n>> -\t\t/* map buffer and fill test patterns */\n>> -\t\tfor (auto &streamConfig : data->streamConfigs_) {\n>> -\t\t\tif (stream == &streamConfig.stream) {\n>> -\t\t\t\tFrameMetadata &fmd = buffer->_d()->metadata();\n>> -\n>> -\t\t\t\tfmd.status = FrameMetadata::Status::FrameSuccess;\n>> -\t\t\t\tfmd.sequence = streamConfig.seq++;\n>> -\t\t\t\tfmd.timestamp = timestamp;\n>> -\n>> -\t\t\t\tfor (const auto [i, p] : utils::enumerate(buffer->planes()))\n>> -\t\t\t\t\tfmd.planes()[i].bytesused = p.length;\n>> -\n>> -\t\t\t\tfound = true;\n>> -\n>> -\t\t\t\tif (streamConfig.frameGenerator->generateFrame(\n>> -\t\t\t\t\t stream->configuration().size, buffer))\n>> -\t\t\t\t\tfmd.status = FrameMetadata::Status::FrameError;\n>> -\n>> -\t\t\t\tcompleteBuffer(request, buffer);\n>> -\t\t\t\tbreak;\n>> -\t\t\t}\n>> -\t\t}\n>> -\t\tASSERT(found);\n>> -\t}\n>> -\n>> \trequest->metadata().set(controls::SensorTimestamp, timestamp);\n>> -\tcompleteRequest(request);\n>> +\tdata->invokeMethod(&VirtualCameraData::queueRequest, ConnectionTypeQueued, request);\n> \n> Line wrap.\n\nOK\n\n\n> \n>> \n>> \treturn 0;\n>> }\n>> diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h\n>> index 683cb82b4..2d83dfe54 100644\n>> --- a/src/libcamera/pipeline/virtual/virtual.h\n>> +++ b/src/libcamera/pipeline/virtual/virtual.h\n>> @@ -11,6 +11,8 @@\n>> #include \n>> #include \n>> \n>> +#include \n>> +#include \n> \n> Missing blank line.\n\nHmmm... checkstyle.py does not complain.\n\n\nRegards,\nBarnabás Pőcze\n\n> \n> With those minor issues addressed,\n> \n> Reviewed-by: Laurent Pinchart \n> \n>> #include \n>> #include \n>> \n>> @@ -25,7 +27,9 @@ namespace libcamera {\n>> \n>> using VirtualFrame = std::variant;\n>> \n>> -class VirtualCameraData : public Camera::Private\n>> +class VirtualCameraData : public Camera::Private,\n>> +\t\t\t public Thread,\n>> +\t\t\t public Object\n>> {\n>> public:\n>> \tconst static unsigned int kMaxStream = 3;\n>> @@ -54,9 +58,12 @@ public:\n>> \n>> \t~VirtualCameraData() = default;\n>> \n>> +\tvoid queueRequest(Request *request);\n>> +\n>> \tConfiguration config_;\n>> \n>> \tstd::vector streamConfigs_;\n>> +\tSignal bufferCompleted;\n>> };\n>> \n>> } /* namespace libcamera */\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 93BB2BDCC1\n\tfor ;\n\tWed, 13 Aug 2025 13:00:27 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 63DFC69249;\n\tWed, 13 Aug 2025 15:00:26 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B76DB61444\n\tfor ;\n\tWed, 13 Aug 2025 15:00:24 +0200 (CEST)","from [192.168.33.21] (185.221.141.188.nat.pool.zt.hu\n\t[185.221.141.188])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0071D3A4;\n\tWed, 13 Aug 2025 14:59:30 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"S9PPj8AV\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1755089971;\n\tbh=q5kH89JjRfoH1mDcOSjZ4ayjh8JpNcK+lAUmmEqvyho=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=S9PPj8AVkKsdynrK+IS4VI5rcrduejxD5ffB6PoHTLm4FKZ7kPrtLtQDSIIV8M38g\n\tGn+G0SAl9YJj/w3ua8Wa0fL0T7VDvey1OF/YyPTlnxF+lBXCurxT7HdQdRlh+awSil\n\tWcZcgZBFYtQm3GNkFAFbYGSwK5bnUlU273ljCYfc=","Message-ID":"<82a0153b-809c-4481-a740-b704de3035f2@ideasonboard.com>","Date":"Wed, 13 Aug 2025 15:00:21 +0200","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [RFC PATCH v2 2/2] libcamera: pipeline: virtual: Move image\n\tgeneration to separate thread","To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","References":"<20250813102501.1645940-1-barnabas.pocze@ideasonboard.com>\n\t<20250813102501.1645940-3-barnabas.pocze@ideasonboard.com>\n\t<20250813113417.GG6440@pendragon.ideasonboard.com>","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"<20250813113417.GG6440@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":35385,"web_url":"https://patchwork.libcamera.org/comment/35385/","msgid":"<20250813130713.GF20174@pendragon.ideasonboard.com>","date":"2025-08-13T13:07:13","subject":"Re: [RFC PATCH v2 2/2] libcamera: pipeline: virtual: Move image\n\tgeneration to separate thread","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Wed, Aug 13, 2025 at 03:00:21PM +0200, Barnabás Pőcze wrote:\n> 2025. 08. 13. 13:34 keltezéssel, Laurent Pinchart írta:\n> > On Wed, Aug 13, 2025 at 12:25:01PM +0200, Barnabás Pőcze wrote:\n> >> Currently the virtual pipeline generates the images synchronously. This is not\n> >> ideal because it blocks the camera manager's internal thread, and because its\n> >> behaviour is different from other existing pipeline handlers, all of which\n> >> complete requests asynchronously.\n> >>\n> >> So move the image generation to a separate thread by deriving `VirtualCameraData`\n> >> from `Thread`, as well as `Object` and using the existing asynchronous signal\n> >> and method call mechanism.\n> >>\n> >> Signed-off-by: Barnabás Pőcze \n> >> ---\n> >> src/libcamera/pipeline/virtual/virtual.cpp | 87 ++++++++++++++--------\n> >> src/libcamera/pipeline/virtual/virtual.h | 9 ++-\n> >> 2 files changed, 66 insertions(+), 30 deletions(-)\n> >>\n> >> diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp\n> >> index 049ebcba5..48857491d 100644\n> >> --- a/src/libcamera/pipeline/virtual/virtual.cpp\n> >> +++ b/src/libcamera/pipeline/virtual/virtual.cpp\n> >> @@ -107,6 +107,14 @@ private:\n> >> \n> >> \tbool initFrameGenerator(Camera *camera);\n> >> \n> >> +\tvoid onBufferCompleted(FrameBuffer *buffer)\n> > \n> > We don't normally prefix signal handlers with \"on\".\n> \n> OK\n> \n> >> +\t{\n> >> +\t\tRequest *request = buffer->request();\n> >> +\n> >> +\t\tif (completeBuffer(request, buffer))\n> >> +\t\t\tcompleteRequest(request);\n> >> +\t}\n> > \n> > I wouldn't have made this an inline function, but no big deal.\n> \n> So defining it out of line? I fear we're going from a short lambda\n> to the longest possible version. :(\n\nThat's why I said it's no big deal :-) I can ignore it.\n\n> >> +\n> >> \tDmaBufAllocator dmaBufAllocator_;\n> >> \n> >> \tbool resetCreated_ = false;\n> >> @@ -129,6 +137,38 @@ VirtualCameraData::VirtualCameraData(PipelineHandler *pipe,\n> >> \n> >> \t/* \\todo Support multiple streams and pass multi_stream_test */\n> >> \tstreamConfigs_.resize(kMaxStream);\n> >> +\n> >> +\tmoveToThread(this);\n> >> +}\n> >> +\n> >> +void VirtualCameraData::queueRequest(Request *request)\n> > \n> > This function doesn't queue a request, but processes it. How about\n> > naming it processRequest() ?\n> \n> OK\n> \n> >> +{\n> >> +\tfor (auto const &[stream, buffer] : request->buffers()) {\n> >> +\t\tbool found = false;\n> >> +\t\t/* map buffer and fill test patterns */\n> >> +\t\tfor (auto &streamConfig : streamConfigs_) {\n> >> +\t\t\tif (stream == &streamConfig.stream) {\n> >> +\t\t\t\tFrameMetadata &fmd = buffer->_d()->metadata();\n> >> +\n> >> +\t\t\t\tfmd.status = FrameMetadata::Status::FrameSuccess;\n> >> +\t\t\t\tfmd.sequence = streamConfig.seq++;\n> >> +\t\t\t\tfmd.timestamp = currentTimestamp();\n> >> +\n> >> +\t\t\t\tfor (const auto [i, p] : utils::enumerate(buffer->planes()))\n> >> +\t\t\t\t\tfmd.planes()[i].bytesused = p.length;\n> >> +\n> >> +\t\t\t\tfound = true;\n> >> +\n> >> +\t\t\t\tif (streamConfig.frameGenerator->generateFrame(\n> >> +\t\t\t\t\t stream->configuration().size, buffer))\n> >> +\t\t\t\t\tfmd.status = FrameMetadata::Status::FrameError;\n> >> +\n> >> +\t\t\t\tbufferCompleted.emit(buffer);\n> >> +\t\t\t\tbreak;\n> >> +\t\t\t}\n> >> +\t\t}\n> >> +\t\tASSERT(found);\n> >> +\t}\n> >> }\n> >> \n> >> VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data)\n> >> @@ -291,11 +331,27 @@ int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera,\n> >> \tfor (auto &s : data->streamConfigs_)\n> >> \t\ts.seq = 0;\n> >> \n> >> +\tdata->bufferCompleted.connect(this, &PipelineHandlerVirtual::onBufferCompleted);\n> >> +\tdata->start();\n> >> +\n> >> \treturn 0;\n> >> }\n> >> \n> >> -void PipelineHandlerVirtual::stopDevice([[maybe_unused]] Camera *camera)\n> >> +void PipelineHandlerVirtual::stopDevice(Camera *camera)\n> >> {\n> >> +\tVirtualCameraData *data = cameraData(camera);\n> >> +\n> >> +\t/* Cancel pending work. */\n> >> +\tdata->exit();\n> >> +\tdata->wait();\n> >> +\tdata->removeMessages(data);\n> >> +\n> >> +\t/* Process pending `bufferCompleted` signals. */\n> >> +\tthread()->dispatchMessages(Message::Type::InvokeMessage, this);\n> >> +\tdata->bufferCompleted.disconnect(this);\n> >> +\n> >> +\twhile (!data->queuedRequests_.empty())\n> >> +\t\tcancelRequest(data->queuedRequests_.front());\n> >> }\n> >> \n> >> int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera,\n> >> @@ -304,35 +360,8 @@ int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera,\n> >> \tVirtualCameraData *data = cameraData(camera);\n> >> \tconst auto timestamp = currentTimestamp();\n> >> \n> >> -\tfor (auto const &[stream, buffer] : request->buffers()) {\n> >> -\t\tbool found = false;\n> >> -\t\t/* map buffer and fill test patterns */\n> >> -\t\tfor (auto &streamConfig : data->streamConfigs_) {\n> >> -\t\t\tif (stream == &streamConfig.stream) {\n> >> -\t\t\t\tFrameMetadata &fmd = buffer->_d()->metadata();\n> >> -\n> >> -\t\t\t\tfmd.status = FrameMetadata::Status::FrameSuccess;\n> >> -\t\t\t\tfmd.sequence = streamConfig.seq++;\n> >> -\t\t\t\tfmd.timestamp = timestamp;\n> >> -\n> >> -\t\t\t\tfor (const auto [i, p] : utils::enumerate(buffer->planes()))\n> >> -\t\t\t\t\tfmd.planes()[i].bytesused = p.length;\n> >> -\n> >> -\t\t\t\tfound = true;\n> >> -\n> >> -\t\t\t\tif (streamConfig.frameGenerator->generateFrame(\n> >> -\t\t\t\t\t stream->configuration().size, buffer))\n> >> -\t\t\t\t\tfmd.status = FrameMetadata::Status::FrameError;\n> >> -\n> >> -\t\t\t\tcompleteBuffer(request, buffer);\n> >> -\t\t\t\tbreak;\n> >> -\t\t\t}\n> >> -\t\t}\n> >> -\t\tASSERT(found);\n> >> -\t}\n> >> -\n> >> \trequest->metadata().set(controls::SensorTimestamp, timestamp);\n> >> -\tcompleteRequest(request);\n> >> +\tdata->invokeMethod(&VirtualCameraData::queueRequest, ConnectionTypeQueued, request);\n> > \n> > Line wrap.\n> \n> OK\n> \n> >> \n> >> \treturn 0;\n> >> }\n> >> diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h\n> >> index 683cb82b4..2d83dfe54 100644\n> >> --- a/src/libcamera/pipeline/virtual/virtual.h\n> >> +++ b/src/libcamera/pipeline/virtual/virtual.h\n> >> @@ -11,6 +11,8 @@\n> >> #include \n> >> #include \n> >> \n> >> +#include \n> >> +#include \n> > \n> > Missing blank line.\n> \n> Hmmm... checkstyle.py does not complain.\n\nI would have expected it to :-/\n\n> > With those minor issues addressed,\n> > \n> > Reviewed-by: Laurent Pinchart \n> > \n> >> #include \n> >> #include \n> >> \n> >> @@ -25,7 +27,9 @@ namespace libcamera {\n> >> \n> >> using VirtualFrame = std::variant;\n> >> \n> >> -class VirtualCameraData : public Camera::Private\n> >> +class VirtualCameraData : public Camera::Private,\n> >> +\t\t\t public Thread,\n> >> +\t\t\t public Object\n> >> {\n> >> public:\n> >> \tconst static unsigned int kMaxStream = 3;\n> >> @@ -54,9 +58,12 @@ public:\n> >> \n> >> \t~VirtualCameraData() = default;\n> >> \n> >> +\tvoid queueRequest(Request *request);\n> >> +\n> >> \tConfiguration config_;\n> >> \n> >> \tstd::vector streamConfigs_;\n> >> +\tSignal bufferCompleted;\n> >> };\n> >> \n> >> } /* namespace libcamera */","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id EC30EBEFBE\n\tfor ;\n\tWed, 13 Aug 2025 13:07:34 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B317E69249;\n\tWed, 13 Aug 2025 15:07:33 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A5AC361444\n\tfor ;\n\tWed, 13 Aug 2025 15:07:32 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E8FA53A4;\n\tWed, 13 Aug 2025 15:06:38 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"FsWe+MVI\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1755090399;\n\tbh=owFWeI0/dOh3oOjA1lAicBq74R52bFixU/E06Gbc7FE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=FsWe+MVIYUeZmnCj7YNj4QYpz2GuiDMmqj2le9yBwqI6Z/z5VWDO01GN5MrEoDvH/\n\tfO1eha3L1J1RCTd2PJQXl/dGIDzkAfGZJZeIppVIwqKP7Y/b11E2fzZKgLFKAaHh6c\n\t26wkGYGASR8afmb4c/TWxX+gtOnCtth+dGBuxskc=","Date":"Wed, 13 Aug 2025 16:07:13 +0300","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [RFC PATCH v2 2/2] libcamera: pipeline: virtual: Move image\n\tgeneration to separate thread","Message-ID":"<20250813130713.GF20174@pendragon.ideasonboard.com>","References":"<20250813102501.1645940-1-barnabas.pocze@ideasonboard.com>\n\t<20250813102501.1645940-3-barnabas.pocze@ideasonboard.com>\n\t<20250813113417.GG6440@pendragon.ideasonboard.com>\n\t<82a0153b-809c-4481-a740-b704de3035f2@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<82a0153b-809c-4481-a740-b704de3035f2@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]