[{"id":39811,"web_url":"https://patchwork.libcamera.org/comment/39811/","msgid":"","date":"2026-07-23T16:17:44","subject":"Re: [RFC PATCH v2 1/1] libcamera: Add independent queue for\n\tControlLists","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Hi everyone\n\nI'd like to give this topic another prod. I talked about it a bit at\nNice, and revisiting this patch seems like a way to start some\ndiscussion, though the precise implementation here is more for\nillustration.\n\nOne of the principal motivations is for things like burst captures,\nwhere you don't want to get held up at the back of the request queue.\n\nDoes this seem like a reasonable thing to do, or are there better alternatives?\n\nThanks\n\nDavid\n\nOn Thu, 12 Mar 2026 at 16:00, David Plowman\n wrote:\n>\n> Add `Camera::queueControls()` whose purpose is to apply controls as\n> soon as possible, without going through `Request::controls()`.\n>\n> A new virtual function `PipelineHandler::queueControlsDevice()` is\n> provided for pipeline handler to implement fast-tracked application of\n> controls. If the pipeline handler does not implement that\n> functionality, or it fails, then a fallback mechanism is used. The\n> controls will be saved for later, and they will be merged into the\n> control list of the next available request sent to the pipeline\n> handler (`Camera::Private::waitingRequests_`).\n>\n> This patch is derived directly from Barnabas's previous verion that\n> implemented the same idea but with a single ControlList, rather than\n> allowing multiple ControlLists to be queued up for consecutive\n> frames.\n>\n> Signed-off-by: David Plowman \n> ---\n> include/libcamera/camera.h | 1 +\n> include/libcamera/internal/camera.h | 1 +\n> include/libcamera/internal/pipeline_handler.h | 7 ++\n> src/libcamera/camera.cpp | 61 +++++++++++++++\n> src/libcamera/pipeline_handler.cpp | 76 +++++++++++++++++++\n> 5 files changed, 146 insertions(+)\n>\n> diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h\n> index b24a2974..93a484e4 100644\n> --- a/include/libcamera/camera.h\n> +++ b/include/libcamera/camera.h\n> @@ -147,6 +147,7 @@ public:\n>\n> std::unique_ptr createRequest(uint64_t cookie = 0);\n> int queueRequest(Request *request);\n> + int queueControls(ControlList &&controls);\n>\n> int start(const ControlList *controls = nullptr);\n> int stop();\n> diff --git a/include/libcamera/internal/camera.h b/include/libcamera/internal/camera.h\n> index 8a2e9ed5..17dda925 100644\n> --- a/include/libcamera/internal/camera.h\n> +++ b/include/libcamera/internal/camera.h\n> @@ -38,6 +38,7 @@ public:\n>\n> std::list queuedRequests_;\n> std::queue waitingRequests_;\n> + std::queue queuedControls_;\n> ControlInfoMap controlInfo_;\n> ControlList properties_;\n>\n> diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h\n> index b4f97477..c25213de 100644\n> --- a/include/libcamera/internal/pipeline_handler.h\n> +++ b/include/libcamera/internal/pipeline_handler.h\n> @@ -57,6 +57,7 @@ public:\n>\n> void registerRequest(Request *request);\n> void queueRequest(Request *request);\n> + int queueControls(Camera *camera, ControlList controls);\n>\n> bool completeBuffer(Request *request, FrameBuffer *buffer);\n> void completeRequest(Request *request);\n> @@ -76,6 +77,12 @@ protected:\n> unsigned int useCount() const { return useCount_; }\n>\n> virtual int queueRequestDevice(Camera *camera, Request *request) = 0;\n> +\n> + virtual int queueControlsDevice([[maybe_unused]] Camera *camera, [[maybe_unused]] const ControlList &controls)\n> + {\n> + return -EOPNOTSUPP;\n> + }\n> +\n> virtual void stopDevice(Camera *camera) = 0;\n>\n> virtual bool acquireDevice(Camera *camera);\n> diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp\n> index f724a1be..f0244707 100644\n> --- a/src/libcamera/camera.cpp\n> +++ b/src/libcamera/camera.cpp\n> @@ -637,6 +637,16 @@ Camera::Private::~Private()\n> * queued requests was reached.\n> */\n>\n> +/**\n> + * \\var Camera::Private::queuedControls_\n> + * \\brief The queue of pending control lists\n> + *\n> + * This queue maintains a list of all the control lists that need to be sent\n> + * to the pipeline handler with subsequent requests. The top item in the queue\n> + * will always be sent with the next request going to\n> + * PipelineHandler::queueRequestDevice().\n> + */\n> +\n> /**\n> * \\var Camera::Private::controlInfo_\n> * \\brief The set of controls supported by the camera\n> @@ -1378,6 +1388,57 @@ int Camera::queueRequest(Request *request)\n> return 0;\n> }\n>\n> +/**\n> + * \\brief Queue controls to be applied as soon as possible\n> + * \\param[in] controls The list of controls to queue\n> + *\n> + * This function tries to ensure that the controls in \\a controls are applied\n> + * to the camera as soon as possible. If there are still pending controls waiting\n> + * to be applied (because of previous calls to Camera::queueControls), then\n> + * these controls will be applied as soon as possible on a frame after those.\n> + *\n> + * The exact guarantees are camera dependent, but it is guaranteed that the\n> + * controls will be applied no later than with the next \\ref Request\"request\"\n> + * that the application \\ref Camera::queueRequest() \"queues\" (after any requests\n> + * have been *used up\" for sending previously queued controls).\n> + *\n> + * \\context This function is \\threadsafe. It may only be called when the camera\n> + * is in the Running state as defined in \\ref camera_operation.\n> + *\n> + * \\return 0 on success or a negative error code otherwise\n> + * \\retval -ENODEV The camera has been disconnected from the system\n> + * \\retval -EACCES The camera is not running\n> + */\n> +int Camera::queueControls(ControlList &&controls)\n> +{\n> + Private *const d = _d();\n> +\n> + /*\n> + * Like requests, controls can't be queued if the camera is not running.\n> + * Controls can be applied immediately when the camera starts using the\n> + * Camera::Start method.\n> + */\n> +\n> + int ret = d->isAccessAllowed(Private::CameraRunning);\n> + if (ret < 0)\n> + return ret;\n> +\n> + /*\n> + * We want to be able to queue empty control lists, as this gives a way of\n> + * forcing another frame with the same controls as last time, before queueing\n> + * another control list that might change them again.\n> + */\n> +\n> + patchControlList(controls);\n> +\n> + /*\n> + * \\todo Or `ConnectionTypeBlocking` to get the return value?\n> + */\n> + d->pipe_->invokeMethod(&PipelineHandler::queueControls, ConnectionTypeQueued, this, std::move(controls));\n> +\n> + return 0;\n> +}\n> +\n> /**\n> * \\brief Start capture from camera\n> * \\param[in] controls Controls to be applied before starting the Camera\n> diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp\n> index 5c469e5b..1a87b28c 100644\n> --- a/src/libcamera/pipeline_handler.cpp\n> +++ b/src/libcamera/pipeline_handler.cpp\n> @@ -398,6 +398,12 @@ void PipelineHandler::stop(Camera *camera)\n> ASSERT(data->queuedRequests_.empty());\n> ASSERT(data->waitingRequests_.empty());\n>\n> + /*\n> + * Clear out any unapplied controls. If an application wants to be\n> + * sure controls have been applied, it should wait before stopping.\n> + */\n> + data->queuedControls_ = {};\n> +\n> data->requestSequence_ = 0;\n> }\n>\n> @@ -477,6 +483,49 @@ void PipelineHandler::queueRequest(Request *request)\n> request->_d()->prepare(300ms);\n> }\n>\n> +/**\n> + * \\brief Queue controls to apply as soon as possible\n> + * \\param[in] camera The camera\n> + * \\param[in] controls The controls to apply\n> + *\n> + * This function tries to queue \\a controls immediately to the device by\n> + * calling queueControlsDevice(). If that fails, then a fallback mechanism\n> + * is used to ensure that \\a controls will be merged into the control list\n> + * of the next available request submitted to the pipeline handler.\n> + *\n> + * \\context This function is called from the CameraManager thread.\n> + */\n> +int PipelineHandler::queueControls(Camera *camera, ControlList controls)\n> +{\n> + Camera::Private *data = camera->_d();\n> + int ret = queueControlsDevice(camera, controls);\n> +\n> + /*\n> + * Don't worry about later request's controls overriding the ones\n> + * sent here - the application needs to deal with that.\n> + */\n> +\n> + if (ret == -EOPNOTSUPP) {\n> + /*\n> + * Fall back to adding the controls to the next request that enters the\n> + * pipeline handler. See PipelineHandler::doQueueRequest().\n> + */\n> + data->queuedControls_.push(std::move(controls));\n> +\n> + /* Counts as \"success\". */\n> + ret = 0;\n> +\n> + } else if (ret < 0) {\n> + /*\n> + * The pipeline handler is claiming to support queueControlsDevice,\n> + * but it has failed. This is an error.\n> + */\n> + LOG(Pipeline, Debug) << \"Fast tracking controls failed: \" << res;\n> + }\n> +\n> + return ret;\n> +}\n> +\n> /**\n> * \\brief Queue one requests to the device\n> */\n> @@ -495,9 +544,21 @@ void PipelineHandler::doQueueRequest(Request *request)\n> return;\n> }\n>\n> + if (!data->queuedControls_.empty()) {\n> + /*\n> + * Note that `ControlList::MergePolicy::KeepExisting` is used. This is\n> + * needed to ensure that if `request` is newer than pendingControls_,\n> + * then its controls take precedence.\n> + */\n> + request->controls().merge(data->queuedControls_.front(),\n> + ControlList::MergePolicy::KeepExisting);\n> + }\n> +\n> int ret = queueRequestDevice(camera, request);\n> if (ret)\n> cancelRequest(request);\n> + else if (!data->queuedControls_.empty())\n> + data->queuedControls_.pop();\n> }\n>\n> /**\n> @@ -543,6 +604,21 @@ void PipelineHandler::doQueueRequests(Camera *camera)\n> * \\return 0 on success or a negative error code otherwise\n> */\n>\n> +/**\n> + * \\fn PipelineHandler::queueControlsDevice()\n> + * \\brief Queue controls to be applied as soon as possible\n> + * \\param[in] camera The camera\n> + * \\param[in] controls The controls to apply\n> + *\n> + * This function queues \\a controls to \\a camera so that they can be\n> + * applied as soon as possible\n> + *\n> + * \\context This function is called from the CameraManager thread.\n> + *\n> + * \\return 0 on success or a negative error code otherwise\n> + * \\return -EOPNOTSUPP if fast-tracking controls is not supported\n> + */\n> +\n> /**\n> * \\brief Complete a buffer for a request\n> * \\param[in] request The request the buffer belongs to\n> --\n> 2.47.3\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6D6F0BDE17\n\tfor ;\n\tThu, 23 Jul 2026 16:17:58 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 76ED067EC4;\n\tThu, 23 Jul 2026 18:17:57 +0200 (CEST)","from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com\n\t[IPv6:2a00:1450:4864:20::62d])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 19FAB67E5C\n\tfor ;\n\tThu, 23 Jul 2026 18:17:56 +0200 (CEST)","by mail-ej1-x62d.google.com with SMTP id\n\ta640c23a62f3a-c1c24ec9525so131375566b.1\n\tfor ;\n\tThu, 23 Jul 2026 09:17:56 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"lTxtzfvI\"; dkim-atps=neutral","ARC-Seal":"i=1; a=rsa-sha256; t=1784823475; cv=none;\n\td=google.com; s=arc-20260327;\n\tb=AW1u4at+BDztRvnMEUVpfWEv+y5/ZBLht93EWQvbPu02k/KB37FIQym0X8HOEzY+6f\n\tiKkAmF0v6eAO0EM+vJyXQL47BVxJXKosxNzZbxhoZNFvIeIp0ydPk6W19FjhESA7GTNv\n\tSzD2odguk1cCffA/jTjqTyVDeiGjXVHv/Brp3Gsj5nWjxaf85ez/UwR7Bv/UlfM+c3EY\n\tCcpSse0wX94SHSqpSavXi2ZkFC2G023mSIrI8kqfAgROAdVj7/e7ZbgIURYzxgOygTl5\n\t10K3KBzQVxJIY619oO6LqI9O4SsI3KX/KH298vTIYsgo8PNXOec7PdpIl37qGoCwqr65\n\toerg==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n\ts=arc-20260327; \n\th=to:subject:message-id:date:from:in-reply-to:references:mime-version\n\t:dkim-signature;\n\tbh=9E1GwF4BiWtLxpTNvvqMimTg+Em78CxiWUTggiKjeTo=;\n\tfh=wtmKwyi0l3IkgfllUcb23GUQ5DBcFLPuxebUMmhSdY0=;\n\tb=TMejbxYX36XyqofHdD6eDKekwx+iqQ+AdOxgPeYvsvVhFhD0bhXUnolfAC+Togo66l\n\tk1TX+5kF7R7FvwyhfHZniKcNPHdRxqUPORdM74CNiReioLR2hxIg1JPohgcAAR47rw9+\n\tqlgeG36uGmmU/mWSUHJYIwJ9BvRY3ggTcUQrk6nGkpkWEerQYMBiOGeFqJVUFrMScSK0\n\t16AKnXwddnTfct8njdVWrWf9RtSSNwFisa4Kc16X43OKj2IMhbVhdf4urrvzjkQpmzkL\n\t8O48f+g6IuA4Sv3cp3DcXz8K08kQlMarlTpGyASnUsS6zuVYN4ztRN/gY5IARsLhAuj1\n\t5GLQ==; darn=lists.libcamera.org","ARC-Authentication-Results":"i=1; mx.google.com; arc=none","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1784823475; x=1785428275;\n\tdarn=lists.libcamera.org; \n\th=content-type:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to\n\t:content-type; bh=9E1GwF4BiWtLxpTNvvqMimTg+Em78CxiWUTggiKjeTo=;\n\tb=lTxtzfvI8vv7KhuFLVV6ZFIhdEUzVfBM0nayrgrSoT6XrMzf/va0AmoYlNVcUfxXEC\n\tRP6bY07mVg/FN599cWWYnDvywjHczEcd+IbPuuBFNrkgIRiX2PbJflYlPa/K1rMHBJqy\n\tBfi6WE2hXxnFcvhTJ4qt617dXKQhdYGxgZNnGQAmF+pLLEbEYC3TGqkLT4lTte9TQB5X\n\tINYxJ17iezb3ccGbPYyVruSaXomzTBAD86tbXhA6zQKuweYXpInfqOAQ0IBo/sfpCfV7\n\tj/aqrV1nAqlnrGc9Z2rQnbLGfENfut27KSRaimqHux0l5+6H59DKG+eKCjh94HUDQN/U\n\tY/aQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20251104; t=1784823475; x=1785428275;\n\th=content-type:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date\n\t:message-id:reply-to:content-type;\n\tbh=9E1GwF4BiWtLxpTNvvqMimTg+Em78CxiWUTggiKjeTo=;\n\tb=iMhWgQocelD+FsQ1xbKxGmMjnbPr6mCaWLznVf55JVL8QTHLFq4aKYFfZL+BGhT9WW\n\tklsoWrvF1qESnFSDoYeO5klEU2JcbiVDQ5ejiEOMYUKURku6DRAiKjyeE5r1kDAO7ek2\n\tpe55Ae9zObiqwiBtFGT3p7QFt7lNRE6z0RVxUYUREcQySqyqIGBmjWAS4A3SDGWBezrA\n\tW2UQsBCd55dfGTaGLz7t3DWP2w+ak941XbtOOiN+7Pe2pYbsxKbhQ+09zF/pxuX+mrUu\n\tLMsPpI/D6lujyo1ucys0IbUYY1+iRlHasvG7MMvTXBx2MyEj+mYR0y7KJK4QupWFnv3g\n\tRj+A==","X-Gm-Message-State":"AOJu0Yx+DFCHeZO7gBaMu2u7qi54aRtSvf8xCUKeRTzbARnL5ZqUXMfm\n\tE03CwSMQ/iXVyNoN8MqYWctLGPKZxI5wvcHwZSYcR9lw0D9Jk8fsD6+Uiyvg+zDzEQtihrJvusE\n\tI1wHY8b6B1owLW5fohe4McnPuzFBgpU6bN0mcJgBmuNrPrzMPFyWMaEY=","X-Gm-Gg":"AR+sD10RCzUOb20XbwsQ/dCwvPP7IST427bT7tOqekf44neZ3KnfxKBbfehQJrizljj\n\t4TM0aAJIVrC+E82s1YhaVsw99TN7ZPbSHxnvd7g8DFWQxEcPpHmtZoEdyR8zj6AokuU7DPa416P\n\tPV+qQIShF4QTj0OPuz+SjeXqNQQGlVxfU77LE7Ey93AF7KzjGHFxFZSnMaKuDBnU7qdXZK9n+M/\n\tBwO7CHo0u/Vnb8oFyS7QCLw/FH4MCTavbDkqOld589RsFnPVpx74C09qGVBZs8JtpDdSPo/PPnD\n\tckleAqlHFEKIj49F+SEB2TbFLE1mwg/9yykftBMbLTb31BqnXt3KBGbeSOw1n4y9knNwz4de28Z\n\t/+p4=","X-Received":"by 2002:a17:906:9fce:b0:c16:a9e0:d5eb with SMTP id\n\ta640c23a62f3a-c1c509a83b2mr192157766b.56.1784823475170;\n\tThu, 23 Jul 2026 09:17:55 -0700 (PDT)","MIME-Version":"1.0","References":"<20260312160009.18654-1-david.plowman@raspberrypi.com>\n\t<20260312160009.18654-2-david.plowman@raspberrypi.com>","In-Reply-To":"<20260312160009.18654-2-david.plowman@raspberrypi.com>","From":"David Plowman ","Date":"Thu, 23 Jul 2026 17:17:44 +0100","X-Gm-Features":"AUfX_mydVJdwuN7_tqq6tVbQpuz-wUHai2W9aPKfeUoPU9KLfgQuC-taDsJg7l4","Message-ID":"","Subject":"Re: [RFC PATCH v2 1/1] libcamera: Add independent queue for\n\tControlLists","To":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"UTF-8\"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]