[{"id":38164,"web_url":"https://patchwork.libcamera.org/comment/38164/","msgid":"<71011e07-5e5b-4c00-85f4-3f6609b286cf@ideasonboard.com>","date":"2026-02-10T15:49:25","subject":"Re: [PATCH 3/3] pipeline/ipa: rpi: vc4: Use extensible parameter\n\tbuffers for ISP configuration","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n2026. 02. 10. 9:25 keltezéssel, Jai Luthra írta:\n> The downstream VC4 (BCM2835) ISP driver used custom V4L2 controls to\n> pass per-frame ISP configuration from userspace, which diverged from\n> upstream ISP drivers (rkisp1, mali-c55, PiSP) which all use parameter\n> buffers. As the driver is being upstreamed now, switch the VC4 IPA and\n> pipeline handler to use the extensible parameter buffer interface\n> instead.\n> \n> Introduce a Bcm2835Params helper class (params.h) that wraps V4L2Params.\n> Each algorithm's applyXYZ function now writes directly into typed\n> parameter blocks of the params buffer using this class.\n> \n> On the pipeline handler side, update the ISP media entity names to match\n> the upstream driver. Register the new bcm2835-isp-params metadata output\n> node as an additional ISP stream.\n> \n> Parameter buffer handling is a bit more complicated compared to PiSP, as\n> we use extensible buffers. So, the buffer is acquired per frame in\n> tryRunPipeline(), and is mapped and passed to the IPA for populating.\n> The IPA signals the completion along with the actual number of bytes\n> used in prepareIspComplete().\n> \n> Also clean up the now unused ISP control related code, and populate the\n> lens shading DMABUF in the IPA itself instead of patching it in the\n> pipeline handler.\n\nI think this unfortunately breaks IPA isolation because in that case they\nhave different file descriptor sets. And I imagine the dma buf will be queried\nfrom the process that queues the parameter buffer?\n\n\nRegards,\nBarnabás Pőcze\n\n\n> \n> Signed-off-by: Jai Luthra \n> ---\n> include/libcamera/ipa/raspberrypi.mojom | 15 +-\n> src/ipa/rpi/common/ipa_base.cpp | 11 +-\n> src/ipa/rpi/common/ipa_base.h | 3 +\n> src/ipa/rpi/vc4/params.h | 77 ++++++++\n> src/ipa/rpi/vc4/vc4.cpp | 232 +++++++++++--------------\n> src/libcamera/pipeline/rpi/common/rpi_stream.h | 1 +\n> src/libcamera/pipeline/rpi/pisp/pisp.cpp | 6 +-\n> src/libcamera/pipeline/rpi/vc4/vc4.cpp | 108 +++++++-----\n> 8 files changed, 260 insertions(+), 193 deletions(-)\n> \n> [...]\n> diff --git a/src/ipa/rpi/vc4/vc4.cpp b/src/ipa/rpi/vc4/vc4.cpp\n> index 2b205b2861bdb5bf81abd61fc538e9d39215293e..b35ece7185136f720e4b8a3afefc1c78f92d42b9 100644\n> --- a/src/ipa/rpi/vc4/vc4.cpp\n> +++ b/src/ipa/rpi/vc4/vc4.cpp\n> @@ -30,6 +30,7 @@\n> [...]\n> -void IpaVc4::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls)\n> +void IpaVc4::applyLS(const struct AlscStatus *lsStatus, Bcm2835Params ¶ms)\n> {\n> \t/*\n> \t * Program lens shading tables into pipeline.\n> @@ -542,24 +508,24 @@ void IpaVc4::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls)\n> \n> \t/* We're going to supply corner sampled tables, 16 bit samples. */\n> \tw++, h++;\n> -\tbcm2835_isp_lens_shading ls = {\n> +\tif (!lsTableHandle_.isValid() || !lsTable_ || w * h * 4 * sizeof(uint16_t) > MaxLsGridSize) {\n> +\t\tLOG(IPARPI, Error) << \"Do not have a correctly allocated lens shading table!\";\n> +\t\treturn;\n> +\t}\n> +\n> +\tauto block = params.block();\n> +\tblock->ls = {\n> \t\t.enabled = 1,\n> \t\t.grid_cell_size = cellSize,\n> \t\t.grid_width = w,\n> \t\t.grid_stride = w,\n> \t\t.grid_height = h,\n> -\t\t/* .dmabuf will be filled in by pipeline handler. */\n> -\t\t.dmabuf = 0,\n> +\t\t.dmabuf = lsTableHandle_.get(),\n> \t\t.ref_transform = 0,\n> \t\t.corner_sampled = 1,\n> \t\t.gain_format = GAIN_FORMAT_U4P10\n> \t};\n> \n> -\tif (!lsTable_ || w * h * 4 * sizeof(uint16_t) > MaxLsGridSize) {\n> -\t\tLOG(IPARPI, Error) << \"Do not have a correctly allocate lens shading table!\";\n> -\t\treturn;\n> -\t}\n> -\n> \tif (lsStatus) {\n> \t\t/* Format will be u4.10 */\n> \t\tuint16_t *grid = static_cast(lsTable_);\n> @@ -570,9 +536,7 @@ void IpaVc4::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls)\n> \t\tresampleTable(grid + 3 * w * h, lsStatus->b, w, h);\n> \t}\n> \n> -\tControlValue c(Span{ reinterpret_cast(&ls),\n> -\t\t\t\t\t sizeof(ls) });\n> -\tctrls.set(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING, c);\n> +\tblock.setEnabled(true);\n> }\n> \n> void IpaVc4::applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls)\n> [...]\n> diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n> index cd1ec4f486990bf620f44bc6cd3ef6ae90c91327..b403992d1f5d32fd79436d7c5652afd9715ada0b 100644\n> --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n> +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n> @@ -32,7 +32,7 @@ using StreamParams = RPi::RPiCameraConfiguration::StreamParams;\n> [...]\n> -void Vc4CameraData::setIspControls(const ControlList &controls)\n> -{\n> -\tControlList ctrls = controls;\n> -\n> -\tif (ctrls.contains(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING)) {\n> -\t\tControlValue &value =\n> -\t\t\tconst_cast(ctrls.get(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING));\n> -\t\tSpan s = value.data();\n> -\t\tbcm2835_isp_lens_shading *ls =\n> -\t\t\treinterpret_cast(s.data());\n> -\t\tls->dmabuf = lsTable_.get();\n> -\t}\n> -\n> -\tisp_[Isp::Input].dev()->setControls(&ctrls);\n> -\thandleState();\n> -}\n> -\n> [...]","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C6A7ABD78E\n\tfor ;\n\tTue, 10 Feb 2026 15:49:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1ADD46218C;\n\tTue, 10 Feb 2026 16:49:31 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 89802620FA\n\tfor ;\n\tTue, 10 Feb 2026 16:49:29 +0100 (CET)","from [192.168.33.64] (185.221.141.206.nat.pool.zt.hu\n\t[185.221.141.206])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B611BE47;\n\tTue, 10 Feb 2026 16:48:42 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"rI/Z1xML\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1770738523;\n\tbh=OUjRpWlJds9eg0fkelvlTHJlO+Kx8b81oTffITyUJng=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=rI/Z1xML04iYgd+TIdrGY4Yu7gxU34jWLjH7ZSCC0yDoAxFBzsKon4JitqDUW96W0\n\tn2wIOd+BrjJoh88I7HawuziFPVO0YTzfF3LbuG2PzGgsRXMzKWMZg3NjNVQ0vtNJMo\n\tdc5z8L5Z3RWhVIOYmIXPCZUizgRP1feeZAh5jApA=","Message-ID":"<71011e07-5e5b-4c00-85f4-3f6609b286cf@ideasonboard.com>","Date":"Tue, 10 Feb 2026 16:49:25 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH 3/3] pipeline/ipa: rpi: vc4: Use extensible parameter\n\tbuffers for ISP configuration","To":"Jai Luthra ,\n\tlibcamera-devel@lists.libcamera.org","Cc":"Naushir Patuck ,\n\tDavid Plowman ,\n\tLaurent Pinchart ,\n\tKieran Bingham ,\n\tJacopo Mondi ,\n\tDaniel Scally ","References":"<20260210-pi4-upstream-v1-0-279841c15fba@ideasonboard.com>\n\t<20260210-pi4-upstream-v1-3-279841c15fba@ideasonboard.com>","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"<20260210-pi4-upstream-v1-3-279841c15fba@ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":38168,"web_url":"https://patchwork.libcamera.org/comment/38168/","msgid":"<177079240307.2245984.792204708132545778@freya>","date":"2026-02-11T06:46:43","subject":"Re: [PATCH 3/3] pipeline/ipa: rpi: vc4: Use extensible parameter\n\tbuffers for ISP configuration","submitter":{"id":223,"url":"https://patchwork.libcamera.org/api/people/223/","name":"Jai Luthra","email":"jai.luthra@ideasonboard.com"},"content":"Hi Barnabás,\n\nThanks for the review.\n\nQuoting Barnabás Pőcze (2026-02-10 21:19:25)\n> Hi\n> \n> 2026. 02. 10. 9:25 keltezéssel, Jai Luthra írta:\n> > The downstream VC4 (BCM2835) ISP driver used custom V4L2 controls to\n> > pass per-frame ISP configuration from userspace, which diverged from\n> > upstream ISP drivers (rkisp1, mali-c55, PiSP) which all use parameter\n> > buffers. As the driver is being upstreamed now, switch the VC4 IPA and\n> > pipeline handler to use the extensible parameter buffer interface\n> > instead.\n> > \n> > Introduce a Bcm2835Params helper class (params.h) that wraps V4L2Params.\n> > Each algorithm's applyXYZ function now writes directly into typed\n> > parameter blocks of the params buffer using this class.\n> > \n> > On the pipeline handler side, update the ISP media entity names to match\n> > the upstream driver. Register the new bcm2835-isp-params metadata output\n> > node as an additional ISP stream.\n> > \n> > Parameter buffer handling is a bit more complicated compared to PiSP, as\n> > we use extensible buffers. So, the buffer is acquired per frame in\n> > tryRunPipeline(), and is mapped and passed to the IPA for populating.\n> > The IPA signals the completion along with the actual number of bytes\n> > used in prepareIspComplete().\n> > \n> > Also clean up the now unused ISP control related code, and populate the\n> > lens shading DMABUF in the IPA itself instead of patching it in the\n> > pipeline handler.\n> \n> I think this unfortunately breaks IPA isolation because in that case they\n> have different file descriptor sets. And I imagine the dma buf will be queried\n> from the process that queues the parameter buffer?\n> \n\nArgh, sorry I didn't read the IPA docs carefully. This worked out in my\ntesting. Now I see that RPi4 does not enforce process isolation between IPA\nand pipeline handler.\n\n(I tried setting LIBCAMERA_IPA_FORCE_ISOLATION=1 now, but that fails to\n even configure the IPA, do you know if that's supposed to be broken?)\n\nIn any case, I will move the FD population back to the pipeline handler in\nv2.\n\nThanks,\n Jai\n\n> \n> Regards,\n> Barnabás Pőcze\n> \n> \n> > \n> > Signed-off-by: Jai Luthra \n> > ---\n> > include/libcamera/ipa/raspberrypi.mojom | 15 +-\n> > src/ipa/rpi/common/ipa_base.cpp | 11 +-\n> > src/ipa/rpi/common/ipa_base.h | 3 +\n> > src/ipa/rpi/vc4/params.h | 77 ++++++++\n> > src/ipa/rpi/vc4/vc4.cpp | 232 +++++++++++--------------\n> > src/libcamera/pipeline/rpi/common/rpi_stream.h | 1 +\n> > src/libcamera/pipeline/rpi/pisp/pisp.cpp | 6 +-\n> > src/libcamera/pipeline/rpi/vc4/vc4.cpp | 108 +++++++-----\n> > 8 files changed, 260 insertions(+), 193 deletions(-)\n> > \n> > [...]\n> > diff --git a/src/ipa/rpi/vc4/vc4.cpp b/src/ipa/rpi/vc4/vc4.cpp\n> > index 2b205b2861bdb5bf81abd61fc538e9d39215293e..b35ece7185136f720e4b8a3afefc1c78f92d42b9 100644\n> > --- a/src/ipa/rpi/vc4/vc4.cpp\n> > +++ b/src/ipa/rpi/vc4/vc4.cpp\n> > @@ -30,6 +30,7 @@\n> > [...]\n> > -void IpaVc4::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls)\n> > +void IpaVc4::applyLS(const struct AlscStatus *lsStatus, Bcm2835Params ¶ms)\n> > {\n> > /*\n> > * Program lens shading tables into pipeline.\n> > @@ -542,24 +508,24 @@ void IpaVc4::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls)\n> > \n> > /* We're going to supply corner sampled tables, 16 bit samples. */\n> > w++, h++;\n> > - bcm2835_isp_lens_shading ls = {\n> > + if (!lsTableHandle_.isValid() || !lsTable_ || w * h * 4 * sizeof(uint16_t) > MaxLsGridSize) {\n> > + LOG(IPARPI, Error) << \"Do not have a correctly allocated lens shading table!\";\n> > + return;\n> > + }\n> > +\n> > + auto block = params.block();\n> > + block->ls = {\n> > .enabled = 1,\n> > .grid_cell_size = cellSize,\n> > .grid_width = w,\n> > .grid_stride = w,\n> > .grid_height = h,\n> > - /* .dmabuf will be filled in by pipeline handler. */\n> > - .dmabuf = 0,\n> > + .dmabuf = lsTableHandle_.get(),\n> > .ref_transform = 0,\n> > .corner_sampled = 1,\n> > .gain_format = GAIN_FORMAT_U4P10\n> > };\n> > \n> > - if (!lsTable_ || w * h * 4 * sizeof(uint16_t) > MaxLsGridSize) {\n> > - LOG(IPARPI, Error) << \"Do not have a correctly allocate lens shading table!\";\n> > - return;\n> > - }\n> > -\n> > if (lsStatus) {\n> > /* Format will be u4.10 */\n> > uint16_t *grid = static_cast(lsTable_);\n> > @@ -570,9 +536,7 @@ void IpaVc4::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls)\n> > resampleTable(grid + 3 * w * h, lsStatus->b, w, h);\n> > }\n> > \n> > - ControlValue c(Span{ reinterpret_cast(&ls),\n> > - sizeof(ls) });\n> > - ctrls.set(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING, c);\n> > + block.setEnabled(true);\n> > }\n> > \n> > void IpaVc4::applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls)\n> > [...]\n> > diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n> > index cd1ec4f486990bf620f44bc6cd3ef6ae90c91327..b403992d1f5d32fd79436d7c5652afd9715ada0b 100644\n> > --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n> > +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n> > @@ -32,7 +32,7 @@ using StreamParams = RPi::RPiCameraConfiguration::StreamParams;\n> > [...]\n> > -void Vc4CameraData::setIspControls(const ControlList &controls)\n> > -{\n> > - ControlList ctrls = controls;\n> > -\n> > - if (ctrls.contains(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING)) {\n> > - ControlValue &value =\n> > - const_cast(ctrls.get(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING));\n> > - Span s = value.data();\n> > - bcm2835_isp_lens_shading *ls =\n> > - reinterpret_cast(s.data());\n> > - ls->dmabuf = lsTable_.get();\n> > - }\n> > -\n> > - isp_[Isp::Input].dev()->setControls(&ctrls);\n> > - handleState();\n> > -}\n> > -\n> > [...]","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 24F55C32EA\n\tfor ;\n\tWed, 11 Feb 2026 06:46:52 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 61011621A2;\n\tWed, 11 Feb 2026 07:46:51 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 75C3661FA0\n\tfor ;\n\tWed, 11 Feb 2026 07:46:50 +0100 (CET)","from mail.ideasonboard.com (unknown\n\t[IPv6:2401:4900:1c30:2edd:807a:f3c0:8d1b:28a])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D87B11AC5;\n\tWed, 11 Feb 2026 07:46:01 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"nY476EKZ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1770792362;\n\tbh=WBAq+E99cfy22jlBrSqOW+EK6FKSmYBwtoREQovcqvg=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=nY476EKZhJ7AOyZv/G0pIRYLhYB0H5SbdO4cEF4ihRFwPZICgi2LnQkgYxqSxu0ae\n\t7brOXsVovUguW0FtrsirC5kr70chuukZpq828MWir9b4Ae+OSOGxPbogKfTzyOnNa3\n\t8Pz3TzlPWoZkdrbcskIRomzPAvV7DNc5lwcOE/tk=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<71011e07-5e5b-4c00-85f4-3f6609b286cf@ideasonboard.com>","References":"<20260210-pi4-upstream-v1-0-279841c15fba@ideasonboard.com>\n\t<20260210-pi4-upstream-v1-3-279841c15fba@ideasonboard.com>\n\t<71011e07-5e5b-4c00-85f4-3f6609b286cf@ideasonboard.com>","Subject":"Re: [PATCH 3/3] pipeline/ipa: rpi: vc4: Use extensible parameter\n\tbuffers for ISP configuration","From":"Jai Luthra ","Cc":"Naushir Patuck ,\n\tDavid Plowman ,\n\tLaurent Pinchart ,\n\tKieran Bingham ,\n\tJacopo Mondi ,\n\tDaniel Scally ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Wed, 11 Feb 2026 12:16:43 +0530","Message-ID":"<177079240307.2245984.792204708132545778@freya>","User-Agent":"alot/0.13.dev20+g31692a239","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]