{"id":20291,"url":"https://patchwork.libcamera.org/api/1.1/patches/20291/?format=json","web_url":"https://patchwork.libcamera.org/patch/20291/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240613132602.1021721-4-dan.scally@ideasonboard.com>","date":"2024-06-13T13:25:55","name":"[03/10] libcamera: mali-c55: Acquire and plumb in 3a params and stats","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"2456b30163c337683a616beb8bfbe0944e7a61fb","submitter":{"id":156,"url":"https://patchwork.libcamera.org/api/1.1/people/156/?format=json","name":"Dan Scally","email":"dan.scally@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/20291/mbox/","series":[{"id":4388,"url":"https://patchwork.libcamera.org/api/1.1/series/4388/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4388","date":"2024-06-13T13:25:52","name":"Add Mali-C55 IPA Module and Algorithms","version":1,"mbox":"https://patchwork.libcamera.org/series/4388/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/20291/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/20291/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D900FC3237\n\tfor ;\n\tThu, 13 Jun 2024 13:26:43 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4B36C65497;\n\tThu, 13 Jun 2024 15:26:36 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 78AD665496\n\tfor ;\n\tThu, 13 Jun 2024 15:26:29 +0200 (CEST)","from mail.ideasonboard.com\n\t(cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 5AFB2114D;\n\tThu, 13 Jun 2024 15:26:15 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"JLpuReH0\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1718285175;\n\tbh=40zI2IvIsvO2RNHw7qz3y5QecMVOF19dUN9xJ93T7t0=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=JLpuReH0cy7MdSBHI2xWLVEv28W/I4lK8D30wIbUAbg8EQiFKu7P8M68aCIaRTbZ9\n\t0UAy+nXgM30HFHqzgsbFo/3nuFb/GRHnjqVB7aluzUYisGksflqZ58oYu2TbJVIi93\n\t0LfCEV2o4dZeyGL1yTK/AXZeYaTO3a3IBzzG5gds=","From":"Daniel Scally ","To":"libcamera-devel@lists.libcamera.org","Cc":"dan.scally@ideasonboard.com, nayden.kanchev@arm.com,\n\tjacopo.mondi@ideasonboard.com","Subject":"[PATCH 03/10] libcamera: mali-c55: Acquire and plumb in 3a params\n\tand stats","Date":"Thu, 13 Jun 2024 14:25:55 +0100","Message-Id":"<20240613132602.1021721-4-dan.scally@ideasonboard.com>","X-Mailer":"git-send-email 2.34.1","In-Reply-To":"<20240613132602.1021721-1-dan.scally@ideasonboard.com>","References":"<20240613132602.1021721-1-dan.scally@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Acquire the mali-c55 3a stats and parameters video devices during\n::match() and plumb them in.\n\nFor this commit we simply allocate and release buffers for the\nstatistics and parameters. Statistics buffers are queue and dequeued\nfrom the stats video device but their contents are for now untouched.\n\nAcked-by: Nayden Kanchev \nCo-developed-by: Jacopo Mondi \nSigned-off-by: Jacopo Mondi \nSigned-off-by: Daniel Scally \n---\n src/libcamera/pipeline/mali-c55/mali-c55.cpp | 153 ++++++++++++++++++-\n 1 file changed, 150 insertions(+), 3 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\nindex f48bbe0d..b9be1207 100644\n--- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n+++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n@@ -12,6 +12,7 @@\n #include \n #include \n \n+#include \n #include \n #include \n \n@@ -26,6 +27,7 @@\n #include \"libcamera/internal/camera.h\"\n #include \"libcamera/internal/camera_sensor.h\"\n #include \"libcamera/internal/device_enumerator.h\"\n+#include \"libcamera/internal/framebuffer.h\"\n #include \"libcamera/internal/media_device.h\"\n #include \"libcamera/internal/pipeline_handler.h\"\n #include \"libcamera/internal/v4l2_subdevice.h\"\n@@ -481,6 +483,8 @@ public:\n \n \tint exportFrameBuffers(Camera *camera, Stream *stream,\n \t\t\t std::vector> *buffers) override;\n+\tint allocateBuffers(Camera *camera);\n+\tvoid freeBuffers(Camera *camera);\n \n \tint start(Camera *camera, const ControlList *controls) override;\n \tvoid stopDevice(Camera *camera) override;\n@@ -488,6 +492,7 @@ public:\n \tint queueRequestDevice(Camera *camera, Request *request) override;\n \n \tvoid bufferReady(FrameBuffer *buffer);\n+\tvoid statsBufferReady(FrameBuffer *buffer);\n \n \tbool match(DeviceEnumerator *enumerator) override;\n \n@@ -547,6 +552,14 @@ private:\n \n \tMediaDevice *media_;\n \tstd::unique_ptr isp_;\n+\tstd::unique_ptr stats_;\n+\tstd::unique_ptr params_;\n+\n+\tstd::vector> statsBuffers_;\n+\tstd::queue availableStatsBuffers_;\n+\n+\tstd::vector> paramsBuffers_;\n+\tstd::queue availableParamsBuffers_;\n \n \tstd::array pipes_;\n \n@@ -803,6 +816,16 @@ int PipelineHandlerMaliC55::configure(Camera *camera,\n \t\t\treturn ret;\n \t}\n \n+\tV4L2DeviceFormat statsFormat;\n+\tret = stats_->getFormat(&statsFormat);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tif (statsFormat.planes[0].size != sizeof(struct mali_c55_stats_buffer)) {\n+\t\tLOG(MaliC55, Error) << \"3a stats buffer size invalid\";\n+\t\treturn -EINVAL;\n+\t}\n+\n \t/*\n \t * Propagate the format to the ISP sink pad and configure the input\n \t * crop rectangle (no crop at the moment).\n@@ -867,27 +890,110 @@ int PipelineHandlerMaliC55::exportFrameBuffers(Camera *camera, Stream *stream,\n \treturn pipe->cap->exportBuffers(count, buffers);\n }\n \n-int PipelineHandlerMaliC55::start([[maybe_unused]] Camera *camera, [[maybe_unused]] const ControlList *controls)\n+void PipelineHandlerMaliC55::freeBuffers([[maybe_unused]] Camera *camera)\n+{\n+\twhile (!availableStatsBuffers_.empty())\n+\t\tavailableStatsBuffers_.pop();\n+\twhile (!availableParamsBuffers_.empty())\n+\t\tavailableParamsBuffers_.pop();\n+\n+\tstatsBuffers_.clear();\n+\tparamsBuffers_.clear();\n+\n+\tif (stats_->releaseBuffers())\n+\t\tLOG(MaliC55, Error) << \"Failed to release stats buffers\";\n+\n+\tif (params_->releaseBuffers())\n+\t\tLOG(MaliC55, Error) << \"Failed to release stats buffers\";\n+\n+\treturn;\n+}\n+\n+int PipelineHandlerMaliC55::allocateBuffers(Camera *camera)\n+{\n+\tMaliC55CameraData *data = cameraData(camera);\n+\tunsigned int bufferCount;\n+\tint ret;\n+\n+\tbufferCount = std::max({\n+\t\tdata->frStream_.configuration().bufferCount,\n+\t\tdata->dsStream_.configuration().bufferCount,\n+\t});\n+\n+\tret = stats_->allocateBuffers(bufferCount, &statsBuffers_);\n+\tif (ret < 0)\n+\t\treturn ret;\n+\n+\tfor (std::unique_ptr &buffer : statsBuffers_)\n+\t\tavailableStatsBuffers_.push(buffer.get());\n+\n+\tret = params_->allocateBuffers(bufferCount, ¶msBuffers_);\n+\tif (ret < 0)\n+\t\treturn ret;\n+\n+\tfor (std::unique_ptr &buffer : paramsBuffers_)\n+\t\tavailableParamsBuffers_.push(buffer.get());\n+\n+\treturn 0;\n+}\n+\n+int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const ControlList *controls)\n {\n+\tint ret;\n+\n+\tret = allocateBuffers(camera);\n+\tif (ret)\n+\t\treturn ret;\n+\n \tfor (MaliC55Pipe &pipe : pipes_) {\n \t\tif (!pipe.stream)\n \t\t\tcontinue;\n \n \t\tStream *stream = pipe.stream;\n \n-\t\tint ret = pipe.cap->importBuffers(stream->configuration().bufferCount);\n+\t\tret = pipe.cap->importBuffers(stream->configuration().bufferCount);\n \t\tif (ret) {\n \t\t\tLOG(MaliC55, Error) << \"Failed to import buffers\";\n+\t\t\tfreeBuffers(camera);\n \t\t\treturn ret;\n \t\t}\n \n \t\tret = pipe.cap->streamOn();\n \t\tif (ret) {\n \t\t\tLOG(MaliC55, Error) << \"Failed to start stream\";\n+\t\t\tfreeBuffers(camera);\n \t\t\treturn ret;\n \t\t}\n \t}\n \n+\tret = stats_->streamOn();\n+\tif (ret) {\n+\t\tLOG(MaliC55, Error) << \"Failed to start stats stream\";\n+\n+\t\tfor (MaliC55Pipe &pipe : pipes_) {\n+\t\t\tif (pipe.stream)\n+\t\t\t\tpipe.cap->streamOff();\n+\t\t}\n+\n+\t\tfreeBuffers(camera);\n+\t\treturn ret;\n+\t}\n+\n+\tret = params_->streamOn();\n+\tif (ret) {\n+\t\tLOG(MaliC55, Error) << \"Failed to start params stream\";\n+\n+\t\tstats_->streamOff();\n+\n+\t\tfor (MaliC55Pipe &pipe : pipes_) {\n+\t\t\tif (pipe.stream)\n+\t\t\t\tpipe.cap->streamOff();\n+\t\t}\n+\n+\t\tfreeBuffers(camera);\n+\t\treturn ret;\n+\t}\n+\n \treturn 0;\n }\n \n@@ -900,6 +1006,10 @@ void PipelineHandlerMaliC55::stopDevice([[maybe_unused]] Camera *camera)\n \t\tpipe.cap->streamOff();\n \t\tpipe.cap->releaseBuffers();\n \t}\n+\n+\tstats_->streamOff();\n+\tparams_->streamOff();\n+\tfreeBuffers(camera);\n }\n \n void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera,\n@@ -1002,8 +1112,24 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera,\n \n int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request)\n {\n+\tFrameBuffer *statsBuffer;\n \tint ret;\n \n+\tif (availableStatsBuffers_.empty()) {\n+\t\tLOG(MaliC55, Error) << \"Stats buffer underrun\";\n+\t\treturn -ENOENT;\n+\t}\n+\n+\tstatsBuffer = availableStatsBuffers_.front();\n+\tavailableStatsBuffers_.pop();\n+\n+\t/*\n+\t * We need to associate the Request to this buffer even though it's a\n+\t * purely internal one because we will need to use request->sequence()\n+\t * later.\n+\t */\n+\tstatsBuffer->_d()->setRequest(request);\n+\n \tfor (auto &[stream, buffer] : request->buffers()) {\n \t\tMaliC55Pipe *pipe = pipeFromStream(cameraData(camera), stream);\n \n@@ -1021,6 +1147,10 @@ int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request)\n \t */\n \tapplyScalerCrop(camera, request->controls());\n \n+\tret = stats_->queueBuffer(statsBuffer);\n+\tif (ret)\n+\t\treturn ret;\n+\n \treturn 0;\n }\n \n@@ -1032,6 +1162,11 @@ void PipelineHandlerMaliC55::bufferReady(FrameBuffer *buffer)\n \t\tcompleteRequest(request);\n }\n \n+void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer)\n+{\n+\tavailableStatsBuffers_.push(buffer);\n+}\n+\n void PipelineHandlerMaliC55::registerMaliCamera(std::unique_ptr data,\n \t\t\t\t\t\tconst std::string &name)\n {\n@@ -1110,7 +1245,7 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator)\n \tconst MediaPad *ispSink;\n \n \t/*\n-\t * We search for just the ISP subdevice and the full resolution pipe.\n+\t * We search for just the always-available elements of the media graph.\n \t * The TPG and the downscale pipe are both optional blocks and may not\n \t * be fitted.\n \t */\n@@ -1118,6 +1253,8 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator)\n \tdm.add(\"mali-c55 isp\");\n \tdm.add(\"mali-c55 resizer fr\");\n \tdm.add(\"mali-c55 fr\");\n+\tdm.add(\"mali-c55 3a stats\");\n+\tdm.add(\"mali-c55 3a params\");\n \n \tmedia_ = acquireMediaDevice(enumerator, dm);\n \tif (!media_)\n@@ -1127,6 +1264,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator)\n \tif (isp_->open() < 0)\n \t\treturn false;\n \n+\tstats_ = V4L2VideoDevice::fromEntityName(media_, \"mali-c55 3a stats\");\n+\tif (stats_->open() < 0)\n+\t\treturn false;\n+\n+\tparams_ = V4L2VideoDevice::fromEntityName(media_, \"mali-c55 3a params\");\n+\tif (params_->open() < 0)\n+\t\treturn false;\n+\n \tMaliC55Pipe *frPipe = &pipes_[MaliC55FR];\n \tfrPipe->resizer = V4L2Subdevice::fromEntityName(media_, \"mali-c55 resizer fr\");\n \tif (frPipe->resizer->open() < 0)\n@@ -1155,6 +1300,8 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator)\n \t\tdsPipe->cap->bufferReady.connect(this, &PipelineHandlerMaliC55::bufferReady);\n \t}\n \n+\tstats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady);\n+\n \tispSink = isp_->entity()->getPadByIndex(0);\n \tif (!ispSink || ispSink->links().empty()) {\n \t\tLOG(MaliC55, Error) << \"ISP sink pad error\";\n","prefixes":["03/10"]}