{"id":10773,"url":"https://patchwork.libcamera.org/api/1.1/patches/10773/?format=json","web_url":"https://patchwork.libcamera.org/patch/10773/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20201229160318.77536-8-niklas.soderlund@ragnatech.se>","date":"2020-12-29T16:03:14","name":"[libcamera-devel,v2,07/11] libcamera: ipu3: Attach to an IPA and allow it to set sensor controls","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"294dd25fd9f9b345ffde8abc8a635170be7ea6a5","submitter":{"id":5,"url":"https://patchwork.libcamera.org/api/1.1/people/5/?format=json","name":"Niklas Söderlund","email":"niklas.soderlund@ragnatech.se"},"delegate":{"id":16,"url":"https://patchwork.libcamera.org/api/1.1/users/16/?format=json","username":"neg","first_name":"Niklas","last_name":"Söderlund","email":"niklas.soderlund@ragnatech.se"},"mbox":"https://patchwork.libcamera.org/patch/10773/mbox/","series":[{"id":1559,"url":"https://patchwork.libcamera.org/api/1.1/series/1559/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1559","date":"2020-12-29T16:03:07","name":"libcamera: ipu3: Attach to an skeleton IPA","version":2,"mbox":"https://patchwork.libcamera.org/series/1559/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/10773/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/10773/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 17690C0F1A\n\tfor ;\n\tTue, 29 Dec 2020 16:04:12 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D9C57615D2;\n\tTue, 29 Dec 2020 17:04:11 +0100 (CET)","from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net\n\t[195.74.38.229])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 090AB615D5\n\tfor ;\n\tTue, 29 Dec 2020 17:04:11 +0100 (CET)","from bismarck.berto.se (p4fca2458.dip0.t-ipconnect.de\n\t[79.202.36.88])\n\tby bin-vsp-out-01.atm.binero.net (Halon) with ESMTPA\n\tid 7c070017-49ef-11eb-a542-005056917a89;\n\tTue, 29 Dec 2020 17:04:09 +0100 (CET)"],"X-Halon-ID":"7c070017-49ef-11eb-a542-005056917a89","Authorized-sender":"niklas.soderlund@fsdn.se","From":"=?utf-8?q?Niklas_S=C3=B6derlund?= ","To":"libcamera-devel@lists.libcamera.org","Date":"Tue, 29 Dec 2020 17:03:14 +0100","Message-Id":"<20201229160318.77536-8-niklas.soderlund@ragnatech.se>","X-Mailer":"git-send-email 2.29.2","In-Reply-To":"<20201229160318.77536-1-niklas.soderlund@ragnatech.se>","References":"<20201229160318.77536-1-niklas.soderlund@ragnatech.se>","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v2 07/11] libcamera: ipu3: Attach to an\n\tIPA and allow it to set sensor controls","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Attach to the IPA and allow it to push V4L2 controls that applies on the\ncamera sensor. The IPA is not fully integrated in the pipeline as\nstatistics and parameters buffers are not yet allocated, processed by\nthe IPA nor queued to the hardware.\n\nSigned-off-by: Niklas Söderlund \n---\n* Changes since v1\n- Rewrite to not use CameraSensor.\n- Fix mistake where configuration sen to IPA was overwritten.\n- Check that IPA configuration was successful before starting.\n- Update commit message.\n---\n src/libcamera/pipeline/ipu3/ipu3.cpp | 103 +++++++++++++++++++++++++++\n 1 file changed, 103 insertions(+)","diff":"diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\nindex a87ce8f3378ba2fe..95f1b75dc8be5d40 100644\n--- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n@@ -14,11 +14,14 @@\n #include \n #include \n #include \n+#include \n #include \n #include \n \n #include \"libcamera/internal/camera_sensor.h\"\n+#include \"libcamera/internal/delayed_controls.h\"\n #include \"libcamera/internal/device_enumerator.h\"\n+#include \"libcamera/internal/ipa_manager.h\"\n #include \"libcamera/internal/log.h\"\n #include \"libcamera/internal/media_device.h\"\n #include \"libcamera/internal/pipeline_handler.h\"\n@@ -53,6 +56,8 @@ public:\n \t{\n \t}\n \n+\tint loadIPA();\n+\n \tvoid imguOutputBufferReady(FrameBuffer *buffer);\n \tvoid cio2BufferReady(FrameBuffer *buffer);\n \n@@ -62,6 +67,11 @@ public:\n \tStream outStream_;\n \tStream vfStream_;\n \tStream rawStream_;\n+\n+\tstd::unique_ptr delayedCtrls_;\n+\n+private:\n+\tvoid actOnIpa(unsigned int id, const IPAOperationData &op);\n };\n \n class IPU3CameraConfiguration : public CameraConfiguration\n@@ -590,6 +600,13 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] ControlList *con\n \tIPU3CameraData *data = cameraData(camera);\n \tCIO2Device *cio2 = &data->cio2_;\n \tImgUDevice *imgu = data->imgu_;\n+\n+\tCameraSensorInfo sensorInfo = {};\n+\tstd::map streamConfig;\n+\tstd::map entityControls;\n+\tIPAOperationData ipaConfig;\n+\tIPAOperationData result = {};\n+\n \tint ret;\n \n \t/* Allocate buffers for internal pipeline usage. */\n@@ -597,6 +614,11 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] ControlList *con\n \tif (ret)\n \t\treturn ret;\n \n+\tIPAOperationData ipaData = {};\n+\tret = data->ipa_->start(ipaData, nullptr);\n+\tif (ret)\n+\t\tgoto error;\n+\n \t/*\n \t * Start the ImgU video devices, buffers will be queued to the\n \t * ImgU output and viewfinder when requests will be queued.\n@@ -612,9 +634,40 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] ControlList *con\n \t\tgoto error;\n \t}\n \n+\t/* Inform IPA of stream configuration and sensor controls. */\n+\tret = data->cio2_.sensor()->sensorInfo(&sensorInfo);\n+\tif (ret) {\n+\t\t/* \\todo Turn to hard failure once sensors info is mandatory. */\n+\t\tLOG(IPU3, Warning) << \"Camera sensor information not available\";\n+\t\tsensorInfo = {};\n+\t\tret = 0;\n+\t}\n+\n+\tstreamConfig[0] = {\n+\t\t.pixelFormat = data->outStream_.configuration().pixelFormat,\n+\t\t.size = data->outStream_.configuration().size,\n+\t};\n+\tstreamConfig[1] = {\n+\t\t.pixelFormat = data->vfStream_.configuration().pixelFormat,\n+\t\t.size = data->vfStream_.configuration().size,\n+\t};\n+\n+\tentityControls.emplace(0, data->cio2_.sensor()->controls());\n+\n+\tdata->ipa_->configure(sensorInfo, streamConfig, entityControls,\n+\t\t\t ipaConfig, &result);\n+\n+\tif ((result.operation != IPU3_IPA_STATUS_CONFIGURATION) ||\n+\t (result.data.size() != 1) || (result.data.at(0) != 1)) {\n+\t\tLOG(IPU3, Warning) << \"Failed to configure IPA\";\n+\t\tret = -EINVAL;\n+\t\tgoto error;\n+\t}\n+\n \treturn 0;\n \n error:\n+\tdata->ipa_->stop();\n \tfreeBuffers(camera);\n \tLOG(IPU3, Error) << \"Failed to start camera \" << camera->id();\n \n@@ -631,6 +684,8 @@ void PipelineHandlerIPU3::stop(Camera *camera)\n \tif (ret)\n \t\tLOG(IPU3, Warning) << \"Failed to stop camera \" << camera->id();\n \n+\tdata->ipa_->stop();\n+\n \tfreeBuffers(camera);\n }\n \n@@ -762,12 +817,32 @@ int PipelineHandlerIPU3::registerCameras()\n \t\tif (ret)\n \t\t\tcontinue;\n \n+\t\tret = data->loadIPA();\n+\t\tif (ret)\n+\t\t\tcontinue;\n+\n \t\t/* Initialize the camera properties. */\n \t\tdata->properties_ = cio2->sensor()->properties();\n \n \t\t/* Initialze the camera controls. */\n \t\tdata->controlInfo_ = IPU3Controls;\n \n+\t\t/*\n+\t\t * \\todo Read dealy values from the sensor itself or from a\n+\t\t * a sensor database. For now use generic values taken from\n+\t\t * the Raspberry Pi and listed as generic values.\n+\t\t */\n+\t\tstd::unordered_map delays = {\n+\t\t\t{ V4L2_CID_ANALOGUE_GAIN, 1 },\n+\t\t\t{ V4L2_CID_EXPOSURE, 2 },\n+\t\t};\n+\n+\t\tdata->delayedCtrls_ =\n+\t\t\tstd::make_unique(cio2->sensor()->device(),\n+\t\t\t\t\t\t\t delays);\n+\t\tdata->cio2_.frameStart().connect(data->delayedCtrls_.get(),\n+\t\t\t\t\t\t &DelayedControls::applyControls);\n+\n \t\t/**\n \t\t * \\todo Dynamically assign ImgU and output devices to each\n \t\t * stream and camera; as of now, limit support to two cameras\n@@ -811,6 +886,34 @@ int PipelineHandlerIPU3::registerCameras()\n \treturn numCameras ? 0 : -ENODEV;\n }\n \n+int IPU3CameraData::loadIPA()\n+{\n+\tipa_ = IPAManager::createIPA(pipe_, 1, 1);\n+\tif (!ipa_)\n+\t\treturn -ENOENT;\n+\n+\tipa_->queueFrameAction.connect(this, &IPU3CameraData::actOnIpa);\n+\n+\tipa_->init(IPASettings{});\n+\n+\treturn 0;\n+}\n+\n+void IPU3CameraData::actOnIpa([[maybe_unused]] unsigned int id,\n+\t\t\t const IPAOperationData &action)\n+{\n+\tswitch (action.operation) {\n+\tcase IPU3_IPA_ACTION_SET_SENSOR_CONTROLS: {\n+\t\tconst ControlList &controls = action.controls[0];\n+\t\tdelayedCtrls_->push(controls);\n+\t\tbreak;\n+\t}\n+\tdefault:\n+\t\tLOG(IPU3, Error) << \"Unknown action \" << action.operation;\n+\t\tbreak;\n+\t}\n+}\n+\n /* -----------------------------------------------------------------------------\n * Buffer Ready slots\n */\n","prefixes":["libcamera-devel","v2","07/11"]}