[{"id":20871,"web_url":"https://patchwork.libcamera.org/comment/20871/","msgid":"<5cc122ca-823a-a929-9bb5-36a7e1469cae@ideasonboard.com>","date":"2021-11-11T16:12:52","subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/people/75/","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"content":"Hi Han-Lin,\n\nOn 11/11/2021 11:49, Han-Lin Chen wrote:\n> Allow IPA to apply controls to the lens device.\n> \n> Signed-off-by: Han-Lin Chen \n> ---\n> meson.build | 6 ++++++\n> src/libcamera/pipeline/ipu3/cio2.cpp | 30 ++++++++++++++++++++++++++++\n> src/libcamera/pipeline/ipu3/cio2.h | 3 +++\n> src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++--\n> 4 files changed, 46 insertions(+), 2 deletions(-)\n> \n> diff --git a/meson.build b/meson.build\n> index 7892a9e3..2a4b68a2 100644\n> --- a/meson.build\n> +++ b/meson.build\n> @@ -108,6 +108,12 @@ if cc.has_argument('-Wno-c99-designator')\n> ]\n> endif\n> \n> +if get_option('android_platform') == 'cros'\n> + common_arguments += [\n> + '-DOS_CHROMEOS',\n> + ]\n> +endif\n\nI am not really aware of specifics regarding this, why can't we have \nsomething not dependent on the OS ?\n\n> +\n> c_arguments += common_arguments\n> cpp_arguments += common_arguments\n> \n> diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp\n> index 59dda56b..233553c2 100644\n> --- a/src/libcamera/pipeline/ipu3/cio2.cpp\n> +++ b/src/libcamera/pipeline/ipu3/cio2.cpp\n> @@ -16,6 +16,7 @@\n> #include \n> #include \n> \n> +#include \"libcamera/internal/camera_lens.h\"\n> #include \"libcamera/internal/camera_sensor.h\"\n> #include \"libcamera/internal/framebuffer.h\"\n> #include \"libcamera/internal/media_device.h\"\n> @@ -159,6 +160,35 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index)\n> \t\treturn -EINVAL;\n> \t}\n> \n> +#if defined(OS_CHROMEOS)\n> +\t/*\n> +\t * \\todo Read the lens model from the sensor itself or from a device database.\n> +\t * For now use default values taken from ChromeOS.\n> +\t */\n> +\tstatic std::unordered_map sensorLens = {\n> +\t\t{ \"ov13858\", \"dw9714\" },\n> +\t\t{ \"imx258\", \"dw9807\" },\n> +\t\t{ \"imx355\", \"ak7375\" }\n> +\t};\n\nSo, we need a list giving the lens model associated to a sensor... How \nis this dependent on chromeos ? Imagine I want to use the sensor lens on \nmy SGo2, which has a ov8865, what is needed to do that ?\n\n> +\n> +\tauto it = sensorLens.find(sensor_->model());\n> +\tif (it != sensorLens.end()) {\n> +\t\tconst std::vector &entities = media->entities();\n> +\t\tfor (auto ent : entities) {\n> +\t\t\tif (ent->function() == MEDIA_ENT_F_LENS) {\n> +\t\t\t\tlens_ = std::make_unique(ent);\n> +\t\t\t\tret = lens_->init();\n> +\t\t\t\tif (!ret && lens_->model() == it->second) {\n> +\t\t\t\t\tbreak;\n> +\t\t\t\t}\n> +\t\t\t\tlens_.reset();\n> +\t\t\t}\n> +\t\t\tif (!lens_)\n> +\t\t\t\tLOG(IPU3, Warning) << \"Lens device \" << it->second << \" not found\";\n> +\t\t}\n> +\t}\n> +#endif\n> +\n> \t/*\n> \t * \\todo Define when to open and close video device nodes, as they\n> \t * might impact on power consumption.\n> diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h\n> index ba8f0052..635566c8 100644\n> --- a/src/libcamera/pipeline/ipu3/cio2.h\n> +++ b/src/libcamera/pipeline/ipu3/cio2.h\n> @@ -18,6 +18,7 @@\n> \n> namespace libcamera {\n> \n> +class CameraLens;\n> class CameraSensor;\n> class FrameBuffer;\n> class MediaDevice;\n> @@ -52,6 +53,7 @@ public:\n> \tint stop();\n> \n> \tCameraSensor *sensor() { return sensor_.get(); }\n> +\tCameraLens *lens() { return lens_.get(); }\n> \tconst CameraSensor *sensor() const { return sensor_.get(); }\n> \n> \tFrameBuffer *queueBuffer(Request *request, FrameBuffer *rawBuffer);\n> @@ -67,6 +69,7 @@ private:\n> \tvoid cio2BufferReady(FrameBuffer *buffer);\n> \n> \tstd::unique_ptr sensor_;\n> +\tstd::unique_ptr lens_;\n> \tstd::unique_ptr csi2_;\n> \tstd::unique_ptr output_;\n> \n> diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> index 97003681..88775f67 100644\n> --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> @@ -24,6 +24,7 @@\n> #include \n> \n> #include \"libcamera/internal/camera.h\"\n> +#include \"libcamera/internal/camera_lens.h\"\n> #include \"libcamera/internal/camera_sensor.h\"\n> #include \"libcamera/internal/delayed_controls.h\"\n> #include \"libcamera/internal/device_enumerator.h\"\n> @@ -1255,8 +1256,12 @@ void IPU3CameraData::queueFrameAction(unsigned int id,\n> {\n> \tswitch (action.op) {\n> \tcase ipa::ipu3::ActionSetSensorControls: {\n> -\t\tconst ControlList &controls = action.sensorControls;\n> -\t\tdelayedCtrls_->push(controls);\n> +\t\tconst ControlList &sensorControls = action.sensorControls;\n> +\t\tdelayedCtrls_->push(sensorControls);\n> +\t\tif (cio2_.lens()) {\n> +\t\t\tControlList lensControls = action.lensControls;\n> +\t\t\tcio2_.lens()->setControls(&lensControls);\n> +\t\t}\n> \t\tbreak;\n> \t}\n> \tcase ipa::ipu3::ActionParamFilled: {\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 29695BF415\n\tfor ;\n\tThu, 11 Nov 2021 16:12:58 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6E5646035A;\n\tThu, 11 Nov 2021 17:12:57 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 33FAB600B5\n\tfor ;\n\tThu, 11 Nov 2021 17:12:56 +0100 (CET)","from [IPV6:2a01:e0a:169:7140:e627:8337:a781:d98] (unknown\n\t[IPv6:2a01:e0a:169:7140:e627:8337:a781:d98])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C85C14A6;\n\tThu, 11 Nov 2021 17:12:55 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"ATJyL1qC\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1636647175;\n\tbh=vIf0vzjRQ+c4A1xxEP0ocKSFVVlcSQIgY85HldyS8oY=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=ATJyL1qCK/u0qBFoRP6y2gF4fQJDNDztCZOhn/F2/IfCji8IDhkSmcKMxzAfk1FcA\n\tMDH55PPEty5CGZ2ZcE2JlKUCJFRJs4RtIx+JQXvFhxCKBynOkBVKTHQCQaeiWAdBFT\n\tQf9j6dX9OT/K7o8kVmwF5ZMKbtIVAkCIrdMr/oek=","Message-ID":"<5cc122ca-823a-a929-9bb5-36a7e1469cae@ideasonboard.com>","Date":"Thu, 11 Nov 2021 17:12:52 +0100","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101\n\tThunderbird/91.2.1","Content-Language":"en-US","To":"Han-Lin Chen ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20211111104958.312070-1-hanlinchen@chromium.org>\n\t<20211111104958.312070-3-hanlinchen@chromium.org>","From":"Jean-Michel Hautbois ","In-Reply-To":"<20211111104958.312070-3-hanlinchen@chromium.org>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20888,"web_url":"https://patchwork.libcamera.org/comment/20888/","msgid":"","date":"2021-11-11T23:57:52","subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Thu, Nov 11, 2021 at 05:12:52PM +0100, Jean-Michel Hautbois wrote:\n> Hi Han-Lin,\n> \n> On 11/11/2021 11:49, Han-Lin Chen wrote:\n> > Allow IPA to apply controls to the lens device.\n> > \n> > Signed-off-by: Han-Lin Chen \n> > ---\n> > meson.build | 6 ++++++\n> > src/libcamera/pipeline/ipu3/cio2.cpp | 30 ++++++++++++++++++++++++++++\n> > src/libcamera/pipeline/ipu3/cio2.h | 3 +++\n> > src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++--\n> > 4 files changed, 46 insertions(+), 2 deletions(-)\n> > \n> > diff --git a/meson.build b/meson.build\n> > index 7892a9e3..2a4b68a2 100644\n> > --- a/meson.build\n> > +++ b/meson.build\n> > @@ -108,6 +108,12 @@ if cc.has_argument('-Wno-c99-designator')\n> > ]\n> > endif\n> > \n> > +if get_option('android_platform') == 'cros'\n> > + common_arguments += [\n> > + '-DOS_CHROMEOS',\n> > + ]\n> > +endif\n> \n> I am not really aware of specifics regarding this, why can't we have \n> something not dependent on the OS ?\n\nThere's a mail thread regarding VCM support on MS Surface machines (see\nhttps://lists.libcamera.org/pipermail/libcamera-devel/2021-November/026526.html).\nRaspberry Pi is also looking into VCM support. It would be best to\ncooperate with all parties involved to define a way to expose the\nrelation between camera sensors and lens controllers to userspace, and\nuse it here. I'm really not keen on having a CrOS-specific hack.\n\n> > +\n> > c_arguments += common_arguments\n> > cpp_arguments += common_arguments\n> > \n> > diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > index 59dda56b..233553c2 100644\n> > --- a/src/libcamera/pipeline/ipu3/cio2.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > @@ -16,6 +16,7 @@\n> > #include \n> > #include \n> > \n> > +#include \"libcamera/internal/camera_lens.h\"\n> > #include \"libcamera/internal/camera_sensor.h\"\n> > #include \"libcamera/internal/framebuffer.h\"\n> > #include \"libcamera/internal/media_device.h\"\n> > @@ -159,6 +160,35 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index)\n> > \t\treturn -EINVAL;\n> > \t}\n> > \n> > +#if defined(OS_CHROMEOS)\n> > +\t/*\n> > +\t * \\todo Read the lens model from the sensor itself or from a device database.\n> > +\t * For now use default values taken from ChromeOS.\n> > +\t */\n> > +\tstatic std::unordered_map sensorLens = {\n> > +\t\t{ \"ov13858\", \"dw9714\" },\n> > +\t\t{ \"imx258\", \"dw9807\" },\n> > +\t\t{ \"imx355\", \"ak7375\" }\n> > +\t};\n> \n> So, we need a list giving the lens model associated to a sensor... How \n> is this dependent on chromeos ? Imagine I want to use the sensor lens on \n> my SGo2, which has a ov8865, what is needed to do that ?\n> \n> > +\n> > +\tauto it = sensorLens.find(sensor_->model());\n> > +\tif (it != sensorLens.end()) {\n> > +\t\tconst std::vector &entities = media->entities();\n> > +\t\tfor (auto ent : entities) {\n> > +\t\t\tif (ent->function() == MEDIA_ENT_F_LENS) {\n> > +\t\t\t\tlens_ = std::make_unique(ent);\n> > +\t\t\t\tret = lens_->init();\n> > +\t\t\t\tif (!ret && lens_->model() == it->second) {\n> > +\t\t\t\t\tbreak;\n> > +\t\t\t\t}\n> > +\t\t\t\tlens_.reset();\n> > +\t\t\t}\n> > +\t\t\tif (!lens_)\n> > +\t\t\t\tLOG(IPU3, Warning) << \"Lens device \" << it->second << \" not found\";\n> > +\t\t}\n> > +\t}\n> > +#endif\n> > +\n> > \t/*\n> > \t * \\todo Define when to open and close video device nodes, as they\n> > \t * might impact on power consumption.\n> > diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h\n> > index ba8f0052..635566c8 100644\n> > --- a/src/libcamera/pipeline/ipu3/cio2.h\n> > +++ b/src/libcamera/pipeline/ipu3/cio2.h\n> > @@ -18,6 +18,7 @@\n> > \n> > namespace libcamera {\n> > \n> > +class CameraLens;\n> > class CameraSensor;\n> > class FrameBuffer;\n> > class MediaDevice;\n> > @@ -52,6 +53,7 @@ public:\n> > \tint stop();\n> > \n> > \tCameraSensor *sensor() { return sensor_.get(); }\n> > +\tCameraLens *lens() { return lens_.get(); }\n> > \tconst CameraSensor *sensor() const { return sensor_.get(); }\n> > \n> > \tFrameBuffer *queueBuffer(Request *request, FrameBuffer *rawBuffer);\n> > @@ -67,6 +69,7 @@ private:\n> > \tvoid cio2BufferReady(FrameBuffer *buffer);\n> > \n> > \tstd::unique_ptr sensor_;\n> > +\tstd::unique_ptr lens_;\n> > \tstd::unique_ptr csi2_;\n> > \tstd::unique_ptr output_;\n> > \n> > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > index 97003681..88775f67 100644\n> > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > @@ -24,6 +24,7 @@\n> > #include \n> > \n> > #include \"libcamera/internal/camera.h\"\n> > +#include \"libcamera/internal/camera_lens.h\"\n> > #include \"libcamera/internal/camera_sensor.h\"\n> > #include \"libcamera/internal/delayed_controls.h\"\n> > #include \"libcamera/internal/device_enumerator.h\"\n> > @@ -1255,8 +1256,12 @@ void IPU3CameraData::queueFrameAction(unsigned int id,\n> > {\n> > \tswitch (action.op) {\n> > \tcase ipa::ipu3::ActionSetSensorControls: {\n> > -\t\tconst ControlList &controls = action.sensorControls;\n> > -\t\tdelayedCtrls_->push(controls);\n> > +\t\tconst ControlList &sensorControls = action.sensorControls;\n> > +\t\tdelayedCtrls_->push(sensorControls);\n> > +\t\tif (cio2_.lens()) {\n> > +\t\t\tControlList lensControls = action.lensControls;\n> > +\t\t\tcio2_.lens()->setControls(&lensControls);\n> > +\t\t}\n> > \t\tbreak;\n> > \t}\n> > \tcase ipa::ipu3::ActionParamFilled: {\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2BB09BF415\n\tfor ;\n\tThu, 11 Nov 2021 23:58:17 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 89F536033C;\n\tFri, 12 Nov 2021 00:58:16 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A224460232\n\tfor ;\n\tFri, 12 Nov 2021 00:58:14 +0100 (CET)","from pendragon.ideasonboard.com\n\t(117.145-247-81.adsl-dyn.isp.belgacom.be [81.247.145.117])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 283FE74C;\n\tFri, 12 Nov 2021 00:58:14 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"pC0e4oMF\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1636675094;\n\tbh=BRmQRQyJ1tnvPFm5ZdMvwmqqspHiDXRs5WB1kfcnGQU=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=pC0e4oMFLz8bemQ0r1z92ty+D0O2n8c4SJEChb7t6IejovnL2ddphU+KIGIpJWPW7\n\tb/KudDMcDsXmKTpjK2+X3gvSOhg6alytruuHHam3SNl/zDTf3KIUVkjT80oKawSDKL\n\tMiS6g9N5gyoN6JQL739bONXOKHmCwBIv5BdNR2nk=","Date":"Fri, 12 Nov 2021 01:57:52 +0200","From":"Laurent Pinchart ","To":"Jean-Michel Hautbois ","Message-ID":"","References":"<20211111104958.312070-1-hanlinchen@chromium.org>\n\t<20211111104958.312070-3-hanlinchen@chromium.org>\n\t<5cc122ca-823a-a929-9bb5-36a7e1469cae@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<5cc122ca-823a-a929-9bb5-36a7e1469cae@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20963,"web_url":"https://patchwork.libcamera.org/comment/20963/","msgid":"","date":"2021-11-16T09:27:29","subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","submitter":{"id":98,"url":"https://patchwork.libcamera.org/api/people/98/","name":"Hanlin Chen","email":"hanlinchen@chromium.org"},"content":"Hi Jean-Michel,\n\nOn Fri, Nov 12, 2021 at 12:12 AM Jean-Michel Hautbois\n wrote:\n>\n> Hi Han-Lin,\n>\n> On 11/11/2021 11:49, Han-Lin Chen wrote:\n> > Allow IPA to apply controls to the lens device.\n> >\n> > Signed-off-by: Han-Lin Chen \n> > ---\n> > meson.build | 6 ++++++\n> > src/libcamera/pipeline/ipu3/cio2.cpp | 30 ++++++++++++++++++++++++++++\n> > src/libcamera/pipeline/ipu3/cio2.h | 3 +++\n> > src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++--\n> > 4 files changed, 46 insertions(+), 2 deletions(-)\n> >\n> > diff --git a/meson.build b/meson.build\n> > index 7892a9e3..2a4b68a2 100644\n> > --- a/meson.build\n> > +++ b/meson.build\n> > @@ -108,6 +108,12 @@ if cc.has_argument('-Wno-c99-designator')\n> > ]\n> > endif\n> >\n> > +if get_option('android_platform') == 'cros'\n> > + common_arguments += [\n> > + '-DOS_CHROMEOS',\n> > + ]\n> > +endif\n>\n> I am not really aware of specifics regarding this, why can't we have\n> something not dependent on the OS ?\n>\n> > +\n> > c_arguments += common_arguments\n> > cpp_arguments += common_arguments\n> >\n> > diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > index 59dda56b..233553c2 100644\n> > --- a/src/libcamera/pipeline/ipu3/cio2.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > @@ -16,6 +16,7 @@\n> > #include \n> > #include \n> >\n> > +#include \"libcamera/internal/camera_lens.h\"\n> > #include \"libcamera/internal/camera_sensor.h\"\n> > #include \"libcamera/internal/framebuffer.h\"\n> > #include \"libcamera/internal/media_device.h\"\n> > @@ -159,6 +160,35 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index)\n> > return -EINVAL;\n> > }\n> >\n> > +#if defined(OS_CHROMEOS)\n> > + /*\n> > + * \\todo Read the lens model from the sensor itself or from a device database.\n> > + * For now use default values taken from ChromeOS.\n> > + */\n> > + static std::unordered_map sensorLens = {\n> > + { \"ov13858\", \"dw9714\" },\n> > + { \"imx258\", \"dw9807\" },\n> > + { \"imx355\", \"ak7375\" }\n> > + };\n>\n> So, we need a list giving the lens model associated to a sensor... How\n> is this dependent on chromeos ? Imagine I want to use the sensor lens on\n> my SGo2, which has a ov8865, what is needed to do that ?\nYou are right. Not only ChromeOS needs the mapping.\nI made it depend on CrOS simply because it's the mapping I can\nconfirm, and I was worried to accidentally broken the other platforms.\nMaybe I was overthinking this... Will remove the directive.\nBesides, the mapping is considered as a temporary solution, which\nshould be removed once we have a way to read the relation from the\nmedia-ctl or a database.\n>\n> > +\n> > + auto it = sensorLens.find(sensor_->model());\n> > + if (it != sensorLens.end()) {\n> > + const std::vector &entities = media->entities();\n> > + for (auto ent : entities) {\n> > + if (ent->function() == MEDIA_ENT_F_LENS) {\n> > + lens_ = std::make_unique(ent);\n> > + ret = lens_->init();\n> > + if (!ret && lens_->model() == it->second) {\n> > + break;\n> > + }\n> > + lens_.reset();\n> > + }\n> > + if (!lens_)\n> > + LOG(IPU3, Warning) << \"Lens device \" << it->second << \" not found\";\n> > + }\n> > + }\n> > +#endif\n> > +\n> > /*\n> > * \\todo Define when to open and close video device nodes, as they\n> > * might impact on power consumption.\n> > diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h\n> > index ba8f0052..635566c8 100644\n> > --- a/src/libcamera/pipeline/ipu3/cio2.h\n> > +++ b/src/libcamera/pipeline/ipu3/cio2.h\n> > @@ -18,6 +18,7 @@\n> >\n> > namespace libcamera {\n> >\n> > +class CameraLens;\n> > class CameraSensor;\n> > class FrameBuffer;\n> > class MediaDevice;\n> > @@ -52,6 +53,7 @@ public:\n> > int stop();\n> >\n> > CameraSensor *sensor() { return sensor_.get(); }\n> > + CameraLens *lens() { return lens_.get(); }\n> > const CameraSensor *sensor() const { return sensor_.get(); }\n> >\n> > FrameBuffer *queueBuffer(Request *request, FrameBuffer *rawBuffer);\n> > @@ -67,6 +69,7 @@ private:\n> > void cio2BufferReady(FrameBuffer *buffer);\n> >\n> > std::unique_ptr sensor_;\n> > + std::unique_ptr lens_;\n> > std::unique_ptr csi2_;\n> > std::unique_ptr output_;\n> >\n> > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > index 97003681..88775f67 100644\n> > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > @@ -24,6 +24,7 @@\n> > #include \n> >\n> > #include \"libcamera/internal/camera.h\"\n> > +#include \"libcamera/internal/camera_lens.h\"\n> > #include \"libcamera/internal/camera_sensor.h\"\n> > #include \"libcamera/internal/delayed_controls.h\"\n> > #include \"libcamera/internal/device_enumerator.h\"\n> > @@ -1255,8 +1256,12 @@ void IPU3CameraData::queueFrameAction(unsigned int id,\n> > {\n> > switch (action.op) {\n> > case ipa::ipu3::ActionSetSensorControls: {\n> > - const ControlList &controls = action.sensorControls;\n> > - delayedCtrls_->push(controls);\n> > + const ControlList &sensorControls = action.sensorControls;\n> > + delayedCtrls_->push(sensorControls);\n> > + if (cio2_.lens()) {\n> > + ControlList lensControls = action.lensControls;\n> > + cio2_.lens()->setControls(&lensControls);\n> > + }\n> > break;\n> > }\n> > case ipa::ipu3::ActionParamFilled: {\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 56F8ABF415\n\tfor ;\n\tTue, 16 Nov 2021 09:27:43 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B479B6037A;\n\tTue, 16 Nov 2021 10:27:42 +0100 (CET)","from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com\n\t[IPv6:2607:f8b0:4864:20::32f])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DBEBE60231\n\tfor ;\n\tTue, 16 Nov 2021 10:27:41 +0100 (CET)","by mail-ot1-x32f.google.com with SMTP id\n\tr10-20020a056830080a00b0055c8fd2cebdso32427890ots.6\n\tfor ;\n\tTue, 16 Nov 2021 01:27:41 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"nx8wsZ2k\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=XP3LvnC6pU+zx8YsKYgDhTCQsizrQ+GVyQoZwM8W3pU=;\n\tb=nx8wsZ2k7PAoNAldoIxy4T8SLLNvve0vPbkR42gh8Z50wKuuNb2nPJzuq3bTpOM7pH\n\tRtGW9rAbEkL0o5WzivUF2kB2xXB37zgTa/RDrooLcV4Cza0Vv8/Ns75FIMZC0LLlEEc5\n\toHoYz1yrU2OSn4vnRPqR1BNgVKO+DNOP0yjME=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=XP3LvnC6pU+zx8YsKYgDhTCQsizrQ+GVyQoZwM8W3pU=;\n\tb=pAhpmxfWHmfqT1akXTYW8vUQ8n9BDItKBYgccudfvn6A2hu5glOC8UWkZcDyXl0piO\n\tOL7Ptzf1n4Oh4J0wOmtjGaJrGQdb4vx1jAYEIA1Jc0nQ2ZTr7koEX/6XKf7T4N5H/GHJ\n\thdVZVTr6/Qcm+jaWgHaOgfG8u/YYr1hsf2IkW58TNml2EZmhygKzv0ZPc+cJ6gleBe66\n\taLWa6BW7A+RH5PXGUxfqUiZhNCyYlUVyAi4LARvQsLupNxZUyH32yF3We8e2ku/Puo1b\n\to2zh0i48xLdcbPofPE/spXpBYxSqxG1u4jAwOEoJn4DEW9/9w5RHoUiH8hJRJa/j8Fr1\n\t7mog==","X-Gm-Message-State":"AOAM531bnoiXi6pAn6zhcPJDW+/5gaVBqpJQqzz0ONtBKgOsx18i0Oy/\n\tUgol+1xCuOnNSeymZyu23zCI/HPalDARUrHah0hxnJx1IqtflQ==","X-Google-Smtp-Source":"ABdhPJyHda6cvg0zlbPdiIabcmdFPLIjzTO0QBtJ5QODB9WcXHvjaYVYQtvoyjHDO/jkSjWb8uses//zMXs/QvvDGEc=","X-Received":"by 2002:a9d:62ce:: with SMTP id z14mr4799646otk.56.1637054860539;\n\tTue, 16 Nov 2021 01:27:40 -0800 (PST)","MIME-Version":"1.0","References":"<20211111104958.312070-1-hanlinchen@chromium.org>\n\t<20211111104958.312070-3-hanlinchen@chromium.org>\n\t<5cc122ca-823a-a929-9bb5-36a7e1469cae@ideasonboard.com>","In-Reply-To":"<5cc122ca-823a-a929-9bb5-36a7e1469cae@ideasonboard.com>","From":"Hanlin Chen ","Date":"Tue, 16 Nov 2021 17:27:29 +0800","Message-ID":"","To":"Jean-Michel Hautbois ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20964,"web_url":"https://patchwork.libcamera.org/comment/20964/","msgid":"","date":"2021-11-16T09:52:49","subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","submitter":{"id":98,"url":"https://patchwork.libcamera.org/api/people/98/","name":"Hanlin Chen","email":"hanlinchen@chromium.org"},"content":"On Fri, Nov 12, 2021 at 7:58 AM Laurent Pinchart\n wrote:\n>\n> On Thu, Nov 11, 2021 at 05:12:52PM +0100, Jean-Michel Hautbois wrote:\n> > Hi Han-Lin,\n> >\n> > On 11/11/2021 11:49, Han-Lin Chen wrote:\n> > > Allow IPA to apply controls to the lens device.\n> > >\n> > > Signed-off-by: Han-Lin Chen \n> > > ---\n> > > meson.build | 6 ++++++\n> > > src/libcamera/pipeline/ipu3/cio2.cpp | 30 ++++++++++++++++++++++++++++\n> > > src/libcamera/pipeline/ipu3/cio2.h | 3 +++\n> > > src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++--\n> > > 4 files changed, 46 insertions(+), 2 deletions(-)\n> > >\n> > > diff --git a/meson.build b/meson.build\n> > > index 7892a9e3..2a4b68a2 100644\n> > > --- a/meson.build\n> > > +++ b/meson.build\n> > > @@ -108,6 +108,12 @@ if cc.has_argument('-Wno-c99-designator')\n> > > ]\n> > > endif\n> > >\n> > > +if get_option('android_platform') == 'cros'\n> > > + common_arguments += [\n> > > + '-DOS_CHROMEOS',\n> > > + ]\n> > > +endif\n> >\n> > I am not really aware of specifics regarding this, why can't we have\n> > something not dependent on the OS ?\n>\n> There's a mail thread regarding VCM support on MS Surface machines (see\n> https://lists.libcamera.org/pipermail/libcamera-devel/2021-November/026526.html).\n> Raspberry Pi is also looking into VCM support. It would be best to\n> cooperate with all parties involved to define a way to expose the\n> relation between camera sensors and lens controllers to userspace, and\n> use it here. I'm really not keen on having a CrOS-specific hack.\nLooking forward to it :-).\n>\n> > > +\n> > > c_arguments += common_arguments\n> > > cpp_arguments += common_arguments\n> > >\n> > > diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > > index 59dda56b..233553c2 100644\n> > > --- a/src/libcamera/pipeline/ipu3/cio2.cpp\n> > > +++ b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > > @@ -16,6 +16,7 @@\n> > > #include \n> > > #include \n> > >\n> > > +#include \"libcamera/internal/camera_lens.h\"\n> > > #include \"libcamera/internal/camera_sensor.h\"\n> > > #include \"libcamera/internal/framebuffer.h\"\n> > > #include \"libcamera/internal/media_device.h\"\n> > > @@ -159,6 +160,35 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index)\n> > > return -EINVAL;\n> > > }\n> > >\n> > > +#if defined(OS_CHROMEOS)\n> > > + /*\n> > > + * \\todo Read the lens model from the sensor itself or from a device database.\n> > > + * For now use default values taken from ChromeOS.\n> > > + */\n> > > + static std::unordered_map sensorLens = {\n> > > + { \"ov13858\", \"dw9714\" },\n> > > + { \"imx258\", \"dw9807\" },\n> > > + { \"imx355\", \"ak7375\" }\n> > > + };\n> >\n> > So, we need a list giving the lens model associated to a sensor... How\n> > is this dependent on chromeos ? Imagine I want to use the sensor lens on\n> > my SGo2, which has a ov8865, what is needed to do that ?\n> >\n> > > +\n> > > + auto it = sensorLens.find(sensor_->model());\n> > > + if (it != sensorLens.end()) {\n> > > + const std::vector &entities = media->entities();\n> > > + for (auto ent : entities) {\n> > > + if (ent->function() == MEDIA_ENT_F_LENS) {\n> > > + lens_ = std::make_unique(ent);\n> > > + ret = lens_->init();\n> > > + if (!ret && lens_->model() == it->second) {\n> > > + break;\n> > > + }\n> > > + lens_.reset();\n> > > + }\n> > > + if (!lens_)\n> > > + LOG(IPU3, Warning) << \"Lens device \" << it->second << \" not found\";\n> > > + }\n> > > + }\n> > > +#endif\n> > > +\n> > > /*\n> > > * \\todo Define when to open and close video device nodes, as they\n> > > * might impact on power consumption.\n> > > diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h\n> > > index ba8f0052..635566c8 100644\n> > > --- a/src/libcamera/pipeline/ipu3/cio2.h\n> > > +++ b/src/libcamera/pipeline/ipu3/cio2.h\n> > > @@ -18,6 +18,7 @@\n> > >\n> > > namespace libcamera {\n> > >\n> > > +class CameraLens;\n> > > class CameraSensor;\n> > > class FrameBuffer;\n> > > class MediaDevice;\n> > > @@ -52,6 +53,7 @@ public:\n> > > int stop();\n> > >\n> > > CameraSensor *sensor() { return sensor_.get(); }\n> > > + CameraLens *lens() { return lens_.get(); }\n> > > const CameraSensor *sensor() const { return sensor_.get(); }\n> > >\n> > > FrameBuffer *queueBuffer(Request *request, FrameBuffer *rawBuffer);\n> > > @@ -67,6 +69,7 @@ private:\n> > > void cio2BufferReady(FrameBuffer *buffer);\n> > >\n> > > std::unique_ptr sensor_;\n> > > + std::unique_ptr lens_;\n> > > std::unique_ptr csi2_;\n> > > std::unique_ptr output_;\n> > >\n> > > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > index 97003681..88775f67 100644\n> > > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > @@ -24,6 +24,7 @@\n> > > #include \n> > >\n> > > #include \"libcamera/internal/camera.h\"\n> > > +#include \"libcamera/internal/camera_lens.h\"\n> > > #include \"libcamera/internal/camera_sensor.h\"\n> > > #include \"libcamera/internal/delayed_controls.h\"\n> > > #include \"libcamera/internal/device_enumerator.h\"\n> > > @@ -1255,8 +1256,12 @@ void IPU3CameraData::queueFrameAction(unsigned int id,\n> > > {\n> > > switch (action.op) {\n> > > case ipa::ipu3::ActionSetSensorControls: {\n> > > - const ControlList &controls = action.sensorControls;\n> > > - delayedCtrls_->push(controls);\n> > > + const ControlList &sensorControls = action.sensorControls;\n> > > + delayedCtrls_->push(sensorControls);\n> > > + if (cio2_.lens()) {\n> > > + ControlList lensControls = action.lensControls;\n> > > + cio2_.lens()->setControls(&lensControls);\n> > > + }\n> > > break;\n> > > }\n> > > case ipa::ipu3::ActionParamFilled: {\n> > >\n>\n> --\n> Regards,\n>\n> Laurent Pinchart","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 23219BDB1C\n\tfor ;\n\tTue, 16 Nov 2021 09:53:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4735860376;\n\tTue, 16 Nov 2021 10:53:03 +0100 (CET)","from mail-oo1-xc31.google.com (mail-oo1-xc31.google.com\n\t[IPv6:2607:f8b0:4864:20::c31])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 74A6260231\n\tfor ;\n\tTue, 16 Nov 2021 10:53:01 +0100 (CET)","by mail-oo1-xc31.google.com with SMTP id\n\tw30-20020a4a355e000000b002c2649b8d5fso7007774oog.10\n\tfor ;\n\tTue, 16 Nov 2021 01:53:01 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"KaQt6DAu\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=r7I8f1hxEdPawv3cMG1vFbOMa4Q+957LvZErvXMSmzY=;\n\tb=KaQt6DAud6EG+RY5iRuMTEeVLv4ijE9fFFgGO6p8FmjEdQfPLyDIRzv5fwN4uLKHjw\n\tkQqOEXvpzDixL74IkbslUmBvy4t4BVnZksIEylRRpVOzXFnQWXUOcg+nXhMJleRHTfYu\n\tacVIw5pjnP6lYF/+yVuJqaRr4UzzErS606JC4=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=r7I8f1hxEdPawv3cMG1vFbOMa4Q+957LvZErvXMSmzY=;\n\tb=p6hGKqmeB6DRNy3OfEQfftHBN9F/oVC+pt+pyyAArklXcfc12Dh7g1/qlL/mQvRuCD\n\tJSKFC9ImGMgEnZvLufXDyoJhvzfA8frZbdJNkivLiIIzPj8Pfs2JrE1bEfPazAFtUfVN\n\tWk709ESUM8ZJulfH97MCnaLPR02QA/PkMUIiB5ePjykp4LexJSgIzv/23JQ0Jkfvt+mo\n\tKzS5V3sX+2KgKxampNmPeHoy0Nmwk5ZRkH2tgBCZLDNRYfWluTmgtxfOa/VZqQs9fdKy\n\tJSFdJoYz63B1ysusuHo41Nfi/D2r89onqQqKsHw2CcjA3FCu+MhATGc/6SpGfXkhb9sI\n\tsRdQ==","X-Gm-Message-State":"AOAM531lAht/pinqnPPhYggHbhs9RmZ2/vhwymjzAsBODKutsXNcHnZq\n\t5LeD1G5hSYWc26tAlxaJ+8VjYageKgSIM6fKqwhajg==","X-Google-Smtp-Source":"ABdhPJzyDq8+yKpX2WEUjKYZzaH3uhoOmn4cDWgvRfwC1dQz5LfDm9e80oTN+9NQNL8MdUxW+tk16TYmj7KKOsILLe8=","X-Received":"by 2002:a4a:ea54:: with SMTP id j20mr3109229ooe.22.1637056379890;\n\tTue, 16 Nov 2021 01:52:59 -0800 (PST)","MIME-Version":"1.0","References":"<20211111104958.312070-1-hanlinchen@chromium.org>\n\t<20211111104958.312070-3-hanlinchen@chromium.org>\n\t<5cc122ca-823a-a929-9bb5-36a7e1469cae@ideasonboard.com>\n\t","In-Reply-To":"","From":"Hanlin Chen ","Date":"Tue, 16 Nov 2021 17:52:49 +0800","Message-ID":"","To":"Laurent Pinchart ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20966,"web_url":"https://patchwork.libcamera.org/comment/20966/","msgid":"","date":"2021-11-16T10:06:46","subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Han-Lin,\n\nOn Tue, Nov 16, 2021 at 05:52:49PM +0800, Hanlin Chen wrote:\n> On Fri, Nov 12, 2021 at 7:58 AM Laurent Pinchart wrote:\n> > On Thu, Nov 11, 2021 at 05:12:52PM +0100, Jean-Michel Hautbois wrote:\n> > > On 11/11/2021 11:49, Han-Lin Chen wrote:\n> > > > Allow IPA to apply controls to the lens device.\n> > > >\n> > > > Signed-off-by: Han-Lin Chen \n> > > > ---\n> > > > meson.build | 6 ++++++\n> > > > src/libcamera/pipeline/ipu3/cio2.cpp | 30 ++++++++++++++++++++++++++++\n> > > > src/libcamera/pipeline/ipu3/cio2.h | 3 +++\n> > > > src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++--\n> > > > 4 files changed, 46 insertions(+), 2 deletions(-)\n> > > >\n> > > > diff --git a/meson.build b/meson.build\n> > > > index 7892a9e3..2a4b68a2 100644\n> > > > --- a/meson.build\n> > > > +++ b/meson.build\n> > > > @@ -108,6 +108,12 @@ if cc.has_argument('-Wno-c99-designator')\n> > > > ]\n> > > > endif\n> > > >\n> > > > +if get_option('android_platform') == 'cros'\n> > > > + common_arguments += [\n> > > > + '-DOS_CHROMEOS',\n> > > > + ]\n> > > > +endif\n> > >\n> > > I am not really aware of specifics regarding this, why can't we have\n> > > something not dependent on the OS ?\n> >\n> > There's a mail thread regarding VCM support on MS Surface machines (see\n> > https://lists.libcamera.org/pipermail/libcamera-devel/2021-November/026526.html).\n> > Raspberry Pi is also looking into VCM support. It would be best to\n> > cooperate with all parties involved to define a way to expose the\n> > relation between camera sensors and lens controllers to userspace, and\n> > use it here. I'm really not keen on having a CrOS-specific hack.\n>\n> Looking forward to it :-).\n\nTo make that happen, may I ask you to reply to David's proposal about\nautofocus controls (https://lists.libcamera.org/pipermail/libcamera-devel/2021-October/025986.html) ?\n\n> > > > +\n> > > > c_arguments += common_arguments\n> > > > cpp_arguments += common_arguments\n> > > >\n> > > > diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > > > index 59dda56b..233553c2 100644\n> > > > --- a/src/libcamera/pipeline/ipu3/cio2.cpp\n> > > > +++ b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > > > @@ -16,6 +16,7 @@\n> > > > #include \n> > > > #include \n> > > >\n> > > > +#include \"libcamera/internal/camera_lens.h\"\n> > > > #include \"libcamera/internal/camera_sensor.h\"\n> > > > #include \"libcamera/internal/framebuffer.h\"\n> > > > #include \"libcamera/internal/media_device.h\"\n> > > > @@ -159,6 +160,35 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index)\n> > > > return -EINVAL;\n> > > > }\n> > > >\n> > > > +#if defined(OS_CHROMEOS)\n> > > > + /*\n> > > > + * \\todo Read the lens model from the sensor itself or from a device database.\n> > > > + * For now use default values taken from ChromeOS.\n> > > > + */\n> > > > + static std::unordered_map sensorLens = {\n> > > > + { \"ov13858\", \"dw9714\" },\n> > > > + { \"imx258\", \"dw9807\" },\n> > > > + { \"imx355\", \"ak7375\" }\n> > > > + };\n> > >\n> > > So, we need a list giving the lens model associated to a sensor... How\n> > > is this dependent on chromeos ? Imagine I want to use the sensor lens on\n> > > my SGo2, which has a ov8865, what is needed to do that ?\n> > >\n> > > > +\n> > > > + auto it = sensorLens.find(sensor_->model());\n> > > > + if (it != sensorLens.end()) {\n> > > > + const std::vector &entities = media->entities();\n> > > > + for (auto ent : entities) {\n> > > > + if (ent->function() == MEDIA_ENT_F_LENS) {\n> > > > + lens_ = std::make_unique(ent);\n> > > > + ret = lens_->init();\n> > > > + if (!ret && lens_->model() == it->second) {\n> > > > + break;\n> > > > + }\n> > > > + lens_.reset();\n> > > > + }\n> > > > + if (!lens_)\n> > > > + LOG(IPU3, Warning) << \"Lens device \" << it->second << \" not found\";\n> > > > + }\n> > > > + }\n> > > > +#endif\n> > > > +\n> > > > /*\n> > > > * \\todo Define when to open and close video device nodes, as they\n> > > > * might impact on power consumption.\n> > > > diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h\n> > > > index ba8f0052..635566c8 100644\n> > > > --- a/src/libcamera/pipeline/ipu3/cio2.h\n> > > > +++ b/src/libcamera/pipeline/ipu3/cio2.h\n> > > > @@ -18,6 +18,7 @@\n> > > >\n> > > > namespace libcamera {\n> > > >\n> > > > +class CameraLens;\n> > > > class CameraSensor;\n> > > > class FrameBuffer;\n> > > > class MediaDevice;\n> > > > @@ -52,6 +53,7 @@ public:\n> > > > int stop();\n> > > >\n> > > > CameraSensor *sensor() { return sensor_.get(); }\n> > > > + CameraLens *lens() { return lens_.get(); }\n> > > > const CameraSensor *sensor() const { return sensor_.get(); }\n> > > >\n> > > > FrameBuffer *queueBuffer(Request *request, FrameBuffer *rawBuffer);\n> > > > @@ -67,6 +69,7 @@ private:\n> > > > void cio2BufferReady(FrameBuffer *buffer);\n> > > >\n> > > > std::unique_ptr sensor_;\n> > > > + std::unique_ptr lens_;\n> > > > std::unique_ptr csi2_;\n> > > > std::unique_ptr output_;\n> > > >\n> > > > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > > index 97003681..88775f67 100644\n> > > > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > > @@ -24,6 +24,7 @@\n> > > > #include \n> > > >\n> > > > #include \"libcamera/internal/camera.h\"\n> > > > +#include \"libcamera/internal/camera_lens.h\"\n> > > > #include \"libcamera/internal/camera_sensor.h\"\n> > > > #include \"libcamera/internal/delayed_controls.h\"\n> > > > #include \"libcamera/internal/device_enumerator.h\"\n> > > > @@ -1255,8 +1256,12 @@ void IPU3CameraData::queueFrameAction(unsigned int id,\n> > > > {\n> > > > switch (action.op) {\n> > > > case ipa::ipu3::ActionSetSensorControls: {\n> > > > - const ControlList &controls = action.sensorControls;\n> > > > - delayedCtrls_->push(controls);\n> > > > + const ControlList &sensorControls = action.sensorControls;\n> > > > + delayedCtrls_->push(sensorControls);\n> > > > + if (cio2_.lens()) {\n> > > > + ControlList lensControls = action.lensControls;\n> > > > + cio2_.lens()->setControls(&lensControls);\n> > > > + }\n> > > > break;\n> > > > }\n> > > > case ipa::ipu3::ActionParamFilled: {\n> > > >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 14B29BF415\n\tfor ;\n\tTue, 16 Nov 2021 10:07:14 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 25BB460376;\n\tTue, 16 Nov 2021 11:07:13 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C838E60231\n\tfor ;\n\tTue, 16 Nov 2021 11:07:11 +0100 (CET)","from pendragon.ideasonboard.com\n\t(117.145-247-81.adsl-dyn.isp.belgacom.be [81.247.145.117])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 25440547;\n\tTue, 16 Nov 2021 11:07:10 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Qt9dUAEK\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1637057230;\n\tbh=6zzDJpzbOGqeaIKg8yKJlqaVNUsgICGxNrCNHyNqTDQ=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Qt9dUAEKFlmYjdSDDvc+c9IplDqdYzQtYov6Kx/hdeCVufxFkzu2PelTlpnSBrB97\n\tdXNrfcnwl42wd/EuFLiJxGT6z4NQuCprds7RI7YgPGGwlURTgNkaAR1BrMsunrl42Q\n\tbBcnoJNNI5Qm90G8Kzdd2835Pa5QxLIDL4KBTUfw=","Date":"Tue, 16 Nov 2021 12:06:46 +0200","From":"Laurent Pinchart ","To":"Hanlin Chen ","Message-ID":"","References":"<20211111104958.312070-1-hanlinchen@chromium.org>\n\t<20211111104958.312070-3-hanlinchen@chromium.org>\n\t<5cc122ca-823a-a929-9bb5-36a7e1469cae@ideasonboard.com>\n\t\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20968,"web_url":"https://patchwork.libcamera.org/comment/20968/","msgid":"","date":"2021-11-16T12:05:10","subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","submitter":{"id":98,"url":"https://patchwork.libcamera.org/api/people/98/","name":"Hanlin Chen","email":"hanlinchen@chromium.org"},"content":"On Tue, Nov 16, 2021 at 6:07 PM Laurent Pinchart\n wrote:\n>\n> Hi Han-Lin,\n>\n> On Tue, Nov 16, 2021 at 05:52:49PM +0800, Hanlin Chen wrote:\n> > On Fri, Nov 12, 2021 at 7:58 AM Laurent Pinchart wrote:\n> > > On Thu, Nov 11, 2021 at 05:12:52PM +0100, Jean-Michel Hautbois wrote:\n> > > > On 11/11/2021 11:49, Han-Lin Chen wrote:\n> > > > > Allow IPA to apply controls to the lens device.\n> > > > >\n> > > > > Signed-off-by: Han-Lin Chen \n> > > > > ---\n> > > > > meson.build | 6 ++++++\n> > > > > src/libcamera/pipeline/ipu3/cio2.cpp | 30 ++++++++++++++++++++++++++++\n> > > > > src/libcamera/pipeline/ipu3/cio2.h | 3 +++\n> > > > > src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++--\n> > > > > 4 files changed, 46 insertions(+), 2 deletions(-)\n> > > > >\n> > > > > diff --git a/meson.build b/meson.build\n> > > > > index 7892a9e3..2a4b68a2 100644\n> > > > > --- a/meson.build\n> > > > > +++ b/meson.build\n> > > > > @@ -108,6 +108,12 @@ if cc.has_argument('-Wno-c99-designator')\n> > > > > ]\n> > > > > endif\n> > > > >\n> > > > > +if get_option('android_platform') == 'cros'\n> > > > > + common_arguments += [\n> > > > > + '-DOS_CHROMEOS',\n> > > > > + ]\n> > > > > +endif\n> > > >\n> > > > I am not really aware of specifics regarding this, why can't we have\n> > > > something not dependent on the OS ?\n> > >\n> > > There's a mail thread regarding VCM support on MS Surface machines (see\n> > > https://lists.libcamera.org/pipermail/libcamera-devel/2021-November/026526.html).\n> > > Raspberry Pi is also looking into VCM support. It would be best to\n> > > cooperate with all parties involved to define a way to expose the\n> > > relation between camera sensors and lens controllers to userspace, and\n> > > use it here. I'm really not keen on having a CrOS-specific hack.\n> >\n> > Looking forward to it :-).\n>\n> To make that happen, may I ask you to reply to David's proposal about\n> autofocus controls (https://lists.libcamera.org/pipermail/libcamera-devel/2021-October/025986.html) ?\n>\nAh yes, thanks for the notice :-) Will do.\n> > > > > +\n> > > > > c_arguments += common_arguments\n> > > > > cpp_arguments += common_arguments\n> > > > >\n> > > > > diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > > > > index 59dda56b..233553c2 100644\n> > > > > --- a/src/libcamera/pipeline/ipu3/cio2.cpp\n> > > > > +++ b/src/libcamera/pipeline/ipu3/cio2.cpp\n> > > > > @@ -16,6 +16,7 @@\n> > > > > #include \n> > > > > #include \n> > > > >\n> > > > > +#include \"libcamera/internal/camera_lens.h\"\n> > > > > #include \"libcamera/internal/camera_sensor.h\"\n> > > > > #include \"libcamera/internal/framebuffer.h\"\n> > > > > #include \"libcamera/internal/media_device.h\"\n> > > > > @@ -159,6 +160,35 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index)\n> > > > > return -EINVAL;\n> > > > > }\n> > > > >\n> > > > > +#if defined(OS_CHROMEOS)\n> > > > > + /*\n> > > > > + * \\todo Read the lens model from the sensor itself or from a device database.\n> > > > > + * For now use default values taken from ChromeOS.\n> > > > > + */\n> > > > > + static std::unordered_map sensorLens = {\n> > > > > + { \"ov13858\", \"dw9714\" },\n> > > > > + { \"imx258\", \"dw9807\" },\n> > > > > + { \"imx355\", \"ak7375\" }\n> > > > > + };\n> > > >\n> > > > So, we need a list giving the lens model associated to a sensor... How\n> > > > is this dependent on chromeos ? Imagine I want to use the sensor lens on\n> > > > my SGo2, which has a ov8865, what is needed to do that ?\n> > > >\n> > > > > +\n> > > > > + auto it = sensorLens.find(sensor_->model());\n> > > > > + if (it != sensorLens.end()) {\n> > > > > + const std::vector &entities = media->entities();\n> > > > > + for (auto ent : entities) {\n> > > > > + if (ent->function() == MEDIA_ENT_F_LENS) {\n> > > > > + lens_ = std::make_unique(ent);\n> > > > > + ret = lens_->init();\n> > > > > + if (!ret && lens_->model() == it->second) {\n> > > > > + break;\n> > > > > + }\n> > > > > + lens_.reset();\n> > > > > + }\n> > > > > + if (!lens_)\n> > > > > + LOG(IPU3, Warning) << \"Lens device \" << it->second << \" not found\";\n> > > > > + }\n> > > > > + }\n> > > > > +#endif\n> > > > > +\n> > > > > /*\n> > > > > * \\todo Define when to open and close video device nodes, as they\n> > > > > * might impact on power consumption.\n> > > > > diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h\n> > > > > index ba8f0052..635566c8 100644\n> > > > > --- a/src/libcamera/pipeline/ipu3/cio2.h\n> > > > > +++ b/src/libcamera/pipeline/ipu3/cio2.h\n> > > > > @@ -18,6 +18,7 @@\n> > > > >\n> > > > > namespace libcamera {\n> > > > >\n> > > > > +class CameraLens;\n> > > > > class CameraSensor;\n> > > > > class FrameBuffer;\n> > > > > class MediaDevice;\n> > > > > @@ -52,6 +53,7 @@ public:\n> > > > > int stop();\n> > > > >\n> > > > > CameraSensor *sensor() { return sensor_.get(); }\n> > > > > + CameraLens *lens() { return lens_.get(); }\n> > > > > const CameraSensor *sensor() const { return sensor_.get(); }\n> > > > >\n> > > > > FrameBuffer *queueBuffer(Request *request, FrameBuffer *rawBuffer);\n> > > > > @@ -67,6 +69,7 @@ private:\n> > > > > void cio2BufferReady(FrameBuffer *buffer);\n> > > > >\n> > > > > std::unique_ptr sensor_;\n> > > > > + std::unique_ptr lens_;\n> > > > > std::unique_ptr csi2_;\n> > > > > std::unique_ptr output_;\n> > > > >\n> > > > > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > > > index 97003681..88775f67 100644\n> > > > > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > > > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > > > > @@ -24,6 +24,7 @@\n> > > > > #include \n> > > > >\n> > > > > #include \"libcamera/internal/camera.h\"\n> > > > > +#include \"libcamera/internal/camera_lens.h\"\n> > > > > #include \"libcamera/internal/camera_sensor.h\"\n> > > > > #include \"libcamera/internal/delayed_controls.h\"\n> > > > > #include \"libcamera/internal/device_enumerator.h\"\n> > > > > @@ -1255,8 +1256,12 @@ void IPU3CameraData::queueFrameAction(unsigned int id,\n> > > > > {\n> > > > > switch (action.op) {\n> > > > > case ipa::ipu3::ActionSetSensorControls: {\n> > > > > - const ControlList &controls = action.sensorControls;\n> > > > > - delayedCtrls_->push(controls);\n> > > > > + const ControlList &sensorControls = action.sensorControls;\n> > > > > + delayedCtrls_->push(sensorControls);\n> > > > > + if (cio2_.lens()) {\n> > > > > + ControlList lensControls = action.lensControls;\n> > > > > + cio2_.lens()->setControls(&lensControls);\n> > > > > + }\n> > > > > break;\n> > > > > }\n> > > > > case ipa::ipu3::ActionParamFilled: {\n> > > > >\n>\n> --\n> Regards,\n>\n> Laurent Pinchart","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0ACE7BF415\n\tfor ;\n\tTue, 16 Nov 2021 12:05:24 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B260260376;\n\tTue, 16 Nov 2021 13:05:23 +0100 (CET)","from mail-oi1-x230.google.com (mail-oi1-x230.google.com\n\t[IPv6:2607:f8b0:4864:20::230])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3996A60120\n\tfor ;\n\tTue, 16 Nov 2021 13:05:22 +0100 (CET)","by mail-oi1-x230.google.com with SMTP id o4so41916168oia.10\n\tfor ;\n\tTue, 16 Nov 2021 04:05:22 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"SUWGmgWt\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=UeIdum16umUtcHGTqBTK0gy6ehJRVp6wXz4xfTrZYgg=;\n\tb=SUWGmgWt5yVx90o2r3K8ki+zp7MSn+vd+dCZea+IRJaGv9mh5A7FewiNwTZnJOgdlT\n\tddJEtZF1in6pJXhpMj8UNkK7OpQBhjs7Jp4XROvosS/hQmvgpnI7ESqAhRdnXR3nZc7K\n\tLHH1FVaQYLp+yOz96rFkcl48ShHrWCrBrDdvI=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=UeIdum16umUtcHGTqBTK0gy6ehJRVp6wXz4xfTrZYgg=;\n\tb=kUYscmYjYMgLZg1eVnXGPgct8o96rlz5/DVMTla5LEApFbHjetDowu2C8qPtbJ+O7w\n\t2PsLGh1JqmeEOMICjlaKl/MxbbzmxE8TZUA5ou9xr33qwdkonBlgnyAie7/U5A54HxDY\n\td7jPsYeURPlKPLP+NExd5YjkDE9SEqgY7qFEBlHcshzSRPr8l8P3xZSVnu6ScifJuZuc\n\tCRhQMSvUbdDRXqH27GZJ4wffO5yEn129N56o6BUiofsjsPpaGGpocsqJk+L0HxCqEKi+\n\tZrHirmZDtmPEfVEi2ruj0Pn/PStyUqkSwF1A5t/7IIP3kh7EmPf/qaHyFOlShxlJLZ7y\n\tfEUw==","X-Gm-Message-State":"AOAM531eVIvdnU6aWWGUBusDJtLae7ZvkhQn7i3qzoCXlowYP99qU8V+\n\tPBmgbz+z7pttz0S2Z6Ppsfw7JsM9n1Vt1ZF516xz8mFqfF0=","X-Google-Smtp-Source":"ABdhPJzZus12RIJSbolUalD05yHV3nBPLcSxpQWcSUIRt9mY9EcI1OgM8GPBtRxjOqZ1nkhWC9uTKrGl82KXMNGgaqs=","X-Received":"by 2002:a54:4401:: with SMTP id\n\tk1mr52820112oiw.143.1637064321025; \n\tTue, 16 Nov 2021 04:05:21 -0800 (PST)","MIME-Version":"1.0","References":"<20211111104958.312070-1-hanlinchen@chromium.org>\n\t<20211111104958.312070-3-hanlinchen@chromium.org>\n\t<5cc122ca-823a-a929-9bb5-36a7e1469cae@ideasonboard.com>\n\t\n\t\n\t","In-Reply-To":"","From":"Hanlin Chen ","Date":"Tue, 16 Nov 2021 20:05:10 +0800","Message-ID":"","To":"Laurent Pinchart ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v3 3/3] ipu3: ipa: Allow IPA to apply\n\tcontrols to the lens device","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]