[{"id":20414,"web_url":"https://patchwork.libcamera.org/comment/20414/","msgid":"<6a1ece74-f602-9e0c-9370-954d0d44aa2e@ideasonboard.com>","date":"2021-10-25T06:29:58","subject":"Re: [libcamera-devel] [RFC PATCH v2] libcamera: pipeline: ipu3:\n\tApply the requested test pattern mode","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/people/75/","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"content":"Hi Hiro,\n\nThanks for the patch !\n\nOn 05/10/2021 07:44, Hirokazu Honda wrote:\n> This enables ipu3 pipeline handler to apply the test pattern mode\n> per frame.\n> \n> Signed-off-by: Hirokazu Honda \n> ---\n> src/libcamera/pipeline/ipu3/ipu3.cpp | 38 ++++++++++++++++++++++++++--\n> 1 file changed, 36 insertions(+), 2 deletions(-)\n> \n> diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> index 92e86925..07fda65f 100644\n> --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> @@ -69,6 +69,7 @@ public:\n> \tvoid statBufferReady(FrameBuffer *buffer);\n> \tvoid queuePendingRequests();\n> \tvoid cancelPendingRequests();\n> +\tvoid frameStart(uint32_t sequence);\n> \n> \tCIO2Device cio2_;\n> \tImgUDevice *imgu_;\n> @@ -88,6 +89,7 @@ public:\n> \tstd::unique_ptr ipa_;\n> \n> \tstd::queue pendingRequests_;\n> +\tstd::queue processingRequests_;\n> \n> \tControlInfoMap ipaControls_;\n> \n> @@ -568,6 +570,8 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c)\n> \tcio2->sensor()->sensorInfo(&sensorInfo);\n> \tdata->cropRegion_ = sensorInfo.analogCrop;\n> \n> +\tcio2->sensor()->setTestPatternMode(controls::draft::TestPatternModeOff);\n\nThere is a dependency on \"[PATCH] libcamera: camera_sensor: Enable to\nset a test pattern mode\" (20211005043723.568685-1-hiroh@chromium.org), \nright ?\nCould you make it a series please :-) ?\n\n> +\n> \t/*\n> \t * Configure the H/V flip controls based on the combination of\n> \t * the sensor and user transform.\n> @@ -801,6 +805,9 @@ void PipelineHandlerIPU3::stop(Camera *camera)\n> \n> \tret |= data->imgu_->stop();\n> \tret |= data->cio2_.stop();\n> +\n> +\tdata->processingRequests_ = {};\n\nShouldn't we do that in cancelPendingRequests() as a mirror of the push \nwhich is done in queuePendingRequests() ?\n\nI am a bit worry that it is done after cio2_->stop()...\n\n> +\n> \tif (ret)\n> \t\tLOG(IPU3, Warning) << \"Failed to stop camera \" << camera->id();\n> \n> @@ -851,6 +858,8 @@ void IPU3CameraData::queuePendingRequests()\n> \n> \t\tinfo->rawBuffer = rawBuffer;\n> \n> +\t\tprocessingRequests_.push(request);\n> +\n> \t\tipa::ipu3::IPU3Event ev;\n> \t\tev.op = ipa::ipu3::EventProcessControls;\n> \t\tev.frame = info->id;\n> @@ -1107,8 +1116,8 @@ int PipelineHandlerIPU3::registerCameras()\n> \t\tdata->delayedCtrls_ =\n> \t\t\tstd::make_unique(cio2->sensor()->device(),\n> \t\t\t\t\t\t\t params);\n> -\t\tdata->cio2_.frameStart().connect(data->delayedCtrls_.get(),\n> -\t\t\t\t\t\t &DelayedControls::applyControls);\n> +\t\tdata->cio2_.frameStart().connect(data.get(),\n> +\t\t\t\t\t\t &IPU3CameraData::frameStart);\n\nThe IPA will receive the ipa::ipu3::EventProcessControls immediately, so \nit could be notified to stop trying to apply the algorithm for instance \n(to be discussed). I think this is good :-).\n\n> \n> \t\t/* Convert the sensor rotation to a transformation */\n> \t\tint32_t rotation = 0;\n> @@ -1399,6 +1408,31 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> \tipa_->processEvent(ev);\n> }\n> \n> +void IPU3CameraData::frameStart(uint32_t sequence)\n> +{\n> +\tif (!processingRequests_.empty()) {\n\nAdd a LOG(IPU3, Debug) here ?\n\n> +\t\t/* Assumes applying the test pattern mode affects immediately. */\n> +\t\tRequest *request = processingRequests_.front();\n> +\t\tprocessingRequests_.pop();\n> +\n> +\t\tif (request->controls().contains(controls::draft::TestPatternMode)) {\n\nSame here, to see which test pattern is applied ?\n\n> +\t\t\tconst int32_t testPatternMode = request->controls().get(\n> +\t\t\t\tcontrols::draft::TestPatternMode);\n> +\n> +\t\t\tint ret = cio2_.sensor()->setTestPatternMode(testPatternMode);\n> +\t\t\tif (ret) {\n> +\t\t\t\tLOG(IPU3, Error) << \"Failed to set test pattern mode: \"\n> +\t\t\t\t\t\t << ret;\n> +\t\t\t} else {\n> +\t\t\t\trequest->metadata().set(controls::draft::TestPatternMode,\n> +\t\t\t\t\t\t\ttestPatternMode);\n> +\t\t\t}\n> +\t\t}\n> +\t}\n> +\n\nAnd a small comment for this line ?\n\n> +\tdelayedCtrls_->applyControls(sequence);\n> +}\n> +\n> REGISTER_PIPELINE_HANDLER(PipelineHandlerIPU3)\n> \n> } /* namespace libcamera */\n> \n\nNow, this patch makes the per-frame control on the IPA point of view \nmore important... How will we handle those ? This is still an open \nquestion...\n\nReviewed-by: Jean-Michel Hautbois ","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 487DCBDB1C\n\tfor ;\n\tMon, 25 Oct 2021 06:30:03 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9246A64870;\n\tMon, 25 Oct 2021 08:30:02 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CA59260237\n\tfor ;\n\tMon, 25 Oct 2021 08:30:00 +0200 (CEST)","from [IPV6:2a01:e0a:169:7140:2d42:18d6:42d3:2848] (unknown\n\t[IPv6:2a01:e0a:169:7140:2d42:18d6:42d3:2848])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6743CE0A;\n\tMon, 25 Oct 2021 08:30:00 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"tSocTQ1W\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1635143400;\n\tbh=BMuzrpt1jS6aUkeX6GsRqvybEa4c9KnxAe03ltDgAko=;\n\th=Date:From:Subject:To:References:In-Reply-To:From;\n\tb=tSocTQ1W7xl4x9ccmkiEBJGk9YD7BRlQj0EMxOVw8DVyKgqYwvIlGiPxNdXvOMFSU\n\tP2k98/PDGF7l0xGDNTcYbNucWga23fuAk3mXxnPoWWo90E01DPLXQaQX04+6leQqB7\n\troP78Ieuj0d03YH9itrw+sWNkmxoBeF4lcfVHpqI=","Message-ID":"<6a1ece74-f602-9e0c-9370-954d0d44aa2e@ideasonboard.com>","Date":"Mon, 25 Oct 2021 08:29:58 +0200","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101\n\tThunderbird/91.1.2","From":"Jean-Michel Hautbois ","To":"Hirokazu Honda , libcamera-devel@lists.libcamera.org","References":"<20211005054414.873711-1-hiroh@chromium.org>","Content-Language":"en-US","In-Reply-To":"<20211005054414.873711-1-hiroh@chromium.org>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [RFC PATCH v2] libcamera: pipeline: ipu3:\n\tApply the requested test pattern mode","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20492,"web_url":"https://patchwork.libcamera.org/comment/20492/","msgid":"","date":"2021-10-26T02:56:45","subject":"Re: [libcamera-devel] [RFC PATCH v2] libcamera: pipeline: ipu3:\n\tApply the requested test pattern mode","submitter":{"id":63,"url":"https://patchwork.libcamera.org/api/people/63/","name":"Hirokazu Honda","email":"hiroh@chromium.org"},"content":"Hi Jean-Michel,\n\nOn Mon, Oct 25, 2021 at 3:30 PM Jean-Michel Hautbois\n wrote:\n>\n> Hi Hiro,\n>\n> Thanks for the patch !\n>\n> On 05/10/2021 07:44, Hirokazu Honda wrote:\n> > This enables ipu3 pipeline handler to apply the test pattern mode\n> > per frame.\n> >\n> > Signed-off-by: Hirokazu Honda \n> > ---\n> > src/libcamera/pipeline/ipu3/ipu3.cpp | 38 ++++++++++++++++++++++++++--\n> > 1 file changed, 36 insertions(+), 2 deletions(-)\n> >\n> > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > index 92e86925..07fda65f 100644\n> > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n> > @@ -69,6 +69,7 @@ public:\n> > void statBufferReady(FrameBuffer *buffer);\n> > void queuePendingRequests();\n> > void cancelPendingRequests();\n> > + void frameStart(uint32_t sequence);\n> >\n> > CIO2Device cio2_;\n> > ImgUDevice *imgu_;\n> > @@ -88,6 +89,7 @@ public:\n> > std::unique_ptr ipa_;\n> >\n> > std::queue pendingRequests_;\n> > + std::queue processingRequests_;\n> >\n> > ControlInfoMap ipaControls_;\n> >\n> > @@ -568,6 +570,8 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c)\n> > cio2->sensor()->sensorInfo(&sensorInfo);\n> > data->cropRegion_ = sensorInfo.analogCrop;\n> >\n> > + cio2->sensor()->setTestPatternMode(controls::draft::TestPatternModeOff);\n>\n> There is a dependency on \"[PATCH] libcamera: camera_sensor: Enable to\n> set a test pattern mode\" (20211005043723.568685-1-hiroh@chromium.org),\n> right ?\n> Could you make it a series please :-) ?\n>\n> > +\n> > /*\n> > * Configure the H/V flip controls based on the combination of\n> > * the sensor and user transform.\n> > @@ -801,6 +805,9 @@ void PipelineHandlerIPU3::stop(Camera *camera)\n> >\n> > ret |= data->imgu_->stop();\n> > ret |= data->cio2_.stop();\n> > +\n> > + data->processingRequests_ = {};\n>\n> Shouldn't we do that in cancelPendingRequests() as a mirror of the push\n> which is done in queuePendingRequests() ?\n>\n> I am a bit worry that it is done after cio2_->stop()...\n>\n> > +\n> > if (ret)\n> > LOG(IPU3, Warning) << \"Failed to stop camera \" << camera->id();\n> >\n> > @@ -851,6 +858,8 @@ void IPU3CameraData::queuePendingRequests()\n> >\n> > info->rawBuffer = rawBuffer;\n> >\n> > + processingRequests_.push(request);\n> > +\n> > ipa::ipu3::IPU3Event ev;\n> > ev.op = ipa::ipu3::EventProcessControls;\n> > ev.frame = info->id;\n> > @@ -1107,8 +1116,8 @@ int PipelineHandlerIPU3::registerCameras()\n> > data->delayedCtrls_ =\n> > std::make_unique(cio2->sensor()->device(),\n> > params);\n> > - data->cio2_.frameStart().connect(data->delayedCtrls_.get(),\n> > - &DelayedControls::applyControls);\n> > + data->cio2_.frameStart().connect(data.get(),\n> > + &IPU3CameraData::frameStart);\n>\n> The IPA will receive the ipa::ipu3::EventProcessControls immediately, so\n> it could be notified to stop trying to apply the algorithm for instance\n> (to be discussed). I think this is good :-).\n>\n> >\n> > /* Convert the sensor rotation to a transformation */\n> > int32_t rotation = 0;\n> > @@ -1399,6 +1408,31 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n> > ipa_->processEvent(ev);\n> > }\n> >\n> > +void IPU3CameraData::frameStart(uint32_t sequence)\n> > +{\n> > + if (!processingRequests_.empty()) {\n>\n> Add a LOG(IPU3, Debug) here ?\n>\n\nI would not do it because processingReqeusts_ is often not empty and\ntherefore adding LOG here is verbose.\n\nThanks,\n-Hiro\n> > + /* Assumes applying the test pattern mode affects immediately. */\n> > + Request *request = processingRequests_.front();\n> > + processingRequests_.pop();\n> > +\n> > + if (request->controls().contains(controls::draft::TestPatternMode)) {\n>\n> Same here, to see which test pattern is applied ?\n>\n> > + const int32_t testPatternMode = request->controls().get(\n> > + controls::draft::TestPatternMode);\n> > +\n> > + int ret = cio2_.sensor()->setTestPatternMode(testPatternMode);\n> > + if (ret) {\n> > + LOG(IPU3, Error) << \"Failed to set test pattern mode: \"\n> > + << ret;\n> > + } else {\n> > + request->metadata().set(controls::draft::TestPatternMode,\n> > + testPatternMode);\n> > + }\n> > + }\n> > + }\n> > +\n>\n> And a small comment for this line ?\n>\n> > + delayedCtrls_->applyControls(sequence);\n> > +}\n> > +\n> > REGISTER_PIPELINE_HANDLER(PipelineHandlerIPU3)\n> >\n> > } /* namespace libcamera */\n> >\n>\n> Now, this patch makes the per-frame control on the IPA point of view\n> more important... How will we handle those ? This is still an open\n> question...\n>\n> Reviewed-by: Jean-Michel Hautbois ","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id EADFBBDB1C\n\tfor ;\n\tTue, 26 Oct 2021 02:56:58 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 031F564878;\n\tTue, 26 Oct 2021 04:56:58 +0200 (CEST)","from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com\n\t[IPv6:2a00:1450:4864:20::52d])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1D1906486B\n\tfor ;\n\tTue, 26 Oct 2021 04:56:56 +0200 (CEST)","by mail-ed1-x52d.google.com with SMTP id j10so7723047eds.12\n\tfor ;\n\tMon, 25 Oct 2021 19:56:56 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"IAXUV4+m\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=OQJ8a6rDkun2/xFeEgPiq2JCW2EYDad1mO0l8EAEcTg=;\n\tb=IAXUV4+ml9a5sACSsPoX+5bmlUqE41s95ap94AKj58F81mI4xQWZL/gMg8oZpeIyxs\n\tBv6uEKIo8Mh5nQ63k5c6Gy5uh0OXTA/uv3NouEf/Tpfl2+MKhbL3xtl/2EkiCFZlqi8t\n\tcfKekCtCMHPP3N2jNSgR8f3pMHqzJFrlL13aQ=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=OQJ8a6rDkun2/xFeEgPiq2JCW2EYDad1mO0l8EAEcTg=;\n\tb=POAbngBr/+T4tgxKR25rhbC/k5me5fwd/3JNEarWhst57MNnLmwZbTUluNIZNx9yTg\n\t9mf4Dn7kOSMwqO7rv2KNEkLW0bYUMggBnVm9CKlhvwLSo78htNLKQPFOf7vvGh9CdW4o\n\tafvwZS9DGJoHhBr4gAqrs2pe+yJDz/Oh1lkHsjRsB9y1P/GhMmqippyUVmPr7rWt/r9C\n\t0NHnefMn7LKfYwKw7J3l3hJpyYprhGzzKiLuM+/BXgQy5ooFrgJUM0Tn8aIv4q+Aw3vm\n\tt728OaBb5hu+DK0Og2MzjDiCAzuEnJrBrrCXSdviGxow6eW2z78Yc97w+JFJFKpq0nZ+\n\tiSvw==","X-Gm-Message-State":"AOAM533tX4OWx1cyjfI8/JOPDQ8mAaT7IFb3fu2j6DolPJBMFQ48JQcy\n\tsgBEfCXwTbD036rUriF+QJ8AIy30STfyXVYPXoWibtxJ7aU=","X-Google-Smtp-Source":"ABdhPJxtVYkPZlryN0Qdc7y8VciQwKFklWKBf3FGHvE3w4UQ4QCAXxVPCnKEcu31vDrEvB+T4LcVWN2gi/CNJXMkJTI=","X-Received":"by 2002:a05:6402:40cd:: with SMTP id\n\tz13mr31045975edb.220.1635217015670; \n\tMon, 25 Oct 2021 19:56:55 -0700 (PDT)","MIME-Version":"1.0","References":"<20211005054414.873711-1-hiroh@chromium.org>\n\t<6a1ece74-f602-9e0c-9370-954d0d44aa2e@ideasonboard.com>","In-Reply-To":"<6a1ece74-f602-9e0c-9370-954d0d44aa2e@ideasonboard.com>","From":"Hirokazu Honda ","Date":"Tue, 26 Oct 2021 11:56:45 +0900","Message-ID":"","To":"Jean-Michel Hautbois ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [RFC PATCH v2] libcamera: pipeline: ipu3:\n\tApply the requested test pattern mode","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]