[{"id":30397,"web_url":"https://patchwork.libcamera.org/comment/30397/","msgid":"","date":"2024-07-13T15:37:15","subject":"Re: [PATCH v2 11/19] libcamera: software_isp: Call\n\tAlgorithm::queueRequest","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/people/86/","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"content":"Hi Milan,\n\n\nOn 03/07/24 11:21 pm, Milan Zamazal wrote:\n> This patch adds Algorithm::queueRequest call for the defined algorithms.\n> This is preparation only since there are currently no Algorithm based\n> algorithms defined.\n\nI think the patch targets adding queueRequest() call to soft ISP and \nsoftIPA and Algorithms here.\n\nSo this has to be reworded to reflect that.\n\n> Signed-off-by: Milan Zamazal \n> ---\n> include/libcamera/internal/software_isp/software_isp.h | 1 +\n> include/libcamera/ipa/soft.mojom | 1 +\n> src/ipa/simple/soft_simple.cpp | 9 +++++++++\n> src/libcamera/pipeline/simple/simple.cpp | 2 ++\n> src/libcamera/software_isp/software_isp.cpp | 10 ++++++++++\n> 5 files changed, 23 insertions(+)\n>\n> diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h\n> index ff26b1d4..1ba9eb62 100644\n> --- a/include/libcamera/internal/software_isp/software_isp.h\n> +++ b/include/libcamera/internal/software_isp/software_isp.h\n> @@ -72,6 +72,7 @@ public:\n> \tint start();\n> \tvoid stop();\n> \n> +\tvoid queueRequest(const uint32_t frame, const ControlList &controls);\n> \tint queueBuffers(uint32_t frame, FrameBuffer *input,\n> \t\t\t const std::map &outputs);\n> \n> diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom\n> index 5e124016..426fee18 100644\n> --- a/include/libcamera/ipa/soft.mojom\n> +++ b/include/libcamera/ipa/soft.mojom\n> @@ -23,6 +23,7 @@ interface IPASoftInterface {\n> \tconfigure(IPAConfigInfo configInfo)\n> \t\t=> (int32 ret);\n> \n> +\tqueueRequest(uint32 frame, libcamera.ControlList sensorControls);\n\nUsually this is [async] , but I suppose that's not the case with Soft \nISP here?\n\nAS this is only queuing later on to Algorithms, I guess it's fine for now.\n\nReviewed-by: Umang Jain \n> \t[async] processStats(uint32 frame, uint32 bufferId, libcamera.ControlList sensorControls);\n> };\n> \n> diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp\n> index 49fff312..bd0dea5f 100644\n> --- a/src/ipa/simple/soft_simple.cpp\n> +++ b/src/ipa/simple/soft_simple.cpp\n> @@ -77,6 +77,7 @@ public:\n> \tint start() override;\n> \tvoid stop() override;\n> \n> +\tvoid queueRequest(const uint32_t frame, const ControlList &controls) override;\n> \tvoid processStats(const uint32_t frame, const uint32_t bufferId,\n> \t\t\t const ControlList &sensorControls) override;\n> \n> @@ -269,6 +270,14 @@ void IPASoftSimple::stop()\n> {\n> }\n> \n> +void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &controls)\n> +{\n> +\tIPAFrameContext &frameContext = context_.frameContexts.alloc(frame);\n> +\n> +\tfor (auto const &algo : algorithms())\n> +\t\talgo->queueRequest(context_, frame, frameContext, controls);\n> +}\n> +\n> void IPASoftSimple::processStats(\n> \t[[maybe_unused]] const uint32_t frame,\n> \t[[maybe_unused]] const uint32_t bufferId,\n> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n> index e0c9fe5c..6fb80209 100644\n> --- a/src/libcamera/pipeline/simple/simple.cpp\n> +++ b/src/libcamera/pipeline/simple/simple.cpp\n> @@ -1430,6 +1430,8 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n> \tif (data->useConversion_)\n> \t\tdata->conversionQueue_.push(std::move(buffers));\n> \n> +\tdata->swIsp_->queueRequest(request->sequence(), request->controls());\n> +\n> \treturn 0;\n> }\n> \n> diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp\n> index 2bbb86da..4e6b3a2e 100644\n> --- a/src/libcamera/software_isp/software_isp.cpp\n> +++ b/src/libcamera/software_isp/software_isp.cpp\n> @@ -279,6 +279,16 @@ int SoftwareIsp::exportBuffers(unsigned int output, unsigned int count,\n> \treturn count;\n> }\n> \n> +/**\n> + * \\brief Queue a request and process the control list from the application\n> + * \\param[in] frame The number of the frame which will be processed next\n> + * \\param[in] controls The controls for the \\a frame\n> + */\n> +void SoftwareIsp::queueRequest(const uint32_t frame, const ControlList &controls)\n> +{\n> +\tipa_->queueRequest(frame, controls);\n> +}\n> +\n> /**\n> * \\brief Queue buffers to Software ISP\n> * \\param[in] frame The frame number","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9B4EEBD87C\n\tfor ;\n\tSat, 13 Jul 2024 15:37:22 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B8AB463371;\n\tSat, 13 Jul 2024 17:37:21 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 96DB463369\n\tfor ;\n\tSat, 13 Jul 2024 17:37:20 +0200 (CEST)","from [IPV6:2405:201:2015:f873:55d7:c02e:b2eb:ee3f] (unknown\n\t[IPv6:2405:201:2015:f873:55d7:c02e:b2eb:ee3f])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B36CDD01;\n\tSat, 13 Jul 2024 17:36:44 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"E8fpveQr\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1720885005;\n\tbh=2WS0oFTgZVOWJpDab7lfIhgvxnLMMeOJn1DsloI+Wv4=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=E8fpveQr4riwvLozH/9G6KQwq7E8xN3XvQ8thtI1y/ZSNRKHqp3aUI+YWzhnhFGSs\n\tVB53BlWqmMwjsXpMmLCjRlKVxTpSC8yAqDRiaDeKDdnOi6+0eKtQSl+rKyEOP7b+Pj\n\tdWUmJMwVTHZKDzxDveK1yos07hTVpib8vOWQlXww=","Message-ID":"","Date":"Sat, 13 Jul 2024 21:07:15 +0530","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH v2 11/19] libcamera: software_isp: Call\n\tAlgorithm::queueRequest","To":"Milan Zamazal , libcamera-devel@lists.libcamera.org","References":"<20240703175119.1872585-1-mzamazal@redhat.com>\n\t<20240703175119.1872585-12-mzamazal@redhat.com>","Content-Language":"en-US","From":"Umang Jain ","In-Reply-To":"<20240703175119.1872585-12-mzamazal@redhat.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":30404,"web_url":"https://patchwork.libcamera.org/comment/30404/","msgid":"<87ikx9hy3j.fsf@redhat.com>","date":"2024-07-13T19:48:48","subject":"Re: [PATCH v2 11/19] libcamera: software_isp: Call\n\tAlgorithm::queueRequest","submitter":{"id":177,"url":"https://patchwork.libcamera.org/api/people/177/","name":"Milan Zamazal","email":"mzamazal@redhat.com"},"content":"Hi Umang,\n\nthank you for review.\n\nUmang Jain writes:\n\n> Hi Milan,\n>\n>\n> On 03/07/24 11:21 pm, Milan Zamazal wrote:\n>> This patch adds Algorithm::queueRequest call for the defined algorithms.\n>> This is preparation only since there are currently no Algorithm based\n>> algorithms defined.\n>\n> I think the patch targets adding queueRequest() call to soft ISP and softIPA and Algorithms here.\n>\n> So this has to be reworded to reflect that.\n\nOK, will do.\n\n>> Signed-off-by: Milan Zamazal \n>> ---\n>> include/libcamera/internal/software_isp/software_isp.h | 1 +\n>> include/libcamera/ipa/soft.mojom | 1 +\n>> src/ipa/simple/soft_simple.cpp | 9 +++++++++\n>> src/libcamera/pipeline/simple/simple.cpp | 2 ++\n>> src/libcamera/software_isp/software_isp.cpp | 10 ++++++++++\n>> 5 files changed, 23 insertions(+)\n>>\n>> diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h\n>> index ff26b1d4..1ba9eb62 100644\n>> --- a/include/libcamera/internal/software_isp/software_isp.h\n>> +++ b/include/libcamera/internal/software_isp/software_isp.h\n>> @@ -72,6 +72,7 @@ public:\n>> \tint start();\n>> \tvoid stop();\n>> +\tvoid queueRequest(const uint32_t frame, const ControlList &controls);\n>> \tint queueBuffers(uint32_t frame, FrameBuffer *input,\n>> \t\t\t const std::map &outputs);\n>> diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom\n>> index 5e124016..426fee18 100644\n>> --- a/include/libcamera/ipa/soft.mojom\n>> +++ b/include/libcamera/ipa/soft.mojom\n>> @@ -23,6 +23,7 @@ interface IPASoftInterface {\n>> \tconfigure(IPAConfigInfo configInfo)\n>> \t\t=> (int32 ret);\n>> +\tqueueRequest(uint32 frame, libcamera.ControlList sensorControls);\n>\n> Usually this is [async] , but I suppose that's not the case with Soft ISP here?\n\nI'm not sure. I don't know what's the reason to make it async as it\nlooks like the algorithms just look at control values and assign their\ncounterparts; maybe because it is called at a sensitive moment? I'll\nchange it to async to be safe and consistent.\n\n> AS this is only queuing later on to Algorithms, I guess it's fine for now.\n>\n> Reviewed-by: Umang Jain \n>> \t[async] processStats(uint32 frame, uint32 bufferId, libcamera.ControlList sensorControls);\n>> };\n>> diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp\n>> index 49fff312..bd0dea5f 100644\n>> --- a/src/ipa/simple/soft_simple.cpp\n>> +++ b/src/ipa/simple/soft_simple.cpp\n>> @@ -77,6 +77,7 @@ public:\n>> \tint start() override;\n>> \tvoid stop() override;\n>> +\tvoid queueRequest(const uint32_t frame, const ControlList &controls) override;\n>> \tvoid processStats(const uint32_t frame, const uint32_t bufferId,\n>> \t\t\t const ControlList &sensorControls) override;\n>> @@ -269,6 +270,14 @@ void IPASoftSimple::stop()\n>> {\n>> }\n>> +void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &controls)\n>> +{\n>> +\tIPAFrameContext &frameContext = context_.frameContexts.alloc(frame);\n>> +\n>> +\tfor (auto const &algo : algorithms())\n>> +\t\talgo->queueRequest(context_, frame, frameContext, controls);\n>> +}\n>> +\n>> void IPASoftSimple::processStats(\n>> \t[[maybe_unused]] const uint32_t frame,\n>> \t[[maybe_unused]] const uint32_t bufferId,\n>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\n>> index e0c9fe5c..6fb80209 100644\n>> --- a/src/libcamera/pipeline/simple/simple.cpp\n>> +++ b/src/libcamera/pipeline/simple/simple.cpp\n>> @@ -1430,6 +1430,8 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n>> \tif (data->useConversion_)\n>> \t\tdata->conversionQueue_.push(std::move(buffers));\n>> +\tdata->swIsp_->queueRequest(request->sequence(), request->controls());\n>> +\n>> \treturn 0;\n>> }\n>> diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp\n>> index 2bbb86da..4e6b3a2e 100644\n>> --- a/src/libcamera/software_isp/software_isp.cpp\n>> +++ b/src/libcamera/software_isp/software_isp.cpp\n>> @@ -279,6 +279,16 @@ int SoftwareIsp::exportBuffers(unsigned int output, unsigned int count,\n>> \treturn count;\n>> }\n>> +/**\n>> + * \\brief Queue a request and process the control list from the application\n>> + * \\param[in] frame The number of the frame which will be processed next\n>> + * \\param[in] controls The controls for the \\a frame\n>> + */\n>> +void SoftwareIsp::queueRequest(const uint32_t frame, const ControlList &controls)\n>> +{\n>> +\tipa_->queueRequest(frame, controls);\n>> +}\n>> +\n>> /**\n>> * \\brief Queue buffers to Software ISP\n>> * \\param[in] frame The frame number","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5D220BDB1C\n\tfor ;\n\tSat, 13 Jul 2024 19:48:58 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 38D226336F;\n\tSat, 13 Jul 2024 21:48:57 +0200 (CEST)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C541563365\n\tfor ;\n\tSat, 13 Jul 2024 21:48:54 +0200 (CEST)","from mail-ej1-f70.google.com (mail-ej1-f70.google.com\n\t[209.85.218.70]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n\tus-mta-505-9rGtJ_1vNhmEbABHGooNRg-1; Sat, 13 Jul 2024 15:48:52 -0400","by mail-ej1-f70.google.com with SMTP id\n\ta640c23a62f3a-a797c5b4f47so212403766b.2\n\tfor ;\n\tSat, 13 Jul 2024 12:48:52 -0700 (PDT)","from nuthatch (ip-77-48-47-2.net.vodafone.cz. [77.48.47.2])\n\tby smtp.gmail.com with ESMTPSA id\n\ta640c23a62f3a-a79bc7f1ceesm73288866b.126.2024.07.13.12.48.49\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tSat, 13 Jul 2024 12:48:49 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"YuB76VQ1\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1720900133;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=JUW/0k2ZHrRPooCZulfpBX7MEpe1jt+cE5oZIZqriEs=;\n\tb=YuB76VQ1072rrJCzfSJo4/YjP0dQPmsDLZzOtLEvgqd+kdteiuK32xKfpUUtg0XjpCDJrN\n\tYvaj5cfFZw713U3iy2xBMTIPgMc/8B5PYEv1uvqd+mVLyX64L/YrkUmiPGFcWaSwurOqiQ\n\tFHNHjPKjP1i3z81zYQGqRmiGFFbjLlY=","X-MC-Unique":"9rGtJ_1vNhmEbABHGooNRg-1","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1720900130; x=1721504930;\n\th=mime-version:user-agent:message-id:date:references:in-reply-to\n\t:subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=JUW/0k2ZHrRPooCZulfpBX7MEpe1jt+cE5oZIZqriEs=;\n\tb=FDTDra4UJN+5NicN4aitDLDRNoMAhWCZ2JUEVtq7ufAGiuNzhXzcqkIXohaSglU2Rz\n\tt3v06GVZteloiGFM0rp58kAR4dD7aNKGM2uHsg5+OOdPAYkePPZsNyPEHQHr9hA2ui6Y\n\thrXAIi+Z6tEUwOTDI6cjav6Pdod/d5qSdfM9td3QgSIFFjqCua3Y7T2JyTHyrWHJJS/j\n\tUvC/7cBTtVqq30yqZrTSm7qMcIhBuP+q015jqolXVZYcmubYeJqnxl9SuiP1zmjj46IP\n\tcMeidU7odfDK+dwyqqBygWNX4y9RjaguYueij4UOEbg89YiIm/JhxCwUCTvP7hJXmV8Y\n\tBS3A==","X-Gm-Message-State":"AOJu0Yy/yV7Ix+5/A5AWopzceUrlQtkpXA4WIxVLdIX9woobFWGJlCO7\n\t8IowcyD9y+QmZG3SgTVpcoGE9TkRd5yZqkilGYWAF0xGpRK6lQZ7Y8rWwqUpznDoTzqVNnXPeng\n\tTKzEKhpTgf90tGbN4LLwEKj3qqiJZzxpBfVQ3+4Bq6FczmyXjPPKur9nROGxC+6WZKb61GlmUyI\n\txfdHwkHF7WVRw2DksgU3rh4jdVZSx2RaIhbpOVO/AzuJnLt/ECV0wPi8Q=","X-Received":["by 2002:a17:906:d89:b0:a77:dcda:1fe1 with SMTP id\n\ta640c23a62f3a-a780b6b1ba2mr813575966b.25.1720900130535; \n\tSat, 13 Jul 2024 12:48:50 -0700 (PDT)","by 2002:a17:906:d89:b0:a77:dcda:1fe1 with SMTP id\n\ta640c23a62f3a-a780b6b1ba2mr813575266b.25.1720900129884; \n\tSat, 13 Jul 2024 12:48:49 -0700 (PDT)"],"X-Google-Smtp-Source":"AGHT+IHOApg5INM0HR5P/fT0GAWumzA3Zkysx3pjsmstcFGHguvnxXA6S8oQnbcfay4bRgWkBKEFPA==","From":"Milan Zamazal ","To":"Umang Jain ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH v2 11/19] libcamera: software_isp: Call\n\tAlgorithm::queueRequest","In-Reply-To":" (Umang\n\tJain's message of \"Sat, 13 Jul 2024 21:07:15 +0530\")","References":"<20240703175119.1872585-1-mzamazal@redhat.com>\n\t<20240703175119.1872585-12-mzamazal@redhat.com>\n\t","Date":"Sat, 13 Jul 2024 21:48:48 +0200","Message-ID":"<87ikx9hy3j.fsf@redhat.com>","User-Agent":"Gnus/5.13 (Gnus v5.13)","MIME-Version":"1.0","X-Mimecast-Spam-Score":"0","X-Mimecast-Originator":"redhat.com","Content-Type":"text/plain","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]