[{"id":32671,"web_url":"https://patchwork.libcamera.org/comment/32671/","msgid":"","date":"2024-12-11T09:29:58","subject":"Re: [PATCH v4 2/7] android: Set StreamBuffer::srcBuffer in\n\tCameraDevice::processCaptureRequest","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/people/143/","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"content":"Hi Harvey\n\nOn Tue, Dec 10, 2024 at 02:23:55PM +0000, Harvey Yang wrote:\n> StreamBuffer::srcBuffer was set right before being processed, while it\n> was already determined when being constructed. This patch sets the value\n> in CameraDevice::processCaptureRequest. `requestedStreams` is refactored\n> from std::set to std::map to find the correct srcBuffer accordingly.\n>\n> The change also helps cleaning up the post process pipeline's dependency\n> on `Camera3RequestDescriptor::pendingStreamsToProcess_`, which is going\n> to be cleaned up in the following patches.\n\nDoes this still apply ?\n\n>\n> Signed-off-by: Han-Lin Chen \n> Co-developed-by: Harvey Yang \n> Signed-off-by: Harvey Yang \n> ---\n> src/android/camera_device.cpp | 28 +++++++++++++++++++---------\n> 1 file changed, 19 insertions(+), 9 deletions(-)\n>\n> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> index 4e3bdc9cc..f6dadaf22 100644\n> --- a/src/android/camera_device.cpp\n> +++ b/src/android/camera_device.cpp\n> @@ -9,7 +9,6 @@\n>\n> #include \n> #include \n> -#include \n> #include \n> #include \n> #include \n> @@ -967,9 +966,9 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> \t * to a libcamera stream. Streams of type Mapped will be handled later.\n> \t *\n> \t * Collect the CameraStream associated to each requested capture stream.\n> -\t * Since requestedStreams is an std:set<>, no duplications can happen.\n> +\t * Since requestedStreams is an std:map<>, no duplications can happen.\n> \t */\n> -\tstd::set requestedStreams;\n> +\tstd::map requestedBuffers;\n> \tfor (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) {\n> \t\tCameraStream *cameraStream = buffer.stream;\n> \t\tcamera3_stream_t *camera3Stream = cameraStream->camera3Stream();\n> @@ -1021,6 +1020,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> \t\t\t */\n> \t\t\tframeBuffer = cameraStream->getBuffer();\n> \t\t\tbuffer.internalBuffer = frameBuffer;\n> +\t\t\tbuffer.srcBuffer = frameBuffer;\n> +\n> \t\t\tLOG(HAL, Debug) << ss.str() << \" (internal)\";\n>\n> \t\t\tdescriptor->pendingStreamsToProcess_.insert(\n> @@ -1037,7 +1038,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> \t\tdescriptor->request_->addBuffer(cameraStream->stream(),\n> \t\t\t\t\t\tframeBuffer, std::move(fence));\n>\n> -\t\trequestedStreams.insert(cameraStream);\n> +\t\trequestedBuffers[cameraStream] = frameBuffer;\n> \t}\n>\n> \t/*\n> @@ -1067,9 +1068,19 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> \t\t * added to the request.\n> \t\t */\n> \t\tCameraStream *sourceStream = cameraStream->sourceStream();\n> -\t\tASSERT(sourceStream);\n> -\t\tif (requestedStreams.find(sourceStream) != requestedStreams.end())\n> +\t\tASSERT(sourceStream && sourceStream->type() == CameraStream::Type::Direct);\n> +\n> +\t\t/*\n> +\t\t * If the buffer for the source stream has been requested as\n> +\t\t * Direct, use its framebuffer as the source buffer for\n> +\t\t * post-processing. No need to recycle the buffer since it's\n> +\t\t * owned by Android.\n> +\t\t */\n> +\t\tauto iterDirectBuffer = requestedBuffers.find(sourceStream);\n> +\t\tif (iterDirectBuffer != requestedBuffers.end()) {\n> +\t\t\tbuffer.srcBuffer = iterDirectBuffer->second;\n> \t\t\tcontinue;\n> +\t\t}\n>\n> \t\t/*\n> \t\t * If that's not the case, we need to add a buffer to the request\n> @@ -1077,11 +1088,12 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> \t\t */\n> \t\tFrameBuffer *frameBuffer = cameraStream->getBuffer();\n> \t\tbuffer.internalBuffer = frameBuffer;\n> +\t\tbuffer.srcBuffer = frameBuffer;\n>\n> \t\tdescriptor->request_->addBuffer(sourceStream->stream(),\n> \t\t\t\t\t\tframeBuffer, nullptr);\n>\n> -\t\trequestedStreams.insert(sourceStream);\n> +\t\trequestedBuffers[sourceStream] = frameBuffer;\n> \t}\n>\n> \t/*\n> @@ -1236,8 +1248,6 @@ void CameraDevice::requestComplete(Request *request)\n> \t\t\tcontinue;\n> \t\t}\n>\n> -\t\tbuffer->srcBuffer = src;\n> -\n\nIn this new version of the series I see that setting srcBuffer earlire\nallows to validate the buffer error status in 5/7, so\n\nReviewed-by: Jacopo Mondi \n\nThanks\n j\n\n> \t\t++iter;\n> \t\tint ret = stream->process(buffer);\n> \t\tif (ret) {\n> --\n> 2.47.0.338.g60cca15819-goog\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2D99DC32DD\n\tfor ;\n\tWed, 11 Dec 2024 09:30:05 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4E39B67EA8;\n\tWed, 11 Dec 2024 10:30:04 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 8F65266136\n\tfor ;\n\tWed, 11 Dec 2024 10:30:01 +0100 (CET)","from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it\n\t[93.61.96.190])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 90B5F352;\n\tWed, 11 Dec 2024 10:29:28 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"jnYEB+OY\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1733909368;\n\tbh=065H0UFT24HQFDkhZfJQMq/8/p8CKCTVr+vDc1HLBZw=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=jnYEB+OYbZmH7TNO18DLhPbO8jMrVOftWXtdB6PQVqe1QTSpnT+ZrLOlZH5hkfpHr\n\t41c24xSZdIYnyFXLTkkw33ulPkknMrGG7wbq1i9R/BmfTJcGRXXJS5eZ+wTelKDO/x\n\tYm4kL/q9qSwX1wps5qKdbwPZagY5IYs2O48QXAC0=","Date":"Wed, 11 Dec 2024 10:29:58 +0100","From":"Jacopo Mondi ","To":"Harvey Yang ","Cc":"libcamera-devel@lists.libcamera.org, \n\tHan-Lin Chen ","Subject":"Re: [PATCH v4 2/7] android: Set StreamBuffer::srcBuffer in\n\tCameraDevice::processCaptureRequest","Message-ID":"","References":"<20241210142557.2886315-1-chenghaoyang@chromium.org>\n\t<20241210142557.2886315-3-chenghaoyang@chromium.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20241210142557.2886315-3-chenghaoyang@chromium.org>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32688,"web_url":"https://patchwork.libcamera.org/comment/32688/","msgid":"","date":"2024-12-12T09:18:54","subject":"Re: [PATCH v4 2/7] android: Set StreamBuffer::srcBuffer in\n\tCameraDevice::processCaptureRequest","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/people/117/","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"content":"Hi Jacopo,\n\nOn Wed, Dec 11, 2024 at 5:30 PM Jacopo Mondi\n wrote:\n>\n> Hi Harvey\n>\n> On Tue, Dec 10, 2024 at 02:23:55PM +0000, Harvey Yang wrote:\n> > StreamBuffer::srcBuffer was set right before being processed, while it\n> > was already determined when being constructed. This patch sets the value\n> > in CameraDevice::processCaptureRequest. `requestedStreams` is refactored\n> > from std::set to std::map to find the correct srcBuffer accordingly.\n> >\n> > The change also helps cleaning up the post process pipeline's dependency\n> > on `Camera3RequestDescriptor::pendingStreamsToProcess_`, which is going\n> > to be cleaned up in the following patches.\n>\n> Does this still apply ?\n\nAh I should say `series`. Updated.\n\n>\n> >\n> > Signed-off-by: Han-Lin Chen \n> > Co-developed-by: Harvey Yang \n> > Signed-off-by: Harvey Yang \n> > ---\n> > src/android/camera_device.cpp | 28 +++++++++++++++++++---------\n> > 1 file changed, 19 insertions(+), 9 deletions(-)\n> >\n> > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > index 4e3bdc9cc..f6dadaf22 100644\n> > --- a/src/android/camera_device.cpp\n> > +++ b/src/android/camera_device.cpp\n> > @@ -9,7 +9,6 @@\n> >\n> > #include \n> > #include \n> > -#include \n> > #include \n> > #include \n> > #include \n> > @@ -967,9 +966,9 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> > * to a libcamera stream. Streams of type Mapped will be handled later.\n> > *\n> > * Collect the CameraStream associated to each requested capture stream.\n> > - * Since requestedStreams is an std:set<>, no duplications can happen.\n> > + * Since requestedStreams is an std:map<>, no duplications can happen.\n> > */\n> > - std::set requestedStreams;\n> > + std::map requestedBuffers;\n> > for (const auto &[i, buffer] : utils::enumerate(descriptor->buffers_)) {\n> > CameraStream *cameraStream = buffer.stream;\n> > camera3_stream_t *camera3Stream = cameraStream->camera3Stream();\n> > @@ -1021,6 +1020,8 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> > */\n> > frameBuffer = cameraStream->getBuffer();\n> > buffer.internalBuffer = frameBuffer;\n> > + buffer.srcBuffer = frameBuffer;\n> > +\n> > LOG(HAL, Debug) << ss.str() << \" (internal)\";\n> >\n> > descriptor->pendingStreamsToProcess_.insert(\n> > @@ -1037,7 +1038,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> > descriptor->request_->addBuffer(cameraStream->stream(),\n> > frameBuffer, std::move(fence));\n> >\n> > - requestedStreams.insert(cameraStream);\n> > + requestedBuffers[cameraStream] = frameBuffer;\n> > }\n> >\n> > /*\n> > @@ -1067,9 +1068,19 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> > * added to the request.\n> > */\n> > CameraStream *sourceStream = cameraStream->sourceStream();\n> > - ASSERT(sourceStream);\n> > - if (requestedStreams.find(sourceStream) != requestedStreams.end())\n> > + ASSERT(sourceStream && sourceStream->type() == CameraStream::Type::Direct);\n> > +\n> > + /*\n> > + * If the buffer for the source stream has been requested as\n> > + * Direct, use its framebuffer as the source buffer for\n> > + * post-processing. No need to recycle the buffer since it's\n> > + * owned by Android.\n> > + */\n> > + auto iterDirectBuffer = requestedBuffers.find(sourceStream);\n> > + if (iterDirectBuffer != requestedBuffers.end()) {\n> > + buffer.srcBuffer = iterDirectBuffer->second;\n> > continue;\n> > + }\n> >\n> > /*\n> > * If that's not the case, we need to add a buffer to the request\n> > @@ -1077,11 +1088,12 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> > */\n> > FrameBuffer *frameBuffer = cameraStream->getBuffer();\n> > buffer.internalBuffer = frameBuffer;\n> > + buffer.srcBuffer = frameBuffer;\n> >\n> > descriptor->request_->addBuffer(sourceStream->stream(),\n> > frameBuffer, nullptr);\n> >\n> > - requestedStreams.insert(sourceStream);\n> > + requestedBuffers[sourceStream] = frameBuffer;\n> > }\n> >\n> > /*\n> > @@ -1236,8 +1248,6 @@ void CameraDevice::requestComplete(Request *request)\n> > continue;\n> > }\n> >\n> > - buffer->srcBuffer = src;\n> > -\n>\n> In this new version of the series I see that setting srcBuffer earlire\n> allows to validate the buffer error status in 5/7, so\n\nYes, this also helps with that :)\n\nBR,\nHarvey\n\n>\n> Reviewed-by: Jacopo Mondi \n>\n> Thanks\n> j\n>\n> > ++iter;\n> > int ret = stream->process(buffer);\n> > if (ret) {\n> > --\n> > 2.47.0.338.g60cca15819-goog\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9668ABD80A\n\tfor ;\n\tThu, 12 Dec 2024 09:19:09 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8F6CD67EBF;\n\tThu, 12 Dec 2024 10:19:08 +0100 (CET)","from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com\n\t[IPv6:2a00:1450:4864:20::22a])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3894B67E6D\n\tfor ;\n\tThu, 12 Dec 2024 10:19:06 +0100 (CET)","by mail-lj1-x22a.google.com with SMTP id\n\t38308e7fff4ca-300479ca5c6so3321001fa.3\n\tfor ;\n\tThu, 12 Dec 2024 01:19:06 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"Q05NQzZR\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=chromium.org; s=google; t=1733995145; x=1734599945;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:cc:to:subject:message-id:date:from\n\t:in-reply-to:references:mime-version:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=1rgYjZBa6r49ihR73D69AZ/3aMbSUos1nvMkWOS9gFk=;\n\tb=Q05NQzZR90aVYJfxXGNG+o0B3d/LnvhGTTwNHPdJ0jxAHrPp9ohEC0HRuA4tjqWtPq\n\t34oS4PujKOfS64EfyF8q6bYqsb5Bi8hEY1gUSOKdxsf9GY9SkZ/Z7Pjm+j0L4FEHI7Sb\n\tOzn5j5LB7ZkWYbNEdhxofBKURe86UruBQKvJY=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1733995145; x=1734599945;\n\th=content-transfer-encoding:cc:to:subject:message-id:date:from\n\t:in-reply-to:references:mime-version:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=1rgYjZBa6r49ihR73D69AZ/3aMbSUos1nvMkWOS9gFk=;\n\tb=bQ0cyLwri49YW0hCfzcce6usfTgS4DHVyRXMFa+fXcQPrc5WiO70LS3NFDU6XynN/H\n\t6ckFCf+FMXFeEPc1T6BwqAUq99AI3O1Pun69sRdWnMuKuUlCeeXA4Ndx5tRiAdpt6uxE\n\tv3uxMM10xM2IKv3/d2U8tdI/HExxs9x+QXBf2+D0bQqDd8G+qt9x3FvkTMJAT9bExmGp\n\tyFkcKAOiCi4Oz2f0Ua7LG40VHdcaFauMgdxzP0osQqZNQ8GIAte49qFpXnlPZhDf6azd\n\tFQcK2HZmyWYXWmY2IEQWELM1Ds2C2dV0VSJSsBoYXM4nfCZkIDgkpaLmR4qaxxB3Zf4R\n\tWOSw==","X-Gm-Message-State":"AOJu0YxaT6cR+mGUhbzdl/zD9LNNnKJgaq0GnDdJLS4TBiY8LDMYJntg\n\tSauu7Rdvb6LS/z3+6J7fhb4KSvyF3r97bQ/gPPN97Ieu9pCO+H5tdWj57IzVG1RN2qwAYEQiMFY\n\tJCeL6+asDR7bCEF74a+Bnd8gbmyy9REQQ1TukUKlIyhhmxoE=","X-Gm-Gg":"ASbGnctbHBZ9JzTbcTlNIvUBONHE1Rkg4E+VrjINnDdkLevI7lugKZMRdlbED5kU2Cb\n\tYVzgHqQO83uZlZXGzAyIaW8aSJM9KUEJ/0Md4BqCq6HV0PQgfqMdmxHNd7sb24XtQqA==","X-Google-Smtp-Source":"AGHT+IFPb8utQ91kuZGziXVXMVsnV5aOvpJm0/j8quiiC9tIY4dBMhNpHqsvDXCX+sytPTXC3PDJniREX4N0SPHoQlE=","X-Received":"by 2002:a2e:bea2:0:b0:300:33b1:f0e1 with SMTP id\n\t38308e7fff4ca-30249d0e67dmr8054261fa.0.1733995145242; Thu, 12 Dec 2024\n\t01:19:05 -0800 (PST)","MIME-Version":"1.0","References":"<20241210142557.2886315-1-chenghaoyang@chromium.org>\n\t<20241210142557.2886315-3-chenghaoyang@chromium.org>\n\t","In-Reply-To":"","From":"Cheng-Hao Yang ","Date":"Thu, 12 Dec 2024 17:18:54 +0800","Message-ID":"","Subject":"Re: [PATCH v4 2/7] android: Set StreamBuffer::srcBuffer in\n\tCameraDevice::processCaptureRequest","To":"Jacopo Mondi ","Cc":"libcamera-devel@lists.libcamera.org, \n\tHan-Lin Chen ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]