{"id":16435,"url":"https://patchwork.libcamera.org/api/patches/16435/?format=json","web_url":"https://patchwork.libcamera.org/patch/16435/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220629103018.4025635-10-chenghaoyang@google.com>","date":"2022-06-29T10:30:18","name":"[libcamera-devel,v3,9/9] ipu3: Fixes frame delay","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"5afbbf58145366fefcaa8c8bdd5bd46aa9db9d39","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/people/117/?format=json","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"delegate":{"id":12,"url":"https://patchwork.libcamera.org/api/users/12/?format=json","username":"uajain","first_name":"Umang","last_name":"Jain","email":"umang.jain@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/patch/16435/mbox/","series":[{"id":3232,"url":"https://patchwork.libcamera.org/api/series/3232/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3232","date":"2022-06-29T10:30:09","name":"Use two imgus in ipu3 pipeline handler","version":3,"mbox":"https://patchwork.libcamera.org/series/3232/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/16435/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/16435/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0CEF2BE173\n\tfor ;\n\tWed, 29 Jun 2022 10:30:41 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9BCEE6564D;\n\tWed, 29 Jun 2022 12:30:40 +0200 (CEST)","from mail-pl1-x631.google.com (mail-pl1-x631.google.com\n\t[IPv6:2607:f8b0:4864:20::631])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E71365643\n\tfor ;\n\tWed, 29 Jun 2022 12:30:35 +0200 (CEST)","by mail-pl1-x631.google.com with SMTP id q18so13664048pld.13\n\tfor ;\n\tWed, 29 Jun 2022 03:30:35 -0700 (PDT)","from chenghaoyang-low.c.googlers.com.com\n\t(208.158.221.35.bc.googleusercontent.com. [35.221.158.208])\n\tby smtp.gmail.com with ESMTPSA id\n\tjb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.32\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 29 Jun 2022 03:30:33 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1656498640;\n\tbh=ly5Y8KqbL8yfCBnsfqyc9LvXVdMjapsBLtxS538uMHw=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=MG1CzcphGpH0x0M61u9hmD67hzWEfTfpzVltUT+kyf6zfOrNu5psUZQY4+5qu74kJ\n\tbojF75iA/LlDqS20099yEVRSo9/5WY4k7Q5Vs0AKeojrEw+lfspjOxkDtGeoc1CWua\n\t0g1qdzwTX+wW39N0XN1i6OIAXGZzCQW8XdwFGDeTq3Jif+8cZP7/2B2IcjmsGzSWtI\n\tNXd0fgLGUR5me7NmCYuWzAAMOos7XemWSuxi5Zun2z8lvdXh4PbH9klt7N8VX1OHn3\n\t/4XhoVhRIRidzx1noPet0effrjJsVS9LJCsDoebw8zKWinXyXJkYBv4bhpUOkf6aHF\n\tHPRnOwzbYJyXg==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=DqmmF0G6VmJrd+CHwU+7cTQv3ufL44zF7g8iuDV1HrA=;\n\tb=dWaUQsLszzcOWnzt4bRlCFGgnNggIF7mL9rNnjkAdXFREg6hyPxMjaMW5d6+0hu6mp\n\tYfgvQ0IXwUdlsrzi2K+BTPnFMG/+ODUC4UbCE92Ec2kiM2Lv7rzHirDLFPoJE6SSmxob\n\tU8lxwDmaqpPDEPRIZUZ6X2HPbkKAAY5IBavt8="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=chromium.org\n\theader.i=@chromium.org header.b=\"dWaUQsLs\"; \n\tdkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=DqmmF0G6VmJrd+CHwU+7cTQv3ufL44zF7g8iuDV1HrA=;\n\tb=rN5UpK7xDfHCWSaedg3UrIBNhBx9o6nSgMtHhBEiSMJI6ik6V68m6I8ypa7wN7Rhlx\n\tBe4DetXxiK8yOtINtqPHEV3zHEOty1rM+o8rDs/x0fyzxuFsEeM0XLif9DtVaaKCnF7j\n\tCaX7IIWLRVM8oLyS/+Lp3dqgQKCbDjKHBlkXVbCyFk6uWyatLkdkaP752WEr7RG0xy93\n\tnSUCvhn/5Mdl2B4Xh+fVh9rLDBY6EVVmpiQrVTu1yS/mqJPrOcVMN+mGYKsZ6R4SLzw4\n\tpScZpv+MX4+N4EzyqbQFDF/H2bzitkxnc+9fKZ6XR0Im74mRNoOiovVz5KJF8zw6LmAh\n\tFafg==","X-Gm-Message-State":"AJIora/+EMOHyZhvLDFsH/wOQ9WYpZXihFAWrS/EeoKyuWSkjJoViB6F\n\tpFXD7kdGIu3eHZJv0js5dJzBq5hNNP3OqZMH","X-Google-Smtp-Source":"AGRyM1tmby4XGwy9PJr1H6plJ0ln4JEmuyNRUCGag1nZ9rqeZLbcSk/8VAL1t6Oa2oQo8oexe8/f/A==","X-Received":"by 2002:a17:902:d50e:b0:16a:13d:30ab with SMTP id\n\tb14-20020a170902d50e00b0016a013d30abmr9819182plg.31.1656498633846; \n\tWed, 29 Jun 2022 03:30:33 -0700 (PDT)","X-Google-Original-From":"Harvey Yang ","To":"libcamera-devel@lists.libcamera.org","Date":"Wed, 29 Jun 2022 10:30:18 +0000","Message-Id":"<20220629103018.4025635-10-chenghaoyang@google.com>","X-Mailer":"git-send-email 2.37.0.rc0.161.g10f37bed90-goog","In-Reply-To":"<20220629103018.4025635-1-chenghaoyang@google.com>","References":"<20220629103018.4025635-1-chenghaoyang@google.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 9/9] ipu3: Fixes frame delay","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Harvey Yang via libcamera-devel ","Reply-To":"Harvey Yang ","Cc":"Harvey Yang ,\n\tHarvey Yang ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Like the shipped ipu3 HAL, handle the frame delay with one extra input,\nneeded for the first frame and StillCapture's non-sequential frame\nrequests.\n\nSigned-off-by: Harvey Yang \n---\n src/libcamera/pipeline/ipu3/ipu3.cpp | 73 +++++++++++++++++++++++-----\n 1 file changed, 62 insertions(+), 11 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\nindex e26c2736..8689cf8b 100644\n--- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n@@ -90,6 +90,9 @@ public:\n \n \tControlInfoMap ipaControls_;\n \n+\tbool firstRequest_ = true;\n+\tint lastRequestSequence_ = -1;\n+\n private:\n \tvoid metadataReady(unsigned int id, const ControlList &metadata);\n \tvoid paramsBufferReady(unsigned int id);\n@@ -565,6 +568,9 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c)\n \tCIO2Device *cio2 = &data->cio2_;\n \tV4L2DeviceFormat outputFormat;\n \n+\tdata->firstRequest_ = true;\n+\tdata->lastRequestSequence_ = -1;\n+\n \tImgUDevice::PipeConfig imguConfig1 = config->imguConfig1();\n \tuseImgu1_ = !imguConfig1.isNull();\n \n@@ -1427,6 +1433,11 @@ void IPU3CameraData::paramsBufferReady(unsigned int id)\n \tif (!info)\n \t\treturn;\n \n+\tint yuvCount = firstRequest_ ? 2 : 1;\n+\tint stillCount = firstRequest_ || (lastRequestSequence_ + 1) != static_cast(info->request->sequence()) ? 2 : 1;\n+\n+\tfirstRequest_ = false;\n+\n \tbool hasYuv = false;\n \tbool hasCapture = false;\n \t/* Queue all buffers from the request aimed for the ImgU. */\n@@ -1436,33 +1447,53 @@ void IPU3CameraData::paramsBufferReady(unsigned int id)\n \n \t\tif (stream == &outStream_) {\n \t\t\thasYuv = true;\n-\t\t\timgu0_->output_->queueBuffer(outbuffer);\n+\n+\t\t\tfor (int i = 0; i < yuvCount; ++i) {\n+\t\t\t\tbufferReturnCounters[outbuffer] += 1;\n+\t\t\t\timgu0_->output_->queueBuffer(outbuffer);\n+\t\t\t}\n \t\t} else if (stream == &vfStream_) {\n \t\t\thasYuv = true;\n-\t\t\timgu0_->viewfinder_->queueBuffer(outbuffer);\n+\n+\t\t\tfor (int i = 0; i < yuvCount; ++i) {\n+\t\t\t\tbufferReturnCounters[outbuffer] += 1;\n+\t\t\t\timgu0_->viewfinder_->queueBuffer(outbuffer);\n+\t\t\t}\n \t\t} else if (stream == &outCaptureStream_) {\n \t\t\thasCapture = true;\n \n-\t\t\timgu1_->output_->queueBuffer(outbuffer);\n+\t\t\tfor (int i = 0; i < stillCount; ++i) {\n+\t\t\t\tbufferReturnCounters[outbuffer] += 1;\n+\t\t\t\timgu1_->output_->queueBuffer(outbuffer);\n+\t\t\t}\n \t\t}\n \t}\n \n \tif (hasYuv) {\n-\t\tbufferReturnCounters[info->rawBuffer] += 1;\n-\n-\t\timgu0_->param_->queueBuffer(info->paramBuffer);\n-\t\timgu0_->stat_->queueBuffer(info->statBuffer);\n-\t\timgu0_->input_->queueBuffer(info->rawBuffer);\n+\t\tfor (int i = 0; i < yuvCount; ++i) {\n+\t\t\tbufferReturnCounters[info->paramBuffer] += 1;\n+\t\t\tbufferReturnCounters[info->statBuffer] += 1;\n+\t\t\tbufferReturnCounters[info->rawBuffer] += 1;\n+\n+\t\t\timgu0_->param_->queueBuffer(info->paramBuffer);\n+\t\t\timgu0_->stat_->queueBuffer(info->statBuffer);\n+\t\t\timgu0_->input_->queueBuffer(info->rawBuffer);\n+\t\t}\n \t} else {\n \t\tinfo->paramDequeued = true;\n \t\tinfo->metadataProcessed = true;\n \t}\n \n \tif (hasCapture) {\n-\t\tbufferReturnCounters[info->rawBuffer] += 1;\n+\t\tlastRequestSequence_ = info->request->sequence();\n \n-\t\timgu1_->param_->queueBuffer(info->captureParamBuffer);\n-\t\timgu1_->input_->queueBuffer(info->rawBuffer);\n+\t\tfor (int i = 0; i < stillCount; ++i) {\n+\t\t\tbufferReturnCounters[info->captureParamBuffer] += 1;\n+\t\t\tbufferReturnCounters[info->rawBuffer] += 1;\n+\n+\t\t\timgu1_->param_->queueBuffer(info->captureParamBuffer);\n+\t\t\timgu1_->input_->queueBuffer(info->rawBuffer);\n+\t\t}\n \t} else {\n \t\tinfo->captureParamDequeued = true;\n \t}\n@@ -1503,6 +1534,11 @@ void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer)\n */\n void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer)\n {\n+\tif (--bufferReturnCounters[buffer] > 0)\n+\t\treturn;\n+\n+\tbufferReturnCounters.erase(buffer);\n+\n \tIPU3Frames::Info *info = frameInfos_.find(buffer);\n \tif (!info)\n \t\treturn;\n@@ -1569,6 +1605,11 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n \n void IPU3CameraData::paramBufferReady(FrameBuffer *buffer)\n {\n+\tif (--bufferReturnCounters[buffer] > 0)\n+\t\treturn;\n+\n+\tbufferReturnCounters.erase(buffer);\n+\n \tIPU3Frames::Info *info = frameInfos_.find(buffer);\n \tif (!info)\n \t\treturn;\n@@ -1589,6 +1630,11 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer)\n \n void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer)\n {\n+\tif (--bufferReturnCounters[buffer] > 0)\n+\t\treturn;\n+\n+\tbufferReturnCounters.erase(buffer);\n+\n \tIPU3Frames::Info *info = frameInfos_.find(buffer);\n \tif (!info)\n \t\treturn;\n@@ -1609,6 +1655,11 @@ void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer)\n \n void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n {\n+\tif (--bufferReturnCounters[buffer] > 0)\n+\t\treturn;\n+\n+\tbufferReturnCounters.erase(buffer);\n+\n \tIPU3Frames::Info *info = frameInfos_.find(buffer);\n \tif (!info)\n \t\treturn;\n","prefixes":["libcamera-devel","v3","9/9"]}