{"id":16058,"url":"https://patchwork.libcamera.org/api/patches/16058/?format=json","web_url":"https://patchwork.libcamera.org/patch/16058/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220526082737.1081262-10-chenghaoyang@google.com>","date":"2022-05-26T08:27:37","name":"[libcamera-devel,v2,9/9] ipu3: Fixes frame delay","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"5afbbf58145366fefcaa8c8bdd5bd46aa9db9d39","submitter":{"id":117,"url":"https://patchwork.libcamera.org/api/people/117/?format=json","name":"Cheng-Hao Yang","email":"chenghaoyang@chromium.org"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/16058/mbox/","series":[{"id":3141,"url":"https://patchwork.libcamera.org/api/series/3141/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3141","date":"2022-05-26T08:27:28","name":"Use two imgus in ipu3 pipeline handler","version":2,"mbox":"https://patchwork.libcamera.org/series/3141/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/16058/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/16058/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 68968C3256\n\tfor ;\n\tThu, 26 May 2022 08:28:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 242126566B;\n\tThu, 26 May 2022 10:28:04 +0200 (CEST)","from mail-pl1-x636.google.com (mail-pl1-x636.google.com\n\t[IPv6:2607:f8b0:4864:20::636])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 411D96567C\n\tfor ;\n\tThu, 26 May 2022 10:28:02 +0200 (CEST)","by mail-pl1-x636.google.com with SMTP id m1so889832plx.3\n\tfor ;\n\tThu, 26 May 2022 01:28:02 -0700 (PDT)","from chenghaoyang-low.c.googlers.com.com\n\t(21.160.199.104.bc.googleusercontent.com. [104.199.160.21])\n\tby smtp.gmail.com with ESMTPSA id\n\tm2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.58\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 26 May 2022 01:27:59 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1653553684;\n\tbh=SolrwT390owr9IE5xukmzOUdJCgi/vpHgCaNN/WYO90=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=cD5av0mpw7sWKRW9vfWGMfElia/JWwpOFUDXM129gO23Sj0/5ftHHnIs14jt/4cZP\n\tORTOpMo/u9N/MXS8Gfwrj9aPYsG2GKUNR7KWsoqaMzmVZHej7YXW5XTEa/NiC8mX+z\n\tx0AjfE3CZKupFkbLbTzVsYmlZKTAgx54HL9PQ75ryRuF4HehxjfOl1fHwJVwpZ6r1w\n\tN1Ii3HrqUSsF4NMVZJfrKLUGbV/6sTyStTW+/9OJqFNT42c7HYE2ns8cBbrC+FwxwZ\n\tIyTCXQcFhUUlyuTtkABe3BAzNzTJd450Swg8j0mrR9wO60SWxFnlvEwMa8gpEyWAcf\n\tgu1GgbAarJsjQ==","v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=qPky7dniWyXB6ul10XWOEbklBqauhe/VyHpy2R3o0Kw=;\n\tb=W3ksR6WY533IuOODdI7RQMeNI4LGN1fK+mzQyAR2mYz8LzPwEY2EqeILqaaaGVd/g+\n\tT4uDYWj/289JvlxA08OM+kcIJqs0+fYTrNUJrQoAqhLrJ2Th+yiVPMY7DtVBLl/PzoTt\n\tdF+1t4WrunU/kEezc9H5Qu5tOJYVAlJD2/G0c="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=chromium.org\n\theader.i=@chromium.org header.b=\"W3ksR6WY\"; \n\tdkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=qPky7dniWyXB6ul10XWOEbklBqauhe/VyHpy2R3o0Kw=;\n\tb=2nM+wP/svnTERQwTKVbwkBW1X3GwAfNsw/5uPsrgsPiBhqv2ABnZu9SUJ/ATWMSM6o\n\t/jHSdqNyDEPODykph1FboMPD8LmwTF6N/DFlWypNMvoH7ICVqCJ7l1gx3kxBh5wmh4tP\n\tCgNoMeQLcPtSd1kVdZZnl5JQwI6sWHWGrhH+Ip8duQpz9yr0JLAnAneZUsFR3WYvs4Bi\n\trPZLcLaVXftDjT+vHEvGiWRMIhrx+V56hElxRRHrNGqTwniQiB7rHI/mgCIj0BpVOQTr\n\toWpHpezpdqxerO7XxZpl7a6qzTI8nFqocI+UQJsIHwA8SYU11UqvWzNM2JRe/Us+8WyK\n\ts2sA==","X-Gm-Message-State":"AOAM5304/DkH1DEzqX9CEwFGP9niwEk3hsEpmQRwO42UHD0hxKQBnivC\n\tZSFRxqTgL/EDIhW2f0a54l3VC2UI+JTOw2JN","X-Google-Smtp-Source":"ABdhPJxrPfwOLoECVWro4eCU5SgiM1xxZSjphvu97nT/5nDLJsVnQBItdrUkdODWehug98BR5qpm8w==","X-Received":"by 2002:a17:90b:164e:b0:1df:99f2:be51 with SMTP id\n\til14-20020a17090b164e00b001df99f2be51mr1396238pjb.59.1653553679949; \n\tThu, 26 May 2022 01:27:59 -0700 (PDT)","X-Google-Original-From":"Harvey Yang ","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 26 May 2022 08:27:37 +0000","Message-Id":"<20220526082737.1081262-10-chenghaoyang@google.com>","X-Mailer":"git-send-email 2.36.1.124.g0e6072fb45-goog","In-Reply-To":"<20220526082737.1081262-1-chenghaoyang@google.com>","References":"<20220526082737.1081262-1-chenghaoyang@google.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v2 9/9] ipu3: Fixes frame delay","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Harvey Yang via libcamera-devel ","Reply-To":"Harvey Yang ","Cc":"Harvey Yang ,\n\tHarvey Yang ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Like the shipped ipu3 HAL, handle the frame delay with one extra input,\nneeded for the first frame and StillCapture's non-sequential frame\nrequests.\n\nSigned-off-by: Harvey Yang \n---\n src/libcamera/pipeline/ipu3/ipu3.cpp | 73 +++++++++++++++++++++++-----\n 1 file changed, 62 insertions(+), 11 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\nindex e26c2736..8689cf8b 100644\n--- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n@@ -90,6 +90,9 @@ public:\n \n \tControlInfoMap ipaControls_;\n \n+\tbool firstRequest_ = true;\n+\tint lastRequestSequence_ = -1;\n+\n private:\n \tvoid metadataReady(unsigned int id, const ControlList &metadata);\n \tvoid paramsBufferReady(unsigned int id);\n@@ -565,6 +568,9 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c)\n \tCIO2Device *cio2 = &data->cio2_;\n \tV4L2DeviceFormat outputFormat;\n \n+\tdata->firstRequest_ = true;\n+\tdata->lastRequestSequence_ = -1;\n+\n \tImgUDevice::PipeConfig imguConfig1 = config->imguConfig1();\n \tuseImgu1_ = !imguConfig1.isNull();\n \n@@ -1427,6 +1433,11 @@ void IPU3CameraData::paramsBufferReady(unsigned int id)\n \tif (!info)\n \t\treturn;\n \n+\tint yuvCount = firstRequest_ ? 2 : 1;\n+\tint stillCount = firstRequest_ || (lastRequestSequence_ + 1) != static_cast(info->request->sequence()) ? 2 : 1;\n+\n+\tfirstRequest_ = false;\n+\n \tbool hasYuv = false;\n \tbool hasCapture = false;\n \t/* Queue all buffers from the request aimed for the ImgU. */\n@@ -1436,33 +1447,53 @@ void IPU3CameraData::paramsBufferReady(unsigned int id)\n \n \t\tif (stream == &outStream_) {\n \t\t\thasYuv = true;\n-\t\t\timgu0_->output_->queueBuffer(outbuffer);\n+\n+\t\t\tfor (int i = 0; i < yuvCount; ++i) {\n+\t\t\t\tbufferReturnCounters[outbuffer] += 1;\n+\t\t\t\timgu0_->output_->queueBuffer(outbuffer);\n+\t\t\t}\n \t\t} else if (stream == &vfStream_) {\n \t\t\thasYuv = true;\n-\t\t\timgu0_->viewfinder_->queueBuffer(outbuffer);\n+\n+\t\t\tfor (int i = 0; i < yuvCount; ++i) {\n+\t\t\t\tbufferReturnCounters[outbuffer] += 1;\n+\t\t\t\timgu0_->viewfinder_->queueBuffer(outbuffer);\n+\t\t\t}\n \t\t} else if (stream == &outCaptureStream_) {\n \t\t\thasCapture = true;\n \n-\t\t\timgu1_->output_->queueBuffer(outbuffer);\n+\t\t\tfor (int i = 0; i < stillCount; ++i) {\n+\t\t\t\tbufferReturnCounters[outbuffer] += 1;\n+\t\t\t\timgu1_->output_->queueBuffer(outbuffer);\n+\t\t\t}\n \t\t}\n \t}\n \n \tif (hasYuv) {\n-\t\tbufferReturnCounters[info->rawBuffer] += 1;\n-\n-\t\timgu0_->param_->queueBuffer(info->paramBuffer);\n-\t\timgu0_->stat_->queueBuffer(info->statBuffer);\n-\t\timgu0_->input_->queueBuffer(info->rawBuffer);\n+\t\tfor (int i = 0; i < yuvCount; ++i) {\n+\t\t\tbufferReturnCounters[info->paramBuffer] += 1;\n+\t\t\tbufferReturnCounters[info->statBuffer] += 1;\n+\t\t\tbufferReturnCounters[info->rawBuffer] += 1;\n+\n+\t\t\timgu0_->param_->queueBuffer(info->paramBuffer);\n+\t\t\timgu0_->stat_->queueBuffer(info->statBuffer);\n+\t\t\timgu0_->input_->queueBuffer(info->rawBuffer);\n+\t\t}\n \t} else {\n \t\tinfo->paramDequeued = true;\n \t\tinfo->metadataProcessed = true;\n \t}\n \n \tif (hasCapture) {\n-\t\tbufferReturnCounters[info->rawBuffer] += 1;\n+\t\tlastRequestSequence_ = info->request->sequence();\n \n-\t\timgu1_->param_->queueBuffer(info->captureParamBuffer);\n-\t\timgu1_->input_->queueBuffer(info->rawBuffer);\n+\t\tfor (int i = 0; i < stillCount; ++i) {\n+\t\t\tbufferReturnCounters[info->captureParamBuffer] += 1;\n+\t\t\tbufferReturnCounters[info->rawBuffer] += 1;\n+\n+\t\t\timgu1_->param_->queueBuffer(info->captureParamBuffer);\n+\t\t\timgu1_->input_->queueBuffer(info->rawBuffer);\n+\t\t}\n \t} else {\n \t\tinfo->captureParamDequeued = true;\n \t}\n@@ -1503,6 +1534,11 @@ void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer)\n */\n void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer)\n {\n+\tif (--bufferReturnCounters[buffer] > 0)\n+\t\treturn;\n+\n+\tbufferReturnCounters.erase(buffer);\n+\n \tIPU3Frames::Info *info = frameInfos_.find(buffer);\n \tif (!info)\n \t\treturn;\n@@ -1569,6 +1605,11 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n \n void IPU3CameraData::paramBufferReady(FrameBuffer *buffer)\n {\n+\tif (--bufferReturnCounters[buffer] > 0)\n+\t\treturn;\n+\n+\tbufferReturnCounters.erase(buffer);\n+\n \tIPU3Frames::Info *info = frameInfos_.find(buffer);\n \tif (!info)\n \t\treturn;\n@@ -1589,6 +1630,11 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer)\n \n void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer)\n {\n+\tif (--bufferReturnCounters[buffer] > 0)\n+\t\treturn;\n+\n+\tbufferReturnCounters.erase(buffer);\n+\n \tIPU3Frames::Info *info = frameInfos_.find(buffer);\n \tif (!info)\n \t\treturn;\n@@ -1609,6 +1655,11 @@ void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer)\n \n void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n {\n+\tif (--bufferReturnCounters[buffer] > 0)\n+\t\treturn;\n+\n+\tbufferReturnCounters.erase(buffer);\n+\n \tIPU3Frames::Info *info = frameInfos_.find(buffer);\n \tif (!info)\n \t\treturn;\n","prefixes":["libcamera-devel","v2","9/9"]}