From patchwork Wed Jun 29 10:30:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16435 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0CEF2BE173 for ; Wed, 29 Jun 2022 10:30:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9BCEE6564D; Wed, 29 Jun 2022 12:30:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1656498640; bh=ly5Y8KqbL8yfCBnsfqyc9LvXVdMjapsBLtxS538uMHw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=MG1CzcphGpH0x0M61u9hmD67hzWEfTfpzVltUT+kyf6zfOrNu5psUZQY4+5qu74kJ bojF75iA/LlDqS20099yEVRSo9/5WY4k7Q5Vs0AKeojrEw+lfspjOxkDtGeoc1CWua 0g1qdzwTX+wW39N0XN1i6OIAXGZzCQW8XdwFGDeTq3Jif+8cZP7/2B2IcjmsGzSWtI NXd0fgLGUR5me7NmCYuWzAAMOos7XemWSuxi5Zun2z8lvdXh4PbH9klt7N8VX1OHn3 /4XhoVhRIRidzx1noPet0effrjJsVS9LJCsDoebw8zKWinXyXJkYBv4bhpUOkf6aHF HPRnOwzbYJyXg== Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E71365643 for ; Wed, 29 Jun 2022 12:30:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="dWaUQsLs"; dkim-atps=neutral Received: by mail-pl1-x631.google.com with SMTP id q18so13664048pld.13 for ; Wed, 29 Jun 2022 03:30:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DqmmF0G6VmJrd+CHwU+7cTQv3ufL44zF7g8iuDV1HrA=; b=dWaUQsLszzcOWnzt4bRlCFGgnNggIF7mL9rNnjkAdXFREg6hyPxMjaMW5d6+0hu6mp YfgvQ0IXwUdlsrzi2K+BTPnFMG/+ODUC4UbCE92Ec2kiM2Lv7rzHirDLFPoJE6SSmxob U8lxwDmaqpPDEPRIZUZ6X2HPbkKAAY5IBavt8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DqmmF0G6VmJrd+CHwU+7cTQv3ufL44zF7g8iuDV1HrA=; b=rN5UpK7xDfHCWSaedg3UrIBNhBx9o6nSgMtHhBEiSMJI6ik6V68m6I8ypa7wN7Rhlx Be4DetXxiK8yOtINtqPHEV3zHEOty1rM+o8rDs/x0fyzxuFsEeM0XLif9DtVaaKCnF7j CaX7IIWLRVM8oLyS/+Lp3dqgQKCbDjKHBlkXVbCyFk6uWyatLkdkaP752WEr7RG0xy93 nSUCvhn/5Mdl2B4Xh+fVh9rLDBY6EVVmpiQrVTu1yS/mqJPrOcVMN+mGYKsZ6R4SLzw4 pScZpv+MX4+N4EzyqbQFDF/H2bzitkxnc+9fKZ6XR0Im74mRNoOiovVz5KJF8zw6LmAh Fafg== X-Gm-Message-State: AJIora/+EMOHyZhvLDFsH/wOQ9WYpZXihFAWrS/EeoKyuWSkjJoViB6F pFXD7kdGIu3eHZJv0js5dJzBq5hNNP3OqZMH X-Google-Smtp-Source: AGRyM1tmby4XGwy9PJr1H6plJ0ln4JEmuyNRUCGag1nZ9rqeZLbcSk/8VAL1t6Oa2oQo8oexe8/f/A== X-Received: by 2002:a17:902:d50e:b0:16a:13d:30ab with SMTP id b14-20020a170902d50e00b0016a013d30abmr9819182plg.31.1656498633846; Wed, 29 Jun 2022 03:30:33 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (208.158.221.35.bc.googleusercontent.com. [35.221.158.208]) by smtp.gmail.com with ESMTPSA id jb4-20020a170903258400b00169071538a0sm10946059plb.267.2022.06.29.03.30.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Jun 2022 03:30:33 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Wed, 29 Jun 2022 10:30:18 +0000 Message-Id: <20220629103018.4025635-10-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220629103018.4025635-1-chenghaoyang@google.com> References: <20220629103018.4025635-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 9/9] ipu3: Fixes frame delay X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Like the shipped ipu3 HAL, handle the frame delay with one extra input, needed for the first frame and StillCapture's non-sequential frame requests. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 73 +++++++++++++++++++++++----- 1 file changed, 62 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e26c2736..8689cf8b 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -90,6 +90,9 @@ public: ControlInfoMap ipaControls_; + bool firstRequest_ = true; + int lastRequestSequence_ = -1; + private: void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); @@ -565,6 +568,9 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) CIO2Device *cio2 = &data->cio2_; V4L2DeviceFormat outputFormat; + data->firstRequest_ = true; + data->lastRequestSequence_ = -1; + ImgUDevice::PipeConfig imguConfig1 = config->imguConfig1(); useImgu1_ = !imguConfig1.isNull(); @@ -1427,6 +1433,11 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (!info) return; + int yuvCount = firstRequest_ ? 2 : 1; + int stillCount = firstRequest_ || (lastRequestSequence_ + 1) != static_cast(info->request->sequence()) ? 2 : 1; + + firstRequest_ = false; + bool hasYuv = false; bool hasCapture = false; /* Queue all buffers from the request aimed for the ImgU. */ @@ -1436,33 +1447,53 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (stream == &outStream_) { hasYuv = true; - imgu0_->output_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->output_->queueBuffer(outbuffer); + } } else if (stream == &vfStream_) { hasYuv = true; - imgu0_->viewfinder_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->viewfinder_->queueBuffer(outbuffer); + } } else if (stream == &outCaptureStream_) { hasCapture = true; - imgu1_->output_->queueBuffer(outbuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu1_->output_->queueBuffer(outbuffer); + } } } if (hasYuv) { - bufferReturnCounters[info->rawBuffer] += 1; - - imgu0_->param_->queueBuffer(info->paramBuffer); - imgu0_->stat_->queueBuffer(info->statBuffer); - imgu0_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[info->paramBuffer] += 1; + bufferReturnCounters[info->statBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); + } } else { info->paramDequeued = true; info->metadataProcessed = true; } if (hasCapture) { - bufferReturnCounters[info->rawBuffer] += 1; + lastRequestSequence_ = info->request->sequence(); - imgu1_->param_->queueBuffer(info->captureParamBuffer); - imgu1_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[info->captureParamBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu1_->param_->queueBuffer(info->captureParamBuffer); + imgu1_->input_->queueBuffer(info->rawBuffer); + } } else { info->captureParamDequeued = true; } @@ -1503,6 +1534,11 @@ void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer) */ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1569,6 +1605,11 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1589,6 +1630,11 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1609,6 +1655,11 @@ void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return;