From patchwork Thu May 26 08:27:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16058 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 68968C3256 for ; Thu, 26 May 2022 08:28:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 242126566B; Thu, 26 May 2022 10:28:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1653553684; bh=SolrwT390owr9IE5xukmzOUdJCgi/vpHgCaNN/WYO90=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=cD5av0mpw7sWKRW9vfWGMfElia/JWwpOFUDXM129gO23Sj0/5ftHHnIs14jt/4cZP ORTOpMo/u9N/MXS8Gfwrj9aPYsG2GKUNR7KWsoqaMzmVZHej7YXW5XTEa/NiC8mX+z x0AjfE3CZKupFkbLbTzVsYmlZKTAgx54HL9PQ75ryRuF4HehxjfOl1fHwJVwpZ6r1w N1Ii3HrqUSsF4NMVZJfrKLUGbV/6sTyStTW+/9OJqFNT42c7HYE2ns8cBbrC+FwxwZ IyTCXQcFhUUlyuTtkABe3BAzNzTJd450Swg8j0mrR9wO60SWxFnlvEwMa8gpEyWAcf gu1GgbAarJsjQ== Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 411D96567C for ; Thu, 26 May 2022 10:28:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="W3ksR6WY"; dkim-atps=neutral Received: by mail-pl1-x636.google.com with SMTP id m1so889832plx.3 for ; Thu, 26 May 2022 01:28:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qPky7dniWyXB6ul10XWOEbklBqauhe/VyHpy2R3o0Kw=; b=W3ksR6WY533IuOODdI7RQMeNI4LGN1fK+mzQyAR2mYz8LzPwEY2EqeILqaaaGVd/g+ T4uDYWj/289JvlxA08OM+kcIJqs0+fYTrNUJrQoAqhLrJ2Th+yiVPMY7DtVBLl/PzoTt dF+1t4WrunU/kEezc9H5Qu5tOJYVAlJD2/G0c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qPky7dniWyXB6ul10XWOEbklBqauhe/VyHpy2R3o0Kw=; b=2nM+wP/svnTERQwTKVbwkBW1X3GwAfNsw/5uPsrgsPiBhqv2ABnZu9SUJ/ATWMSM6o /jHSdqNyDEPODykph1FboMPD8LmwTF6N/DFlWypNMvoH7ICVqCJ7l1gx3kxBh5wmh4tP CgNoMeQLcPtSd1kVdZZnl5JQwI6sWHWGrhH+Ip8duQpz9yr0JLAnAneZUsFR3WYvs4Bi rPZLcLaVXftDjT+vHEvGiWRMIhrx+V56hElxRRHrNGqTwniQiB7rHI/mgCIj0BpVOQTr oWpHpezpdqxerO7XxZpl7a6qzTI8nFqocI+UQJsIHwA8SYU11UqvWzNM2JRe/Us+8WyK s2sA== X-Gm-Message-State: AOAM5304/DkH1DEzqX9CEwFGP9niwEk3hsEpmQRwO42UHD0hxKQBnivC ZSFRxqTgL/EDIhW2f0a54l3VC2UI+JTOw2JN X-Google-Smtp-Source: ABdhPJxrPfwOLoECVWro4eCU5SgiM1xxZSjphvu97nT/5nDLJsVnQBItdrUkdODWehug98BR5qpm8w== X-Received: by 2002:a17:90b:164e:b0:1df:99f2:be51 with SMTP id il14-20020a17090b164e00b001df99f2be51mr1396238pjb.59.1653553679949; Thu, 26 May 2022 01:27:59 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (21.160.199.104.bc.googleusercontent.com. [104.199.160.21]) by smtp.gmail.com with ESMTPSA id m2-20020a170902bb8200b0016191b843e2sm834429pls.235.2022.05.26.01.27.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 May 2022 01:27:59 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Thu, 26 May 2022 08:27:37 +0000 Message-Id: <20220526082737.1081262-10-chenghaoyang@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog In-Reply-To: <20220526082737.1081262-1-chenghaoyang@google.com> References: <20220526082737.1081262-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 9/9] ipu3: Fixes frame delay X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Like the shipped ipu3 HAL, handle the frame delay with one extra input, needed for the first frame and StillCapture's non-sequential frame requests. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 73 +++++++++++++++++++++++----- 1 file changed, 62 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e26c2736..8689cf8b 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -90,6 +90,9 @@ public: ControlInfoMap ipaControls_; + bool firstRequest_ = true; + int lastRequestSequence_ = -1; + private: void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); @@ -565,6 +568,9 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) CIO2Device *cio2 = &data->cio2_; V4L2DeviceFormat outputFormat; + data->firstRequest_ = true; + data->lastRequestSequence_ = -1; + ImgUDevice::PipeConfig imguConfig1 = config->imguConfig1(); useImgu1_ = !imguConfig1.isNull(); @@ -1427,6 +1433,11 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (!info) return; + int yuvCount = firstRequest_ ? 2 : 1; + int stillCount = firstRequest_ || (lastRequestSequence_ + 1) != static_cast(info->request->sequence()) ? 2 : 1; + + firstRequest_ = false; + bool hasYuv = false; bool hasCapture = false; /* Queue all buffers from the request aimed for the ImgU. */ @@ -1436,33 +1447,53 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (stream == &outStream_) { hasYuv = true; - imgu0_->output_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->output_->queueBuffer(outbuffer); + } } else if (stream == &vfStream_) { hasYuv = true; - imgu0_->viewfinder_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->viewfinder_->queueBuffer(outbuffer); + } } else if (stream == &outCaptureStream_) { hasCapture = true; - imgu1_->output_->queueBuffer(outbuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu1_->output_->queueBuffer(outbuffer); + } } } if (hasYuv) { - bufferReturnCounters[info->rawBuffer] += 1; - - imgu0_->param_->queueBuffer(info->paramBuffer); - imgu0_->stat_->queueBuffer(info->statBuffer); - imgu0_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[info->paramBuffer] += 1; + bufferReturnCounters[info->statBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); + } } else { info->paramDequeued = true; info->metadataProcessed = true; } if (hasCapture) { - bufferReturnCounters[info->rawBuffer] += 1; + lastRequestSequence_ = info->request->sequence(); - imgu1_->param_->queueBuffer(info->captureParamBuffer); - imgu1_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[info->captureParamBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu1_->param_->queueBuffer(info->captureParamBuffer); + imgu1_->input_->queueBuffer(info->rawBuffer); + } } else { info->captureParamDequeued = true; } @@ -1503,6 +1534,11 @@ void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer) */ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1569,6 +1605,11 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1589,6 +1630,11 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1609,6 +1655,11 @@ void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return;