From patchwork Tue Aug 2 10:29:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16908 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BB405C3275 for ; Tue, 2 Aug 2022 10:30:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 89AFA6332B; Tue, 2 Aug 2022 12:30:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436205; bh=/8WAQCAA70hTG+TU7mjus/Hc4ZjB7tRiVTIREFuJRAk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ZD9Mu1fwi5PfN3E0wRgg4kXTj6NerheL/LhM/YGl9af0mYydahZUTPjptGZFI3OiX xRLu4yiWLlaNtdD8IjXNbbU05NHbJWOLVZTwP8Zamj2Rnwn8ZZxiyIImVTirKapBwH R8OG6FNMSrTBfXwZO3ZSlm6Uk6NXMggZFehm8P4tbyVuxm6A0QFA0uNCnj7HFiKPbV DFAoYW4LWLTG4IULPCZbzaldyeCtRFABlV5O4uqzLOJx2QuJ3el6qK92utLYq8bYzQ b5p+H5dmcWdp0VujM19BgPaE8/m14alM7jZ95aVjqJOjC82fOvj8tZQrumJ4c3NhAS sEXrHiHBANgEQ== Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1571063312 for ; Tue, 2 Aug 2022 12:30:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="TCKgnWxw"; dkim-atps=neutral Received: by mail-pl1-x635.google.com with SMTP id y15so13038540plp.10 for ; Tue, 02 Aug 2022 03:30:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=d19T4fGNv59vqQ2SXElDkwk2dTDjZrY0C4F34OxC9Ls=; b=TCKgnWxwo+kzsSfQdMbXWiW8WsUE3G91hzATrpTT0F/zmU4NpE99spUT7O+4bOKgQo tCmlET/WdSWAQHSWMk9qCvCzyhwjVUChP8lz25eJGTesOKbGdCYebSGphIsbXGMF2shA 9Dps9M9qJz7gJss+SNz2IyzkvpOYMR4ksNf5o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d19T4fGNv59vqQ2SXElDkwk2dTDjZrY0C4F34OxC9Ls=; b=3uA3N/zfI7iUTpyfWP7kUB161GgCNoNVldI1+3lbqAIw0dnFPQpvTZsRfMs0na+wXU QTHB7I/hdBaybK8v67xPD1jJMQipn8BhWFLsSZhZ7ywyS4b4rYSW0JA9OF9rP5nf3el0 24bd4xVfH9moU9CYgfNCX8DqsagFxJL/pQaHwqDsgxjUsqCv4KmgOzcT4hVpgz+D2II9 6u+4o0r8SYZW53YehQLmCTrc0DUISo14L5rUUxQPvfjR8JfRk7JX5HM/2dx9dcvRSadb DA7JtxKly5UYcH+SatWvx0epuJG2pXdJWPySEOiQE86BhCkUMriBjTB2X7XsmrtokO5p ZdPw== X-Gm-Message-State: ACgBeo38qh/1521BkmvTtsDpUEIn74Nx5Aj6CK3/Ebs9R5VscUThNlPW Ugochxe32pDwOhX2z7ITbLNJeTjlIJRAAg== X-Google-Smtp-Source: AA6agR5MaF6zQm4byCfXncKr7q2V1eYPFuj0mwFeJL+plyJOR9mGrrfLLNIze7Q6HOm16XLOVYNZqQ== X-Received: by 2002:a17:90a:5207:b0:1ee:dfa1:afa9 with SMTP id v7-20020a17090a520700b001eedfa1afa9mr23195652pjh.246.1659436203211; Tue, 02 Aug 2022 03:30:03 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.30.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:30:02 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:43 +0000 Message-Id: <20220802102943.3221109-10-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 9/9] ipu3: Fixes frame delay X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Like the shipped ipu3 HAL [1], handle the frame delay with one extra input, needed for the first frame and StillCapture's non-sequential frame requests. [1]: https://source.corp.google.com/chromeos_public/src/platform/camera/hal/intel/ipu3/psl/ipu3/ImguUnit.cpp;l=773 Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 73 +++++++++++++++++++++++----- 1 file changed, 62 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index f8101c6c..36ca47c2 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -90,6 +90,9 @@ public: ControlInfoMap ipaControls_; + bool firstRequest_ = true; + int lastCaptureRequestSeq_ = -1; + private: void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); @@ -884,6 +887,9 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis CIO2Device *cio2 = &data->cio2_; int ret; + data->firstRequest_ = true; + data->lastCaptureRequestSeq_ = -1; + imgu0_.input_->bufferReady.connect(data, &IPU3CameraData::tryReturnBuffer); imgu0_.output_->bufferReady.connect(data, @@ -1422,6 +1428,11 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (!info) return; + int yuvCount = firstRequest_ ? 2 : 1; + int stillCount = firstRequest_ || (lastCaptureRequestSeq_ + 1) != static_cast(info->request->sequence()) ? 2 : 1; + + firstRequest_ = false; + bool hasYuv = false; bool hasCapture = false; /* Queue all buffers from the request aimed for the ImgU. */ @@ -1431,33 +1442,53 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (stream == &outStream_) { hasYuv = true; - imgu0_->output_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->output_->queueBuffer(outbuffer); + } } else if (stream == &vfStream_) { hasYuv = true; - imgu0_->viewfinder_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->viewfinder_->queueBuffer(outbuffer); + } } else if (stream == &outCaptureStream_) { hasCapture = true; - imgu1_->output_->queueBuffer(outbuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu1_->output_->queueBuffer(outbuffer); + } } } if (hasYuv) { - bufferReturnCounters[info->rawBuffer] += 1; - - imgu0_->param_->queueBuffer(info->paramBuffer); - imgu0_->stat_->queueBuffer(info->statBuffer); - imgu0_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[info->paramBuffer] += 1; + bufferReturnCounters[info->statBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); + } } else { info->paramDequeued = true; info->metadataProcessed = true; } if (hasCapture) { - bufferReturnCounters[info->rawBuffer] += 1; + lastCaptureRequestSeq_ = info->request->sequence(); - imgu1_->param_->queueBuffer(info->captureParamBuffer); - imgu1_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[info->captureParamBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu1_->param_->queueBuffer(info->captureParamBuffer); + imgu1_->input_->queueBuffer(info->rawBuffer); + } } else { info->captureParamDequeued = true; } @@ -1498,6 +1529,11 @@ void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer) */ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1564,6 +1600,11 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1584,6 +1625,11 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1604,6 +1650,11 @@ void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return;