From patchwork Tue Aug 2 10:29:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Harvey Yang X-Patchwork-Id: 16904 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 247C2C3275 for ; Tue, 2 Aug 2022 10:30:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DA0C863321; Tue, 2 Aug 2022 12:30:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436200; bh=4WzB0CSclEuEB8tkJJRrAqgZQS3c9xGlo2hHnYBTO4A=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ysBkxhOjat14cdjY7tr9l1sbBTs6rYAGg2Rf8MT36wkGbFeAb/rnVPuTi6Mwfj9cN fx5haCsLS/eZbs5LacmMAhxy18UiWC/UNvEs8XvMcw727ZXYgs0AXPRGscsg9xTJvQ wwqgULQ0dmSmodL7LXpY4AWAve8Bt+JJE65ChJOjkUamrMF2sOxgP7oIZjRUSxgHl8 I7k48e0t+PIr9Kdr4B5CHb7LvwhHmOkcIvhq9LtMB/6azwtXrNHfARJCaNvzwv5x1u 2R6SY+UGjxf2MlgMZKOiJ96Kmkt0s9ZJL72tGjcGgq8Qvb5cps0MdPYM8ArUsynNBY V/zmwQM2D/PGQ== Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D9E5A63321 for ; Tue, 2 Aug 2022 12:29:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="CoR9JRKn"; dkim-atps=neutral Received: by mail-pf1-x429.google.com with SMTP id 17so13214445pfy.0 for ; Tue, 02 Aug 2022 03:29:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lwZSNK4m47bsqn6MMOTV9F42bJ3SBDlJaP+3cftDLIM=; b=CoR9JRKntpL6KRKpMUBN/lwcO/vDC9Rs+wLqxhpmnghLM/wTJzC5KSic1Ec67eQv1J xeFZE+wXGHTZZ00kA1Hl8d0JPD4U0koAHqiHyGvWAxhcReaArgwPo/72aSRvqhLzk6+O G1v4oY/JIgi5DfCheSXc8nG8UQEb7a1Ug105k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lwZSNK4m47bsqn6MMOTV9F42bJ3SBDlJaP+3cftDLIM=; b=0C4dKo6mZyVNfe+r2aQhD5nn+nvwnPlGmzNmpRsNKTBOi4J/OB0/GXJIyZn1VIuMEj 1yW4DoLgpf/cDAsbo6iPmeQu4NB573f0hXhUcvUEItT8eywVFQkWfiFA9bxngI7FGsPV qBhMCc+xGxiEvCV6wdgIwCEXsz2SKCanUQS4qWubwfEN4kIvgWZwa+zczTVXKaJO7dDH MnpNihYGe4TiiL14QMdqy2S5qXxe4riDWnYbprqUuDX2nJDOOsBff6YPGpUOjzfMhPzq AL/8/pM9na3ema0f9tyjqCpvHjgZD9Lfhi3q6HjMbo6aAp+ne8A/daYFPIi8NZCpznkA sMPg== X-Gm-Message-State: ACgBeo2QozzHOlu4mlqsKw3/2bffQIBCMJ9qQTZT55U276M4TSkU+JXc xUNYjGmsSbgjd2azm1Whcd+lRPP73FrIYw== X-Google-Smtp-Source: AA6agR4qepdhsmxz4wkLdctLsG6q0C/ICQQBq0qPpB03EA7FC+YkXxHFhsvrM15jWiGEQLcVwdKO0g== X-Received: by 2002:a63:4b1b:0:b0:41c:863:8ccf with SMTP id y27-20020a634b1b000000b0041c08638ccfmr7492413pga.509.1659436196574; Tue, 02 Aug 2022 03:29:56 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.29.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:29:56 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:39 +0000 Message-Id: <20220802102943.3221109-6-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 5/9] ipu3: Update IPAIPU3Interface::fillParamsBuffer with captureBufferId X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Harvey Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang This patch names the ipa interface |IPAIPU3Interface::fillParamsBuffer| as |IPAIPU3Interface::fillParamsBuffers|, and add input |captureBufferId| to fill the param buffer for the StillCapture stream as well. Signed-off-by: Harvey Yang --- include/libcamera/ipa/ipu3.mojom | 2 +- src/ipa/ipu3/ipu3-ipa-design-guide.rst | 6 +++--- src/ipa/ipu3/ipu3.cpp | 25 ++++++++++++++++++++----- src/libcamera/pipeline/ipu3/ipu3.cpp | 3 ++- 4 files changed, 26 insertions(+), 10 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index d1b1c6b8..1f337b1d 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -31,7 +31,7 @@ interface IPAIPU3Interface { unmapBuffers(array ids); [async] queueRequest(uint32 frame, libcamera.ControlList controls); - [async] fillParamsBuffer(uint32 frame, uint32 bufferId); + [async] fillParamsBuffers(uint32 frame, uint32 bufferId, uint32 captureBufferId); [async] processStatsBuffer(uint32 frame, int64 frameTimestamp, uint32 bufferId, libcamera.ControlList sensorControls); }; diff --git a/src/ipa/ipu3/ipu3-ipa-design-guide.rst b/src/ipa/ipu3/ipu3-ipa-design-guide.rst index e724fdda..6e106a03 100644 --- a/src/ipa/ipu3/ipu3-ipa-design-guide.rst +++ b/src/ipa/ipu3/ipu3-ipa-design-guide.rst @@ -25,7 +25,7 @@ from applications, and managing events from the pipeline handler. └─┬───┬───┬──────┬────┬────┬────┬─┴────▼─┬──┘ 1: init() │ │ │ │ ▲ │ ▲ │ ▲ │ ▲ │ 2: configure() │1 │2 │3 │4│ │4│ │4│ │4│ │5 3: mapBuffers(), start() - │ │ │ │ │ │ │ │ │ │ │ │ 4: (▼) queueRequest(), fillParamsBuffer(), processStatsBuffer() + │ │ │ │ │ │ │ │ │ │ │ │ 4: (▼) queueRequest(), fillParamsBuffers(), processStatsBuffer() ▼ ▼ ▼ ▼ │ ▼ │ ▼ │ ▼ │ ▼ (▲) setSensorControls, paramsBufferReady, metadataReady Signals ┌──────────────────┴────┴────┴────┴─────────┐ 5: stop(), unmapBuffers() │ IPU3 IPA │ @@ -102,7 +102,7 @@ to operate when running: - configure() - queueRequest() -- fillParamsBuffer() +- fillParamsBuffers() - processStatsBuffer() The configuration phase allows the pipeline-handler to inform the IPA of @@ -117,7 +117,7 @@ When configured, the IPA is notified by the pipeline handler of the Camera ``start()`` event, after which incoming requests will be queued for processing, requiring a parameter buffer (``ipu3_uapi_params``) to be populated for the ImgU. This is given to the IPA through -``fillParamsBuffer()``, and then passed directly to each algorithm +``fillParamsBuffers()``, and then passed directly to each algorithm through the ``prepare()`` call allowing the ISP configuration to be updated for the needs of each component that the algorithm is responsible for. diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index dd6cfd79..35c87785 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -82,14 +82,14 @@ namespace ipa::ipu3 { * parameter buffer, and adapting the settings of the sensor attached to the * IPU3 CIO2 through sensor-specific V4L2 controls. * - * In fillParamsBuffer(), we populate the ImgU parameter buffer with + * In fillParamsBuffers(), we populate the ImgU parameter buffer with * settings to configure the device in preparation for handling the frame * queued in the Request. * * When the frame has completed processing, the ImgU will generate a statistics * buffer which is given to the IPA with processStatsBuffer(). In this we run the * algorithms to parse the statistics and cache any results for the next - * fillParamsBuffer() call. + * fillParamsBuffers() call. * * The individual algorithms are split into modular components that are called * iteratively to allow them to process statistics from the ImgU in a defined @@ -146,7 +146,8 @@ public: void unmapBuffers(const std::vector &ids) override; void queueRequest(const uint32_t frame, const ControlList &controls) override; - void fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) override; + void fillParamsBuffers(const uint32_t frame, const uint32_t bufferId, + const uint32_t captureBufferId) override; void processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, const uint32_t bufferId, const ControlList &sensorControls) override; @@ -508,12 +509,14 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) /** * \brief Fill and return a buffer with ISP processing parameters for a frame * \param[in] frame The frame number - * \param[in] bufferId ID of the parameter buffer to fill + * \param[in] bufferId ID of the video parameter buffer to fill + * \param[in] captureBufferId ID of the capture parameter buffer to fill * * Algorithms are expected to fill the IPU3 parameter buffer for the next * frame given their most recent processing of the ImgU statistics. */ -void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) +void IPAIPU3::fillParamsBuffers(const uint32_t frame, const uint32_t bufferId, + const uint32_t captureBufferId) { auto it = buffers_.find(bufferId); if (it == buffers_.end()) { @@ -536,6 +539,18 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; + for (auto const &algo : algorithms_) + algo->prepare(context_, params); + + // TODO: use different algorithms to set StillCapture params. + it = buffers_.find(captureBufferId); + if (it == buffers_.end()) { + LOG(IPAIPU3, Error) << "Could not find capture param buffer!"; + return; + } + mem = it->second.planes()[0]; + params = reinterpret_cast(mem.data()); + params->use = {}; for (auto const &algo : algorithms_) algo->prepare(context_, params); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 4efa1019..d0da146c 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1416,7 +1416,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) if (request->findBuffer(&rawStream_)) pipe()->completeBuffer(request, buffer); - ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); + ipa_->fillParamsBuffers(info->id, info->paramBuffer->cookie(), + info->captureParamBuffer->cookie()); } void IPU3CameraData::paramBufferReady(FrameBuffer *buffer)