From patchwork Tue Aug 2 10:29:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16900 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 85B2AC3275 for ; Tue, 2 Aug 2022 10:29:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5059163317; Tue, 2 Aug 2022 12:29:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436194; bh=tw4t1Vhx/4aMCyf9lekQ1nSSMUPqSV40jp//wFh1paw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=r1C6BsFjmSxynS2cgTJYPb0Bjp6u8aZZbR/AUUms8XpPx+lfyINFnzlY5veHTDvz2 SFL2MtsAgRBmNclOJA6IN+LE7hOteMY9H0r4t86W7QSzPSxKO03Ky31Y66BwXYFv8l 3AkR4xSl3dLeoMg949zlvGn7AGew+N33lVQgF5M/ATd7L0pyepPmNEzlz+4VxbXjYw SDaTyGmhQkEpix43vG3iuiCNbeEoGnlcKKz5y1sfbY1UdBPHkVdynXXUozgBaNziNq f9Q2pIFyOCccftyDGzW8cA5ehDzYzEn8TfI5aauYxbd9ev7I6H8Y2AU/a2JnDDE7Lx De7awZo7Yixjg== Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 36A8463317 for ; Tue, 2 Aug 2022 12:29:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="O+PVqQBQ"; dkim-atps=neutral Received: by mail-pj1-x102f.google.com with SMTP id 15-20020a17090a098f00b001f305b453feso18037577pjo.1 for ; Tue, 02 Aug 2022 03:29:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SDLw6TYV7tOVe7A+8xUir38x5u4H1X4D+n4cs03UeJ0=; b=O+PVqQBQKjX66ndwqEMSBmpbSi3FOWE1B6QexGA7ga9BZ8y85ALZTajAs3GoyzbuJC di4A+uZwS30X4e0bCbjCK7zT3Dk7Y8VlhTLl3cirtEQzD4oGO8VvHUZ4xCCKgNQWcHJU BatfkGg5UeCghfMvWVXx+L/kFl2mho+Zl/gnk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SDLw6TYV7tOVe7A+8xUir38x5u4H1X4D+n4cs03UeJ0=; b=agg26YfOG2zNA4BZ6h7bjHJHrHH5o1DT3iyzaDzWwo8qdhX2Bbpglq61elaOgoGtB1 HEDVgtYLkD8CBzWPlxt8b2AiT4E+rLVqkbAXAQcEvGA7kUukcQ6nJFp6H+lzSsErPR6F CdWdKdDXAMKsxS9tc+tsRGN4PjTHwShr6NqHVB18bWY5x1+fuChjcUgKBLmQF0Q2SrHB AASp3jp12/xqxKQs8jQV6U3Q95cEGH+CJqKlVw3qN/iTrtTzGtn2JPdNdVfp06/wmIj0 aPgJFMWdlpkSeioBlvVl6FsNazVoqd+85vfCnJHTVJXhG8XZsDNtYSfj14LnTGBt7o5h 7XdQ== X-Gm-Message-State: ACgBeo0y4KtyxEksX0v2S4GQJo1oaeoFGmAadLgeDnzScTszKbHL8pmH i+H2AKN8DZh8ZXYND96lj00gyfq9n+wLvg== X-Google-Smtp-Source: AA6agR76rIugt/sPCiwM1TLYgm/TX249QnZdR+nElsmPWJReK/xAiUmVQhvnU++l/lDu9I38tBYB6g== X-Received: by 2002:a17:90b:3887:b0:1f2:cc6a:500d with SMTP id mu7-20020a17090b388700b001f2cc6a500dmr23475846pjb.56.1659436190379; Tue, 02 Aug 2022 03:29:50 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.29.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:29:49 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:35 +0000 Message-Id: <20220802102943.3221109-2-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 1/9] Add StreamRole in StreamConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang To let ipu3 pipeline handler understand the usage of streams, this patch adds StreamRole member variable in StreamConfiguration, which should be set by the (Android) adapter. Signed-off-by: Harvey Yang --- include/libcamera/stream.h | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index f0ae7e62..c9e79226 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -22,6 +22,15 @@ namespace libcamera { class Camera; class Stream; +enum StreamRole { + Raw, + StillCapture, + VideoRecording, + Viewfinder, +}; + +using StreamRoles = std::vector; + class StreamFormats { public: @@ -50,6 +59,8 @@ struct StreamConfiguration { std::optional colorSpace; + StreamRole streamRole; + Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } @@ -61,15 +72,6 @@ private: StreamFormats formats_; }; -enum StreamRole { - Raw, - StillCapture, - VideoRecording, - Viewfinder, -}; - -using StreamRoles = std::vector; - class Stream { public: From patchwork Tue Aug 2 10:29:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16901 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 557FBC3276 for ; Tue, 2 Aug 2022 10:29:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B7C316330E; Tue, 2 Aug 2022 12:29:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436194; bh=K+d5Jo6cNsZjUngIlCIyJUfRNsIoGQk31HD1Z6KeIJ0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=TIiPR2MF6bz+mk1Mt2dBIpyY7i3q4cQjQNtSuLzDAiPWFO69dgxrDlfm+pLpg0570 HwOSlHYFzDsDrUODZ/nZ1HTTYohdTR6/3PzmKRQWbkAaCF6tQUxf8M7Htp+O0bsHJ9 m4ckG6BhV/BKkTWA9Vd0aLcwj3w5XjIzLJZnKUlQ5T+BlZcCeCcJAfm9otgb7G8EJ8 aysq9AdHXInJrWgRmeesPsToQ0CTMliguxDXKljujwlasYNe4wbHWzCy7apmkYVmki QyZLlQ6gyNFCh2d8Q5kY15wIoXt1Wy9NvVlnFGGG+K3GdHmV/qdl930sNknOaSF1kW 8YQhrwDF9ytpw== Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BD22F603E7 for ; Tue, 2 Aug 2022 12:29:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Yu1+A24U"; dkim-atps=neutral Received: by mail-pg1-x52b.google.com with SMTP id bh13so11996132pgb.4 for ; Tue, 02 Aug 2022 03:29:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AEuPodY8SqK4FTOvnbJG9NYQfsr6wvmyDvFxGw5IoUs=; b=Yu1+A24UVJvubefo9indYLi9H/wJrHrRHHetheIwJyb1i58MlOHpujAe1/j2Ik6Yhr 4g0FPYPLjOhfhDpnF3O5iCAaQqykJEbQWOj9bah13rfF58m/5SA8NNJU0wH3V5wNQxZF 2/NQ1UofnWoR1tlPXtrmRrCoQWpF/ztCftF2s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AEuPodY8SqK4FTOvnbJG9NYQfsr6wvmyDvFxGw5IoUs=; b=UInPYUeSBjR1ztttwudzBDi9Oxsy2Aewj0Zh1zG8fmqtOl9VJoIAn7F3znvSft63sl Acg7uAdDxQxsIhOGtvHVoAdOg5p76p+Bdtjn8DILOI1rQUAU4nMO/BGWvn4EVc+xK4iL YdfLihN/eeqe1XXk2Gtyy6QkP3OhQqfFh4awCAwN7WsB2cZItHbv7s+cJCodBGVgLZ08 1sxtYDL/EEo/fFj+0aisjLd5n6eAlZNXg7+TM5G3rcgst81rPM2heTD6huGIH1H9oEAK ahfRV+jzRq1hmImWfM8bkyn+zMUCSThYZFgYbdnBVqI/Q3FxjhMEtJHs5KBuyLa+Q/f/ O7IA== X-Gm-Message-State: AJIora/O5ZJt9rLG5VPMX6Nb5hx48CGdSqzwZ6MDKnOctJrp1bZ1CH6D qoJMr8d45YBhsek/qkZSiDKUApfUoihE+w== X-Google-Smtp-Source: AGRyM1sY0reyi3e5V+naC6d9rQ0vJiNdr5F6VyTLQBNLiiIYfdSLCjL9pQmDWRg4xzEaOU2HL7tjhw== X-Received: by 2002:a63:4847:0:b0:419:c8f4:4f88 with SMTP id x7-20020a634847000000b00419c8f44f88mr16081361pgk.603.1659436191773; Tue, 02 Aug 2022 03:29:51 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.29.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:29:51 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:36 +0000 Message-Id: <20220802102943.3221109-3-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 2/9] ipu3: Allow only one camera being started X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang As we hardly have use cases/applications that need both cameras at the same time, this patch adds a rule that only one camera can be started one time. This also allows the following patches that use both imgus to process frames from one single camera. Signed-off-by: Harvey Yang --- include/libcamera/internal/pipeline_handler.h | 5 ++++ src/libcamera/camera.cpp | 11 ++++++++ src/libcamera/pipeline/ipu3/ipu3.cpp | 18 +++++++++++++ src/libcamera/pipeline_handler.cpp | 26 +++++++++++++++++++ 4 files changed, 60 insertions(+) diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index c3e4c258..8277ceda 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -55,6 +55,9 @@ public: virtual int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) = 0; + virtual bool acquire(Camera *camera); + virtual void release(Camera *camera); + virtual int start(Camera *camera, const ControlList *controls) = 0; void stop(Camera *camera); bool hasPendingRequests(const Camera *camera) const; @@ -76,6 +79,8 @@ protected: CameraManager *manager_; + std::set acquiredCameras_; + private: void mediaDeviceDisconnected(MediaDevice *media); virtual void disconnect(); diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index 713543fd..a31f8769 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -826,6 +826,8 @@ int Camera::exportFrameBuffers(Stream *stream, * \return 0 on success or a negative error code otherwise * \retval -ENODEV The camera has been disconnected from the system * \retval -EBUSY The camera is not free and can't be acquired by the caller + * \retval -EUSERS The maximum number of cameras that can be opened concurrently + * were opened already */ int Camera::acquire() { @@ -845,6 +847,14 @@ int Camera::acquire() return -EBUSY; } + if (!d->pipe_->acquire(this)) { + LOG(Camera, Info) + << "The maximum number of cameras that can be opened" + "concurrently were opened already"; + d->pipe_->unlock(); + return -EUSERS; + } + d->setState(Private::CameraAcquired); return 0; @@ -873,6 +883,7 @@ int Camera::release() if (ret < 0) return ret == -EACCES ? -EBUSY : ret; + d->pipe_->release(this); d->pipe_->unlock(); d->setState(Private::CameraAvailable); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index fd989e61..091a40e1 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -141,6 +141,8 @@ public: int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) override; + bool acquire(Camera *camera) override; + int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; @@ -763,8 +765,24 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) return 0; } +bool PipelineHandlerIPU3::acquire(Camera *camera) { + /* + * Enforce that only a single camera can be used at a time to use both + * ImgUs on the camera, so that StillCapture stream can adopt another + * set of configuration. + */ + if (!acquiredCameras_.empty()) + return false; + + return PipelineHandler::acquire(camera); + +} + int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlList *controls) { + if (acquiredCameras_.empty() || *acquiredCameras_.begin() != camera) + return -EUSERS; + IPU3CameraData *data = cameraData(camera); CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index 7ebd76ad..fff9ee59 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -270,6 +270,32 @@ void PipelineHandler::unlock() * otherwise */ +/** + * \fn PipelineHandler::acquire() + * \brief Check if \a camera can be acquired and supported with the current + * pipeline handler usage. If \a camera has already been acquired (by the same + * or another process), return false. + * \param[in] camera The camera + */ +bool PipelineHandler::acquire(Camera *camera) +{ + if (acquiredCameras_.find(camera) != acquiredCameras_.end()) + return false; + + acquiredCameras_.emplace(camera); + return true; +} + +/** + * \fn PipelineHandler::release() + * \brief Release exclusive access to \a camera + * \param[in] camera The camera + */ +void PipelineHandler::release(Camera *camera) +{ + acquiredCameras_.erase(camera); +} + /** * \fn PipelineHandler::start() * \brief Start capturing from a group of streams From patchwork Tue Aug 2 10:29:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16902 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9F2DFC3275 for ; Tue, 2 Aug 2022 10:29:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 59D95603E7; Tue, 2 Aug 2022 12:29:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436197; bh=8owVkMvTwYV4t1VAR7564R18ozCoPyP6Nwle53sKb3E=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=WG0EISoi6COsJ6uKuBNJMVmuHwGyG4rTxKS/hg5sXlKsCYC3bew6JcQPUkkgOK6uf z+/lM81I4XhgDvR5u6v6DuQbL4q4PJ+a0+NsWsLV7XhyM7lHC33ccFt/gLgMEJO7cV nuZWF5EqGdaFWlFwcAHKrrHqhX6BvsMpi/aQuerMWFfMPbRfvfKIabQGfl2OXv46Y0 zTcJ/xbzo5QIvPjklIXDBG5buXglppdMLdL+Se/gmEIRvY5YlURaIeknDgA3lNI24c 5ils0Wbm1BgxzjYJLj3kEQfqsavqRmwamY4YQHpz9ie6cMtPmJF5ian2PaX/l7uibd bI59jtWwwxO1Q== Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7DB63603E7 for ; Tue, 2 Aug 2022 12:29:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="QuqWowKZ"; dkim-atps=neutral Received: by mail-pj1-x1030.google.com with SMTP id b10so13264813pjq.5 for ; Tue, 02 Aug 2022 03:29:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4CKqAm0iSaW6Vegjv6AqPctRCQxnb1ayR2zOADFpnnM=; b=QuqWowKZX2UXkhUbQC2ftMvLLpqI1L+GQ0Zy66OmFfVX0pJCKvUJHewvf0Fb9g6GKD uqyqTmMCXeaND/DTaDXFxG3N86LuDr7LbfOb1NreXAz5C0gS102Hcu/HQD7pKQ215Njl wXVAJCVUHA4nCb0uP+DURWaMwpgsZF+XmJ8Bo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4CKqAm0iSaW6Vegjv6AqPctRCQxnb1ayR2zOADFpnnM=; b=oEZu+KeQfio6qHq/qB2qMmMXE+ymtfevxvKi1Nw9F/4xXKcU9UosEILyNzPG2G3A77 nwaCnIobmlpT4GaNE4dFWd2d+C5k/eDRwtkmbaJ98wLFTTRzF1pf3hOlbSU4rIN8zS1n 9v9F0rfniNp5lLsfd9x+AyuiEJvEVUIf8p7yqe2Jc2mSueeO1Sni8PJBRIzN4mfzGG1j 61XKgoTltvtz3Rs8Bu60cFi2B4xfXcDpTF9ysMcUXqTQOq8h8yBpdrai+B0W2zA/WKtZ W5Id50/zmt7xCMQ6UTj2Raovv1N6cSMhDPOv/3c4y/dJ0hZQl5r5uy9lu/wW2bg0U6M4 bHqg== X-Gm-Message-State: ACgBeo0lgF4pelWCTAiTIJ5Enud8H9KWLcyeyXm3mNcoUhXIeCROHMsY kK26jtJRBW/5jUULJi6B2cTsNN03/hnp9A== X-Google-Smtp-Source: AA6agR4gkBvVG4Vp6j4Z3BuGJeU6x8P/7W7iKPRSXDu/ZZdCOp3JLMW9JVaK5o+okfZKssK3uaiMUQ== X-Received: by 2002:a17:90b:3711:b0:1f5:179c:ad64 with SMTP id mg17-20020a17090b371100b001f5179cad64mr5400547pjb.11.1659436193295; Tue, 02 Aug 2022 03:29:53 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.29.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:29:52 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:37 +0000 Message-Id: <20220802102943.3221109-4-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 3/9] ipu3: Use imgu0 as default X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang With only one camera being started, we can always use imgu0 to process frames (for video/preview). In the following patches, we'll use imgu1 for still capture if needed. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/imgu.cpp | 13 ++++ src/libcamera/pipeline/ipu3/imgu.h | 2 + src/libcamera/pipeline/ipu3/ipu3.cpp | 96 ++++++++++++++-------------- 3 files changed, 63 insertions(+), 48 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp index e5bbc382..aa72b1ec 100644 --- a/src/libcamera/pipeline/ipu3/imgu.cpp +++ b/src/libcamera/pipeline/ipu3/imgu.cpp @@ -765,4 +765,17 @@ int ImgUDevice::enableLinks(bool enable) return linkSetup(name_, PAD_STAT, statName, 0, enable); } +/** + * \brief Disconnect bufferReady signals from |input_|, |param_|, |output_|, + * |viewfinder_|, and |stat_|. + */ +void ImgUDevice::disconnectSignals() +{ + input_->bufferReady.disconnect(); + param_->bufferReady.disconnect(); + output_->bufferReady.disconnect(); + viewfinder_->bufferReady.disconnect(); + stat_->bufferReady.disconnect(); +} + } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h index 0af4dd8a..aa88e36e 100644 --- a/src/libcamera/pipeline/ipu3/imgu.h +++ b/src/libcamera/pipeline/ipu3/imgu.h @@ -92,6 +92,8 @@ public: int enableLinks(bool enable); + void disconnectSignals(); + std::unique_ptr imgu_; std::unique_ptr input_; std::unique_ptr param_; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 091a40e1..c3e90f22 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -64,7 +64,8 @@ public: void frameStart(uint32_t sequence); CIO2Device cio2_; - ImgUDevice *imgu_; + ImgUDevice *imgu0_; + ImgUDevice *imgu1_; Stream outStream_; Stream vfStream_; @@ -406,7 +407,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() /* Only compute the ImgU configuration if a YUV stream has been requested. */ if (yuvCount) { - pipeConfig_ = data_->imgu_->calculatePipeConfig(&pipe); + pipeConfig_ = data_->imgu0_->calculatePipeConfig(&pipe); if (pipeConfig_.isNull()) { LOG(IPU3, Error) << "Failed to calculate pipe configuration: " << "unsupported resolutions."; @@ -518,16 +519,12 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) Stream *outStream = &data->outStream_; Stream *vfStream = &data->vfStream_; CIO2Device *cio2 = &data->cio2_; - ImgUDevice *imgu = data->imgu_; V4L2DeviceFormat outputFormat; int ret; /* - * FIXME: enabled links in one ImgU pipe interfere with capture - * operations on the other one. This can be easily triggered by - * capturing from one camera and then trying to capture from the other - * one right after, without disabling media links on the first used - * pipe. + * Enabled links in one ImgU pipe interfere with capture + * operations on the other one. * * The tricky part here is where to disable links on the ImgU instance * which is currently not in use: @@ -543,11 +540,11 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * configuring the device, to allow alternate the usage of the two * ImgU pipes. * - * As a consequence, a Camera using an ImgU shall be configured before - * any start()/stop() sequence. An application that wants to - * pre-configure all the camera and then start/stop them alternatively - * without going through any re-configuration (a sequence that is - * allowed by the Camera state machine) would now fail on the IPU3. + * Now that only one camera is allowed to be used at the same time, we + * don't need to handle the complicated interference between the two + * ImgU links. Therefore, an application still cannot pre-configure all + * the cameras and then start/stop them alternatively without going + * throught any re-configuration. */ ret = imguMediaDev_->disableLinks(); if (ret) @@ -560,7 +557,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * stream which is for raw capture, in which case no buffers will * ever be queued to the ImgU. */ - ret = data->imgu_->enableLinks(true); + ret = imgu0_.enableLinks(true); if (ret) return ret; @@ -610,7 +607,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (imguConfig.isNull()) return 0; - ret = imgu->configure(imguConfig, &cio2Format); + ret = imgu0_.configure(imguConfig, &cio2Format); if (ret) return ret; @@ -624,12 +621,12 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (stream == outStream) { mainCfg = &cfg; - ret = imgu->configureOutput(cfg, &outputFormat); + ret = imgu0_.configureOutput(cfg, &outputFormat); if (ret) return ret; } else if (stream == vfStream) { vfCfg = &cfg; - ret = imgu->configureViewfinder(cfg, &outputFormat); + ret = imgu0_.configureViewfinder(cfg, &outputFormat); if (ret) return ret; } @@ -641,13 +638,13 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * be at least one active stream in the configuration request). */ if (!vfCfg) { - ret = imgu->configureViewfinder(*mainCfg, &outputFormat); + ret = imgu0_.configureViewfinder(*mainCfg, &outputFormat); if (ret) return ret; } /* Apply the "pipe_mode" control to the ImgU subdevice. */ - ControlList ctrls(imgu->imgu_->controls()); + ControlList ctrls(imgu0_.imgu_->controls()); /* * Set the ImgU pipe mode to 'Video' unconditionally to have statistics * generated. @@ -657,7 +654,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) */ ctrls.set(V4L2_CID_IPU3_PIPE_MODE, static_cast(IPU3PipeModeVideo)); - ret = imgu->imgu_->setControls(&ctrls); + ret = imgu0_.imgu_->setControls(&ctrls); if (ret) { LOG(IPU3, Error) << "Unable to set pipe_mode control"; return ret; @@ -691,9 +688,9 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, unsigned int count = stream->configuration().bufferCount; if (stream == &data->outStream_) - return data->imgu_->output_->exportBuffers(count, buffers); + return imgu0_.output_->exportBuffers(count, buffers); else if (stream == &data->vfStream_) - return data->imgu_->viewfinder_->exportBuffers(count, buffers); + return imgu0_.viewfinder_->exportBuffers(count, buffers); else if (stream == &data->rawStream_) return data->cio2_.exportBuffers(count, buffers); @@ -711,7 +708,6 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, int PipelineHandlerIPU3::allocateBuffers(Camera *camera) { IPU3CameraData *data = cameraData(camera); - ImgUDevice *imgu = data->imgu_; unsigned int bufferCount; int ret; @@ -721,26 +717,26 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) data->rawStream_.configuration().bufferCount, }); - ret = imgu->allocateBuffers(bufferCount); + ret = imgu0_.allocateBuffers(bufferCount); if (ret < 0) return ret; /* Map buffers to the IPA. */ unsigned int ipaBufferId = 1; - for (const std::unique_ptr &buffer : imgu->paramBuffers_) { + for (const std::unique_ptr &buffer : imgu0_.paramBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } - for (const std::unique_ptr &buffer : imgu->statBuffers_) { + for (const std::unique_ptr &buffer : imgu0_.statBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } data->ipa_->mapBuffers(ipaBuffers_); - data->frameInfos_.init(imgu->paramBuffers_, imgu->statBuffers_); + data->frameInfos_.init(imgu0_.paramBuffers_, imgu0_.statBuffers_); data->frameInfos_.bufferAvailable.connect( data, &IPU3CameraData::queuePendingRequests); @@ -760,7 +756,7 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) data->ipa_->unmapBuffers(ids); ipaBuffers_.clear(); - data->imgu_->freeBuffers(); + imgu0_.freeBuffers(); return 0; } @@ -785,9 +781,18 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis IPU3CameraData *data = cameraData(camera); CIO2Device *cio2 = &data->cio2_; - ImgUDevice *imgu = data->imgu_; int ret; + imgu0_.input_->bufferReady.connect(&data->cio2_, + &CIO2Device::tryReturnBuffer); + imgu0_.output_->bufferReady.connect(data, + &IPU3CameraData::imguOutputBufferReady); + imgu0_.viewfinder_->bufferReady.connect(data, + &IPU3CameraData::imguOutputBufferReady); + imgu0_.param_->bufferReady.connect(data, + &IPU3CameraData::paramBufferReady); + imgu0_.stat_->bufferReady.connect(data, + &IPU3CameraData::statBufferReady); /* Disable test pattern mode on the sensor, if any. */ ret = cio2->sensor()->setTestPatternMode( controls::draft::TestPatternModeEnum::TestPatternModeOff); @@ -813,19 +818,21 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; - ret = imgu->start(); + ret = imgu0_.start(); if (ret) goto error; return 0; error: - imgu->stop(); + imgu0_.stop(); cio2->stop(); data->ipa_->stop(); freeBuffers(camera); LOG(IPU3, Error) << "Failed to start camera " << camera->id(); + imgu0_.disconnectSignals(); + return ret; } @@ -838,12 +845,14 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) data->ipa_->stop(); - ret |= data->imgu_->stop(); + ret |= imgu0_.stop(); ret |= data->cio2_.stop(); if (ret) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); freeBuffers(camera); + + imgu0_.disconnectSignals(); } void IPU3CameraData::cancelPendingRequests() @@ -1186,7 +1195,8 @@ int PipelineHandlerIPU3::registerCameras() * only, and assign imgu0 to the first one and imgu1 to the * second. */ - data->imgu_ = numCameras ? &imgu1_ : &imgu0_; + data->imgu0_ = &imgu0_; + data->imgu1_ = &imgu1_; /* * Connect video devices' 'bufferReady' signals to their @@ -1200,16 +1210,6 @@ int PipelineHandlerIPU3::registerCameras() &IPU3CameraData::cio2BufferReady); data->cio2_.bufferAvailable.connect( data.get(), &IPU3CameraData::queuePendingRequests); - data->imgu_->input_->bufferReady.connect(&data->cio2_, - &CIO2Device::tryReturnBuffer); - data->imgu_->output_->bufferReady.connect(data.get(), - &IPU3CameraData::imguOutputBufferReady); - data->imgu_->viewfinder_->bufferReady.connect(data.get(), - &IPU3CameraData::imguOutputBufferReady); - data->imgu_->param_->bufferReady.connect(data.get(), - &IPU3CameraData::paramBufferReady); - data->imgu_->stat_->bufferReady.connect(data.get(), - &IPU3CameraData::statBufferReady); /* Create and register the Camera instance. */ const std::string &cameraId = cio2->sensor()->id(); @@ -1302,14 +1302,14 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) FrameBuffer *outbuffer = it.second; if (stream == &outStream_) - imgu_->output_->queueBuffer(outbuffer); + imgu0_->output_->queueBuffer(outbuffer); else if (stream == &vfStream_) - imgu_->viewfinder_->queueBuffer(outbuffer); + imgu0_->viewfinder_->queueBuffer(outbuffer); } - imgu_->param_->queueBuffer(info->paramBuffer); - imgu_->stat_->queueBuffer(info->statBuffer); - imgu_->input_->queueBuffer(info->rawBuffer); + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) From patchwork Tue Aug 2 10:29:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16903 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 51D3BC3276 for ; Tue, 2 Aug 2022 10:29:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0F5246331C; Tue, 2 Aug 2022 12:29:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436198; bh=FtFd22lL6VR968rYEokrScVZxJZYgXXbbuNzHEkOY4w=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Pt0kopp1g9hNtfJ+rL5GyvpdNp/sGcaLQG8M7xu3biiQRB9gI95/V5kLLFc8WxMdc E3A6ifywcZn+yJCTl+zOvErbiPZZYcjPJ115f/1Ve5oR3qffcRNsZHhpzFKSD25kyF 7PYGbVXyw1rM3PBZ2QX+2aik+D8KWK1HOwIGIdaNpNEYrFTkQ0w8gtK187o1SbjpeJ K+B+hALupTLDJq/m96Mf2/TN6eRLwhhWFqc+65hvu6u9XrPwWsbTcPyonInBcm+NFg p+HO355tm4k+RpP1RjyHZX5qDtZY871LWct5wWbU06JnDnHvRU6gqDUoh0X/Qmjo8k VI0M5Pujm7BFw== Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DA7F86331B for ; Tue, 2 Aug 2022 12:29:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="TqJKO6sH"; dkim-atps=neutral Received: by mail-pg1-x52f.google.com with SMTP id d7so8949308pgc.13 for ; Tue, 02 Aug 2022 03:29:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Jbu0snkTEI4cpm0JAWs7QyslODoBJyLzd9cT/bBu3p8=; b=TqJKO6sHdAVQnSl5XEzzBzWpHxLfK3iOW1TwtUk8ocGRSQN7Me50Wn+Hzxy37zFlE7 sRkLha/EKt2/kjuGRqQb6QUdOSXA0rlGoYwTB2a7hPnAhSncEYom1OggBW/LBhWGT3jw nokv/LAQOB5GWNkrIM2mma9T1XwdwcFotxisI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Jbu0snkTEI4cpm0JAWs7QyslODoBJyLzd9cT/bBu3p8=; b=aVig1wFGs4Kr8cbaasi9RFC13tVfKYNkY6+cvutRUEtlzSCnWKSt5l9sbt/xwCvW/J B/GvGLluvNDd7fMpY9ardFcUw89X0UdCnT2UUi6tBqcH6MZ6M0OGVVGZUnFgW1IrjvUy b1GfxHg7Zpfoj9XmjXYTikYVvihcyvC6JbxvNL1CUlOTdhCwiyuJYAuKMfCXGgW/BlQf 3LtmDtlq/evRcHiuRvFjZGRzxcAdLlSTRyed8hEv65OqCSXwqDuwC4E2H6f05Ph49XXF 6bZ99+MTuU2VHmEgi54ocAaaSOBzMpJsawVXFM3ZFt3DnVAWc2Qpu6q57gFy4PvxMa/T CS2w== X-Gm-Message-State: AJIora86Jp9QJlnPTq+tH1QJb+uQwV9MpbRw60prNQn60zWIutaBYWMz bkU1a+gABglVOSlCUQXa6hWouQWww4Zadw== X-Google-Smtp-Source: AGRyM1uyXNcUn/uagE6Ic0VubDA6blZ/Qn2VKURU9OQPQmC9KJdJLsUk5EGULNbYFOxzOsd7PaetjA== X-Received: by 2002:a63:1510:0:b0:41b:3901:2c13 with SMTP id v16-20020a631510000000b0041b39012c13mr15835487pgl.341.1659436194957; Tue, 02 Aug 2022 03:29:54 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.29.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:29:54 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:38 +0000 Message-Id: <20220802102943.3221109-5-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 4/9] ipu3: Add StillCapture stream and imgu1 param buffers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang This patch adds the StillCapture stream and ImgU1 param buffers. ImgU1 statistics buffers will be ignored, as they're not needed for 3A. The following patches will enable imgu1 to handle StillCapture stream specifically, when the imgu0 is needed to handle video/preview streams. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/frames.cpp | 23 +++++++++++++++++++++-- src/libcamera/pipeline/ipu3/frames.h | 8 +++++++- src/libcamera/pipeline/ipu3/ipu3.cpp | 20 +++++++++++++++++++- 3 files changed, 47 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/frames.cpp b/src/libcamera/pipeline/ipu3/frames.cpp index a4c3477c..f9b705a5 100644 --- a/src/libcamera/pipeline/ipu3/frames.cpp +++ b/src/libcamera/pipeline/ipu3/frames.cpp @@ -23,7 +23,8 @@ IPU3Frames::IPU3Frames() } void IPU3Frames::init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers) + const std::vector> &statBuffers, + const std::vector> &captureParamBuffers) { for (const std::unique_ptr &buffer : paramBuffers) availableParamBuffers_.push(buffer.get()); @@ -31,6 +32,9 @@ void IPU3Frames::init(const std::vector> ¶mBuff for (const std::unique_ptr &buffer : statBuffers) availableStatBuffers_.push(buffer.get()); + for (const std::unique_ptr &buffer : captureParamBuffers) + availableCaptureParamBuffers_.push(buffer.get()); + frameInfo_.clear(); } @@ -38,6 +42,7 @@ void IPU3Frames::clear() { availableParamBuffers_ = {}; availableStatBuffers_ = {}; + availableCaptureParamBuffers_ = {}; } IPU3Frames::Info *IPU3Frames::create(Request *request) @@ -54,14 +59,22 @@ IPU3Frames::Info *IPU3Frames::create(Request *request) return nullptr; } + if (availableCaptureParamBuffers_.empty()) { + LOG(IPU3, Debug) << "Capture parameters buffer underrun"; + return nullptr; + } + FrameBuffer *paramBuffer = availableParamBuffers_.front(); FrameBuffer *statBuffer = availableStatBuffers_.front(); + FrameBuffer *captureParamBuffer = availableCaptureParamBuffers_.front(); paramBuffer->_d()->setRequest(request); statBuffer->_d()->setRequest(request); + captureParamBuffer->_d()->setRequest(request); availableParamBuffers_.pop(); availableStatBuffers_.pop(); + availableCaptureParamBuffers_.pop(); /* \todo Remove the dynamic allocation of Info */ std::unique_ptr info = std::make_unique(); @@ -71,7 +84,9 @@ IPU3Frames::Info *IPU3Frames::create(Request *request) info->rawBuffer = nullptr; info->paramBuffer = paramBuffer; info->statBuffer = statBuffer; + info->captureParamBuffer = captureParamBuffer; info->paramDequeued = false; + info->captureParamDequeued = false; info->metadataProcessed = false; frameInfo_[id] = std::move(info); @@ -84,6 +99,7 @@ void IPU3Frames::remove(IPU3Frames::Info *info) /* Return params and stat buffer for reuse. */ availableParamBuffers_.push(info->paramBuffer); availableStatBuffers_.push(info->statBuffer); + availableCaptureParamBuffers_.push(info->captureParamBuffer); /* Delete the extended frame information. */ frameInfo_.erase(info->id); @@ -102,6 +118,9 @@ bool IPU3Frames::tryComplete(IPU3Frames::Info *info) if (!info->paramDequeued) return false; + if (!info->captureParamDequeued) + return false; + remove(info); bufferAvailable.emit(); @@ -131,7 +150,7 @@ IPU3Frames::Info *IPU3Frames::find(FrameBuffer *buffer) return info; if (info->rawBuffer == buffer || info->paramBuffer == buffer || - info->statBuffer == buffer) + info->statBuffer == buffer || info->captureParamBuffer == buffer) return info; } diff --git a/src/libcamera/pipeline/ipu3/frames.h b/src/libcamera/pipeline/ipu3/frames.h index 6e3cb915..8fcb8a14 100644 --- a/src/libcamera/pipeline/ipu3/frames.h +++ b/src/libcamera/pipeline/ipu3/frames.h @@ -36,16 +36,20 @@ public: FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + FrameBuffer *captureParamBuffer; + ControlList effectiveSensorControls; bool paramDequeued; + bool captureParamDequeued; bool metadataProcessed; }; IPU3Frames(); void init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers); + const std::vector> &statBuffers, + const std::vector> &captureParamBuffers); void clear(); Info *create(Request *request); @@ -61,6 +65,8 @@ private: std::queue availableParamBuffers_; std::queue availableStatBuffers_; + std::queue availableCaptureParamBuffers_; + std::map> frameInfo_; }; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index c3e90f22..4efa1019 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -70,6 +70,7 @@ public: Stream outStream_; Stream vfStream_; Stream rawStream_; + Stream outCaptureStream_; Rectangle cropRegion_; bool supportsFlips_; @@ -693,6 +694,8 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, return imgu0_.viewfinder_->exportBuffers(count, buffers); else if (stream == &data->rawStream_) return data->cio2_.exportBuffers(count, buffers); + else if (stream == &data->outCaptureStream_) + return imgu1_.output_->exportBuffers(count, buffers); return -EINVAL; } @@ -715,11 +718,17 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) data->outStream_.configuration().bufferCount, data->vfStream_.configuration().bufferCount, data->rawStream_.configuration().bufferCount, + data->outCaptureStream_.configuration().bufferCount, }); ret = imgu0_.allocateBuffers(bufferCount); if (ret < 0) return ret; + ret = imgu1_.allocateBuffers(bufferCount); + if (ret < 0) { + imgu0_.freeBuffers(); + return ret; + } /* Map buffers to the IPA. */ unsigned int ipaBufferId = 1; @@ -734,9 +743,14 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); } + for (const std::unique_ptr &buffer : imgu1_.paramBuffers_) { + buffer->setCookie(ipaBufferId++); + ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); + } + data->ipa_->mapBuffers(ipaBuffers_); - data->frameInfos_.init(imgu0_.paramBuffers_, imgu0_.statBuffers_); + data->frameInfos_.init(imgu0_.paramBuffers_, imgu0_.statBuffers_, imgu1_.paramBuffers_); data->frameInfos_.bufferAvailable.connect( data, &IPU3CameraData::queuePendingRequests); @@ -757,6 +771,7 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) ipaBuffers_.clear(); imgu0_.freeBuffers(); + imgu1_.freeBuffers(); return 0; } @@ -1135,6 +1150,7 @@ int PipelineHandlerIPU3::registerCameras() &data->outStream_, &data->vfStream_, &data->rawStream_, + &data->outCaptureStream_, }; CIO2Device *cio2 = &data->cio2_; @@ -1310,6 +1326,8 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) imgu0_->param_->queueBuffer(info->paramBuffer); imgu0_->stat_->queueBuffer(info->statBuffer); imgu0_->input_->queueBuffer(info->rawBuffer); + + info->captureParamDequeued = true; } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) From patchwork Tue Aug 2 10:29:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16904 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 247C2C3275 for ; Tue, 2 Aug 2022 10:30:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DA0C863321; Tue, 2 Aug 2022 12:30:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436200; bh=4WzB0CSclEuEB8tkJJRrAqgZQS3c9xGlo2hHnYBTO4A=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ysBkxhOjat14cdjY7tr9l1sbBTs6rYAGg2Rf8MT36wkGbFeAb/rnVPuTi6Mwfj9cN fx5haCsLS/eZbs5LacmMAhxy18UiWC/UNvEs8XvMcw727ZXYgs0AXPRGscsg9xTJvQ wwqgULQ0dmSmodL7LXpY4AWAve8Bt+JJE65ChJOjkUamrMF2sOxgP7oIZjRUSxgHl8 I7k48e0t+PIr9Kdr4B5CHb7LvwhHmOkcIvhq9LtMB/6azwtXrNHfARJCaNvzwv5x1u 2R6SY+UGjxf2MlgMZKOiJ96Kmkt0s9ZJL72tGjcGgq8Qvb5cps0MdPYM8ArUsynNBY V/zmwQM2D/PGQ== Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D9E5A63321 for ; Tue, 2 Aug 2022 12:29:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="CoR9JRKn"; dkim-atps=neutral Received: by mail-pf1-x429.google.com with SMTP id 17so13214445pfy.0 for ; Tue, 02 Aug 2022 03:29:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lwZSNK4m47bsqn6MMOTV9F42bJ3SBDlJaP+3cftDLIM=; b=CoR9JRKntpL6KRKpMUBN/lwcO/vDC9Rs+wLqxhpmnghLM/wTJzC5KSic1Ec67eQv1J xeFZE+wXGHTZZ00kA1Hl8d0JPD4U0koAHqiHyGvWAxhcReaArgwPo/72aSRvqhLzk6+O G1v4oY/JIgi5DfCheSXc8nG8UQEb7a1Ug105k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lwZSNK4m47bsqn6MMOTV9F42bJ3SBDlJaP+3cftDLIM=; b=0C4dKo6mZyVNfe+r2aQhD5nn+nvwnPlGmzNmpRsNKTBOi4J/OB0/GXJIyZn1VIuMEj 1yW4DoLgpf/cDAsbo6iPmeQu4NB573f0hXhUcvUEItT8eywVFQkWfiFA9bxngI7FGsPV qBhMCc+xGxiEvCV6wdgIwCEXsz2SKCanUQS4qWubwfEN4kIvgWZwa+zczTVXKaJO7dDH MnpNihYGe4TiiL14QMdqy2S5qXxe4riDWnYbprqUuDX2nJDOOsBff6YPGpUOjzfMhPzq AL/8/pM9na3ema0f9tyjqCpvHjgZD9Lfhi3q6HjMbo6aAp+ne8A/daYFPIi8NZCpznkA sMPg== X-Gm-Message-State: ACgBeo2QozzHOlu4mlqsKw3/2bffQIBCMJ9qQTZT55U276M4TSkU+JXc xUNYjGmsSbgjd2azm1Whcd+lRPP73FrIYw== X-Google-Smtp-Source: AA6agR4qepdhsmxz4wkLdctLsG6q0C/ICQQBq0qPpB03EA7FC+YkXxHFhsvrM15jWiGEQLcVwdKO0g== X-Received: by 2002:a63:4b1b:0:b0:41c:863:8ccf with SMTP id y27-20020a634b1b000000b0041c08638ccfmr7492413pga.509.1659436196574; Tue, 02 Aug 2022 03:29:56 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.29.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:29:56 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:39 +0000 Message-Id: <20220802102943.3221109-6-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 5/9] ipu3: Update IPAIPU3Interface::fillParamsBuffer with captureBufferId X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Harvey Yang This patch names the ipa interface |IPAIPU3Interface::fillParamsBuffer| as |IPAIPU3Interface::fillParamsBuffers|, and add input |captureBufferId| to fill the param buffer for the StillCapture stream as well. Signed-off-by: Harvey Yang --- include/libcamera/ipa/ipu3.mojom | 2 +- src/ipa/ipu3/ipu3-ipa-design-guide.rst | 6 +++--- src/ipa/ipu3/ipu3.cpp | 25 ++++++++++++++++++++----- src/libcamera/pipeline/ipu3/ipu3.cpp | 3 ++- 4 files changed, 26 insertions(+), 10 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index d1b1c6b8..1f337b1d 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -31,7 +31,7 @@ interface IPAIPU3Interface { unmapBuffers(array ids); [async] queueRequest(uint32 frame, libcamera.ControlList controls); - [async] fillParamsBuffer(uint32 frame, uint32 bufferId); + [async] fillParamsBuffers(uint32 frame, uint32 bufferId, uint32 captureBufferId); [async] processStatsBuffer(uint32 frame, int64 frameTimestamp, uint32 bufferId, libcamera.ControlList sensorControls); }; diff --git a/src/ipa/ipu3/ipu3-ipa-design-guide.rst b/src/ipa/ipu3/ipu3-ipa-design-guide.rst index e724fdda..6e106a03 100644 --- a/src/ipa/ipu3/ipu3-ipa-design-guide.rst +++ b/src/ipa/ipu3/ipu3-ipa-design-guide.rst @@ -25,7 +25,7 @@ from applications, and managing events from the pipeline handler. └─┬───┬───┬──────┬────┬────┬────┬─┴────▼─┬──┘ 1: init() │ │ │ │ ▲ │ ▲ │ ▲ │ ▲ │ 2: configure() │1 │2 │3 │4│ │4│ │4│ │4│ │5 3: mapBuffers(), start() - │ │ │ │ │ │ │ │ │ │ │ │ 4: (▼) queueRequest(), fillParamsBuffer(), processStatsBuffer() + │ │ │ │ │ │ │ │ │ │ │ │ 4: (▼) queueRequest(), fillParamsBuffers(), processStatsBuffer() ▼ ▼ ▼ ▼ │ ▼ │ ▼ │ ▼ │ ▼ (▲) setSensorControls, paramsBufferReady, metadataReady Signals ┌──────────────────┴────┴────┴────┴─────────┐ 5: stop(), unmapBuffers() │ IPU3 IPA │ @@ -102,7 +102,7 @@ to operate when running: - configure() - queueRequest() -- fillParamsBuffer() +- fillParamsBuffers() - processStatsBuffer() The configuration phase allows the pipeline-handler to inform the IPA of @@ -117,7 +117,7 @@ When configured, the IPA is notified by the pipeline handler of the Camera ``start()`` event, after which incoming requests will be queued for processing, requiring a parameter buffer (``ipu3_uapi_params``) to be populated for the ImgU. This is given to the IPA through -``fillParamsBuffer()``, and then passed directly to each algorithm +``fillParamsBuffers()``, and then passed directly to each algorithm through the ``prepare()`` call allowing the ISP configuration to be updated for the needs of each component that the algorithm is responsible for. diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index dd6cfd79..35c87785 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -82,14 +82,14 @@ namespace ipa::ipu3 { * parameter buffer, and adapting the settings of the sensor attached to the * IPU3 CIO2 through sensor-specific V4L2 controls. * - * In fillParamsBuffer(), we populate the ImgU parameter buffer with + * In fillParamsBuffers(), we populate the ImgU parameter buffer with * settings to configure the device in preparation for handling the frame * queued in the Request. * * When the frame has completed processing, the ImgU will generate a statistics * buffer which is given to the IPA with processStatsBuffer(). In this we run the * algorithms to parse the statistics and cache any results for the next - * fillParamsBuffer() call. + * fillParamsBuffers() call. * * The individual algorithms are split into modular components that are called * iteratively to allow them to process statistics from the ImgU in a defined @@ -146,7 +146,8 @@ public: void unmapBuffers(const std::vector &ids) override; void queueRequest(const uint32_t frame, const ControlList &controls) override; - void fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) override; + void fillParamsBuffers(const uint32_t frame, const uint32_t bufferId, + const uint32_t captureBufferId) override; void processStatsBuffer(const uint32_t frame, const int64_t frameTimestamp, const uint32_t bufferId, const ControlList &sensorControls) override; @@ -508,12 +509,14 @@ void IPAIPU3::unmapBuffers(const std::vector &ids) /** * \brief Fill and return a buffer with ISP processing parameters for a frame * \param[in] frame The frame number - * \param[in] bufferId ID of the parameter buffer to fill + * \param[in] bufferId ID of the video parameter buffer to fill + * \param[in] captureBufferId ID of the capture parameter buffer to fill * * Algorithms are expected to fill the IPU3 parameter buffer for the next * frame given their most recent processing of the ImgU statistics. */ -void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) +void IPAIPU3::fillParamsBuffers(const uint32_t frame, const uint32_t bufferId, + const uint32_t captureBufferId) { auto it = buffers_.find(bufferId); if (it == buffers_.end()) { @@ -536,6 +539,18 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; + for (auto const &algo : algorithms_) + algo->prepare(context_, params); + + // TODO: use different algorithms to set StillCapture params. + it = buffers_.find(captureBufferId); + if (it == buffers_.end()) { + LOG(IPAIPU3, Error) << "Could not find capture param buffer!"; + return; + } + mem = it->second.planes()[0]; + params = reinterpret_cast(mem.data()); + params->use = {}; for (auto const &algo : algorithms_) algo->prepare(context_, params); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 4efa1019..d0da146c 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1416,7 +1416,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) if (request->findBuffer(&rawStream_)) pipe()->completeBuffer(request, buffer); - ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); + ipa_->fillParamsBuffers(info->id, info->paramBuffer->cookie(), + info->captureParamBuffer->cookie()); } void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) From patchwork Tue Aug 2 10:29:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16905 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 958A0C3275 for ; Tue, 2 Aug 2022 10:30:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4E70E63318; Tue, 2 Aug 2022 12:30:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436202; bh=83xpDlctzj1RUgQpAg+lrTEvEQIsJbIJzIzTvnpO39g=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=grmVLGr/EdgjNDYHgxTQH/H20yUiMm8JjDCRE1tSVyVawUrm8hwVoEZjQZHXPsrGd 2gfSVTJiZW/1wZAJkjuzPuvTvpAPbUdFm+fL6LBnqGv52egBuhJh505hm/xlHcpO5N 7OFs28ovBb/oLy92Ise+Cy+JA1wSHbDcNFwENGN7mMRlurdO5N5xLIqBDiPXDHbRgP g/4+MFKtSq9/epKRcz4tQSdgiuhlqytwGIUDlrv2cK/TzSJ4hkm+85OzYPMwlBWkbS v/TgjYUfa83Oy2T1zGLhY3XdywlAW5vAZOVi1xQk40LCw5BS5zgBBMhPThU6Zhfu6G hbn/dOjJabTaw== Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 50AB363315 for ; Tue, 2 Aug 2022 12:30:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="csitd7FH"; dkim-atps=neutral Received: by mail-pl1-x635.google.com with SMTP id y15so13038363plp.10 for ; Tue, 02 Aug 2022 03:30:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=08iCI83PXScs5BYnm7xF9MtyotxHsmZxVh87Ykl1QyQ=; b=csitd7FHScnN/XK8dkIu6++xgGgMvemwqBrjft2mPup1YqWm7L3ulm8wyV+/dRzpur 2IzyYG6MuXlA4POphXx5r8Wkq2RiwAV9Syty3PcU/WdUUbRCWrPdHsUc4DHSC1OXfDLW zYnK2CY/OGuMgHs8ToOiaNtQeOlUPBj8LdTkY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=08iCI83PXScs5BYnm7xF9MtyotxHsmZxVh87Ykl1QyQ=; b=eV/YceyZsrBoDjedtNdJZC7qN+BU+kA8mGGZDiqbteuou3MnamRMbvdoKYsEISWFRM 52ITwYy+6lNJXMMiBW6loHZyIHGxoBYcIS/KwkWWIHi9Hv2jqiefXYtnCnKnuy/SbacB HwY0uPrCDeRUPcCIYfeo/6fc3/ZUWB6pwOAmPItquRovjYQ3XtMczVLSUCXo9ArJztbE yk0JNDj4izyXgnUrdh8KqDjg2L167jmj4OPjqRTX7dEifW4eG2J/pF8iY6DyVrqrbSUo BmltTIfjA7DMz+KTgGCXwkEMBoArxtKYlrd3JMqzek1t503209Jh5roYRu0M2Mqh/m7m i8Iw== X-Gm-Message-State: ACgBeo2TbHsF0LjGIxewLjo1irp+Ds7LLoHbs50/+hIAWAQo8vFRqnY6 MTe8JRbtl7Kh+NTOGflJR8Ky+qn/VkQKcA== X-Google-Smtp-Source: AA6agR5lFWuoBUS1DqJXZmwdbfFCKttxId/KsCipRkCbbiMITHgLY8quSAUfj5c7SHaXW3MzBGViYw== X-Received: by 2002:a17:90a:8a8d:b0:1f3:155:3324 with SMTP id x13-20020a17090a8a8d00b001f301553324mr23102458pjn.89.1659436198229; Tue, 02 Aug 2022 03:29:58 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.29.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:29:57 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:40 +0000 Message-Id: <20220802102943.3221109-7-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 6/9] ipu3: Configure imgu1 when necessary X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch configure and setup imgu1 when still capture and other streams are enabled. If only still capture is enabled, imgu0 is still being used with IPU3PipeModeVideo (i.e. VideoSnapshot) to ensure 3A is working. Still capture stream will be enabled in the following patches. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 170 +++++++++++++++++++++++---- 1 file changed, 146 insertions(+), 24 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index d0da146c..73f361f8 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -55,9 +55,11 @@ public: int loadIPA(); + void tryReturnBuffer(FrameBuffer *buffer); void imguOutputBufferReady(FrameBuffer *buffer); void cio2BufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); + void captureParamBufferReady(FrameBuffer *buffer); void statBufferReady(FrameBuffer *buffer); void queuePendingRequests(); void cancelPendingRequests(); @@ -93,6 +95,8 @@ private: void paramsBufferReady(unsigned int id); void setSensorControls(unsigned int id, const ControlList &sensorControls, const ControlList &lensControls); + + std::map bufferReturnCounters; }; class IPU3CameraConfiguration : public CameraConfiguration @@ -106,7 +110,8 @@ public: Status validate() override; const StreamConfiguration &cio2Format() const { return cio2Configuration_; } - const ImgUDevice::PipeConfig imguConfig() const { return pipeConfig_; } + const ImgUDevice::PipeConfig imguConfig0() const { return pipeConfig0_; } + const ImgUDevice::PipeConfig imguConfig1() const { return pipeConfig1_; } /* Cache the combinedTransform_ that will be applied to the sensor */ Transform combinedTransform_; @@ -120,7 +125,8 @@ private: const IPU3CameraData *data_; StreamConfiguration cio2Configuration_; - ImgUDevice::PipeConfig pipeConfig_; + ImgUDevice::PipeConfig pipeConfig0_; + ImgUDevice::PipeConfig pipeConfig1_; }; class PipelineHandlerIPU3 : public PipelineHandler @@ -170,6 +176,8 @@ private: MediaDevice *cio2MediaDev_; MediaDevice *imguMediaDev_; + bool useImgu1_ = false; + std::vector ipaBuffers_; }; @@ -408,8 +416,8 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() /* Only compute the ImgU configuration if a YUV stream has been requested. */ if (yuvCount) { - pipeConfig_ = data_->imgu0_->calculatePipeConfig(&pipe); - if (pipeConfig_.isNull()) { + pipeConfig0_ = data_->imgu0_->calculatePipeConfig(&pipe); + if (pipeConfig0_.isNull()) { LOG(IPU3, Error) << "Failed to calculate pipe configuration: " << "unsupported resolutions."; return Invalid; @@ -519,8 +527,13 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) IPU3CameraData *data = cameraData(camera); Stream *outStream = &data->outStream_; Stream *vfStream = &data->vfStream_; + Stream *outCaptureStream = &data->outCaptureStream_; CIO2Device *cio2 = &data->cio2_; V4L2DeviceFormat outputFormat; + + ImgUDevice::PipeConfig imguConfig1 = config->imguConfig1(); + useImgu1_ = !imguConfig1.isNull(); + int ret; /* @@ -562,6 +575,12 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + if (useImgu1_) { + ret = imgu1_.enableLinks(true); + if (ret) + return ret; + } + /* * Pass the requested stream size to the CIO2 unit and get back the * adjusted format to be propagated to the ImgU output devices. @@ -604,14 +623,20 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) * stream has been requested: return here to skip the ImgU configuration * part. */ - ImgUDevice::PipeConfig imguConfig = config->imguConfig(); - if (imguConfig.isNull()) + ImgUDevice::PipeConfig imguConfig0 = config->imguConfig0(); + if (imguConfig0.isNull()) return 0; - ret = imgu0_.configure(imguConfig, &cio2Format); + ret = imgu0_.configure(imguConfig0, &cio2Format); if (ret) return ret; + if (useImgu1_) { + ret = imgu1_.configure(imguConfig1, &cio2Format); + if (ret) + return ret; + } + /* Apply the format to the configured streams output devices. */ StreamConfiguration *mainCfg = nullptr; StreamConfiguration *vfCfg = nullptr; @@ -630,6 +655,15 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) ret = imgu0_.configureViewfinder(cfg, &outputFormat); if (ret) return ret; + } else if (stream == outCaptureStream) { + ASSERT(useImgu1_); + ret = imgu1_.configureOutput(cfg, &outputFormat); + if (ret) + return ret; + + ret = imgu1_.configureViewfinder(cfg, &outputFormat); + if (ret) + return ret; } } @@ -645,22 +679,34 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) } /* Apply the "pipe_mode" control to the ImgU subdevice. */ - ControlList ctrls(imgu0_.imgu_->controls()); + ControlList ctrls0(imgu0_.imgu_->controls()); /* * Set the ImgU pipe mode to 'Video' unconditionally to have statistics * generated. - * - * \todo Figure out what the 'Still Capture' mode is meant for, and use - * it accordingly. */ - ctrls.set(V4L2_CID_IPU3_PIPE_MODE, - static_cast(IPU3PipeModeVideo)); - ret = imgu0_.imgu_->setControls(&ctrls); + ctrls0.set(V4L2_CID_IPU3_PIPE_MODE, + static_cast(IPU3PipeModeVideo)); + ret = imgu0_.imgu_->setControls(&ctrls0); if (ret) { LOG(IPU3, Error) << "Unable to set pipe_mode control"; return ret; } + if (useImgu1_) { + ControlList ctrls1(imgu1_.imgu_->controls()); + /* + * \todo Figure out what the 'Still Capture' mode is meant for, + * and use it accordingly. + */ + ctrls1.set(V4L2_CID_IPU3_PIPE_MODE, + static_cast(IPU3PipeModeStillCapture)); + ret = imgu1_.imgu_->setControls(&ctrls1); + if (ret) { + LOG(IPU3, Error) << "Unable to set pipe_mode control"; + return ret; + } + } + ipa::ipu3::IPAConfigInfo configInfo; configInfo.sensorControls = data->cio2_.sensor()->controls(); @@ -669,8 +715,8 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) configInfo.lensControls = lens->controls(); configInfo.sensorInfo = sensorInfo; - configInfo.bdsOutputSize = config->imguConfig().bds; - configInfo.iif = config->imguConfig().iif; + configInfo.bdsOutputSize = config->imguConfig0().bds; + configInfo.iif = config->imguConfig0().iif; ret = data->ipa_->configure(configInfo, &data->ipaControls_); if (ret) { @@ -798,8 +844,8 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis CIO2Device *cio2 = &data->cio2_; int ret; - imgu0_.input_->bufferReady.connect(&data->cio2_, - &CIO2Device::tryReturnBuffer); + imgu0_.input_->bufferReady.connect(data, + &IPU3CameraData::tryReturnBuffer); imgu0_.output_->bufferReady.connect(data, &IPU3CameraData::imguOutputBufferReady); imgu0_.viewfinder_->bufferReady.connect(data, @@ -808,6 +854,16 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis &IPU3CameraData::paramBufferReady); imgu0_.stat_->bufferReady.connect(data, &IPU3CameraData::statBufferReady); + + if (useImgu1_) { + imgu1_.input_->bufferReady.connect(data, + &IPU3CameraData::tryReturnBuffer); + imgu1_.output_->bufferReady.connect(data, + &IPU3CameraData::imguOutputBufferReady); + imgu1_.param_->bufferReady.connect(data, + &IPU3CameraData::captureParamBufferReady); + } + /* Disable test pattern mode on the sensor, if any. */ ret = cio2->sensor()->setTestPatternMode( controls::draft::TestPatternModeEnum::TestPatternModeOff); @@ -837,16 +893,26 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; + if (useImgu1_) { + ret = imgu1_.start(); + if (ret) + goto error; + } + return 0; error: imgu0_.stop(); + imgu1_.stop(); cio2->stop(); data->ipa_->stop(); freeBuffers(camera); LOG(IPU3, Error) << "Failed to start camera " << camera->id(); imgu0_.disconnectSignals(); + imgu1_.disconnectSignals(); + + useImgu1_ = false; return ret; } @@ -861,6 +927,7 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) data->ipa_->stop(); ret |= imgu0_.stop(); + ret |= imgu1_.stop(); ret |= data->cio2_.stop(); if (ret) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); @@ -868,6 +935,9 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) freeBuffers(camera); imgu0_.disconnectSignals(); + imgu1_.disconnectSignals(); + + useImgu1_ = false; } void IPU3CameraData::cancelPendingRequests() @@ -1312,22 +1382,45 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (!info) return; + bool hasYuv = false; + bool hasCapture = false; /* Queue all buffers from the request aimed for the ImgU. */ for (auto it : info->request->buffers()) { const Stream *stream = it.first; FrameBuffer *outbuffer = it.second; - if (stream == &outStream_) + if (stream == &outStream_) { + hasYuv = true; imgu0_->output_->queueBuffer(outbuffer); - else if (stream == &vfStream_) + } else if (stream == &vfStream_) { + hasYuv = true; imgu0_->viewfinder_->queueBuffer(outbuffer); + } else if (stream == &outCaptureStream_) { + hasCapture = true; + + imgu1_->output_->queueBuffer(outbuffer); + } } - imgu0_->param_->queueBuffer(info->paramBuffer); - imgu0_->stat_->queueBuffer(info->statBuffer); - imgu0_->input_->queueBuffer(info->rawBuffer); + if (hasYuv) { + bufferReturnCounters[info->rawBuffer] += 1; - info->captureParamDequeued = true; + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); + } else { + info->paramDequeued = true; + info->metadataProcessed = true; + } + + if (hasCapture) { + bufferReturnCounters[info->rawBuffer] += 1; + + imgu1_->param_->queueBuffer(info->captureParamBuffer); + imgu1_->input_->queueBuffer(info->rawBuffer); + } else { + info->captureParamDequeued = true; + } } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) @@ -1348,6 +1441,15 @@ void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) * Buffer Ready slots */ +void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer) +{ + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + cio2_.tryReturnBuffer(buffer); +} + /** * \brief Handle buffers completion at the ImgU output * \param[in] buffer The completed buffer @@ -1440,6 +1542,26 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) pipe()->completeRequest(request); } +void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) +{ + IPU3Frames::Info *info = frameInfos_.find(buffer); + if (!info) + return; + + info->captureParamDequeued = true; + + /* + * tryComplete() will delete info if it completes the IPU3Frame. + * In that event, we must have obtained the Request before hand. + * + * \todo Improve the FrameInfo API to avoid this type of issue + */ + Request *request = info->request; + + if (frameInfos_.tryComplete(info)) + pipe()->completeRequest(request); +} + void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { IPU3Frames::Info *info = frameInfos_.find(buffer); From patchwork Tue Aug 2 10:29:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16906 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0E077C3276 for ; Tue, 2 Aug 2022 10:30:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B963D63322; Tue, 2 Aug 2022 12:30:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436202; bh=fftKRZ8Zftn+0drHl4Az7GPNsW4B5X+NpZrUkBSDbxI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=PvvSWWqK+tzi/qpTx/RMS5FXXn5iHWKD52gfjAePxaDDqtCY9F2ehY3hk/IzWx6+u PknWHi73A6c5TdZzEuceoljVB9ZCpEnKB7pdTKm3XmIG/qdJy4zzUFjZeTirN8BERE jqIoNgfb2Tng9IojTT9NZTnteGk3ScN0xheGi41er6aoSvpKBBP/Gnnouj+Jqy8OHe PM6tBDccUTCOaFtRMk+z+EjE08Zot1wGa95P47AOLBA5fhN2yPVwNYWq/o6ZpKaWhT ENPWAbsMiyOgUqFuea/Y0194dxS9pmdyvK3WZjVaqZC9ItXZsdlcfCtfE+ZCMHMbtr 9HNGWGPo3ATDQ== Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AAA9C63322 for ; Tue, 2 Aug 2022 12:30:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="deV0JFxB"; dkim-atps=neutral Received: by mail-pg1-x52b.google.com with SMTP id bh13so11996353pgb.4 for ; Tue, 02 Aug 2022 03:30:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=h4dGYpA4iZvZ1T7cXsIMETguxzq6lHaCODno5/mhFrM=; b=deV0JFxBMPSokv+92/1Ohs2evFpGWohbbh3CmbGR+RbHJLQ/jpHkyrPQhVo1XnIiMR hQbby1y6GaSJ9KHdfpdtxl2YuuhM+OFvI9d/Oi5txvZ3mfzJL5+7mYny2oFVqG3rWwW6 kyLUe/x6rJuWqVyb1IDyX6QUARf8VpGjLaMlY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h4dGYpA4iZvZ1T7cXsIMETguxzq6lHaCODno5/mhFrM=; b=NtHC1E49Th3dRydUHynU6rYfG5HcI5k8d4+uxO3kUX7Tq9i3eWgT5pipd9n9FbIijA amiCEfBH5WnAL9JZz7u3dCElJ4xdIVhoIyNCT31mXIXWULhvmp2vP+DUCbVrl2nVU5rz QUBg4g04DIODKYVVy085K5Uuz4UWqie0ci9igTHvBfh3vdpRihXbHXAllssR1HMUno/G xVqgwpNMuAdPaqVkYhRoTMVxUt/jxtugqMh+hQ19l5Nhe8O7qE8o+ruSJLuUC/cyzxLI lq/Vj7tjhVTQU5fuK11PB+cn+fZ9MZimNQhj+ocl7CzJBNreLsWC38lU+YjhYa/+dDo1 kV5g== X-Gm-Message-State: AJIora9PJw82Jx6+g605kONkXW4kRB16/wYgWFYkJ2bLpSe0DToCeeKI s7ISRP9diypfYBnA3xFz8SCCGDaiORjBMA== X-Google-Smtp-Source: AGRyM1t7oMKaEhBfm9L7TA5ovfeS2V+MLAl9AlMT2LRL21LQmcFN2IfxanKGdVQDMsUyvkg3WinVDg== X-Received: by 2002:a63:211b:0:b0:41b:8f73:576d with SMTP id h27-20020a63211b000000b0041b8f73576dmr16199913pgh.106.1659436199823; Tue, 02 Aug 2022 03:29:59 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.29.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:29:59 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:41 +0000 Message-Id: <20220802102943.3221109-8-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 7/9] ipu3: Assign |outCaptureStream| to StillCapture configuration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When StillCapture and other non-raw configurations are requested, assigns |outCaptureStream| instead of |outStream| to the StillCapture configuration. Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 56 ++++++++++++++++++++++++---- 1 file changed, 48 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 73f361f8..f8101c6c 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -254,8 +254,10 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() * \todo Clarify the IF and BDS margins requirements. */ unsigned int rawCount = 0; - unsigned int yuvCount = 0; + unsigned int videoCount = 0; + unsigned int stillCount = 0; Size maxYuvSize; + Size maxVideoSize; Size rawSize; for (const StreamConfiguration &cfg : config_) { @@ -264,16 +266,34 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { rawCount++; rawSize.expandTo(cfg.size); + } else if (cfg.streamRole == StreamRole::StillCapture) { + if (stillCount != 0) + return Invalid; + + stillCount++; + maxYuvSize.expandTo(cfg.size); } else { - yuvCount++; + videoCount++; maxYuvSize.expandTo(cfg.size); + maxVideoSize.expandTo(cfg.size); } } - if (rawCount > 1 || yuvCount > 2) { + if (videoCount == 0 && stillCount == 1) { + /* + * If only the StillCapture stream is requested, it cannot be + * supported natively, as we need a video stream that enables + * the 3A. Therefore, use video snapshot instead. + */ + stillCount = 0; + videoCount = 1; + maxVideoSize.expandTo(maxYuvSize); + } + + if (rawCount > 1 || videoCount > 2) { LOG(IPU3, Debug) << "Camera configuration not supported"; return Invalid; - } else if (rawCount && !yuvCount) { + } else if (rawCount && !stillCount && !videoCount) { /* * Disallow raw-only camera configuration. Currently, ImgU does * not get configured for raw-only streams and has early return @@ -316,6 +336,9 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() ImgUDevice::Pipe pipe{}; pipe.input = cio2Configuration_.size; + ImgUDevice::Pipe pipe1{}; + pipe1.input = cio2Configuration_.size; + /* * Adjust the configurations if needed and assign streams while * iterating them. @@ -380,18 +403,35 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() cfg->stride = info.stride(cfg->size.width, 0, 1); cfg->frameSize = info.frameSize(cfg->size, 1); + if (cfg->streamRole == StreamRole::StillCapture) { + ASSERT(stillCount == 1); + LOG(IPU3, Debug) << "Assigned " + << cfg->toString() + << " to the imgu1 main output"; + cfg->setStream(const_cast(&data_->outCaptureStream_)); + + pipe1.main = cfg->size; + pipe1.viewfinder = pipe1.main; + + pipeConfig1_ = data_->imgu1_->calculatePipeConfig(&pipe1); + if (pipeConfig1_.isNull()) { + LOG(IPU3, Error) << "Failed to calculate pipe configuration: " + << "unsupported resolutions."; + return Invalid; + } /* * Use the main output stream in case only one stream is * requested or if the current configuration is the one * with the maximum YUV output size. */ - if (mainOutputAvailable && - (originalCfg.size == maxYuvSize || yuvCount == 1)) { + } else if (mainOutputAvailable && + (originalCfg.size == maxVideoSize || + videoCount == 1)) { cfg->setStream(const_cast(&data_->outStream_)); mainOutputAvailable = false; pipe.main = cfg->size; - if (yuvCount == 1) + if (videoCount == 1) pipe.viewfinder = pipe.main; LOG(IPU3, Debug) << "Assigned " << cfg->toString() @@ -415,7 +455,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() } /* Only compute the ImgU configuration if a YUV stream has been requested. */ - if (yuvCount) { + if (videoCount) { pipeConfig0_ = data_->imgu0_->calculatePipeConfig(&pipe); if (pipeConfig0_.isNull()) { LOG(IPU3, Error) << "Failed to calculate pipe configuration: " From patchwork Tue Aug 2 10:29:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16907 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6DA71C3275 for ; Tue, 2 Aug 2022 10:30:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 30E5A6331E; Tue, 2 Aug 2022 12:30:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436204; bh=cVUZrqn5rxXptGZixyEE7N62IrSbeb/6ouOj91M4N0Q=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=qY8FLLl0LCFg4IF8lhpYBF1QcxGpP0Y+K481K/oa8pwI6zXN/aMjQejFCP1eih2WO uGDJ8vMk0I1w9IkdmuESVE8D2zKvl2XVcsoyHdi2DSrzLhD3MWo8IR4fshbPWnwTTp QYmsqBN9aPmEsd7H7IbiDE2ZHORwTrAG3/EZvdVDa5PB7cb6zXqpB2YDN2vA6IUqsv D93kv6yIyV0+PoXqyPkB0sNlKZbsdBzpfPVW5f9IYdi8RCN3j1Ll6jCbAkTS4bV1cD 1ap2dGWMXMnm+FfZ6MPd5uhAm38xg8b1RT4q802dTbb2QdpXP1ZW4QVYTmafQm6e6G D8lsO89W3LmtQ== Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6C50263327 for ; Tue, 2 Aug 2022 12:30:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Mup07Rnk"; dkim-atps=neutral Received: by mail-pf1-x429.google.com with SMTP id 17so13214614pfy.0 for ; Tue, 02 Aug 2022 03:30:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l+ECUe5UPfDYukWy9zAiCTnRz4IAiHnvawL2cEiDqPU=; b=Mup07Rnk9xMNnPzaJaO8dAFvn+91lNXQZRoxdqZGNjKzSjOIwXBw9cRa13lAvXepBV QbOq8H7THm/6vLXai609935V/tl/7RR6t4iPAsYVBWdtDoXGYXA4XQedRNrTwYR+2IXk OuXWtbzkgQKyYSLBEw+WP9W7omKjDAD3AFpoE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l+ECUe5UPfDYukWy9zAiCTnRz4IAiHnvawL2cEiDqPU=; b=KdhRsKPZRbuKFKgRwu+LDfyduJ4EI02py7XLfVvvFW7z6jqhJxufzguZH650eRSgRT j0XOJ1Pe8dQhFHJTFjvRSljBljms97fCdtlfYE3PftprCZ+AlB6t6B11lhsp47t0LkQi Ftjsk9ddj51rTpQW8h3Aunj+pldaM8FgwhhUYmT5pVRLQrwsSbYW/TXhL9UMM0mCzU/q JR44TXoy2HMbKPEjLY14NZt0G8Ev++hrz4q+IvJeBZ21VgRMbqiLpx7i/cBvFVR/dgeA niXzda91F1W/vr+rzXc3+qNEqd9/B/xCUtDVrzWkIDbZSv5FvdDHgnes9wZYNksbsq+k mStQ== X-Gm-Message-State: AJIora91GGua0F7yJd9JVS0d91K5JHVi/G6p+xn1hnHcAZQyuBIDsnId m5TBgJ1rA9d5RG6oywfzoqoyz6rU5fj6qg== X-Google-Smtp-Source: AGRyM1tXdUFcdz+0ZN6ThZPaXkYbFEOKxPiB6bgNl2/A0LkQ3OQQz5wdlCr9ozdIgA16hkTlGhPC+Q== X-Received: by 2002:a63:4913:0:b0:40d:8235:2d1c with SMTP id w19-20020a634913000000b0040d82352d1cmr16259816pga.584.1659436201438; Tue, 02 Aug 2022 03:30:01 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.30.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:30:00 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:42 +0000 Message-Id: <20220802102943.3221109-9-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 8/9] Request StillCapture stream as default in Android adapter X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If the pipeline handler accepts, the Android adapter requests to handle the StillCapture stream separately, instead of being mapped to an existing video/preview stream or creating a new video stream. Signed-off-by: Harvey Yang --- src/android/camera_device.cpp | 235 +++++++++++++++++++--------------- src/android/camera_device.h | 21 +++ 2 files changed, 154 insertions(+), 102 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 00d48471..bbb72615 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -36,23 +36,6 @@ LOG_DECLARE_CATEGORY(HAL) namespace { -/* - * \struct Camera3StreamConfig - * \brief Data to store StreamConfiguration associated with camera3_stream(s) - * \var streams List of the pairs of a stream requested by Android HAL client - * and CameraStream::Type associated with the stream - * \var config StreamConfiguration for streams - */ -struct Camera3StreamConfig { - struct Camera3Stream { - camera3_stream_t *stream; - CameraStream::Type type; - }; - - std::vector streams; - StreamConfiguration config; -}; - /* * Reorder the configurations so that libcamera::Camera can accept them as much * as possible. The sort rule is as follows. @@ -536,16 +519,6 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) return -EINVAL; #endif - /* - * Generate an empty configuration, and construct a StreamConfiguration - * for each camera3_stream to add to it. - */ - std::unique_ptr config = camera_->generateConfiguration(); - if (!config) { - LOG(HAL, Error) << "Failed to generate camera configuration"; - return -EINVAL; - } - /* * Clear and remove any existing configuration from previous calls, and * ensure the required entries are available without further @@ -614,93 +587,102 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) stream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; } + std::unique_ptr config; + /* Now handle the MJPEG streams, adding a new stream if required. */ if (jpegStream) { - CameraStream::Type type; - int index = -1; - - /* Search for a compatible stream in the non-JPEG ones. */ - for (size_t i = 0; i < streamConfigs.size(); ++i) { - Camera3StreamConfig &streamConfig = streamConfigs[i]; - const auto &cfg = streamConfig.config; - - /* - * \todo The PixelFormat must also be compatible with - * the encoder. - */ - if (cfg.size.width != jpegStream->width || - cfg.size.height != jpegStream->height) - continue; - - LOG(HAL, Info) - << "Android JPEG stream mapped to libcamera stream " << i; - - type = CameraStream::Type::Mapped; - index = i; - - /* - * The source stream will be read by software to - * produce the JPEG stream. - */ - camera3_stream_t *stream = streamConfig.streams[0].stream; - stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; - break; - } + /* This stream will be produced by hardware. */ + jpegStream->usage |= GRALLOC_USAGE_HW_CAMERA_WRITE; /* - * Without a compatible match for JPEG encoding we must - * introduce a new stream to satisfy the request requirements. + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. */ - if (index < 0) { + Camera3StreamConfig sConfig; + sConfig.config.streamRole = libcamera::StreamRole::StillCapture; + sConfig.config.size.width = jpegStream->width; + sConfig.config.size.height = jpegStream->height; + sConfig.config.pixelFormat = formats::NV12; + + std::vector streamConfigsCopy(streamConfigs); + streamConfigsCopy.push_back(std::move(sConfig)); + + LOG(HAL, Info) << "Adding " << sConfig.config.toString() + << " for MJPEG support"; + + streamConfigsCopy.back().streams.push_back({ jpegStream, CameraStream::Type::Internal }); + + config = tryValidation(streamConfigsCopy, jpegStream); + if (config) { + streamConfigs = streamConfigsCopy; + } else { + /* The JPEG stream will be produced by software. */ + jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; + + CameraStream::Type type; + int index = -1; + /* Search for a compatible stream in the non-JPEG ones. */ + for (size_t i = 0; i < streamConfigs.size(); ++i) { + Camera3StreamConfig &streamConfig = streamConfigs[i]; + const auto &cfg = streamConfig.config; + + /* + * \todo The PixelFormat must also be compatible with + * the encoder. + */ + if (cfg.size.width != jpegStream->width || + cfg.size.height != jpegStream->height) + continue; + + LOG(HAL, Info) + << "Android JPEG stream mapped to libcamera stream " << i; + index = i; + type = CameraStream::Type::Mapped; + + /* + * The source stream will be read by software to + * produce the JPEG stream. + */ + camera3_stream_t *stream = streamConfig.streams[0].stream; + stream->usage |= GRALLOC_USAGE_SW_READ_OFTEN; + break; + } + /* - * \todo The pixelFormat should be a 'best-fit' choice - * and may require a validation cycle. This is not yet - * handled, and should be considered as part of any - * stream configuration reworks. + * Without a compatible match for JPEG encoding we must + * introduce a new stream to satisfy the request requirements. */ - Camera3StreamConfig streamConfig; - streamConfig.config.size.width = jpegStream->width; - streamConfig.config.size.height = jpegStream->height; - streamConfig.config.pixelFormat = formats::NV12; - streamConfigs.push_back(std::move(streamConfig)); - - LOG(HAL, Info) << "Adding " << streamConfig.config.toString() - << " for MJPEG support"; - - type = CameraStream::Type::Internal; - index = streamConfigs.size() - 1; - } - - /* The JPEG stream will be produced by software. */ - jpegStream->usage |= GRALLOC_USAGE_SW_WRITE_OFTEN; - - streamConfigs[index].streams.push_back({ jpegStream, type }); - } - - sortCamera3StreamConfigs(streamConfigs, jpegStream); - for (const auto &streamConfig : streamConfigs) { - config->addConfiguration(streamConfig.config); + if (index < 0) { + /* + * \todo The pixelFormat should be a 'best-fit' choice + * and may require a validation cycle. This is not yet + * handled, and should be considered as part of any + * stream configuration reworks. + */ + Camera3StreamConfig streamConfig; + streamConfig.config.size.width = jpegStream->width; + streamConfig.config.size.height = jpegStream->height; + streamConfig.config.pixelFormat = formats::NV12; + streamConfigs.push_back(std::move(streamConfig)); + + LOG(HAL, Info) << "Adding " << streamConfig.config.toString() + << " for MJPEG support"; + + type = CameraStream::Type::Internal; + index = streamConfigs.size() - 1; + } - for (auto &stream : streamConfig.streams) { - streams_.emplace_back(this, config.get(), stream.type, - stream.stream, config->size() - 1); - stream.stream->priv = static_cast(&streams_.back()); + streamConfigs[index].streams.push_back({ jpegStream, type }); } } - switch (config->validate()) { - case CameraConfiguration::Valid: - break; - case CameraConfiguration::Adjusted: - LOG(HAL, Info) << "Camera configuration adjusted"; - - for (const StreamConfiguration &cfg : *config) - LOG(HAL, Info) << " - " << cfg.toString(); + if (!config) { + config = tryValidation(streamConfigs, jpegStream); + if (!config) + return -EINVAL; - return -EINVAL; - case CameraConfiguration::Invalid: - LOG(HAL, Info) << "Camera configuration invalid"; - return -EINVAL; } /* @@ -1527,3 +1509,52 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons return resultMetadata; } + +std::unique_ptr CameraDevice::tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream) +{ + /* + * Generate an empty configuration, and construct a StreamConfiguration + * for each camera3_stream to add to it. + */ + std::unique_ptr config = camera_->generateConfiguration(); + if (!config) { + LOG(HAL, Error) << "Failed to generate camera configuration"; + return nullptr; + } + + sortCamera3StreamConfigs(streamConfigs, jpegStream); + for (const auto &streamConfig : streamConfigs) { + config->addConfiguration(streamConfig.config); + + for (auto &stream : streamConfig.streams) { + streams_.emplace_back(this, config.get(), stream.type, + stream.stream, config->size() - 1); + stream.stream->priv = static_cast(&streams_.back()); + } + } + + switch (config->validate()) { + case CameraConfiguration::Valid: + break; + case CameraConfiguration::Adjusted: + LOG(HAL, Info) << "Camera configuration adjusted"; + + for (const StreamConfiguration &cfg : *config) + LOG(HAL, Info) << " - " << cfg.toString(); + + config.reset(); + break; + case CameraConfiguration::Invalid: + LOG(HAL, Info) << "Camera configuration invalid"; + + config.reset(); + break; + } + + if (!config) + streams_.clear(); + + return config; +} diff --git a/src/android/camera_device.h b/src/android/camera_device.h index 64050416..8540e3f5 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -34,6 +34,23 @@ class Camera3RequestDescriptor; struct CameraConfigData; +/* + * \struct Camera3StreamConfig + * \brief Data to store StreamConfiguration associated with camera3_stream(s) + * \var streams List of the pairs of a stream requested by Android HAL client + * and CameraStream::Type associated with the stream + * \var config StreamConfiguration for streams + */ +struct Camera3StreamConfig { + struct Camera3Stream { + camera3_stream_t *stream; + CameraStream::Type type; + }; + + std::vector streams; + libcamera::StreamConfiguration config; +}; + class CameraDevice : protected libcamera::Loggable { public: @@ -101,6 +118,10 @@ private: std::unique_ptr getResultMetadata( const Camera3RequestDescriptor &descriptor) const; + std::unique_ptr tryValidation( + std::vector &streamConfigs, + const camera3_stream_t *jpegStream); + unsigned int id_; camera3_device_t camera3Device_; From patchwork Tue Aug 2 10:29:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Hao Yang X-Patchwork-Id: 16908 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BB405C3275 for ; Tue, 2 Aug 2022 10:30:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 89AFA6332B; Tue, 2 Aug 2022 12:30:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1659436205; bh=/8WAQCAA70hTG+TU7mjus/Hc4ZjB7tRiVTIREFuJRAk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ZD9Mu1fwi5PfN3E0wRgg4kXTj6NerheL/LhM/YGl9af0mYydahZUTPjptGZFI3OiX xRLu4yiWLlaNtdD8IjXNbbU05NHbJWOLVZTwP8Zamj2Rnwn8ZZxiyIImVTirKapBwH R8OG6FNMSrTBfXwZO3ZSlm6Uk6NXMggZFehm8P4tbyVuxm6A0QFA0uNCnj7HFiKPbV DFAoYW4LWLTG4IULPCZbzaldyeCtRFABlV5O4uqzLOJx2QuJ3el6qK92utLYq8bYzQ b5p+H5dmcWdp0VujM19BgPaE8/m14alM7jZ95aVjqJOjC82fOvj8tZQrumJ4c3NhAS sEXrHiHBANgEQ== Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1571063312 for ; Tue, 2 Aug 2022 12:30:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="TCKgnWxw"; dkim-atps=neutral Received: by mail-pl1-x635.google.com with SMTP id y15so13038540plp.10 for ; Tue, 02 Aug 2022 03:30:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=d19T4fGNv59vqQ2SXElDkwk2dTDjZrY0C4F34OxC9Ls=; b=TCKgnWxwo+kzsSfQdMbXWiW8WsUE3G91hzATrpTT0F/zmU4NpE99spUT7O+4bOKgQo tCmlET/WdSWAQHSWMk9qCvCzyhwjVUChP8lz25eJGTesOKbGdCYebSGphIsbXGMF2shA 9Dps9M9qJz7gJss+SNz2IyzkvpOYMR4ksNf5o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d19T4fGNv59vqQ2SXElDkwk2dTDjZrY0C4F34OxC9Ls=; b=3uA3N/zfI7iUTpyfWP7kUB161GgCNoNVldI1+3lbqAIw0dnFPQpvTZsRfMs0na+wXU QTHB7I/hdBaybK8v67xPD1jJMQipn8BhWFLsSZhZ7ywyS4b4rYSW0JA9OF9rP5nf3el0 24bd4xVfH9moU9CYgfNCX8DqsagFxJL/pQaHwqDsgxjUsqCv4KmgOzcT4hVpgz+D2II9 6u+4o0r8SYZW53YehQLmCTrc0DUISo14L5rUUxQPvfjR8JfRk7JX5HM/2dx9dcvRSadb DA7JtxKly5UYcH+SatWvx0epuJG2pXdJWPySEOiQE86BhCkUMriBjTB2X7XsmrtokO5p ZdPw== X-Gm-Message-State: ACgBeo38qh/1521BkmvTtsDpUEIn74Nx5Aj6CK3/Ebs9R5VscUThNlPW Ugochxe32pDwOhX2z7ITbLNJeTjlIJRAAg== X-Google-Smtp-Source: AA6agR5MaF6zQm4byCfXncKr7q2V1eYPFuj0mwFeJL+plyJOR9mGrrfLLNIze7Q6HOm16XLOVYNZqQ== X-Received: by 2002:a17:90a:5207:b0:1ee:dfa1:afa9 with SMTP id v7-20020a17090a520700b001eedfa1afa9mr23195652pjh.246.1659436203211; Tue, 02 Aug 2022 03:30:03 -0700 (PDT) Received: from chenghaoyang-low.c.googlers.com.com (231.137.80.34.bc.googleusercontent.com. [34.80.137.231]) by smtp.gmail.com with ESMTPSA id t15-20020a170902e84f00b0016db88f69a2sm1955128plg.141.2022.08.02.03.30.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 03:30:02 -0700 (PDT) X-Google-Original-From: Harvey Yang To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Aug 2022 10:29:43 +0000 Message-Id: <20220802102943.3221109-10-chenghaoyang@google.com> X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog In-Reply-To: <20220802102943.3221109-1-chenghaoyang@google.com> References: <20220802102943.3221109-1-chenghaoyang@google.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 9/9] ipu3: Fixes frame delay X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Harvey Yang via libcamera-devel From: Cheng-Hao Yang Reply-To: Harvey Yang Cc: Harvey Yang , Harvey Yang Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Like the shipped ipu3 HAL [1], handle the frame delay with one extra input, needed for the first frame and StillCapture's non-sequential frame requests. [1]: https://source.corp.google.com/chromeos_public/src/platform/camera/hal/intel/ipu3/psl/ipu3/ImguUnit.cpp;l=773 Signed-off-by: Harvey Yang --- src/libcamera/pipeline/ipu3/ipu3.cpp | 73 +++++++++++++++++++++++----- 1 file changed, 62 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index f8101c6c..36ca47c2 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -90,6 +90,9 @@ public: ControlInfoMap ipaControls_; + bool firstRequest_ = true; + int lastCaptureRequestSeq_ = -1; + private: void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); @@ -884,6 +887,9 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis CIO2Device *cio2 = &data->cio2_; int ret; + data->firstRequest_ = true; + data->lastCaptureRequestSeq_ = -1; + imgu0_.input_->bufferReady.connect(data, &IPU3CameraData::tryReturnBuffer); imgu0_.output_->bufferReady.connect(data, @@ -1422,6 +1428,11 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (!info) return; + int yuvCount = firstRequest_ ? 2 : 1; + int stillCount = firstRequest_ || (lastCaptureRequestSeq_ + 1) != static_cast(info->request->sequence()) ? 2 : 1; + + firstRequest_ = false; + bool hasYuv = false; bool hasCapture = false; /* Queue all buffers from the request aimed for the ImgU. */ @@ -1431,33 +1442,53 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) if (stream == &outStream_) { hasYuv = true; - imgu0_->output_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->output_->queueBuffer(outbuffer); + } } else if (stream == &vfStream_) { hasYuv = true; - imgu0_->viewfinder_->queueBuffer(outbuffer); + + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu0_->viewfinder_->queueBuffer(outbuffer); + } } else if (stream == &outCaptureStream_) { hasCapture = true; - imgu1_->output_->queueBuffer(outbuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[outbuffer] += 1; + imgu1_->output_->queueBuffer(outbuffer); + } } } if (hasYuv) { - bufferReturnCounters[info->rawBuffer] += 1; - - imgu0_->param_->queueBuffer(info->paramBuffer); - imgu0_->stat_->queueBuffer(info->statBuffer); - imgu0_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < yuvCount; ++i) { + bufferReturnCounters[info->paramBuffer] += 1; + bufferReturnCounters[info->statBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu0_->param_->queueBuffer(info->paramBuffer); + imgu0_->stat_->queueBuffer(info->statBuffer); + imgu0_->input_->queueBuffer(info->rawBuffer); + } } else { info->paramDequeued = true; info->metadataProcessed = true; } if (hasCapture) { - bufferReturnCounters[info->rawBuffer] += 1; + lastCaptureRequestSeq_ = info->request->sequence(); - imgu1_->param_->queueBuffer(info->captureParamBuffer); - imgu1_->input_->queueBuffer(info->rawBuffer); + for (int i = 0; i < stillCount; ++i) { + bufferReturnCounters[info->captureParamBuffer] += 1; + bufferReturnCounters[info->rawBuffer] += 1; + + imgu1_->param_->queueBuffer(info->captureParamBuffer); + imgu1_->input_->queueBuffer(info->rawBuffer); + } } else { info->captureParamDequeued = true; } @@ -1498,6 +1529,11 @@ void IPU3CameraData::tryReturnBuffer(FrameBuffer *buffer) */ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1564,6 +1600,11 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1584,6 +1625,11 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return; @@ -1604,6 +1650,11 @@ void IPU3CameraData::captureParamBufferReady(FrameBuffer *buffer) void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { + if (--bufferReturnCounters[buffer] > 0) + return; + + bufferReturnCounters.erase(buffer); + IPU3Frames::Info *info = frameInfos_.find(buffer); if (!info) return;