From patchwork Wed Feb 21 17:40:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19523 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2F625C0F1B for ; Wed, 21 Feb 2024 17:40:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0A5C262819; Wed, 21 Feb 2024 18:40:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qjE3qpmr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CAE4A62805 for ; Wed, 21 Feb 2024 18:40:27 +0100 (CET) Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AA10E720; Wed, 21 Feb 2024 18:40:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1708537219; bh=C9zQWpF2Pk3tAkuTtTdIwy50nUFri2LcYCqo1c4smko=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qjE3qpmrYZqmSyAaG5zJsOzxLjIrNqlx+B1mHUfuuTB0NbYvwag5iTJLo5wpnviGS BK5ZyRvMKPWR0KVG0QjoqyJTYZiZIyPCosKzaicg4yoM+jBmzUbMLBIoLiMVOEE12P 8BxcnCUlKFsSrX1kulOpcuQswppAODCpYxEDpqLI= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Subject: [PATCH 1/5] libcamera: Allow pipeline to provide a Private request Date: Wed, 21 Feb 2024 18:40:09 +0100 Message-ID: <20240221174015.52958-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> References: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to allow each pipeline handler to create a Request::Private derived class to store ancillary data associated with a capture request, modify the way a Request is created by the PipelineHandler base class. Introduce a virtual PipelineHandler::createRequestDevice() that pipeline handler implementation can optionally override to provide a custom private data type to initialize the Request with. As we can't anymore create a Request without going through an active camera, drop the queueRequest() checks from the Camera statemachine test. Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally Tested-by: Daniel Scally Reviewed-by: Stefan Klug --- include/libcamera/internal/pipeline_handler.h | 5 ++- include/libcamera/request.h | 3 +- src/libcamera/camera.cpp | 8 +--- src/libcamera/pipeline_handler.cpp | 38 ++++++++++++++++--- src/libcamera/request.cpp | 15 +++++--- test/camera/statemachine.cpp | 12 ------ 6 files changed, 50 insertions(+), 31 deletions(-) diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index c96944f4ecc4..bbe74f22e5ae 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -21,6 +21,7 @@ #include #include "libcamera/internal/ipa_proxy.h" +#include "libcamera/internal/request.h" namespace libcamera { @@ -59,7 +60,7 @@ public: void stop(Camera *camera); bool hasPendingRequests(const Camera *camera) const; - void registerRequest(Request *request); + std::unique_ptr createRequest(Camera *camera, uint64_t cookie); void queueRequest(Request *request); bool completeBuffer(Request *request, FrameBuffer *buffer); @@ -74,6 +75,8 @@ protected: void registerCamera(std::shared_ptr camera); void hotplugMediaDevice(MediaDevice *media); + virtual std::unique_ptr createRequestDevice(Camera *camera, + uint64_t cookie); virtual int queueRequestDevice(Camera *camera, Request *request) = 0; virtual void stopDevice(Camera *camera) = 0; diff --git a/include/libcamera/request.h b/include/libcamera/request.h index dffde1536cad..e16e61a93873 100644 --- a/include/libcamera/request.h +++ b/include/libcamera/request.h @@ -45,9 +45,10 @@ public: using BufferMap = std::map; - Request(Camera *camera, uint64_t cookie = 0); + Request(std::unique_ptr d, uint64_t cookie = 0); ~Request(); + static std::unique_ptr create(std::unique_ptr d, uint64_t); void reuse(ReuseFlag flags = Default); ControlList &controls() { return *controls_; } diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index a71dc933b911..b7053576fe42 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -1236,12 +1236,8 @@ std::unique_ptr Camera::createRequest(uint64_t cookie) if (ret < 0) return nullptr; - std::unique_ptr request = std::make_unique(this, cookie); - - /* Associate the request with the pipeline handler. */ - d->pipe_->registerRequest(request.get()); - - return request; + /* Create a Request from the pipeline handler. */ + return d->pipe_->createRequest(this, cookie); } /** diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index 29e0c98a6db5..f2a8cdac0408 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -371,20 +371,25 @@ bool PipelineHandler::hasPendingRequests(const Camera *camera) const } /** - * \fn PipelineHandler::registerRequest() - * \brief Register a request for use by the pipeline handler - * \param[in] request The request to register + * \fn PipelineHandler::createRequest() + * \brief Create a request and register it for use by the pipeline handler + * \param[in] camera The camera the Request belongs to + * \param[in] cookie The Request unique identifier * - * This function is called when the request is created, and allows the pipeline - * handler to perform any one-time initialization it requries for the request. + * This function is called to create a Request by calling createRequestDevice() + * which can be optionally provided by the PipelineHandler derived classes. */ -void PipelineHandler::registerRequest(Request *request) +std::unique_ptr PipelineHandler::createRequest(Camera *camera, uint64_t cookie) { + std::unique_ptr request = createRequestDevice(camera, cookie); + /* * Connect the request prepared signal to notify the pipeline handler * when a request is ready to be processed. */ request->_d()->prepared.connect(this, &PipelineHandler::doQueueRequests); + + return request; } /** @@ -462,6 +467,27 @@ void PipelineHandler::doQueueRequests() } } +/** + * \brief Create a Request from the pipeline handler + * \param[in] camera The camera the Request belongs to + * \param[in] cookie The Request unique identifier + * + * A virtual function that PipelineHandler derived classes are free to override + * in order to initialize a Request with a custom Request::Private derived + * class. + * + * This is the base class implementation that use Request::Private to + * initialize the Request. + * + * \return A unique pointer to a newly created Request + */ +std::unique_ptr +PipelineHandler::createRequestDevice(Camera *camera, uint64_t cookie) +{ + auto d = std::make_unique(camera); + return Request::create(std::move(d), cookie); +} + /** * \fn PipelineHandler::queueRequestDevice() * \brief Queue a request to the device diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index 949c556fa437..d1051ad3d25e 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -336,7 +336,7 @@ void Request::Private::timeout() /** * \brief Create a capture request for a camera - * \param[in] camera The camera that creates the request + * \param[in] d The request private data * \param[in] cookie Opaque cookie for application use * * The \a cookie is stored in the request and is accessible through the @@ -344,12 +344,17 @@ void Request::Private::timeout() * the request to an external resource in the request completion handler, and is * completely opaque to libcamera. */ -Request::Request(Camera *camera, uint64_t cookie) - : Extensible(std::make_unique(camera)), - cookie_(cookie), status_(RequestPending) +std::unique_ptr Request::create(std::unique_ptr d, + uint64_t cookie) +{ + return std::make_unique(std::move(d), cookie); +} + +Request::Request(std::unique_ptr d, uint64_t cookie) + : Extensible(std::move(d)), cookie_(cookie), status_(RequestPending) { controls_ = new ControlList(controls::controls, - camera->_d()->validator()); + _d()->camera()->_d()->validator()); /** * \todo Add a validator for metadata controls. diff --git a/test/camera/statemachine.cpp b/test/camera/statemachine.cpp index 9c2b0c6a7d99..5714061f88b5 100644 --- a/test/camera/statemachine.cpp +++ b/test/camera/statemachine.cpp @@ -38,10 +38,6 @@ protected: if (camera_->start() != -EACCES) return TestFail; - Request request(camera_.get()); - if (camera_->queueRequest(&request) != -EACCES) - return TestFail; - /* Test operations which should pass. */ if (camera_->release()) return TestFail; @@ -68,10 +64,6 @@ protected: if (camera_->start() != -EACCES) return TestFail; - Request request(camera_.get()); - if (camera_->queueRequest(&request) != -EACCES) - return TestFail; - /* Test operations which should pass. */ if (camera_->stop()) return TestFail; @@ -95,10 +87,6 @@ protected: if (camera_->acquire() != -EBUSY) return TestFail; - Request request1(camera_.get()); - if (camera_->queueRequest(&request1) != -EACCES) - return TestFail; - /* Test operations which should pass. */ std::unique_ptr request2 = camera_->createRequest(); if (!request2) From patchwork Wed Feb 21 17:40:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19524 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 55201C3263 for ; Wed, 21 Feb 2024 17:40:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C8D7462816; Wed, 21 Feb 2024 18:40:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="NQVhpOPS"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0CE1561C9D for ; Wed, 21 Feb 2024 18:40:28 +0100 (CET) Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0483F2B3; Wed, 21 Feb 2024 18:40:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1708537220; bh=3z7DvESK+liXlMgHMPObKGRjmwfcjoLZ0B7SC62+AIY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NQVhpOPS/TLzH+8w3MeRSvnYZdPhV4cuEd4zUEGaH9aYEYFezdMiM9yiV0+QIxJ/v u/emnfWF/kNCwH7pXHQvZto7KV9JI9HezVm8CZZpJ92ZrjRpNZ2P1F+gOdW9wERfYi B2lAtZ7+p6HM/qmONw33dRJ2heD3hYEU9JRlx3i4= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Subject: [PATCH 2/5] libcamera: rkisp1: Create a request with a custom private class Date: Wed, 21 Feb 2024 18:40:10 +0100 Message-ID: <20240221174015.52958-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> References: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally Tested-by: Daniel Scally Reviewed-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 586b46d64630..e981e60758f7 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -39,6 +39,7 @@ #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -83,6 +84,15 @@ private: std::map frameInfo_; }; +class RkISP1Request : public Request::Private +{ +public: + RkISP1Request(Camera *camera) + : Request::Private(camera) + { + } +}; + class RkISP1CameraData : public Camera::Private { public: @@ -157,6 +167,8 @@ public: int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; + std::unique_ptr createRequestDevice(Camera *camera, + uint64_t cookie) override; int queueRequestDevice(Camera *camera, Request *request) override; bool match(DeviceEnumerator *enumerator) override; @@ -1020,6 +1032,13 @@ void PipelineHandlerRkISP1::stopDevice(Camera *camera) activeCamera_ = nullptr; } +std::unique_ptr PipelineHandlerRkISP1::createRequestDevice(Camera *camera, + uint64_t cookie) +{ + auto request = std::make_unique(camera); + return Request::create(std::move(request), cookie); +} + int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) { RkISP1CameraData *data = cameraData(camera); From patchwork Wed Feb 21 17:40:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19525 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E9096C0F1B for ; Wed, 21 Feb 2024 17:40:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5EE0F62813; Wed, 21 Feb 2024 18:40:32 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PZC7h+c6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6596E62819 for ; Wed, 21 Feb 2024 18:40:28 +0100 (CET) Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 52560720; Wed, 21 Feb 2024 18:40:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1708537220; bh=U7bG37XpJKu3HPF5MsigoAx1N4BONFkpnY9U0KaM2nc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PZC7h+c625e0y24bIqahWZsahZTYS4+k3A3O6CqZwaYZ1Q1CZ06SGMMx57WH+p9IB DSs1S1xrYAi+OZ2NJPIOVqZCiHz4utwdOkhY0184uqYlRa5qYCm7MM8kOttHLKUy0y qXcHe+5F53NhMo4w2B2DVCI09lWSH2SodQZ7QzXQ= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Subject: [PATCH 3/5] libcamera: rkisp1: Replace usage of RkISP1FrameInfo Date: Wed, 21 Feb 2024 18:40:11 +0100 Message-ID: <20240221174015.52958-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> References: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the pipeline handler can create a private derived class of Request::Private, use it to store the pipeline specific data. In the case of RkISP1 we associate the statistic and paramters buffers with a Request. As the IPA sends notifications for paramters and statistics buffer by identifying them by frame id, associate the frame ids and the Request in the RkISP1CameraData class. This replaces the functionalities of RkISP1FrameInfo which can now be removed. Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally Tested-by: Daniel Scally Reviewed-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 358 ++++++++--------------- 1 file changed, 128 insertions(+), 230 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index e981e60758f7..f2163f528251 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -49,76 +49,63 @@ namespace libcamera { LOG_DEFINE_CATEGORY(RkISP1) -class PipelineHandlerRkISP1; -class RkISP1CameraData; +class RkISP1Request : public Request::Private +{ +public: + RkISP1Request(Camera *camera) + : Request::Private(camera) + { + } -struct RkISP1FrameInfo { - unsigned int frame; - Request *request; + bool hasPendingBuffers(bool isRaw) const; - FrameBuffer *paramBuffer; FrameBuffer *statBuffer; - FrameBuffer *mainPathBuffer; - FrameBuffer *selfPathBuffer; + FrameBuffer *paramBuffer; + + /* The frame number this request is associated with. */ + unsigned int frame; bool paramDequeued; bool metadataProcessed; }; -class RkISP1Frames -{ -public: - RkISP1Frames(PipelineHandler *pipe); - - RkISP1FrameInfo *create(const RkISP1CameraData *data, Request *request, - bool isRaw); - int destroy(unsigned int frame); - void clear(); - - RkISP1FrameInfo *find(unsigned int frame); - RkISP1FrameInfo *find(FrameBuffer *buffer); - RkISP1FrameInfo *find(Request *request); - -private: - PipelineHandlerRkISP1 *pipe_; - std::map frameInfo_; -}; - -class RkISP1Request : public Request::Private +bool RkISP1Request::hasPendingBuffers(bool isRaw) const { -public: - RkISP1Request(Camera *camera) - : Request::Private(camera) - { - } -}; + return Request::Private::hasPendingBuffers() || + !metadataProcessed || (!isRaw && !paramDequeued); +} +class PipelineHandlerRkISP1; class RkISP1CameraData : public Camera::Private { public: RkISP1CameraData(PipelineHandler *pipe, RkISP1MainPath *mainPath, RkISP1SelfPath *selfPath) - : Camera::Private(pipe), frame_(0), frameInfo_(pipe), - mainPath_(mainPath), selfPath_(selfPath) + : Camera::Private(pipe), frame_(0), mainPath_(mainPath), + selfPath_(selfPath) { } PipelineHandlerRkISP1 *pipe(); int loadIPA(unsigned int hwRevision); + void addRequest(RkISP1Request *request); + Stream mainPathStream_; Stream selfPathStream_; std::unique_ptr sensor_; std::unique_ptr delayedCtrls_; unsigned int frame_; std::vector ipaBuffers_; - RkISP1Frames frameInfo_; RkISP1MainPath *mainPath_; RkISP1SelfPath *selfPath_; std::unique_ptr ipa_; + /* Associate a frame id with a Request. */ + std::map requestMap_; + private: void paramFilled(unsigned int frame); void setSensorControls(unsigned int frame, @@ -181,13 +168,17 @@ private: return static_cast(camera->_d()); } + RkISP1Request *cameraRequest(Request *request) + { + return static_cast(request->_d()); + } + friend RkISP1CameraData; - friend RkISP1Frames; int initLinks(Camera *camera, const CameraSensor *sensor, const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); - void tryCompleteRequest(RkISP1FrameInfo *info); + void tryCompleteRequest(RkISP1Request *request); void bufferReady(FrameBuffer *buffer); void paramReady(FrameBuffer *buffer); void statReady(FrameBuffer *buffer); @@ -218,129 +209,6 @@ private: const MediaPad *ispSink_; }; -RkISP1Frames::RkISP1Frames(PipelineHandler *pipe) - : pipe_(static_cast(pipe)) -{ -} - -RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *request, - bool isRaw) -{ - unsigned int frame = data->frame_; - - FrameBuffer *paramBuffer = nullptr; - FrameBuffer *statBuffer = nullptr; - - if (!isRaw) { - if (pipe_->availableParamBuffers_.empty()) { - LOG(RkISP1, Error) << "Parameters buffer underrun"; - return nullptr; - } - - if (pipe_->availableStatBuffers_.empty()) { - LOG(RkISP1, Error) << "Statistic buffer underrun"; - return nullptr; - } - - paramBuffer = pipe_->availableParamBuffers_.front(); - pipe_->availableParamBuffers_.pop(); - - statBuffer = pipe_->availableStatBuffers_.front(); - pipe_->availableStatBuffers_.pop(); - } - - FrameBuffer *mainPathBuffer = request->findBuffer(&data->mainPathStream_); - FrameBuffer *selfPathBuffer = request->findBuffer(&data->selfPathStream_); - - RkISP1FrameInfo *info = new RkISP1FrameInfo; - - info->frame = frame; - info->request = request; - info->paramBuffer = paramBuffer; - info->mainPathBuffer = mainPathBuffer; - info->selfPathBuffer = selfPathBuffer; - info->statBuffer = statBuffer; - info->paramDequeued = false; - info->metadataProcessed = false; - - frameInfo_[frame] = info; - - return info; -} - -int RkISP1Frames::destroy(unsigned int frame) -{ - RkISP1FrameInfo *info = find(frame); - if (!info) - return -ENOENT; - - pipe_->availableParamBuffers_.push(info->paramBuffer); - pipe_->availableStatBuffers_.push(info->statBuffer); - - frameInfo_.erase(info->frame); - - delete info; - - return 0; -} - -void RkISP1Frames::clear() -{ - for (const auto &entry : frameInfo_) { - RkISP1FrameInfo *info = entry.second; - - pipe_->availableParamBuffers_.push(info->paramBuffer); - pipe_->availableStatBuffers_.push(info->statBuffer); - - delete info; - } - - frameInfo_.clear(); -} - -RkISP1FrameInfo *RkISP1Frames::find(unsigned int frame) -{ - auto itInfo = frameInfo_.find(frame); - - if (itInfo != frameInfo_.end()) - return itInfo->second; - - LOG(RkISP1, Fatal) << "Can't locate info from frame"; - - return nullptr; -} - -RkISP1FrameInfo *RkISP1Frames::find(FrameBuffer *buffer) -{ - for (auto &itInfo : frameInfo_) { - RkISP1FrameInfo *info = itInfo.second; - - if (info->paramBuffer == buffer || - info->statBuffer == buffer || - info->mainPathBuffer == buffer || - info->selfPathBuffer == buffer) - return info; - } - - LOG(RkISP1, Fatal) << "Can't locate info from buffer"; - - return nullptr; -} - -RkISP1FrameInfo *RkISP1Frames::find(Request *request) -{ - for (auto &itInfo : frameInfo_) { - RkISP1FrameInfo *info = itInfo.second; - - if (info->request == request) - return info; - } - - LOG(RkISP1, Fatal) << "Can't locate info from request"; - - return nullptr; -} - PipelineHandlerRkISP1 *RkISP1CameraData::pipe() { return static_cast(Camera::Private::pipe()); @@ -391,23 +259,34 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) return 0; } +void RkISP1CameraData::addRequest(RkISP1Request *request) +{ + /* Associate the request and the frame number. */ + request->frame = frame_; + requestMap_[frame_] = request; + frame_++; +} + void RkISP1CameraData::paramFilled(unsigned int frame) { PipelineHandlerRkISP1 *pipe = RkISP1CameraData::pipe(); - RkISP1FrameInfo *info = frameInfo_.find(frame); - if (!info) - return; + RkISP1Request *request = requestMap_.at(frame); + ASSERT(request); - info->paramBuffer->_d()->metadata().planes()[0].bytesused = + request->paramBuffer->_d()->metadata().planes()[0].bytesused = sizeof(struct rkisp1_params_cfg); - pipe->param_->queueBuffer(info->paramBuffer); - pipe->stat_->queueBuffer(info->statBuffer); - - if (info->mainPathBuffer) - mainPath_->queueBuffer(info->mainPathBuffer); - - if (selfPath_ && info->selfPathBuffer) - selfPath_->queueBuffer(info->selfPathBuffer); + pipe->param_->queueBuffer(request->paramBuffer); + pipe->stat_->queueBuffer(request->statBuffer); + + FrameBuffer *mainPathBuffer = + request->_o()->findBuffer(&mainPathStream_); + if (mainPathBuffer) + mainPath_->queueBuffer(mainPathBuffer); + + FrameBuffer *selfPathBuffer = + request->_o()->findBuffer(&selfPathStream_); + if (selfPath_ && selfPathBuffer) + selfPath_->queueBuffer(selfPathBuffer); } void RkISP1CameraData::setSensorControls([[maybe_unused]] unsigned int frame, @@ -418,14 +297,13 @@ void RkISP1CameraData::setSensorControls([[maybe_unused]] unsigned int frame, void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &metadata) { - RkISP1FrameInfo *info = frameInfo_.find(frame); - if (!info) - return; + RkISP1Request *request = requestMap_.at(frame); + ASSERT(request); - info->request->metadata().merge(metadata); - info->metadataProcessed = true; + request->_o()->metadata().merge(metadata); + request->metadataProcessed = true; - pipe()->tryCompleteRequest(info); + pipe()->tryCompleteRequest(request); } /* ----------------------------------------------------------------------------- @@ -1025,7 +903,7 @@ void PipelineHandlerRkISP1::stopDevice(Camera *camera) } ASSERT(data->queuedRequests_.empty()); - data->frameInfo_.clear(); + data->requestMap_.clear(); freeBuffers(camera); @@ -1042,24 +920,60 @@ std::unique_ptr PipelineHandlerRkISP1::createRequestDevice(Camera *came int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) { RkISP1CameraData *data = cameraData(camera); + RkISP1Request *rkisp1Request = cameraRequest(request); - RkISP1FrameInfo *info = data->frameInfo_.create(data, request, isRaw_); - if (!info) - return -ENOENT; + data->addRequest(rkisp1Request); - data->ipa_->queueRequest(data->frame_, request->controls()); + /* + * If we're operating in RAW mode (only one RAW stream is captured) + * then we simply queue buffers to the video devices as we don't + * need to run the IPA. + */ if (isRaw_) { - if (info->mainPathBuffer) - data->mainPath_->queueBuffer(info->mainPathBuffer); + FrameBuffer *mainPathBuffer = + request->findBuffer(&data->mainPathStream_); + if (mainPathBuffer) + data->mainPath_->queueBuffer(mainPathBuffer); - if (data->selfPath_ && info->selfPathBuffer) - data->selfPath_->queueBuffer(info->selfPathBuffer); - } else { - data->ipa_->fillParamsBuffer(data->frame_, - info->paramBuffer->cookie()); + FrameBuffer *selfPathBuffer = + request->findBuffer(&data->selfPathStream_); + if (data->selfPath_ && selfPathBuffer) + data->selfPath_->queueBuffer(selfPathBuffer); + + return 0; + } + + /* + * If we run the IPA we need to associate a parameters and a statistics + * buffer with the Request and associate the request with the current + * frame number. + * + * Associate the stat and frame buffers to a Request (if available) + * and then run the IPA. + */ + if (availableParamBuffers_.empty()) { + LOG(RkISP1, Error) << "Parameters buffer underrun"; + return -ENOENT; } - data->frame_++; + if (availableStatBuffers_.empty()) { + LOG(RkISP1, Error) << "Statistic buffer underrun"; + return -ENOENT; + } + + rkisp1Request->paramBuffer = availableParamBuffers_.front(); + rkisp1Request->paramDequeued = false; + rkisp1Request->paramBuffer->_d()->setRequest(request); + availableParamBuffers_.pop(); + + rkisp1Request->statBuffer = availableStatBuffers_.front(); + rkisp1Request->metadataProcessed = false; + rkisp1Request->statBuffer->_d()->setRequest(request); + availableStatBuffers_.pop(); + + data->ipa_->queueRequest(rkisp1Request->frame, request->controls()); + data->ipa_->fillParamsBuffer(rkisp1Request->frame, + rkisp1Request->paramBuffer->cookie()); return 0; } @@ -1251,37 +1165,28 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) * Buffer Handling */ -void PipelineHandlerRkISP1::tryCompleteRequest(RkISP1FrameInfo *info) +void PipelineHandlerRkISP1::tryCompleteRequest(RkISP1Request *request) { RkISP1CameraData *data = cameraData(activeCamera_); - Request *request = info->request; - - if (request->hasPendingBuffers()) - return; - - if (!info->metadataProcessed) - return; - if (!isRaw_ && !info->paramDequeued) + if (request->hasPendingBuffers(isRaw_)) return; - data->frameInfo_.destroy(info->frame); + /* Return the stat and param buffers to the pipeline. */ + availableParamBuffers_.push(request->paramBuffer); + availableStatBuffers_.push(request->statBuffer); + data->requestMap_.erase(request->frame); - completeRequest(request); + completeRequest(request->_o()); } void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) { ASSERT(activeCamera_); RkISP1CameraData *data = cameraData(activeCamera_); - - RkISP1FrameInfo *info = data->frameInfo_.find(buffer); - if (!info) - return; + RkISP1Request *request = cameraRequest(buffer->request()); const FrameMetadata &metadata = buffer->metadata(); - Request *request = buffer->request(); - if (metadata.status != FrameMetadata::FrameCancelled) { /* * Record the sensor's timestamp in the request metadata. @@ -1289,55 +1194,48 @@ void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal. */ - request->metadata().set(controls::SensorTimestamp, - metadata.timestamp); + request->_o()->metadata().set(controls::SensorTimestamp, + metadata.timestamp); if (isRaw_) { const ControlList &ctrls = data->delayedCtrls_->get(metadata.sequence); - data->ipa_->processStatsBuffer(info->frame, 0, ctrls); + data->ipa_->processStatsBuffer(request->frame, 0, ctrls); } } else { if (isRaw_) - info->metadataProcessed = true; + request->metadataProcessed = true; } - completeBuffer(request, buffer); - tryCompleteRequest(info); + completeBuffer(request->_o(), buffer); + tryCompleteRequest(request); } void PipelineHandlerRkISP1::paramReady(FrameBuffer *buffer) { ASSERT(activeCamera_); - RkISP1CameraData *data = cameraData(activeCamera_); + RkISP1Request *request = cameraRequest(buffer->request()); - RkISP1FrameInfo *info = data->frameInfo_.find(buffer); - if (!info) - return; - - info->paramDequeued = true; - tryCompleteRequest(info); + request->paramDequeued = true; + tryCompleteRequest(request); } void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) { ASSERT(activeCamera_); RkISP1CameraData *data = cameraData(activeCamera_); - - RkISP1FrameInfo *info = data->frameInfo_.find(buffer); - if (!info) - return; + RkISP1Request *request = cameraRequest(buffer->request()); if (buffer->metadata().status == FrameMetadata::FrameCancelled) { - info->metadataProcessed = true; - tryCompleteRequest(info); + request->metadataProcessed = true; + tryCompleteRequest(request); return; } if (data->frame_ <= buffer->metadata().sequence) data->frame_ = buffer->metadata().sequence + 1; - data->ipa_->processStatsBuffer(info->frame, info->statBuffer->cookie(), + data->ipa_->processStatsBuffer(request->frame, request->statBuffer->cookie(), data->delayedCtrls_->get(buffer->metadata().sequence)); } From patchwork Wed Feb 21 17:40:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19526 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 718A5C32C3 for ; Wed, 21 Feb 2024 17:40:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 402CE62822; Wed, 21 Feb 2024 18:40:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="QW+pHLgR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C766C6281D for ; Wed, 21 Feb 2024 18:40:28 +0100 (CET) Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A9FCA2B3; Wed, 21 Feb 2024 18:40:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1708537220; bh=Y9w5i6JRZWaf6u+qYrUbj4vtvLWg+cZKde67FOjM1K4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QW+pHLgRnoA7XxPwSasyX5hPH9GFC7Esy63GWnTAKcrK3jXritXU+OORFbkArN0xu K9CL4UgTT0H7zcB6o+1AvDi+v03D6KnBQMkZQIrKoisEKSPkGgOwXqanBoMe8k10GO nMobBFEvvtGmWSseEE2816GXfs7dRIEryrJSDVtc= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Subject: [PATCH 4/5] libcamera: ipu3: Replace IPU3FrameInfo Date: Wed, 21 Feb 2024 18:40:12 +0100 Message-ID: <20240221174015.52958-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> References: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the pipeline handler can create a private derived class of Request::Private, use it to store the pipeline specific data. In the case of IPU3 we associate statistics and paramters buffer with a Request and a raw buffer which gets dequeued from the CIO2 and input to the ImgU input. This replaces the functionalities of IPU3FrameInfo which can now be removed. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/frames.cpp | 143 --------------- src/libcamera/pipeline/ipu3/frames.h | 67 ------- src/libcamera/pipeline/ipu3/ipu3.cpp | 221 ++++++++++++++++-------- src/libcamera/pipeline/ipu3/meson.build | 1 - 4 files changed, 146 insertions(+), 286 deletions(-) delete mode 100644 src/libcamera/pipeline/ipu3/frames.cpp delete mode 100644 src/libcamera/pipeline/ipu3/frames.h diff --git a/src/libcamera/pipeline/ipu3/frames.cpp b/src/libcamera/pipeline/ipu3/frames.cpp deleted file mode 100644 index a4c3477cd9ef..000000000000 --- a/src/libcamera/pipeline/ipu3/frames.cpp +++ /dev/null @@ -1,143 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2020, Google Inc. - * - * frames.cpp - Intel IPU3 Frames helper - */ - -#include "frames.h" - -#include -#include - -#include "libcamera/internal/framebuffer.h" -#include "libcamera/internal/pipeline_handler.h" -#include "libcamera/internal/v4l2_videodevice.h" - -namespace libcamera { - -LOG_DECLARE_CATEGORY(IPU3) - -IPU3Frames::IPU3Frames() -{ -} - -void IPU3Frames::init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers) -{ - for (const std::unique_ptr &buffer : paramBuffers) - availableParamBuffers_.push(buffer.get()); - - for (const std::unique_ptr &buffer : statBuffers) - availableStatBuffers_.push(buffer.get()); - - frameInfo_.clear(); -} - -void IPU3Frames::clear() -{ - availableParamBuffers_ = {}; - availableStatBuffers_ = {}; -} - -IPU3Frames::Info *IPU3Frames::create(Request *request) -{ - unsigned int id = request->sequence(); - - if (availableParamBuffers_.empty()) { - LOG(IPU3, Debug) << "Parameters buffer underrun"; - return nullptr; - } - - if (availableStatBuffers_.empty()) { - LOG(IPU3, Debug) << "Statistics buffer underrun"; - return nullptr; - } - - FrameBuffer *paramBuffer = availableParamBuffers_.front(); - FrameBuffer *statBuffer = availableStatBuffers_.front(); - - paramBuffer->_d()->setRequest(request); - statBuffer->_d()->setRequest(request); - - availableParamBuffers_.pop(); - availableStatBuffers_.pop(); - - /* \todo Remove the dynamic allocation of Info */ - std::unique_ptr info = std::make_unique(); - - info->id = id; - info->request = request; - info->rawBuffer = nullptr; - info->paramBuffer = paramBuffer; - info->statBuffer = statBuffer; - info->paramDequeued = false; - info->metadataProcessed = false; - - frameInfo_[id] = std::move(info); - - return frameInfo_[id].get(); -} - -void IPU3Frames::remove(IPU3Frames::Info *info) -{ - /* Return params and stat buffer for reuse. */ - availableParamBuffers_.push(info->paramBuffer); - availableStatBuffers_.push(info->statBuffer); - - /* Delete the extended frame information. */ - frameInfo_.erase(info->id); -} - -bool IPU3Frames::tryComplete(IPU3Frames::Info *info) -{ - Request *request = info->request; - - if (request->hasPendingBuffers()) - return false; - - if (!info->metadataProcessed) - return false; - - if (!info->paramDequeued) - return false; - - remove(info); - - bufferAvailable.emit(); - - return true; -} - -IPU3Frames::Info *IPU3Frames::find(unsigned int id) -{ - const auto &itInfo = frameInfo_.find(id); - - if (itInfo != frameInfo_.end()) - return itInfo->second.get(); - - LOG(IPU3, Fatal) << "Can't find tracking information for frame " << id; - - return nullptr; -} - -IPU3Frames::Info *IPU3Frames::find(FrameBuffer *buffer) -{ - for (auto const &itInfo : frameInfo_) { - Info *info = itInfo.second.get(); - - for (auto const itBuffers : info->request->buffers()) - if (itBuffers.second == buffer) - return info; - - if (info->rawBuffer == buffer || info->paramBuffer == buffer || - info->statBuffer == buffer) - return info; - } - - LOG(IPU3, Fatal) << "Can't find tracking information from buffer"; - - return nullptr; -} - -} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/frames.h b/src/libcamera/pipeline/ipu3/frames.h deleted file mode 100644 index 6e3cb915c7b8..000000000000 --- a/src/libcamera/pipeline/ipu3/frames.h +++ /dev/null @@ -1,67 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2020, Google Inc. - * - * frames.h - Intel IPU3 Frames helper - */ - -#pragma once - -#include -#include -#include -#include - -#include - -#include - -namespace libcamera { - -class FrameBuffer; -class IPAProxy; -class PipelineHandler; -class Request; -class V4L2VideoDevice; -struct IPABuffer; - -class IPU3Frames -{ -public: - struct Info { - unsigned int id; - Request *request; - - FrameBuffer *rawBuffer; - FrameBuffer *paramBuffer; - FrameBuffer *statBuffer; - - ControlList effectiveSensorControls; - - bool paramDequeued; - bool metadataProcessed; - }; - - IPU3Frames(); - - void init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers); - void clear(); - - Info *create(Request *request); - void remove(Info *info); - bool tryComplete(Info *info); - - Info *find(unsigned int id); - Info *find(FrameBuffer *buffer); - - Signal<> bufferAvailable; - -private: - std::queue availableParamBuffers_; - std::queue availableStatBuffers_; - - std::map> frameInfo_; -}; - -} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index fa4bd0bb73e2..f61e4a10892e 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -34,9 +34,9 @@ #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" #include "cio2.h" -#include "frames.h" #include "imgu.h" namespace libcamera { @@ -47,6 +47,24 @@ static const ControlInfoMap::Map IPU3Controls = { { &controls::draft::PipelineDepth, ControlInfo(2, 3) }, }; +class IPU3Request : public Request::Private +{ +public: + IPU3Request(Camera *camera) + : Request::Private(camera) + { + } + + FrameBuffer *rawBuffer; + FrameBuffer *paramBuffer; + FrameBuffer *statBuffer; + + ControlList effectiveSensorControls; + + bool paramDequeued; + bool metadataProcessed; +}; + class IPU3CameraData : public Camera::Private { public: @@ -57,6 +75,7 @@ public: int loadIPA(); + void tryCompleteRequest(IPU3Request *request); void imguOutputBufferReady(FrameBuffer *buffer); void cio2BufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); @@ -75,7 +94,6 @@ public: Rectangle cropRegion_; std::unique_ptr delayedCtrls_; - IPU3Frames frameInfos_; std::unique_ptr ipa_; @@ -86,7 +104,17 @@ public: ControlInfoMap ipaControls_; + std::map requestMap_; + + std::queue availableParamBuffers_; + std::queue availableStatBuffers_; + private: + IPU3Request *cameraRequest(Request *request) + { + return static_cast(request->_d()); + } + void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); void setSensorControls(unsigned int id, const ControlList &sensorControls, @@ -144,6 +172,8 @@ public: int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; + std::unique_ptr createRequestDevice(Camera *camera, + uint64_t cookie) override; int queueRequestDevice(Camera *camera, Request *request) override; bool match(DeviceEnumerator *enumerator) override; @@ -154,6 +184,11 @@ private: return static_cast(camera->_d()); } + IPU3Request *cameraRequest(Request *request) + { + return static_cast(request->_d()); + } + int initControls(IPU3CameraData *data); int updateControls(IPU3CameraData *data); int registerCameras(); @@ -680,19 +715,17 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) for (const std::unique_ptr &buffer : imgu->paramBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); + data->availableParamBuffers_.push(buffer.get()); } for (const std::unique_ptr &buffer : imgu->statBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); + data->availableStatBuffers_.push(buffer.get()); } data->ipa_->mapBuffers(ipaBuffers_); - data->frameInfos_.init(imgu->paramBuffers_, imgu->statBuffers_); - data->frameInfos_.bufferAvailable.connect( - data, &IPU3CameraData::queuePendingRequests); - return 0; } @@ -700,8 +733,6 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) { IPU3CameraData *data = cameraData(camera); - data->frameInfos_.clear(); - std::vector ids; for (IPABuffer &ipabuf : ipaBuffers_) ids.push_back(ipabuf.id); @@ -777,6 +808,7 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); freeBuffers(camera); + data->requestMap_.clear(); } void IPU3CameraData::cancelPendingRequests() @@ -801,37 +833,80 @@ void IPU3CameraData::queuePendingRequests() { while (!pendingRequests_.empty()) { Request *request = pendingRequests_.front(); + IPU3Request *ipu3Request = cameraRequest(request); - IPU3Frames::Info *info = frameInfos_.create(request); - if (!info) - break; + /* + * The IPU3 pipeline associates buffers with the request + * sequence. + */ + requestMap_[request->sequence()] = ipu3Request; /* * Queue a buffer on the CIO2, using the raw stream buffer * provided in the request, if any, or a CIO2 internal buffer * otherwise. + * + * No need to set a buffer->setRequest() as + * a) If the buffer is provided by the application that's + * already been done + * b) If the buffer comes from the CIO2 internal pool, + * CIO2Device::queueBuffer() does that for us */ FrameBuffer *reqRawBuffer = request->findBuffer(&rawStream_); FrameBuffer *rawBuffer = cio2_.queueBuffer(request, reqRawBuffer); + /* * \todo If queueBuffer fails in queuing a buffer to the device, * report the request as error by cancelling the request and * calling PipelineHandler::completeRequest(). */ - if (!rawBuffer) { - frameInfos_.remove(info); + if (!rawBuffer) + break; + + /* + * Store the raw buffer queued to the CIO2 to queue it to the + * Imgu input when the IPA has prepared the paramters buffer. + */ + ipu3Request->rawBuffer = rawBuffer; + + /* + * Prepare the stats and parameters buffer and associate it + * with the currently queued request. + */ + if (availableParamBuffers_.empty()) { + LOG(IPU3, Debug) << "Parameters buffer underrun"; + break; + } + + if (availableStatBuffers_.empty()) { + LOG(IPU3, Debug) << "Statistics buffer underrun"; break; } - info->rawBuffer = rawBuffer; + ipu3Request->paramBuffer = availableParamBuffers_.front(); + ipu3Request->paramDequeued = false; + ipu3Request->paramBuffer->_d()->setRequest(request); + availableParamBuffers_.pop(); - ipa_->queueRequest(info->id, request->controls()); + ipu3Request->statBuffer = availableStatBuffers_.front(); + ipu3Request->metadataProcessed = false; + ipu3Request->statBuffer->_d()->setRequest(request); + availableStatBuffers_.pop(); + + ipa_->queueRequest(request->sequence(), request->controls()); pendingRequests_.pop(); processingRequests_.push(request); } } +std::unique_ptr PipelineHandlerIPU3::createRequestDevice(Camera *camera, + uint64_t cookie) +{ + auto request = std::make_unique(camera); + return Request::create(std::move(request), cookie); +} + int PipelineHandlerIPU3::queueRequestDevice(Camera *camera, Request *request) { IPU3CameraData *data = cameraData(camera); @@ -1200,6 +1275,26 @@ int IPU3CameraData::loadIPA() return 0; } +void IPU3CameraData::tryCompleteRequest(IPU3Request *request) +{ + if (request->hasPendingBuffers()) + return; + + if (!request->metadataProcessed) + return; + + if (!request->paramDequeued) + return; + + availableParamBuffers_.push(request->paramBuffer); + availableStatBuffers_.push(request->statBuffer); + + pipe()->completeRequest(request->_o()); + + /* Try queue another request now that this one has completed. */ + queuePendingRequests(); +} + void IPU3CameraData::setSensorControls([[maybe_unused]] unsigned int id, const ControlList &sensorControls, const ControlList &lensControls) @@ -1220,12 +1315,10 @@ void IPU3CameraData::setSensorControls([[maybe_unused]] unsigned int id, void IPU3CameraData::paramsBufferReady(unsigned int id) { - IPU3Frames::Info *info = frameInfos_.find(id); - if (!info) - return; + IPU3Request *request = requestMap_[id]; /* Queue all buffers from the request aimed for the ImgU. */ - for (auto it : info->request->buffers()) { + for (auto it : request->_o()->buffers()) { const Stream *stream = it.first; FrameBuffer *outbuffer = it.second; @@ -1235,25 +1328,21 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) imgu_->viewfinder_->queueBuffer(outbuffer); } - info->paramBuffer->_d()->metadata().planes()[0].bytesused = + request->paramBuffer->_d()->metadata().planes()[0].bytesused = sizeof(struct ipu3_uapi_params); - imgu_->param_->queueBuffer(info->paramBuffer); - imgu_->stat_->queueBuffer(info->statBuffer); - imgu_->input_->queueBuffer(info->rawBuffer); + imgu_->param_->queueBuffer(request->paramBuffer); + imgu_->stat_->queueBuffer(request->statBuffer); + imgu_->input_->queueBuffer(request->rawBuffer); } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) { - IPU3Frames::Info *info = frameInfos_.find(id); - if (!info) - return; + IPU3Request *request = requestMap_[id]; - Request *request = info->request; - request->metadata().merge(metadata); + request->_o()->metadata().merge(metadata); - info->metadataProcessed = true; - if (frameInfos_.tryComplete(info)) - pipe()->completeRequest(request); + request->metadataProcessed = true; + tryCompleteRequest(request); } /* ----------------------------------------------------------------------------- @@ -1268,11 +1357,7 @@ void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) */ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) { - IPU3Frames::Info *info = frameInfos_.find(buffer); - if (!info) - return; - - Request *request = info->request; + Request *request = buffer->request(); pipe()->completeBuffer(request, buffer); @@ -1283,8 +1368,7 @@ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) cropRegion_ = *scalerCrop; request->metadata().set(controls::ScalerCrop, cropRegion_); - if (frameInfos_.tryComplete(info)) - pipe()->completeRequest(request); + tryCompleteRequest(cameraRequest(request)); } /** @@ -1296,22 +1380,20 @@ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) */ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) { - IPU3Frames::Info *info = frameInfos_.find(buffer); - if (!info) - return; - - Request *request = info->request; + IPU3Request *request = cameraRequest(buffer->request()); /* If the buffer is cancelled force a complete of the whole request. */ if (buffer->metadata().status == FrameMetadata::FrameCancelled) { - for (auto it : request->buffers()) { + for (auto it : request->_o()->buffers()) { FrameBuffer *b = it.second; b->_d()->cancel(); - pipe()->completeBuffer(request, b); + pipe()->completeBuffer(request->_o(), b); } - frameInfos_.remove(info); - pipe()->completeRequest(request); + availableParamBuffers_.push(request->paramBuffer); + availableStatBuffers_.push(request->statBuffer); + + pipe()->completeRequest(request->_o()); return; } @@ -1321,24 +1403,23 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal. */ - request->metadata().set(controls::SensorTimestamp, - buffer->metadata().timestamp); + request->_o()->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); - info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); + request->effectiveSensorControls = + delayedCtrls_->get(buffer->metadata().sequence); - if (request->findBuffer(&rawStream_)) - pipe()->completeBuffer(request, buffer); + if (request->_o()->findBuffer(&rawStream_)) + pipe()->completeBuffer(request->_o(), buffer); - ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); + ipa_->fillParamsBuffer(request->_o()->sequence(), + request->paramBuffer->cookie()); } void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) { - IPU3Frames::Info *info = frameInfos_.find(buffer); - if (!info) - return; - - info->paramDequeued = true; + IPU3Request *request = cameraRequest(buffer->request()); + request->paramDequeued = true; /* * tryComplete() will delete info if it completes the IPU3Frame. @@ -1346,35 +1427,25 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) * * \todo Improve the FrameInfo API to avoid this type of issue */ - Request *request = info->request; - if (frameInfos_.tryComplete(info)) - pipe()->completeRequest(request); + tryCompleteRequest(request); } void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { - IPU3Frames::Info *info = frameInfos_.find(buffer); - if (!info) - return; - - Request *request = info->request; + IPU3Request *request = cameraRequest(buffer->request()); if (buffer->metadata().status == FrameMetadata::FrameCancelled) { - info->metadataProcessed = true; + request->metadataProcessed = true; - /* - * tryComplete() will delete info if it completes the IPU3Frame. - * In that event, we must have obtained the Request before hand. - */ - if (frameInfos_.tryComplete(info)) - pipe()->completeRequest(request); + tryCompleteRequest(request); return; } - ipa_->processStatsBuffer(info->id, request->metadata().get(controls::SensorTimestamp).value_or(0), - info->statBuffer->cookie(), info->effectiveSensorControls); + ipa_->processStatsBuffer(request->_o()->sequence(), + request->_o()->metadata().get(controls::SensorTimestamp).value_or(0), + request->statBuffer->cookie(), request->effectiveSensorControls); } /* diff --git a/src/libcamera/pipeline/ipu3/meson.build b/src/libcamera/pipeline/ipu3/meson.build index a1b0b31ac5bc..d60e07ae6cca 100644 --- a/src/libcamera/pipeline/ipu3/meson.build +++ b/src/libcamera/pipeline/ipu3/meson.build @@ -2,7 +2,6 @@ libcamera_sources += files([ 'cio2.cpp', - 'frames.cpp', 'imgu.cpp', 'ipu3.cpp', ]) From patchwork Wed Feb 21 17:40:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19527 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DF3B8C3263 for ; Wed, 21 Feb 2024 17:40:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B4F4A6281F; Wed, 21 Feb 2024 18:40:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="G5DzlPop"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0F2CC62805 for ; Wed, 21 Feb 2024 18:40:29 +0100 (CET) Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 03F119EC; Wed, 21 Feb 2024 18:40:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1708537221; bh=+XaQ5Ov4BVfEVahJ7Eafb+W0a15MLRT1NfZGI7e/+UU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G5DzlPopJGktXOMY0iAVbyEtOaU/qfPNX3gem/TSIw4OHVziWwPDVGdezrrJwzmVw 59V6VdpeHpgRETcaj6Bw8SyCMMwfts8UjXL/qvpQ4Up9eelW6HeOLVZbxFv4G1TyOe v+7relkLsn/xAgUPNa2WKE7hSUAeyHxorQCkoMVQ= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Subject: [PATCH 5/5] libcamera: ipu3: Return Raw buffers on error Date: Wed, 21 Feb 2024 18:40:13 +0100 Message-ID: <20240221174015.52958-6-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> References: <20240221174015.52958-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a Request is cancelled, all its application-provided buffers are completed and the paramters and stats buffer are made available again to the pipeline handler. If the RAW buffer doesn't come from the application (no RAW stream requested) it comes from the CIO2Device internal pool. Return it to the CIO2 pool in case of error. Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally --- src/libcamera/pipeline/ipu3/ipu3.cpp | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index f61e4a10892e..bcbcfd3dc290 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1393,6 +1393,9 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) availableParamBuffers_.push(request->paramBuffer); availableStatBuffers_.push(request->statBuffer); + /* Return RAW buffer to the CIO2. */ + cio2_.tryReturnBuffer(buffer); + pipe()->completeRequest(request->_o()); return; }