From patchwork Mon Mar 11 12:32:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19657 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9565CBD1F1 for ; Mon, 11 Mar 2024 12:32:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 84F7962C84; Mon, 11 Mar 2024 13:32:47 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BejItgeX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CA9FF6286F for ; Mon, 11 Mar 2024 13:32:44 +0100 (CET) Received: from localhost.localdomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7362AF0A; Mon, 11 Mar 2024 13:32:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1710160343; bh=fl3hRCbieAa2K1342VeSQlfCx4iM3xTYClxBJBCMfq0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BejItgeX/6W0Wl062CJDnqunQiOMfG9hLYWPyjGzoKd8g7GQPZY+NUGMwR0BcJ3sr PyJXejU5Iww12ziy0xhp/9ESJbYeGiwv3CRBkZsxN6STSc/QLMI6hRYIkDMexCYRS8 I3O0wznHcfppzLLXu5m+kCTnChxHhicvAw017luc= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Subject: [PATCH v2 1/4] libcamera: Allow pipeline to provide a Private request Date: Mon, 11 Mar 2024 13:32:29 +0100 Message-ID: <20240311123234.32925-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240311123234.32925-1-jacopo.mondi@ideasonboard.com> References: <20240311123234.32925-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to allow each pipeline handler to create a Request::Private derived class to store ancillary data associated with a capture request, modify the way a Request is created by the PipelineHandler base class. Introduce a virtual PipelineHandler::createRequestDevice() that pipeline handler implementation can optionally override to provide a custom private data type to initialize the Request with and make the existing Request() public constructor not usable by applications. As we can't anymore create a Request without going through an active camera, drop the queueRequest() checks from the Camera statemachine test. Signed-off-by: Jacopo Mondi Reviewed-by: Stefan Klug Reviewed-by: Daniel Scally Tested-by: Daniel Scally --- include/libcamera/internal/pipeline_handler.h | 5 ++- include/libcamera/request.h | 3 +- src/libcamera/camera.cpp | 8 +--- src/libcamera/pipeline_handler.cpp | 38 ++++++++++++++++--- src/libcamera/request.cpp | 15 +++++--- test/camera/statemachine.cpp | 12 ------ 6 files changed, 50 insertions(+), 31 deletions(-) diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index c96944f4ecc4..bbe74f22e5ae 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -21,6 +21,7 @@ #include #include "libcamera/internal/ipa_proxy.h" +#include "libcamera/internal/request.h" namespace libcamera { @@ -59,7 +60,7 @@ public: void stop(Camera *camera); bool hasPendingRequests(const Camera *camera) const; - void registerRequest(Request *request); + std::unique_ptr createRequest(Camera *camera, uint64_t cookie); void queueRequest(Request *request); bool completeBuffer(Request *request, FrameBuffer *buffer); @@ -74,6 +75,8 @@ protected: void registerCamera(std::shared_ptr camera); void hotplugMediaDevice(MediaDevice *media); + virtual std::unique_ptr createRequestDevice(Camera *camera, + uint64_t cookie); virtual int queueRequestDevice(Camera *camera, Request *request) = 0; virtual void stopDevice(Camera *camera) = 0; diff --git a/include/libcamera/request.h b/include/libcamera/request.h index dffde1536cad..e16e61a93873 100644 --- a/include/libcamera/request.h +++ b/include/libcamera/request.h @@ -45,9 +45,10 @@ public: using BufferMap = std::map; - Request(Camera *camera, uint64_t cookie = 0); + Request(std::unique_ptr d, uint64_t cookie = 0); ~Request(); + static std::unique_ptr create(std::unique_ptr d, uint64_t); void reuse(ReuseFlag flags = Default); ControlList &controls() { return *controls_; } diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index a71dc933b911..b7053576fe42 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -1236,12 +1236,8 @@ std::unique_ptr Camera::createRequest(uint64_t cookie) if (ret < 0) return nullptr; - std::unique_ptr request = std::make_unique(this, cookie); - - /* Associate the request with the pipeline handler. */ - d->pipe_->registerRequest(request.get()); - - return request; + /* Create a Request from the pipeline handler. */ + return d->pipe_->createRequest(this, cookie); } /** diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index 29e0c98a6db5..f2a8cdac0408 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -371,20 +371,25 @@ bool PipelineHandler::hasPendingRequests(const Camera *camera) const } /** - * \fn PipelineHandler::registerRequest() - * \brief Register a request for use by the pipeline handler - * \param[in] request The request to register + * \fn PipelineHandler::createRequest() + * \brief Create a request and register it for use by the pipeline handler + * \param[in] camera The camera the Request belongs to + * \param[in] cookie The Request unique identifier * - * This function is called when the request is created, and allows the pipeline - * handler to perform any one-time initialization it requries for the request. + * This function is called to create a Request by calling createRequestDevice() + * which can be optionally provided by the PipelineHandler derived classes. */ -void PipelineHandler::registerRequest(Request *request) +std::unique_ptr PipelineHandler::createRequest(Camera *camera, uint64_t cookie) { + std::unique_ptr request = createRequestDevice(camera, cookie); + /* * Connect the request prepared signal to notify the pipeline handler * when a request is ready to be processed. */ request->_d()->prepared.connect(this, &PipelineHandler::doQueueRequests); + + return request; } /** @@ -462,6 +467,27 @@ void PipelineHandler::doQueueRequests() } } +/** + * \brief Create a Request from the pipeline handler + * \param[in] camera The camera the Request belongs to + * \param[in] cookie The Request unique identifier + * + * A virtual function that PipelineHandler derived classes are free to override + * in order to initialize a Request with a custom Request::Private derived + * class. + * + * This is the base class implementation that use Request::Private to + * initialize the Request. + * + * \return A unique pointer to a newly created Request + */ +std::unique_ptr +PipelineHandler::createRequestDevice(Camera *camera, uint64_t cookie) +{ + auto d = std::make_unique(camera); + return Request::create(std::move(d), cookie); +} + /** * \fn PipelineHandler::queueRequestDevice() * \brief Queue a request to the device diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index 949c556fa437..d1051ad3d25e 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -336,7 +336,7 @@ void Request::Private::timeout() /** * \brief Create a capture request for a camera - * \param[in] camera The camera that creates the request + * \param[in] d The request private data * \param[in] cookie Opaque cookie for application use * * The \a cookie is stored in the request and is accessible through the @@ -344,12 +344,17 @@ void Request::Private::timeout() * the request to an external resource in the request completion handler, and is * completely opaque to libcamera. */ -Request::Request(Camera *camera, uint64_t cookie) - : Extensible(std::make_unique(camera)), - cookie_(cookie), status_(RequestPending) +std::unique_ptr Request::create(std::unique_ptr d, + uint64_t cookie) +{ + return std::make_unique(std::move(d), cookie); +} + +Request::Request(std::unique_ptr d, uint64_t cookie) + : Extensible(std::move(d)), cookie_(cookie), status_(RequestPending) { controls_ = new ControlList(controls::controls, - camera->_d()->validator()); + _d()->camera()->_d()->validator()); /** * \todo Add a validator for metadata controls. diff --git a/test/camera/statemachine.cpp b/test/camera/statemachine.cpp index 9c2b0c6a7d99..5714061f88b5 100644 --- a/test/camera/statemachine.cpp +++ b/test/camera/statemachine.cpp @@ -38,10 +38,6 @@ protected: if (camera_->start() != -EACCES) return TestFail; - Request request(camera_.get()); - if (camera_->queueRequest(&request) != -EACCES) - return TestFail; - /* Test operations which should pass. */ if (camera_->release()) return TestFail; @@ -68,10 +64,6 @@ protected: if (camera_->start() != -EACCES) return TestFail; - Request request(camera_.get()); - if (camera_->queueRequest(&request) != -EACCES) - return TestFail; - /* Test operations which should pass. */ if (camera_->stop()) return TestFail; @@ -95,10 +87,6 @@ protected: if (camera_->acquire() != -EBUSY) return TestFail; - Request request1(camera_.get()); - if (camera_->queueRequest(&request1) != -EACCES) - return TestFail; - /* Test operations which should pass. */ std::unique_ptr request2 = camera_->createRequest(); if (!request2) From patchwork Mon Mar 11 12:32:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19658 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E1C31C32A3 for ; Mon, 11 Mar 2024 12:32:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0B0CC62C89; Mon, 11 Mar 2024 13:32:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ojFxlMA0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 285B76286F for ; Mon, 11 Mar 2024 13:32:45 +0100 (CET) Received: from localhost.localdomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DA4DDC85; Mon, 11 Mar 2024 13:32:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1710160344; bh=hMSBLGSj2sGGPKdpA0Ln0LMOtcUanFa0GBH/0ec3ZFI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ojFxlMA042u5lcbX+Y6m4otsJwjj7497fifZecJViI/edeacOMtqg7+T01oj/rTH9 iL971FkVveQhWgVF2oua8PBFjZPG6zYLtssJQKws28HtMac88YCC/ln9m9xorfgue9 dOm6Rv10JJEspM1WMPN7PG/yECJEH46EydxTF3pE= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Subject: [PATCH v2 2/4] libcamera: rkisp1: Replace usage of RkISP1FrameInfo Date: Mon, 11 Mar 2024 13:32:30 +0100 Message-ID: <20240311123234.32925-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240311123234.32925-1-jacopo.mondi@ideasonboard.com> References: <20240311123234.32925-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the pipeline handler can create a private derived class of Request::Private, use it to store the pipeline specific data. In the case of RkISP1 we associate the statistic and paramters buffers with a Request. As the IPA sends notifications for paramters and statistics buffer by identifying them by frame id, associate the frame ids and the Request in the RkISP1CameraData class. This replaces the functionalities of RkISP1FrameInfo which can now be removed. Signed-off-by: Jacopo Mondi Reviewed-by: Stefan Klug Reviewed-by: Daniel Scally Tested-by: Daniel Scally --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 359 +++++++++-------------- 1 file changed, 138 insertions(+), 221 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 586b46d64630..0479e243d392 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -39,6 +39,7 @@ #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -48,67 +49,63 @@ namespace libcamera { LOG_DEFINE_CATEGORY(RkISP1) -class PipelineHandlerRkISP1; -class RkISP1CameraData; +class RkISP1Request : public Request::Private +{ +public: + RkISP1Request(Camera *camera) + : Request::Private(camera) + { + } -struct RkISP1FrameInfo { - unsigned int frame; - Request *request; + bool hasPendingBuffers(bool isRaw) const; - FrameBuffer *paramBuffer; FrameBuffer *statBuffer; - FrameBuffer *mainPathBuffer; - FrameBuffer *selfPathBuffer; + FrameBuffer *paramBuffer; + + /* The frame number this request is associated with. */ + unsigned int frame; bool paramDequeued; bool metadataProcessed; }; -class RkISP1Frames +bool RkISP1Request::hasPendingBuffers(bool isRaw) const { -public: - RkISP1Frames(PipelineHandler *pipe); - - RkISP1FrameInfo *create(const RkISP1CameraData *data, Request *request, - bool isRaw); - int destroy(unsigned int frame); - void clear(); - - RkISP1FrameInfo *find(unsigned int frame); - RkISP1FrameInfo *find(FrameBuffer *buffer); - RkISP1FrameInfo *find(Request *request); - -private: - PipelineHandlerRkISP1 *pipe_; - std::map frameInfo_; -}; + return Request::Private::hasPendingBuffers() || + !metadataProcessed || (!isRaw && !paramDequeued); +} +class PipelineHandlerRkISP1; class RkISP1CameraData : public Camera::Private { public: RkISP1CameraData(PipelineHandler *pipe, RkISP1MainPath *mainPath, RkISP1SelfPath *selfPath) - : Camera::Private(pipe), frame_(0), frameInfo_(pipe), - mainPath_(mainPath), selfPath_(selfPath) + : Camera::Private(pipe), frame_(0), mainPath_(mainPath), + selfPath_(selfPath) { } PipelineHandlerRkISP1 *pipe(); int loadIPA(unsigned int hwRevision); + void addRequest(RkISP1Request *request); + Stream mainPathStream_; Stream selfPathStream_; std::unique_ptr sensor_; std::unique_ptr delayedCtrls_; unsigned int frame_; std::vector ipaBuffers_; - RkISP1Frames frameInfo_; RkISP1MainPath *mainPath_; RkISP1SelfPath *selfPath_; std::unique_ptr ipa_; + /* Associate a frame id with a Request. */ + std::map requestMap_; + private: void paramFilled(unsigned int frame); void setSensorControls(unsigned int frame, @@ -157,6 +154,8 @@ public: int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; + std::unique_ptr createRequestDevice(Camera *camera, + uint64_t cookie) override; int queueRequestDevice(Camera *camera, Request *request) override; bool match(DeviceEnumerator *enumerator) override; @@ -169,13 +168,17 @@ private: return static_cast(camera->_d()); } + RkISP1Request *cameraRequest(Request *request) + { + return static_cast(request->_d()); + } + friend RkISP1CameraData; - friend RkISP1Frames; int initLinks(Camera *camera, const CameraSensor *sensor, const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); - void tryCompleteRequest(RkISP1FrameInfo *info); + void tryCompleteRequest(RkISP1Request *request); void bufferReady(FrameBuffer *buffer); void paramReady(FrameBuffer *buffer); void statReady(FrameBuffer *buffer); @@ -206,129 +209,6 @@ private: const MediaPad *ispSink_; }; -RkISP1Frames::RkISP1Frames(PipelineHandler *pipe) - : pipe_(static_cast(pipe)) -{ -} - -RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *request, - bool isRaw) -{ - unsigned int frame = data->frame_; - - FrameBuffer *paramBuffer = nullptr; - FrameBuffer *statBuffer = nullptr; - - if (!isRaw) { - if (pipe_->availableParamBuffers_.empty()) { - LOG(RkISP1, Error) << "Parameters buffer underrun"; - return nullptr; - } - - if (pipe_->availableStatBuffers_.empty()) { - LOG(RkISP1, Error) << "Statistic buffer underrun"; - return nullptr; - } - - paramBuffer = pipe_->availableParamBuffers_.front(); - pipe_->availableParamBuffers_.pop(); - - statBuffer = pipe_->availableStatBuffers_.front(); - pipe_->availableStatBuffers_.pop(); - } - - FrameBuffer *mainPathBuffer = request->findBuffer(&data->mainPathStream_); - FrameBuffer *selfPathBuffer = request->findBuffer(&data->selfPathStream_); - - RkISP1FrameInfo *info = new RkISP1FrameInfo; - - info->frame = frame; - info->request = request; - info->paramBuffer = paramBuffer; - info->mainPathBuffer = mainPathBuffer; - info->selfPathBuffer = selfPathBuffer; - info->statBuffer = statBuffer; - info->paramDequeued = false; - info->metadataProcessed = false; - - frameInfo_[frame] = info; - - return info; -} - -int RkISP1Frames::destroy(unsigned int frame) -{ - RkISP1FrameInfo *info = find(frame); - if (!info) - return -ENOENT; - - pipe_->availableParamBuffers_.push(info->paramBuffer); - pipe_->availableStatBuffers_.push(info->statBuffer); - - frameInfo_.erase(info->frame); - - delete info; - - return 0; -} - -void RkISP1Frames::clear() -{ - for (const auto &entry : frameInfo_) { - RkISP1FrameInfo *info = entry.second; - - pipe_->availableParamBuffers_.push(info->paramBuffer); - pipe_->availableStatBuffers_.push(info->statBuffer); - - delete info; - } - - frameInfo_.clear(); -} - -RkISP1FrameInfo *RkISP1Frames::find(unsigned int frame) -{ - auto itInfo = frameInfo_.find(frame); - - if (itInfo != frameInfo_.end()) - return itInfo->second; - - LOG(RkISP1, Fatal) << "Can't locate info from frame"; - - return nullptr; -} - -RkISP1FrameInfo *RkISP1Frames::find(FrameBuffer *buffer) -{ - for (auto &itInfo : frameInfo_) { - RkISP1FrameInfo *info = itInfo.second; - - if (info->paramBuffer == buffer || - info->statBuffer == buffer || - info->mainPathBuffer == buffer || - info->selfPathBuffer == buffer) - return info; - } - - LOG(RkISP1, Fatal) << "Can't locate info from buffer"; - - return nullptr; -} - -RkISP1FrameInfo *RkISP1Frames::find(Request *request) -{ - for (auto &itInfo : frameInfo_) { - RkISP1FrameInfo *info = itInfo.second; - - if (info->request == request) - return info; - } - - LOG(RkISP1, Fatal) << "Can't locate info from request"; - - return nullptr; -} - PipelineHandlerRkISP1 *RkISP1CameraData::pipe() { return static_cast(Camera::Private::pipe()); @@ -379,23 +259,34 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) return 0; } +void RkISP1CameraData::addRequest(RkISP1Request *request) +{ + /* Associate the request and the frame number. */ + request->frame = frame_; + requestMap_[frame_] = request; + frame_++; +} + void RkISP1CameraData::paramFilled(unsigned int frame) { PipelineHandlerRkISP1 *pipe = RkISP1CameraData::pipe(); - RkISP1FrameInfo *info = frameInfo_.find(frame); - if (!info) - return; + RkISP1Request *request = requestMap_.at(frame); + ASSERT(request); - info->paramBuffer->_d()->metadata().planes()[0].bytesused = + request->paramBuffer->_d()->metadata().planes()[0].bytesused = sizeof(struct rkisp1_params_cfg); - pipe->param_->queueBuffer(info->paramBuffer); - pipe->stat_->queueBuffer(info->statBuffer); - - if (info->mainPathBuffer) - mainPath_->queueBuffer(info->mainPathBuffer); - - if (selfPath_ && info->selfPathBuffer) - selfPath_->queueBuffer(info->selfPathBuffer); + pipe->param_->queueBuffer(request->paramBuffer); + pipe->stat_->queueBuffer(request->statBuffer); + + FrameBuffer *mainPathBuffer = + request->_o()->findBuffer(&mainPathStream_); + if (mainPathBuffer) + mainPath_->queueBuffer(mainPathBuffer); + + FrameBuffer *selfPathBuffer = + request->_o()->findBuffer(&selfPathStream_); + if (selfPath_ && selfPathBuffer) + selfPath_->queueBuffer(selfPathBuffer); } void RkISP1CameraData::setSensorControls([[maybe_unused]] unsigned int frame, @@ -406,14 +297,13 @@ void RkISP1CameraData::setSensorControls([[maybe_unused]] unsigned int frame, void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &metadata) { - RkISP1FrameInfo *info = frameInfo_.find(frame); - if (!info) - return; + RkISP1Request *request = requestMap_.at(frame); + ASSERT(request); - info->request->metadata().merge(metadata); - info->metadataProcessed = true; + request->_o()->metadata().merge(metadata); + request->metadataProcessed = true; - pipe()->tryCompleteRequest(info); + pipe()->tryCompleteRequest(request); } /* ----------------------------------------------------------------------------- @@ -1013,34 +903,77 @@ void PipelineHandlerRkISP1::stopDevice(Camera *camera) } ASSERT(data->queuedRequests_.empty()); - data->frameInfo_.clear(); + data->requestMap_.clear(); freeBuffers(camera); activeCamera_ = nullptr; } +std::unique_ptr PipelineHandlerRkISP1::createRequestDevice(Camera *camera, + uint64_t cookie) +{ + auto request = std::make_unique(camera); + return Request::create(std::move(request), cookie); +} + int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) { RkISP1CameraData *data = cameraData(camera); + RkISP1Request *rkisp1Request = cameraRequest(request); - RkISP1FrameInfo *info = data->frameInfo_.create(data, request, isRaw_); - if (!info) - return -ENOENT; + data->addRequest(rkisp1Request); + data->ipa_->queueRequest(rkisp1Request->frame, request->controls()); - data->ipa_->queueRequest(data->frame_, request->controls()); + /* + * If we're operating in RAW mode (only one RAW stream is captured) + * then we simply queue buffers to the video devices as we don't + * need to run the IPA. + */ if (isRaw_) { - if (info->mainPathBuffer) - data->mainPath_->queueBuffer(info->mainPathBuffer); + FrameBuffer *mainPathBuffer = + request->findBuffer(&data->mainPathStream_); + if (mainPathBuffer) + data->mainPath_->queueBuffer(mainPathBuffer); - if (data->selfPath_ && info->selfPathBuffer) - data->selfPath_->queueBuffer(info->selfPathBuffer); - } else { - data->ipa_->fillParamsBuffer(data->frame_, - info->paramBuffer->cookie()); + FrameBuffer *selfPathBuffer = + request->findBuffer(&data->selfPathStream_); + if (data->selfPath_ && selfPathBuffer) + data->selfPath_->queueBuffer(selfPathBuffer); + + return 0; + } + + /* + * If we run the IPA we need to associate a parameters and a statistics + * buffer with the Request and associate the request with the current + * frame number. + * + * Associate the stat and frame buffers to a Request (if available) + * and then run the IPA. + */ + if (availableParamBuffers_.empty()) { + LOG(RkISP1, Error) << "Parameters buffer underrun"; + return -ENOENT; + } + + if (availableStatBuffers_.empty()) { + LOG(RkISP1, Error) << "Statistic buffer underrun"; + return -ENOENT; } - data->frame_++; + rkisp1Request->paramBuffer = availableParamBuffers_.front(); + rkisp1Request->paramDequeued = false; + rkisp1Request->paramBuffer->_d()->setRequest(request); + availableParamBuffers_.pop(); + + rkisp1Request->statBuffer = availableStatBuffers_.front(); + rkisp1Request->metadataProcessed = false; + rkisp1Request->statBuffer->_d()->setRequest(request); + availableStatBuffers_.pop(); + + data->ipa_->fillParamsBuffer(rkisp1Request->frame, + rkisp1Request->paramBuffer->cookie()); return 0; } @@ -1232,37 +1165,28 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) * Buffer Handling */ -void PipelineHandlerRkISP1::tryCompleteRequest(RkISP1FrameInfo *info) +void PipelineHandlerRkISP1::tryCompleteRequest(RkISP1Request *request) { RkISP1CameraData *data = cameraData(activeCamera_); - Request *request = info->request; - if (request->hasPendingBuffers()) + if (request->hasPendingBuffers(isRaw_)) return; - if (!info->metadataProcessed) - return; + /* Return the stat and param buffers to the pipeline. */ + availableParamBuffers_.push(request->paramBuffer); + availableStatBuffers_.push(request->statBuffer); + data->requestMap_.erase(request->frame); - if (!isRaw_ && !info->paramDequeued) - return; - - data->frameInfo_.destroy(info->frame); - - completeRequest(request); + completeRequest(request->_o()); } void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) { ASSERT(activeCamera_); RkISP1CameraData *data = cameraData(activeCamera_); - - RkISP1FrameInfo *info = data->frameInfo_.find(buffer); - if (!info) - return; + RkISP1Request *request = cameraRequest(buffer->request()); const FrameMetadata &metadata = buffer->metadata(); - Request *request = buffer->request(); - if (metadata.status != FrameMetadata::FrameCancelled) { /* * Record the sensor's timestamp in the request metadata. @@ -1270,55 +1194,48 @@ void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal. */ - request->metadata().set(controls::SensorTimestamp, - metadata.timestamp); + request->_o()->metadata().set(controls::SensorTimestamp, + metadata.timestamp); if (isRaw_) { const ControlList &ctrls = data->delayedCtrls_->get(metadata.sequence); - data->ipa_->processStatsBuffer(info->frame, 0, ctrls); + data->ipa_->processStatsBuffer(request->frame, 0, ctrls); } } else { if (isRaw_) - info->metadataProcessed = true; + request->metadataProcessed = true; } - completeBuffer(request, buffer); - tryCompleteRequest(info); + completeBuffer(request->_o(), buffer); + tryCompleteRequest(request); } void PipelineHandlerRkISP1::paramReady(FrameBuffer *buffer) { ASSERT(activeCamera_); - RkISP1CameraData *data = cameraData(activeCamera_); + RkISP1Request *request = cameraRequest(buffer->request()); - RkISP1FrameInfo *info = data->frameInfo_.find(buffer); - if (!info) - return; - - info->paramDequeued = true; - tryCompleteRequest(info); + request->paramDequeued = true; + tryCompleteRequest(request); } void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) { ASSERT(activeCamera_); RkISP1CameraData *data = cameraData(activeCamera_); - - RkISP1FrameInfo *info = data->frameInfo_.find(buffer); - if (!info) - return; + RkISP1Request *request = cameraRequest(buffer->request()); if (buffer->metadata().status == FrameMetadata::FrameCancelled) { - info->metadataProcessed = true; - tryCompleteRequest(info); + request->metadataProcessed = true; + tryCompleteRequest(request); return; } if (data->frame_ <= buffer->metadata().sequence) data->frame_ = buffer->metadata().sequence + 1; - data->ipa_->processStatsBuffer(info->frame, info->statBuffer->cookie(), + data->ipa_->processStatsBuffer(request->frame, request->statBuffer->cookie(), data->delayedCtrls_->get(buffer->metadata().sequence)); } From patchwork Mon Mar 11 12:32:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19659 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6BADFC32C3 for ; Mon, 11 Mar 2024 12:32:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 33C3E62C80; Mon, 11 Mar 2024 13:32:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kXLaAOa+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7592562C84 for ; Mon, 11 Mar 2024 13:32:45 +0100 (CET) Received: from localhost.localdomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4F03AF0A; Mon, 11 Mar 2024 13:32:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1710160344; bh=smJtlzff6rYQgdOz9xjdVA74F+C733Ls/z/j3MGeREA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kXLaAOa+GXbX1djUtSf+0ZL3w14hmMUe82Uv50t9QOaKei3glQOJ2YlT0094KWKUY WpvG1aXIRNFfoY/Yvv47S26fw8ZLkEKwvKTudi1VXLj4zGrippcKCpdmL6tzU19dog oRXYOXuRrzT12XNWcRhTlWz06L6zJHX8/tHivsjM= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Subject: [PATCH v2 3/4] libcamera: ipu3: Replace IPU3FrameInfo Date: Mon, 11 Mar 2024 13:32:31 +0100 Message-ID: <20240311123234.32925-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240311123234.32925-1-jacopo.mondi@ideasonboard.com> References: <20240311123234.32925-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the pipeline handler can create a private derived class of Request::Private, use it to store the pipeline specific data. In the case of IPU3 we associate statistics and paramters buffer with a Request and a raw buffer which gets dequeued from the CIO2 and input to the ImgU input. This replaces the functionalities of IPU3FrameInfo which can now be removed. Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally Tested-by: Daniel Scally Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/ipu3/frames.cpp | 143 ---------------- src/libcamera/pipeline/ipu3/frames.h | 67 -------- src/libcamera/pipeline/ipu3/ipu3.cpp | 212 +++++++++++++++--------- src/libcamera/pipeline/ipu3/meson.build | 1 - 4 files changed, 137 insertions(+), 286 deletions(-) delete mode 100644 src/libcamera/pipeline/ipu3/frames.cpp delete mode 100644 src/libcamera/pipeline/ipu3/frames.h diff --git a/src/libcamera/pipeline/ipu3/frames.cpp b/src/libcamera/pipeline/ipu3/frames.cpp deleted file mode 100644 index a4c3477cd9ef..000000000000 --- a/src/libcamera/pipeline/ipu3/frames.cpp +++ /dev/null @@ -1,143 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2020, Google Inc. - * - * frames.cpp - Intel IPU3 Frames helper - */ - -#include "frames.h" - -#include -#include - -#include "libcamera/internal/framebuffer.h" -#include "libcamera/internal/pipeline_handler.h" -#include "libcamera/internal/v4l2_videodevice.h" - -namespace libcamera { - -LOG_DECLARE_CATEGORY(IPU3) - -IPU3Frames::IPU3Frames() -{ -} - -void IPU3Frames::init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers) -{ - for (const std::unique_ptr &buffer : paramBuffers) - availableParamBuffers_.push(buffer.get()); - - for (const std::unique_ptr &buffer : statBuffers) - availableStatBuffers_.push(buffer.get()); - - frameInfo_.clear(); -} - -void IPU3Frames::clear() -{ - availableParamBuffers_ = {}; - availableStatBuffers_ = {}; -} - -IPU3Frames::Info *IPU3Frames::create(Request *request) -{ - unsigned int id = request->sequence(); - - if (availableParamBuffers_.empty()) { - LOG(IPU3, Debug) << "Parameters buffer underrun"; - return nullptr; - } - - if (availableStatBuffers_.empty()) { - LOG(IPU3, Debug) << "Statistics buffer underrun"; - return nullptr; - } - - FrameBuffer *paramBuffer = availableParamBuffers_.front(); - FrameBuffer *statBuffer = availableStatBuffers_.front(); - - paramBuffer->_d()->setRequest(request); - statBuffer->_d()->setRequest(request); - - availableParamBuffers_.pop(); - availableStatBuffers_.pop(); - - /* \todo Remove the dynamic allocation of Info */ - std::unique_ptr info = std::make_unique(); - - info->id = id; - info->request = request; - info->rawBuffer = nullptr; - info->paramBuffer = paramBuffer; - info->statBuffer = statBuffer; - info->paramDequeued = false; - info->metadataProcessed = false; - - frameInfo_[id] = std::move(info); - - return frameInfo_[id].get(); -} - -void IPU3Frames::remove(IPU3Frames::Info *info) -{ - /* Return params and stat buffer for reuse. */ - availableParamBuffers_.push(info->paramBuffer); - availableStatBuffers_.push(info->statBuffer); - - /* Delete the extended frame information. */ - frameInfo_.erase(info->id); -} - -bool IPU3Frames::tryComplete(IPU3Frames::Info *info) -{ - Request *request = info->request; - - if (request->hasPendingBuffers()) - return false; - - if (!info->metadataProcessed) - return false; - - if (!info->paramDequeued) - return false; - - remove(info); - - bufferAvailable.emit(); - - return true; -} - -IPU3Frames::Info *IPU3Frames::find(unsigned int id) -{ - const auto &itInfo = frameInfo_.find(id); - - if (itInfo != frameInfo_.end()) - return itInfo->second.get(); - - LOG(IPU3, Fatal) << "Can't find tracking information for frame " << id; - - return nullptr; -} - -IPU3Frames::Info *IPU3Frames::find(FrameBuffer *buffer) -{ - for (auto const &itInfo : frameInfo_) { - Info *info = itInfo.second.get(); - - for (auto const itBuffers : info->request->buffers()) - if (itBuffers.second == buffer) - return info; - - if (info->rawBuffer == buffer || info->paramBuffer == buffer || - info->statBuffer == buffer) - return info; - } - - LOG(IPU3, Fatal) << "Can't find tracking information from buffer"; - - return nullptr; -} - -} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/frames.h b/src/libcamera/pipeline/ipu3/frames.h deleted file mode 100644 index 6e3cb915c7b8..000000000000 --- a/src/libcamera/pipeline/ipu3/frames.h +++ /dev/null @@ -1,67 +0,0 @@ -/* SPDX-License-Identifier: LGPL-2.1-or-later */ -/* - * Copyright (C) 2020, Google Inc. - * - * frames.h - Intel IPU3 Frames helper - */ - -#pragma once - -#include -#include -#include -#include - -#include - -#include - -namespace libcamera { - -class FrameBuffer; -class IPAProxy; -class PipelineHandler; -class Request; -class V4L2VideoDevice; -struct IPABuffer; - -class IPU3Frames -{ -public: - struct Info { - unsigned int id; - Request *request; - - FrameBuffer *rawBuffer; - FrameBuffer *paramBuffer; - FrameBuffer *statBuffer; - - ControlList effectiveSensorControls; - - bool paramDequeued; - bool metadataProcessed; - }; - - IPU3Frames(); - - void init(const std::vector> ¶mBuffers, - const std::vector> &statBuffers); - void clear(); - - Info *create(Request *request); - void remove(Info *info); - bool tryComplete(Info *info); - - Info *find(unsigned int id); - Info *find(FrameBuffer *buffer); - - Signal<> bufferAvailable; - -private: - std::queue availableParamBuffers_; - std::queue availableStatBuffers_; - - std::map> frameInfo_; -}; - -} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index fa4bd0bb73e2..0c9d3167d2e6 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -34,9 +34,9 @@ #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" #include "cio2.h" -#include "frames.h" #include "imgu.h" namespace libcamera { @@ -47,6 +47,24 @@ static const ControlInfoMap::Map IPU3Controls = { { &controls::draft::PipelineDepth, ControlInfo(2, 3) }, }; +class IPU3Request : public Request::Private +{ +public: + IPU3Request(Camera *camera) + : Request::Private(camera) + { + } + + FrameBuffer *rawBuffer; + FrameBuffer *paramBuffer; + FrameBuffer *statBuffer; + + ControlList effectiveSensorControls; + + bool paramDequeued; + bool metadataProcessed; +}; + class IPU3CameraData : public Camera::Private { public: @@ -57,6 +75,7 @@ public: int loadIPA(); + void tryCompleteRequest(IPU3Request *request); void imguOutputBufferReady(FrameBuffer *buffer); void cio2BufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); @@ -75,7 +94,6 @@ public: Rectangle cropRegion_; std::unique_ptr delayedCtrls_; - IPU3Frames frameInfos_; std::unique_ptr ipa_; @@ -86,7 +104,17 @@ public: ControlInfoMap ipaControls_; + std::map requestMap_; + + std::queue availableParamBuffers_; + std::queue availableStatBuffers_; + private: + IPU3Request *cameraRequest(Request *request) + { + return static_cast(request->_d()); + } + void metadataReady(unsigned int id, const ControlList &metadata); void paramsBufferReady(unsigned int id); void setSensorControls(unsigned int id, const ControlList &sensorControls, @@ -144,6 +172,8 @@ public: int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; + std::unique_ptr createRequestDevice(Camera *camera, + uint64_t cookie) override; int queueRequestDevice(Camera *camera, Request *request) override; bool match(DeviceEnumerator *enumerator) override; @@ -680,19 +710,17 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) for (const std::unique_ptr &buffer : imgu->paramBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); + data->availableParamBuffers_.push(buffer.get()); } for (const std::unique_ptr &buffer : imgu->statBuffers_) { buffer->setCookie(ipaBufferId++); ipaBuffers_.emplace_back(buffer->cookie(), buffer->planes()); + data->availableStatBuffers_.push(buffer.get()); } data->ipa_->mapBuffers(ipaBuffers_); - data->frameInfos_.init(imgu->paramBuffers_, imgu->statBuffers_); - data->frameInfos_.bufferAvailable.connect( - data, &IPU3CameraData::queuePendingRequests); - return 0; } @@ -700,8 +728,6 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) { IPU3CameraData *data = cameraData(camera); - data->frameInfos_.clear(); - std::vector ids; for (IPABuffer &ipabuf : ipaBuffers_) ids.push_back(ipabuf.id); @@ -777,6 +803,7 @@ void PipelineHandlerIPU3::stopDevice(Camera *camera) LOG(IPU3, Warning) << "Failed to stop camera " << camera->id(); freeBuffers(camera); + data->requestMap_.clear(); } void IPU3CameraData::cancelPendingRequests() @@ -801,37 +828,76 @@ void IPU3CameraData::queuePendingRequests() { while (!pendingRequests_.empty()) { Request *request = pendingRequests_.front(); + IPU3Request *ipu3Request = cameraRequest(request); - IPU3Frames::Info *info = frameInfos_.create(request); - if (!info) - break; + requestMap_[request->sequence()] = ipu3Request; /* * Queue a buffer on the CIO2, using the raw stream buffer * provided in the request, if any, or a CIO2 internal buffer * otherwise. + * + * No need to set a buffer->setRequest() as + * a) If the buffer is provided by the application that's + * already been done + * b) If the buffer comes from the CIO2 internal pool, + * CIO2Device::queueBuffer() does that for us */ FrameBuffer *reqRawBuffer = request->findBuffer(&rawStream_); FrameBuffer *rawBuffer = cio2_.queueBuffer(request, reqRawBuffer); + /* * \todo If queueBuffer fails in queuing a buffer to the device, * report the request as error by cancelling the request and * calling PipelineHandler::completeRequest(). */ - if (!rawBuffer) { - frameInfos_.remove(info); + if (!rawBuffer) + break; + + /* + * Store the raw buffer queued to the CIO2 to queue it to the + * Imgu input when the IPA has prepared the parameters buffer. + */ + ipu3Request->rawBuffer = rawBuffer; + + /* + * Prepare the stats and parameters buffer and associate them + * with the currently queued request. + */ + if (availableParamBuffers_.empty()) { + LOG(IPU3, Warning) << "Parameters buffer underrun"; break; } - info->rawBuffer = rawBuffer; + if (availableStatBuffers_.empty()) { + LOG(IPU3, Warning) << "Statistics buffer underrun"; + break; + } - ipa_->queueRequest(info->id, request->controls()); + ipu3Request->paramBuffer = availableParamBuffers_.front(); + ipu3Request->paramDequeued = false; + ipu3Request->paramBuffer->_d()->setRequest(request); + availableParamBuffers_.pop(); + + ipu3Request->statBuffer = availableStatBuffers_.front(); + ipu3Request->metadataProcessed = false; + ipu3Request->statBuffer->_d()->setRequest(request); + availableStatBuffers_.pop(); + + ipa_->queueRequest(request->sequence(), request->controls()); pendingRequests_.pop(); processingRequests_.push(request); } } +std::unique_ptr PipelineHandlerIPU3::createRequestDevice(Camera *camera, + uint64_t cookie) +{ + auto request = std::make_unique(camera); + return Request::create(std::move(request), cookie); +} + int PipelineHandlerIPU3::queueRequestDevice(Camera *camera, Request *request) { IPU3CameraData *data = cameraData(camera); @@ -1200,6 +1266,26 @@ int IPU3CameraData::loadIPA() return 0; } +void IPU3CameraData::tryCompleteRequest(IPU3Request *request) +{ + if (request->hasPendingBuffers()) + return; + + if (!request->metadataProcessed) + return; + + if (!request->paramDequeued) + return; + + availableParamBuffers_.push(request->paramBuffer); + availableStatBuffers_.push(request->statBuffer); + + pipe()->completeRequest(request->_o()); + + /* Try queue another request now that this one has completed. */ + queuePendingRequests(); +} + void IPU3CameraData::setSensorControls([[maybe_unused]] unsigned int id, const ControlList &sensorControls, const ControlList &lensControls) @@ -1220,12 +1306,10 @@ void IPU3CameraData::setSensorControls([[maybe_unused]] unsigned int id, void IPU3CameraData::paramsBufferReady(unsigned int id) { - IPU3Frames::Info *info = frameInfos_.find(id); - if (!info) - return; + IPU3Request *request = requestMap_[id]; /* Queue all buffers from the request aimed for the ImgU. */ - for (auto it : info->request->buffers()) { + for (auto it : request->_o()->buffers()) { const Stream *stream = it.first; FrameBuffer *outbuffer = it.second; @@ -1235,25 +1319,21 @@ void IPU3CameraData::paramsBufferReady(unsigned int id) imgu_->viewfinder_->queueBuffer(outbuffer); } - info->paramBuffer->_d()->metadata().planes()[0].bytesused = + request->paramBuffer->_d()->metadata().planes()[0].bytesused = sizeof(struct ipu3_uapi_params); - imgu_->param_->queueBuffer(info->paramBuffer); - imgu_->stat_->queueBuffer(info->statBuffer); - imgu_->input_->queueBuffer(info->rawBuffer); + imgu_->param_->queueBuffer(request->paramBuffer); + imgu_->stat_->queueBuffer(request->statBuffer); + imgu_->input_->queueBuffer(request->rawBuffer); } void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) { - IPU3Frames::Info *info = frameInfos_.find(id); - if (!info) - return; + IPU3Request *request = requestMap_[id]; - Request *request = info->request; - request->metadata().merge(metadata); + request->_o()->metadata().merge(metadata); - info->metadataProcessed = true; - if (frameInfos_.tryComplete(info)) - pipe()->completeRequest(request); + request->metadataProcessed = true; + tryCompleteRequest(request); } /* ----------------------------------------------------------------------------- @@ -1268,11 +1348,7 @@ void IPU3CameraData::metadataReady(unsigned int id, const ControlList &metadata) */ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) { - IPU3Frames::Info *info = frameInfos_.find(buffer); - if (!info) - return; - - Request *request = info->request; + Request *request = buffer->request(); pipe()->completeBuffer(request, buffer); @@ -1283,8 +1359,7 @@ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) cropRegion_ = *scalerCrop; request->metadata().set(controls::ScalerCrop, cropRegion_); - if (frameInfos_.tryComplete(info)) - pipe()->completeRequest(request); + tryCompleteRequest(cameraRequest(request)); } /** @@ -1296,22 +1371,20 @@ void IPU3CameraData::imguOutputBufferReady(FrameBuffer *buffer) */ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) { - IPU3Frames::Info *info = frameInfos_.find(buffer); - if (!info) - return; - - Request *request = info->request; + IPU3Request *request = cameraRequest(buffer->request()); /* If the buffer is cancelled force a complete of the whole request. */ if (buffer->metadata().status == FrameMetadata::FrameCancelled) { - for (auto it : request->buffers()) { + for (auto it : request->_o()->buffers()) { FrameBuffer *b = it.second; b->_d()->cancel(); - pipe()->completeBuffer(request, b); + pipe()->completeBuffer(request->_o(), b); } - frameInfos_.remove(info); - pipe()->completeRequest(request); + availableParamBuffers_.push(request->paramBuffer); + availableStatBuffers_.push(request->statBuffer); + + pipe()->completeRequest(request->_o()); return; } @@ -1321,24 +1394,23 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal. */ - request->metadata().set(controls::SensorTimestamp, - buffer->metadata().timestamp); + request->_o()->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); - info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); + request->effectiveSensorControls = + delayedCtrls_->get(buffer->metadata().sequence); - if (request->findBuffer(&rawStream_)) - pipe()->completeBuffer(request, buffer); + if (request->_o()->findBuffer(&rawStream_)) + pipe()->completeBuffer(request->_o(), buffer); - ipa_->fillParamsBuffer(info->id, info->paramBuffer->cookie()); + ipa_->fillParamsBuffer(request->_o()->sequence(), + request->paramBuffer->cookie()); } void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) { - IPU3Frames::Info *info = frameInfos_.find(buffer); - if (!info) - return; - - info->paramDequeued = true; + IPU3Request *request = cameraRequest(buffer->request()); + request->paramDequeued = true; /* * tryComplete() will delete info if it completes the IPU3Frame. @@ -1346,35 +1418,25 @@ void IPU3CameraData::paramBufferReady(FrameBuffer *buffer) * * \todo Improve the FrameInfo API to avoid this type of issue */ - Request *request = info->request; - if (frameInfos_.tryComplete(info)) - pipe()->completeRequest(request); + tryCompleteRequest(request); } void IPU3CameraData::statBufferReady(FrameBuffer *buffer) { - IPU3Frames::Info *info = frameInfos_.find(buffer); - if (!info) - return; - - Request *request = info->request; + IPU3Request *request = cameraRequest(buffer->request()); if (buffer->metadata().status == FrameMetadata::FrameCancelled) { - info->metadataProcessed = true; + request->metadataProcessed = true; - /* - * tryComplete() will delete info if it completes the IPU3Frame. - * In that event, we must have obtained the Request before hand. - */ - if (frameInfos_.tryComplete(info)) - pipe()->completeRequest(request); + tryCompleteRequest(request); return; } - ipa_->processStatsBuffer(info->id, request->metadata().get(controls::SensorTimestamp).value_or(0), - info->statBuffer->cookie(), info->effectiveSensorControls); + ipa_->processStatsBuffer(request->_o()->sequence(), + request->_o()->metadata().get(controls::SensorTimestamp).value_or(0), + request->statBuffer->cookie(), request->effectiveSensorControls); } /* diff --git a/src/libcamera/pipeline/ipu3/meson.build b/src/libcamera/pipeline/ipu3/meson.build index a1b0b31ac5bc..d60e07ae6cca 100644 --- a/src/libcamera/pipeline/ipu3/meson.build +++ b/src/libcamera/pipeline/ipu3/meson.build @@ -2,7 +2,6 @@ libcamera_sources += files([ 'cio2.cpp', - 'frames.cpp', 'imgu.cpp', 'ipu3.cpp', ]) From patchwork Mon Mar 11 12:32:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19660 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CDB0BBD1F1 for ; Mon, 11 Mar 2024 12:32:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E0F4B62C8D; Mon, 11 Mar 2024 13:32:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AG787P3l"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E136462C8B for ; Mon, 11 Mar 2024 13:32:45 +0100 (CET) Received: from localhost.localdomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9E9C2C85; Mon, 11 Mar 2024 13:32:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1710160344; bh=sF0DrTYu2Cc+ovujEymNlCdkBtQrKRo8jyD5vbstFlw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AG787P3l+lA6h/FaEp05MA5FjR6ptvpYruB3Mr3tZ1js6xAwi4eecilhl/dWzjS/j cBxEUAbSQ7cIvQnmJN8ymdOehjNBuYokLAkVexuuevHvmbCvbX4oH9dBS57lQftBPv HU3pcw0cGXFT+2Qt1002Uxriu40FvPIxd1pQOzE8= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Subject: [PATCH v2 4/4] libcamera: ipu3: Return Raw buffers on error Date: Mon, 11 Mar 2024 13:32:32 +0100 Message-ID: <20240311123234.32925-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240311123234.32925-1-jacopo.mondi@ideasonboard.com> References: <20240311123234.32925-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a Request is cancelled, all its application-provided buffers are completed and the paramters and stats buffer are made available again to the pipeline handler. If the RAW buffer doesn't come from the application (no RAW stream requested) it comes from the CIO2Device internal pool. Return it to the CIO2 pool in case of error. Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/ipu3/ipu3.cpp | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 0c9d3167d2e6..57d0d6ed4758 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1384,6 +1384,9 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) availableParamBuffers_.push(request->paramBuffer); availableStatBuffers_.push(request->statBuffer); + /* Return RAW buffer to the CIO2. */ + cio2_.tryReturnBuffer(buffer); + pipe()->completeRequest(request->_o()); return; }