From patchwork Tue Jun 2 01:39:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3890 Return-Path: Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3ABA061012 for ; Tue, 2 Jun 2020 03:39:28 +0200 (CEST) X-Halon-ID: d444b550-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d444b550-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:38:58 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:00 +0200 Message-Id: <20200602013909.3170593-2-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 01/10] libcamera: stream: Initialize stride and bufferCount X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:28 -0000 The two public unsigned integer fields of StreamConfiguration, stride and bufferCount where not initialized, fix this to match other members being initialized to their 'zero' state. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- src/libcamera/stream.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index f34348f2aea022a0..3c4f64daf69d68a4 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -279,7 +279,7 @@ SizeRange StreamFormats::range(const PixelFormat &pixelformat) const * handlers provied StreamFormats. */ StreamConfiguration::StreamConfiguration() - : pixelFormat(0), stream_(nullptr) + : pixelFormat(0), stride(0), bufferCount(0), stream_(nullptr) { } @@ -287,7 +287,8 @@ StreamConfiguration::StreamConfiguration() * \brief Construct a configuration with stream formats */ StreamConfiguration::StreamConfiguration(const StreamFormats &formats) - : pixelFormat(0), stream_(nullptr), formats_(formats) + : pixelFormat(0), stride(0), bufferCount(0), stream_(nullptr), + formats_(formats) { } From patchwork Tue Jun 2 01:39:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3891 Return-Path: Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9044461027 for ; Tue, 2 Jun 2020 03:39:28 +0200 (CEST) X-Halon-ID: d4899f0a-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d4899f0a-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:38:59 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:01 +0200 Message-Id: <20200602013909.3170593-3-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 02/10] libcamera: ipu3: Remove id from camera names X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:28 -0000 The id in the camera name is confusing and is of little use for users. Camera names are not (yet) required to be unique and appending which numerical CIO2 unit the sensor is attached to is just as good as depending on the i2c bus information already present in the entity name. Before this change, $ cam -l Available cameras: 1: ov13858 2-0010 0 2: ov5670 4-0036 1 After this change, $ cam -l Available cameras: 1: ov13858 2-0010 2: ov5670 4-0036 Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/ipu3/ipu3.cpp | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index b805fea71c2d812d..6df1e29281941ebf 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -949,8 +949,7 @@ int PipelineHandlerIPU3::registerCameras() &IPU3CameraData::imguOutputBufferReady); /* Create and register the Camera instance. */ - std::string cameraName = cio2->sensor_->entity()->name() + " " - + std::to_string(id); + std::string cameraName = cio2->sensor_->entity()->name(); std::shared_ptr camera = Camera::create(this, cameraName, streams); From patchwork Tue Jun 2 01:39:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3892 Return-Path: Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0EF8661012 for ; Tue, 2 Jun 2020 03:39:29 +0200 (CEST) X-Halon-ID: d4cf2dd9-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d4cf2dd9-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:38:59 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:02 +0200 Message-Id: <20200602013909.3170593-4-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 03/10] libcamera: ipu3: Fold mediaBusToFormat() into only caller X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:29 -0000 Make the code easier to read and refactor. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/ipu3.cpp | 38 ++++++++++++++-------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 6df1e29281941ebf..f7363244e1d2d0ff 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -133,8 +133,6 @@ public: int start(); int stop(); - static V4L2PixelFormat mediaBusToFormat(unsigned int code); - V4L2VideoDevice *output_; V4L2Subdevice *csi2_; CameraSensor *sensor_; @@ -1504,7 +1502,25 @@ int CIO2Device::configure(const Size &size, if (ret) return ret; - outputFormat->fourcc = mediaBusToFormat(sensorFormat.mbus_code); + V4L2PixelFormat v4l2Format; + switch (sensorFormat.mbus_code) { + case MEDIA_BUS_FMT_SBGGR10_1X10: + v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SBGGR10); + break; + case MEDIA_BUS_FMT_SGBRG10_1X10: + v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGBRG10); + break; + case MEDIA_BUS_FMT_SGRBG10_1X10: + v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGRBG10); + break; + case MEDIA_BUS_FMT_SRGGB10_1X10: + v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SRGGB10); + break; + default: + return -EINVAL; + } + + outputFormat->fourcc = v4l2Format; outputFormat->size = sensorFormat.size; outputFormat->planesCount = 1; @@ -1578,22 +1594,6 @@ int CIO2Device::stop() return output_->streamOff(); } -V4L2PixelFormat CIO2Device::mediaBusToFormat(unsigned int code) -{ - switch (code) { - case MEDIA_BUS_FMT_SBGGR10_1X10: - return V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SBGGR10); - case MEDIA_BUS_FMT_SGBRG10_1X10: - return V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGBRG10); - case MEDIA_BUS_FMT_SGRBG10_1X10: - return V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGRBG10); - case MEDIA_BUS_FMT_SRGGB10_1X10: - return V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SRGGB10); - default: - return {}; - } -} - REGISTER_PIPELINE_HANDLER(PipelineHandlerIPU3); } /* namespace libcamera */ From patchwork Tue Jun 2 01:39:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3893 Return-Path: Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EEE8861012 for ; Tue, 2 Jun 2020 03:39:29 +0200 (CEST) X-Halon-ID: d519e034-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d519e034-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:39:00 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:03 +0200 Message-Id: <20200602013909.3170593-5-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 04/10] libcamera: ipu3: Breakout stream assignment to new function X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:30 -0000 Picking which stream that is most suitable for the requested configuration is mixed with adjusting the requested format when validating configurations. This is hard to read and got worse when support for Bayer formats where added, break it out into a separate function. Signed-off-by: Niklas Söderlund --- src/libcamera/pipeline/ipu3/ipu3.cpp | 87 +++++++++++++++++----------- 1 file changed, 53 insertions(+), 34 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index f7363244e1d2d0ff..0e7555c716b36749 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -190,6 +190,7 @@ private: static constexpr unsigned int IPU3_BUFFER_COUNT = 4; static constexpr unsigned int IPU3_MAX_STREAMS = 3; + int updateStreams(); void adjustStream(StreamConfiguration &cfg, bool scale); /* @@ -256,6 +257,51 @@ IPU3CameraConfiguration::IPU3CameraConfiguration(Camera *camera, data_ = data; } +int IPU3CameraConfiguration::updateStreams() +{ + std::set availableStreams = { + &data_->outStream_, + &data_->vfStream_, + &data_->rawStream_, + }; + + /* Pick the stream most suitable for the requested configuration. */ + std::vector streams; + for (unsigned int i = 0; i < config_.size(); ++i) { + const StreamConfiguration &cfg = config_[i]; + const IPU3Stream *stream; + + /* + * Only the raw stream can support Bayer formats. + */ + if (cfg.pixelFormat.modifier() == IPU3_FORMAT_MOD_PACKED) + stream = &data_->rawStream_; + /* + * Output stream can't scale so can only be used if the size + * matches the size of the sensor. + * + * NOTE: It can crop but is not supported. + */ + else if (cfg.size == sensorFormat_.size) + stream = &data_->outStream_; + /* + * Pick the view finder stream last as it may scale. + */ + else + stream = &data_->vfStream_; + + if (availableStreams.find(stream) == availableStreams.end()) + return -EINVAL; + + streams.push_back(stream); + availableStreams.erase(stream); + } + + streams_ = streams; + + return 0; +} + void IPU3CameraConfiguration::adjustStream(StreamConfiguration &cfg, bool scale) { /* The only pixel format the driver supports is NV12. */ @@ -342,40 +388,16 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() if (!sensorFormat_.size.width || !sensorFormat_.size.height) sensorFormat_.size = sensor->resolution(); - /* - * Verify and update all configuration entries, and assign a stream to - * each of them. The viewfinder stream can scale, while the output - * stream can crop only, so select the output stream when the requested - * resolution is equal to the sensor resolution, and the viewfinder - * stream otherwise. - */ - std::set availableStreams = { - &data_->outStream_, - &data_->vfStream_, - &data_->rawStream_, - }; - streams_.clear(); - streams_.reserve(config_.size()); + /* Assign streams to each configuration entry. */ + if (updateStreams()) + return Invalid; + /* Verify and adjust configuration if needed. */ for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; - const PixelFormat pixelFormat = cfg.pixelFormat; - const Size size = cfg.size; - const IPU3Stream *stream; - - if (cfg.pixelFormat.modifier() == IPU3_FORMAT_MOD_PACKED) - stream = &data_->rawStream_; - else if (cfg.size == sensorFormat_.size) - stream = &data_->outStream_; - else - stream = &data_->vfStream_; - - if (availableStreams.find(stream) == availableStreams.end()) - stream = *availableStreams.begin(); - - LOG(IPU3, Debug) - << "Assigned '" << stream->name_ << "' to stream " << i; + const StreamConfiguration org = cfg; + const IPU3Stream *stream = streams_[i]; if (stream->raw_) { const auto &itFormat = @@ -392,15 +414,12 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() cfg.bufferCount = IPU3_BUFFER_COUNT; - if (cfg.pixelFormat != pixelFormat || cfg.size != size) { + if (cfg.pixelFormat != org.pixelFormat || cfg.size != org.size) { LOG(IPU3, Debug) << "Stream " << i << " configuration adjusted to " << cfg.toString(); status = Adjusted; } - - streams_.push_back(stream); - availableStreams.erase(stream); } return status; From patchwork Tue Jun 2 01:39:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3894 Return-Path: Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6665661080 for ; Tue, 2 Jun 2020 03:39:30 +0200 (CEST) X-Halon-ID: d57809e7-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d57809e7-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:39:00 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:04 +0200 Message-Id: <20200602013909.3170593-6-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 05/10] libcamera: ipu3: Calculate number of buffers for ImgU X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:31 -0000 Decouple the number of buffers to allocate for the ImgU from the number of buffers allocated for the CIO2. Instead of blindly following the CIO2 pick the maximum number of buffers requested for any stream facing applications. This is potentially wasteful, as each stream could allocate just as many buffers as requested by the application instead of the maximum from the set. But this is not more wasteful then whats already used by the pipeline and should be fixed on top after the decoupling of the two processing units. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/ipu3.cpp | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 0e7555c716b36749..f4759715c6ae7572 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -729,14 +729,16 @@ int PipelineHandlerIPU3::allocateBuffers(Camera *camera) IPU3CameraData *data = cameraData(camera); CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; - unsigned int bufferCount; + unsigned int bufferCount = 0; int ret; ret = cio2->allocateBuffers(); if (ret < 0) return ret; - bufferCount = ret; + bufferCount = std::max(data->outStream_.configuration().bufferCount, bufferCount); + bufferCount = std::max(data->vfStream_.configuration().bufferCount, bufferCount); + bufferCount = std::max(data->rawStream_.configuration().bufferCount, bufferCount); ret = imgu->allocateBuffers(data, bufferCount); if (ret < 0) { From patchwork Tue Jun 2 01:39:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3895 Return-Path: Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7BA8861080 for ; Tue, 2 Jun 2020 03:39:31 +0200 (CEST) X-Halon-ID: d5c542fa-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d5c542fa-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:39:01 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:05 +0200 Message-Id: <20200602013909.3170593-7-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 06/10] libcamera: ipu3: cio2: Move the CIO2Device class to separate files X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:33 -0000 In preparation of refactoring the IPU3 pipeline handler breakout the CIO2Device into its own .cpp and .h file, no functional change. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/ipu3/cio2.cpp | 233 ++++++++++++++++++++++ src/libcamera/pipeline/ipu3/cio2.h | 64 ++++++ src/libcamera/pipeline/ipu3/ipu3.cpp | 249 +----------------------- src/libcamera/pipeline/ipu3/meson.build | 1 + 4 files changed, 300 insertions(+), 247 deletions(-) create mode 100644 src/libcamera/pipeline/ipu3/cio2.cpp create mode 100644 src/libcamera/pipeline/ipu3/cio2.h diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp new file mode 100644 index 0000000000000000..113486e3e3d0f2f1 --- /dev/null +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -0,0 +1,233 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * cio2.cpp - Intel IPU3 CIO2 + */ + +#include "cio2.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(IPU3) + +/** + * \brief Initialize components of the CIO2 device with \a index + * \param[in] media The CIO2 media device + * \param[in] index The CIO2 device index + * + * Create and open the video device and subdevices in the CIO2 instance at \a + * index, if a supported image sensor is connected to the CSI-2 receiver of + * this CIO2 instance. Enable the media links connecting the CIO2 components + * to prepare for capture operations and cached the sensor maximum size. + * + * \return 0 on success or a negative error code otherwise + * \retval -ENODEV No supported image sensor is connected to this CIO2 instance + */ +int CIO2Device::init(const MediaDevice *media, unsigned int index) +{ + int ret; + + /* + * Verify that a sensor subdevice is connected to this CIO2 instance + * and enable the media link between the two. + */ + std::string csi2Name = "ipu3-csi2 " + std::to_string(index); + MediaEntity *csi2Entity = media->getEntityByName(csi2Name); + const std::vector &pads = csi2Entity->pads(); + if (pads.empty()) + return -ENODEV; + + /* IPU3 CSI-2 receivers have a single sink pad at index 0. */ + MediaPad *sink = pads[0]; + const std::vector &links = sink->links(); + if (links.empty()) + return -ENODEV; + + MediaLink *link = links[0]; + MediaEntity *sensorEntity = link->source()->entity(); + sensor_ = new CameraSensor(sensorEntity); + ret = sensor_->init(); + if (ret) + return ret; + + ret = link->setEnabled(true); + if (ret) + return ret; + + /* + * Make sure the sensor produces at least one format compatible with + * the CIO2 requirements. + * + * utils::set_overlap requires the ranges to be sorted, keep the + * cio2Codes vector sorted in ascending order. + */ + const std::vector cio2Codes{ MEDIA_BUS_FMT_SBGGR10_1X10, + MEDIA_BUS_FMT_SGRBG10_1X10, + MEDIA_BUS_FMT_SGBRG10_1X10, + MEDIA_BUS_FMT_SRGGB10_1X10 }; + const std::vector &sensorCodes = sensor_->mbusCodes(); + if (!utils::set_overlap(sensorCodes.begin(), sensorCodes.end(), + cio2Codes.begin(), cio2Codes.end())) { + LOG(IPU3, Error) + << "Sensor " << sensor_->entity()->name() + << " has not format compatible with the IPU3"; + return -EINVAL; + } + + /* + * \todo Define when to open and close video device nodes, as they + * might impact on power consumption. + */ + + csi2_ = new V4L2Subdevice(csi2Entity); + ret = csi2_->open(); + if (ret) + return ret; + + std::string cio2Name = "ipu3-cio2 " + std::to_string(index); + output_ = V4L2VideoDevice::fromEntityName(media, cio2Name); + ret = output_->open(); + if (ret) + return ret; + + return 0; +} + +/** + * \brief Configure the CIO2 unit + * \param[in] size The requested CIO2 output frame size + * \param[out] outputFormat The CIO2 unit output image format + * \return 0 on success or a negative error code otherwise + */ +int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) +{ + V4L2SubdeviceFormat sensorFormat; + int ret; + + /* + * Apply the selected format to the sensor, the CSI-2 receiver and + * the CIO2 output device. + */ + sensorFormat = sensor_->getFormat({ MEDIA_BUS_FMT_SBGGR10_1X10, + MEDIA_BUS_FMT_SGBRG10_1X10, + MEDIA_BUS_FMT_SGRBG10_1X10, + MEDIA_BUS_FMT_SRGGB10_1X10 }, + size); + ret = sensor_->setFormat(&sensorFormat); + if (ret) + return ret; + + ret = csi2_->setFormat(0, &sensorFormat); + if (ret) + return ret; + + V4L2PixelFormat v4l2Format; + switch (sensorFormat.mbus_code) { + case MEDIA_BUS_FMT_SBGGR10_1X10: + v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SBGGR10); + break; + case MEDIA_BUS_FMT_SGBRG10_1X10: + v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGBRG10); + break; + case MEDIA_BUS_FMT_SGRBG10_1X10: + v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGRBG10); + break; + case MEDIA_BUS_FMT_SRGGB10_1X10: + v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SRGGB10); + break; + default: + return -EINVAL; + } + + outputFormat->fourcc = v4l2Format; + outputFormat->size = sensorFormat.size; + outputFormat->planesCount = 1; + + ret = output_->setFormat(outputFormat); + if (ret) + return ret; + + LOG(IPU3, Debug) << "CIO2 output format " << outputFormat->toString(); + + return 0; +} + +/** + * \brief Allocate frame buffers for the CIO2 output + * + * Allocate frame buffers in the CIO2 video device to be used to capture frames + * from the CIO2 output. The buffers are stored in the CIO2Device::buffers_ + * vector. + * + * \return Number of buffers allocated or negative error code + */ +int CIO2Device::allocateBuffers() +{ + int ret = output_->allocateBuffers(CIO2_BUFFER_COUNT, &buffers_); + if (ret < 0) + return ret; + + for (std::unique_ptr &buffer : buffers_) + availableBuffers_.push(buffer.get()); + + return ret; +} + +void CIO2Device::freeBuffers() +{ + /* The default std::queue constructor is explicit with gcc 5 and 6. */ + availableBuffers_ = std::queue{}; + + buffers_.clear(); + + if (output_->releaseBuffers()) + LOG(IPU3, Error) << "Failed to release CIO2 buffers"; +} + +FrameBuffer *CIO2Device::getBuffer() +{ + if (availableBuffers_.empty()) { + LOG(IPU3, Error) << "CIO2 buffer underrun"; + return nullptr; + } + + FrameBuffer *buffer = availableBuffers_.front(); + + availableBuffers_.pop(); + + return buffer; +} + +void CIO2Device::putBuffer(FrameBuffer *buffer) +{ + availableBuffers_.push(buffer); +} + +int CIO2Device::start() +{ + return output_->streamOn(); +} + +int CIO2Device::stop() +{ + return output_->streamOff(); +} + +V4L2PixelFormat CIO2Device::mediaBusToFormat(unsigned int code) +{ + switch (code) { + case MEDIA_BUS_FMT_SBGGR10_1X10: + return V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SBGGR10); + case MEDIA_BUS_FMT_SGBRG10_1X10: + return V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGBRG10); + case MEDIA_BUS_FMT_SGRBG10_1X10: + return V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGRBG10); + case MEDIA_BUS_FMT_SRGGB10_1X10: + return V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SRGGB10); + default: + return {}; + } +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h new file mode 100644 index 0000000000000000..d923038bb4ba356f --- /dev/null +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * cio2.h - Intel IPU3 CIO2 + */ +#ifndef __LIBCAMERA_PIPELINE_IPU3_CIO2_H__ +#define __LIBCAMERA_PIPELINE_IPU3_CIO2_H__ + +#include +#include + +#include + +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +class CIO2Device +{ +public: + static constexpr unsigned int CIO2_BUFFER_COUNT = 4; + + CIO2Device() + : output_(nullptr), csi2_(nullptr), sensor_(nullptr) + { + } + + ~CIO2Device() + { + delete output_; + delete csi2_; + delete sensor_; + } + + int init(const MediaDevice *media, unsigned int index); + int configure(const Size &size, V4L2DeviceFormat *outputFormat); + + int allocateBuffers(); + void freeBuffers(); + + FrameBuffer *getBuffer(); + void putBuffer(FrameBuffer *buffer); + + int start(); + int stop(); + + static V4L2PixelFormat mediaBusToFormat(unsigned int code); + + V4L2VideoDevice *output_; + V4L2Subdevice *csi2_; + CameraSensor *sensor_; + +private: + std::vector> buffers_; + std::queue availableBuffers_; +}; + +} /* namespace libcamera */ + +#endif /* __LIBCAMERA_PIPELINE_IPU3_CIO2_H__ */ diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index f4759715c6ae7572..2047deac299dbf75 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -27,6 +27,8 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "cio2.h" + namespace libcamera { LOG_DEFINE_CATEGORY(IPU3) @@ -103,45 +105,6 @@ public: /* \todo Add param video device for 3A tuning */ }; -class CIO2Device -{ -public: - static constexpr unsigned int CIO2_BUFFER_COUNT = 4; - - CIO2Device() - : output_(nullptr), csi2_(nullptr), sensor_(nullptr) - { - } - - ~CIO2Device() - { - delete output_; - delete csi2_; - delete sensor_; - } - - int init(const MediaDevice *media, unsigned int index); - int configure(const Size &size, - V4L2DeviceFormat *outputFormat); - - int allocateBuffers(); - void freeBuffers(); - - FrameBuffer *getBuffer(); - void putBuffer(FrameBuffer *buffer); - - int start(); - int stop(); - - V4L2VideoDevice *output_; - V4L2Subdevice *csi2_; - CameraSensor *sensor_; - -private: - std::vector> buffers_; - std::queue availableBuffers_; -}; - class IPU3Stream : public Stream { public: @@ -1407,214 +1370,6 @@ int ImgUDevice::enableLinks(bool enable) return linkSetup(name_, PAD_STAT, statName, 0, enable); } -/*------------------------------------------------------------------------------ - * CIO2 Device - */ - -/** - * \brief Initialize components of the CIO2 device with \a index - * \param[in] media The CIO2 media device - * \param[in] index The CIO2 device index - * - * Create and open the video device and subdevices in the CIO2 instance at \a - * index, if a supported image sensor is connected to the CSI-2 receiver of - * this CIO2 instance. Enable the media links connecting the CIO2 components - * to prepare for capture operations and cached the sensor maximum size. - * - * \return 0 on success or a negative error code otherwise - * \retval -ENODEV No supported image sensor is connected to this CIO2 instance - */ -int CIO2Device::init(const MediaDevice *media, unsigned int index) -{ - int ret; - - /* - * Verify that a sensor subdevice is connected to this CIO2 instance - * and enable the media link between the two. - */ - std::string csi2Name = "ipu3-csi2 " + std::to_string(index); - MediaEntity *csi2Entity = media->getEntityByName(csi2Name); - const std::vector &pads = csi2Entity->pads(); - if (pads.empty()) - return -ENODEV; - - /* IPU3 CSI-2 receivers have a single sink pad at index 0. */ - MediaPad *sink = pads[0]; - const std::vector &links = sink->links(); - if (links.empty()) - return -ENODEV; - - MediaLink *link = links[0]; - MediaEntity *sensorEntity = link->source()->entity(); - sensor_ = new CameraSensor(sensorEntity); - ret = sensor_->init(); - if (ret) - return ret; - - ret = link->setEnabled(true); - if (ret) - return ret; - - /* - * Make sure the sensor produces at least one format compatible with - * the CIO2 requirements. - * - * utils::set_overlap requires the ranges to be sorted, keep the - * cio2Codes vector sorted in ascending order. - */ - const std::vector cio2Codes{ MEDIA_BUS_FMT_SBGGR10_1X10, - MEDIA_BUS_FMT_SGRBG10_1X10, - MEDIA_BUS_FMT_SGBRG10_1X10, - MEDIA_BUS_FMT_SRGGB10_1X10 }; - const std::vector &sensorCodes = sensor_->mbusCodes(); - if (!utils::set_overlap(sensorCodes.begin(), sensorCodes.end(), - cio2Codes.begin(), cio2Codes.end())) { - LOG(IPU3, Error) - << "Sensor " << sensor_->entity()->name() - << " has not format compatible with the IPU3"; - return -EINVAL; - } - - /* - * \todo Define when to open and close video device nodes, as they - * might impact on power consumption. - */ - - csi2_ = new V4L2Subdevice(csi2Entity); - ret = csi2_->open(); - if (ret) - return ret; - - std::string cio2Name = "ipu3-cio2 " + std::to_string(index); - output_ = V4L2VideoDevice::fromEntityName(media, cio2Name); - ret = output_->open(); - if (ret) - return ret; - - return 0; -} - -/** - * \brief Configure the CIO2 unit - * \param[in] size The requested CIO2 output frame size - * \param[out] outputFormat The CIO2 unit output image format - * \return 0 on success or a negative error code otherwise - */ -int CIO2Device::configure(const Size &size, - V4L2DeviceFormat *outputFormat) -{ - V4L2SubdeviceFormat sensorFormat; - int ret; - - /* - * Apply the selected format to the sensor, the CSI-2 receiver and - * the CIO2 output device. - */ - sensorFormat = sensor_->getFormat({ MEDIA_BUS_FMT_SBGGR10_1X10, - MEDIA_BUS_FMT_SGBRG10_1X10, - MEDIA_BUS_FMT_SGRBG10_1X10, - MEDIA_BUS_FMT_SRGGB10_1X10 }, - size); - ret = sensor_->setFormat(&sensorFormat); - if (ret) - return ret; - - ret = csi2_->setFormat(0, &sensorFormat); - if (ret) - return ret; - - V4L2PixelFormat v4l2Format; - switch (sensorFormat.mbus_code) { - case MEDIA_BUS_FMT_SBGGR10_1X10: - v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SBGGR10); - break; - case MEDIA_BUS_FMT_SGBRG10_1X10: - v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGBRG10); - break; - case MEDIA_BUS_FMT_SGRBG10_1X10: - v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SGRBG10); - break; - case MEDIA_BUS_FMT_SRGGB10_1X10: - v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_IPU3_SRGGB10); - break; - default: - return -EINVAL; - } - - outputFormat->fourcc = v4l2Format; - outputFormat->size = sensorFormat.size; - outputFormat->planesCount = 1; - - ret = output_->setFormat(outputFormat); - if (ret) - return ret; - - LOG(IPU3, Debug) << "CIO2 output format " << outputFormat->toString(); - - return 0; -} - -/** - * \brief Allocate frame buffers for the CIO2 output - * - * Allocate frame buffers in the CIO2 video device to be used to capture frames - * from the CIO2 output. The buffers are stored in the CIO2Device::buffers_ - * vector. - * - * \return Number of buffers allocated or negative error code - */ -int CIO2Device::allocateBuffers() -{ - int ret = output_->allocateBuffers(CIO2_BUFFER_COUNT, &buffers_); - if (ret < 0) - return ret; - - for (std::unique_ptr &buffer : buffers_) - availableBuffers_.push(buffer.get()); - - return ret; -} - -void CIO2Device::freeBuffers() -{ - /* The default std::queue constructor is explicit with gcc 5 and 6. */ - availableBuffers_ = std::queue{}; - - buffers_.clear(); - - if (output_->releaseBuffers()) - LOG(IPU3, Error) << "Failed to release CIO2 buffers"; -} - -FrameBuffer *CIO2Device::getBuffer() -{ - if (availableBuffers_.empty()) { - LOG(IPU3, Error) << "CIO2 buffer underrun"; - return nullptr; - } - - FrameBuffer *buffer = availableBuffers_.front(); - - availableBuffers_.pop(); - - return buffer; -} - -void CIO2Device::putBuffer(FrameBuffer *buffer) -{ - availableBuffers_.push(buffer); -} - -int CIO2Device::start() -{ - return output_->streamOn(); -} - -int CIO2Device::stop() -{ - return output_->streamOff(); -} - REGISTER_PIPELINE_HANDLER(PipelineHandlerIPU3); } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/meson.build b/src/libcamera/pipeline/ipu3/meson.build index 0e8c5a14f2b3317b..b2602d30123f908d 100644 --- a/src/libcamera/pipeline/ipu3/meson.build +++ b/src/libcamera/pipeline/ipu3/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_sources += files([ + 'cio2.cpp', 'ipu3.cpp', ]) From patchwork Tue Jun 2 01:39:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3896 Return-Path: Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4971061130 for ; Tue, 2 Jun 2020 03:39:32 +0200 (CEST) X-Halon-ID: d690d55f-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d690d55f-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:39:02 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:06 +0200 Message-Id: <20200602013909.3170593-8-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 07/10] libcamera: ipu3: cio2: Add function to generate configuration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:33 -0000 Collect the code used to generate configurations for the CIO2 block in the CIO2Device class. This allows for both simplifying the code while extending it's functionality. With this change applications can now switch which Bayer format pattern are used instead being more or less forced to use SBGGR10. Signed-off-by: Niklas Söderlund --- src/libcamera/pipeline/ipu3/cio2.cpp | 44 +++++++++++++++++++++ src/libcamera/pipeline/ipu3/cio2.h | 5 +++ src/libcamera/pipeline/ipu3/ipu3.cpp | 57 ++++++---------------------- 3 files changed, 60 insertions(+), 46 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 113486e3e3d0f2f1..2263d6530ec6b672 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -11,6 +11,13 @@ namespace libcamera { LOG_DECLARE_CATEGORY(IPU3) +static const std::map sensorMbusToPixel = { + { MEDIA_BUS_FMT_SBGGR10_1X10, PixelFormat(DRM_FORMAT_SBGGR10, IPU3_FORMAT_MOD_PACKED) }, + { MEDIA_BUS_FMT_SGBRG10_1X10, PixelFormat(DRM_FORMAT_SGBRG10, IPU3_FORMAT_MOD_PACKED) }, + { MEDIA_BUS_FMT_SGRBG10_1X10, PixelFormat(DRM_FORMAT_SGRBG10, IPU3_FORMAT_MOD_PACKED) }, + { MEDIA_BUS_FMT_SRGGB10_1X10, PixelFormat(DRM_FORMAT_SRGGB10, IPU3_FORMAT_MOD_PACKED) }, +}; + /** * \brief Initialize components of the CIO2 device with \a index * \param[in] media The CIO2 media device @@ -153,6 +160,43 @@ int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) return 0; } +StreamConfiguration +CIO2Device::generateConfiguration(const PixelFormat desiredPixelFormat, + const Size desiredSize) const +{ + StreamConfiguration cfg; + + /* If no desired pixelformat allow all supported.*/ + std::vector mbusCodes = { + MEDIA_BUS_FMT_SBGGR10_1X10, + MEDIA_BUS_FMT_SGBRG10_1X10, + MEDIA_BUS_FMT_SGRBG10_1X10, + MEDIA_BUS_FMT_SRGGB10_1X10 + }; + if (desiredPixelFormat.isValid()) { + for (const auto &iter : sensorMbusToPixel) { + if (iter.second == desiredPixelFormat) { + mbusCodes = { iter.first }; + break; + } + } + } + + /* If no desired size use the sensor resolution. */ + Size size = sensor_->resolution(); + if (desiredSize.width && desiredSize.height) + size = desiredSize; + + /* Query the sensor static information for closest match. */ + V4L2SubdeviceFormat sensorFormat = sensor_->getFormat(mbusCodes, size); + + cfg.size = sensorFormat.size; + cfg.pixelFormat = sensorMbusToPixel.at(sensorFormat.mbus_code); + cfg.bufferCount = CIO2_BUFFER_COUNT; + + return cfg; +} + /** * \brief Allocate frame buffers for the CIO2 output * diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index d923038bb4ba356f..2e268a7154b2d241 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -12,6 +12,8 @@ #include +#include + #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/v4l2_subdevice.h" @@ -39,6 +41,9 @@ public: int init(const MediaDevice *media, unsigned int index); int configure(const Size &size, V4L2DeviceFormat *outputFormat); + StreamConfiguration generateConfiguration(const PixelFormat desiredPixelFormat = {}, + const Size desiredSize = {}) const; + int allocateBuffers(); void freeBuffers(); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 2047deac299dbf75..56cc3ca10414f0d2 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -35,13 +35,6 @@ LOG_DEFINE_CATEGORY(IPU3) class IPU3CameraData; -static const std::map sensorMbusToPixel = { - { MEDIA_BUS_FMT_SBGGR10_1X10, PixelFormat(DRM_FORMAT_SBGGR10, IPU3_FORMAT_MOD_PACKED) }, - { MEDIA_BUS_FMT_SGBRG10_1X10, PixelFormat(DRM_FORMAT_SGBRG10, IPU3_FORMAT_MOD_PACKED) }, - { MEDIA_BUS_FMT_SGRBG10_1X10, PixelFormat(DRM_FORMAT_SGRBG10, IPU3_FORMAT_MOD_PACKED) }, - { MEDIA_BUS_FMT_SRGGB10_1X10, PixelFormat(DRM_FORMAT_SRGGB10, IPU3_FORMAT_MOD_PACKED) }, -}; - class ImgUDevice { public: @@ -146,7 +139,7 @@ public: Status validate() override; - const V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; } + const StreamConfiguration &sensorFormat() const { return sensorFormat_; }; const std::vector &streams() { return streams_; } private: @@ -164,7 +157,7 @@ private: std::shared_ptr camera_; const IPU3CameraData *data_; - V4L2SubdeviceFormat sensorFormat_; + StreamConfiguration sensorFormat_; std::vector streams_; }; @@ -313,7 +306,6 @@ void IPU3CameraConfiguration::adjustStream(StreamConfiguration &cfg, bool scale) CameraConfiguration::Status IPU3CameraConfiguration::validate() { - const CameraSensor *sensor = data_->cio2_.sensor_; Status status = Valid; if (config_.empty()) @@ -325,32 +317,21 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() status = Adjusted; } - /* - * Select the sensor format by collecting the maximum width and height - * and picking the closest larger match, as the IPU3 can downscale - * only. If no resolution is requested for any stream, or if no sensor - * resolution is large enough, pick the largest one. - */ + /* Find largets size and raw format (if any) in the configuration. */ Size size = {}; - + PixelFormat pixelFormat = {}; for (const StreamConfiguration &cfg : config_) { if (cfg.size.width > size.width) size.width = cfg.size.width; if (cfg.size.height > size.height) size.height = cfg.size.height; + + if (cfg.pixelFormat.modifier() == IPU3_FORMAT_MOD_PACKED) + pixelFormat = cfg.pixelFormat; } - if (!size.width || !size.height) - size = sensor->resolution(); - - sensorFormat_ = sensor->getFormat({ MEDIA_BUS_FMT_SBGGR10_1X10, - MEDIA_BUS_FMT_SGBRG10_1X10, - MEDIA_BUS_FMT_SGRBG10_1X10, - MEDIA_BUS_FMT_SRGGB10_1X10 }, - size); - if (!sensorFormat_.size.width || !sensorFormat_.size.height) - sensorFormat_.size = sensor->resolution(); - + /* Generate raw configuration from CIO2. */ + sensorFormat_ = data_->cio2_.generateConfiguration(pixelFormat, size); /* Assign streams to each configuration entry. */ if (updateStreams()) @@ -363,13 +344,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() const IPU3Stream *stream = streams_[i]; if (stream->raw_) { - const auto &itFormat = - sensorMbusToPixel.find(sensorFormat_.mbus_code); - if (itFormat == sensorMbusToPixel.end()) - return Invalid; - - cfg.pixelFormat = itFormat->second; - cfg.size = sensorFormat_.size; + cfg = sensorFormat_; } else { bool scale = stream == &data_->vfStream_; adjustStream(config_[i], scale); @@ -452,17 +427,7 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, } stream = &data->rawStream_; - - cfg.size = data->cio2_.sensor_->resolution(); - - V4L2SubdeviceFormat sensorFormat = - data->cio2_.sensor_->getFormat({ MEDIA_BUS_FMT_SBGGR10_1X10, - MEDIA_BUS_FMT_SGBRG10_1X10, - MEDIA_BUS_FMT_SGRBG10_1X10, - MEDIA_BUS_FMT_SRGGB10_1X10 }, - cfg.size); - cfg.pixelFormat = - sensorMbusToPixel.at(sensorFormat.mbus_code); + cfg = data->cio2_.generateConfiguration(); break; } From patchwork Tue Jun 2 01:39:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3897 Return-Path: Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0DD03610BF for ; Tue, 2 Jun 2020 03:39:33 +0200 (CEST) X-Halon-ID: d70968d6-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d70968d6-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:39:03 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:07 +0200 Message-Id: <20200602013909.3170593-9-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 08/10] libcamera: ipu3: cio2: Make the V4L2 devices private X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:33 -0000 In order to make the CIO2 easier to extend with new features make the V4L2 deices (sensor, CIO2 and video device) private members. This requires a few helper functions to be added to allow for the IPU3 driver to still be able to interact with all parts of the CIO2. These helper functions will later be extended to add new features to the IPU3 pipeline. Signed-off-by: Niklas Söderlund --- src/libcamera/pipeline/ipu3/cio2.cpp | 18 ++++++++++++++++++ src/libcamera/pipeline/ipu3/cio2.h | 19 +++++++++++++++---- src/libcamera/pipeline/ipu3/ipu3.cpp | 16 +++++++--------- 3 files changed, 40 insertions(+), 13 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 2263d6530ec6b672..63a46959f3cac06a 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -98,6 +98,8 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index) if (ret) return ret; + output_->bufferReady.connect(this, &CIO2Device::cio2BufferReady); + return 0; } @@ -218,6 +220,12 @@ int CIO2Device::allocateBuffers() return ret; } +int CIO2Device::exportBuffers(unsigned int count, + std::vector> *buffers) +{ + return output_->exportBuffers(count, buffers); +} + void CIO2Device::freeBuffers() { /* The default std::queue constructor is explicit with gcc 5 and 6. */ @@ -258,6 +266,11 @@ int CIO2Device::stop() return output_->streamOff(); } +int CIO2Device::queueBuffer(FrameBuffer *buffer) +{ + return output_->queueBuffer(buffer); +} + V4L2PixelFormat CIO2Device::mediaBusToFormat(unsigned int code) { switch (code) { @@ -274,4 +287,9 @@ V4L2PixelFormat CIO2Device::mediaBusToFormat(unsigned int code) } } +void CIO2Device::cio2BufferReady(FrameBuffer *buffer) +{ + bufferReady.emit(buffer); +} + } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index 2e268a7154b2d241..465c0778f27e4066 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -27,7 +27,7 @@ public: static constexpr unsigned int CIO2_BUFFER_COUNT = 4; CIO2Device() - : output_(nullptr), csi2_(nullptr), sensor_(nullptr) + : sensor_(nullptr), csi2_(nullptr), output_(nullptr) { } @@ -45,6 +45,8 @@ public: const Size desiredSize = {}) const; int allocateBuffers(); + int exportBuffers(unsigned int count, + std::vector> *buffers); void freeBuffers(); FrameBuffer *getBuffer(); @@ -53,13 +55,22 @@ public: int start(); int stop(); + const std::string &name() const { return sensor_->entity()->name(); } + Size resolution() const { return sensor_->resolution(); } + ControlList properties() const { return sensor_->properties(); } + + int queueBuffer(FrameBuffer *buffer); + Signal bufferReady; + +private: static V4L2PixelFormat mediaBusToFormat(unsigned int code); - V4L2VideoDevice *output_; - V4L2Subdevice *csi2_; + void cio2BufferReady(FrameBuffer *buffer); + CameraSensor *sensor_; + V4L2Subdevice *csi2_; + V4L2VideoDevice *output_; -private: std::vector> buffers_; std::queue availableBuffers_; }; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 56cc3ca10414f0d2..2d636f0944587a17 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -454,7 +454,7 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, * available sensor resolution and to the IPU3 * alignment constraints. */ - const Size &res = data->cio2_.sensor_->resolution(); + const Size &res = data->cio2_.resolution(); unsigned int width = std::min(1280U, res.width); unsigned int height = std::min(720U, res.height); cfg.size = { width & ~7, height & ~3 }; @@ -634,13 +634,11 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, IPU3CameraData *data = cameraData(camera); IPU3Stream *ipu3stream = static_cast(stream); unsigned int count = stream->configuration().bufferCount; - V4L2VideoDevice *video; if (ipu3stream->raw_) - video = data->cio2_.output_; - else - video = ipu3stream->device_->dev; + return data->cio2_.exportBuffers(count, buffers); + V4L2VideoDevice *video = ipu3stream->device_->dev; return video->exportBuffers(count, buffers); } @@ -749,7 +747,7 @@ int PipelineHandlerIPU3::queueRequestDevice(Camera *camera, Request *request) return -EINVAL; buffer->setRequest(request); - data->cio2_.output_->queueBuffer(buffer); + data->cio2_.queueBuffer(buffer); for (auto it : request->buffers()) { IPU3Stream *stream = static_cast(it.first); @@ -862,7 +860,7 @@ int PipelineHandlerIPU3::registerCameras() continue; /* Initialize the camera properties. */ - data->properties_ = cio2->sensor_->properties(); + data->properties_ = cio2->properties(); /** * \todo Dynamically assign ImgU and output devices to each @@ -886,7 +884,7 @@ int PipelineHandlerIPU3::registerCameras() * associated ImgU input where they get processed and * returned through the ImgU main and secondary outputs. */ - data->cio2_.output_->bufferReady.connect(data.get(), + data->cio2_.bufferReady.connect(data.get(), &IPU3CameraData::cio2BufferReady); data->imgu_->input_->bufferReady.connect(data.get(), &IPU3CameraData::imguInputBufferReady); @@ -896,7 +894,7 @@ int PipelineHandlerIPU3::registerCameras() &IPU3CameraData::imguOutputBufferReady); /* Create and register the Camera instance. */ - std::string cameraName = cio2->sensor_->entity()->name(); + std::string cameraName = cio2->name(); std::shared_ptr camera = Camera::create(this, cameraName, streams); From patchwork Tue Jun 2 01:39:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3898 Return-Path: Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BA3316112C for ; Tue, 2 Jun 2020 03:39:33 +0200 (CEST) X-Halon-ID: d77c88ed-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d77c88ed-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:39:04 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:08 +0200 Message-Id: <20200602013909.3170593-10-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 09/10] libcamera: ipu3: cio2: Hide buffer allocation and freeing from users X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:34 -0000 The allocation and freeing of buffers for the CIO2 is handled in the IPU3 pipeline handlers start() and stop() functions. These functions also calls CIO2Device start() and stop() at the appropriate times so move the CIO2 buffer allocation/freeing inside the CIO2Device and reduce the complexity of the exposed interface. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/cio2.cpp | 53 +++++++++++----------------- src/libcamera/pipeline/ipu3/cio2.h | 2 -- src/libcamera/pipeline/ipu3/ipu3.cpp | 10 +----- 3 files changed, 21 insertions(+), 44 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 63a46959f3cac06a..895848d2b3a1fba9 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -199,44 +199,12 @@ CIO2Device::generateConfiguration(const PixelFormat desiredPixelFormat, return cfg; } -/** - * \brief Allocate frame buffers for the CIO2 output - * - * Allocate frame buffers in the CIO2 video device to be used to capture frames - * from the CIO2 output. The buffers are stored in the CIO2Device::buffers_ - * vector. - * - * \return Number of buffers allocated or negative error code - */ -int CIO2Device::allocateBuffers() -{ - int ret = output_->allocateBuffers(CIO2_BUFFER_COUNT, &buffers_); - if (ret < 0) - return ret; - - for (std::unique_ptr &buffer : buffers_) - availableBuffers_.push(buffer.get()); - - return ret; -} - int CIO2Device::exportBuffers(unsigned int count, std::vector> *buffers) { return output_->exportBuffers(count, buffers); } -void CIO2Device::freeBuffers() -{ - /* The default std::queue constructor is explicit with gcc 5 and 6. */ - availableBuffers_ = std::queue{}; - - buffers_.clear(); - - if (output_->releaseBuffers()) - LOG(IPU3, Error) << "Failed to release CIO2 buffers"; -} - FrameBuffer *CIO2Device::getBuffer() { if (availableBuffers_.empty()) { @@ -258,12 +226,31 @@ void CIO2Device::putBuffer(FrameBuffer *buffer) int CIO2Device::start() { + int ret = output_->allocateBuffers(CIO2_BUFFER_COUNT, &buffers_); + if (ret < 0) + return ret; + + for (std::unique_ptr &buffer : buffers_) + availableBuffers_.push(buffer.get()); + return output_->streamOn(); } int CIO2Device::stop() { - return output_->streamOff(); + int ret; + + ret = output_->streamOff(); + + /* The default std::queue constructor is explicit with gcc 5 and 6. */ + availableBuffers_ = std::queue{}; + + buffers_.clear(); + + if (output_->releaseBuffers()) + LOG(IPU3, Error) << "Failed to release CIO2 buffers"; + + return ret; } int CIO2Device::queueBuffer(FrameBuffer *buffer) diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index 465c0778f27e4066..032b91c082889a63 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -44,10 +44,8 @@ public: StreamConfiguration generateConfiguration(const PixelFormat desiredPixelFormat = {}, const Size desiredSize = {}) const; - int allocateBuffers(); int exportBuffers(unsigned int count, std::vector> *buffers); - void freeBuffers(); FrameBuffer *getBuffer(); void putBuffer(FrameBuffer *buffer); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 2d636f0944587a17..636c1c54627b5777 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -653,24 +653,17 @@ int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, int PipelineHandlerIPU3::allocateBuffers(Camera *camera) { IPU3CameraData *data = cameraData(camera); - CIO2Device *cio2 = &data->cio2_; ImgUDevice *imgu = data->imgu_; unsigned int bufferCount = 0; int ret; - ret = cio2->allocateBuffers(); - if (ret < 0) - return ret; - bufferCount = std::max(data->outStream_.configuration().bufferCount, bufferCount); bufferCount = std::max(data->vfStream_.configuration().bufferCount, bufferCount); bufferCount = std::max(data->rawStream_.configuration().bufferCount, bufferCount); ret = imgu->allocateBuffers(data, bufferCount); - if (ret < 0) { - cio2->freeBuffers(); + if (ret < 0) return ret; - } return 0; } @@ -679,7 +672,6 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) { IPU3CameraData *data = cameraData(camera); - data->cio2_.freeBuffers(); data->imgu_->freeBuffers(data); return 0; From patchwork Tue Jun 2 01:39:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 3899 Return-Path: Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 866196113D for ; Tue, 2 Jun 2020 03:39:34 +0200 (CEST) X-Halon-ID: d7e63e55-a471-11ea-a73e-0050569116f7 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (p4fca2392.dip0.t-ipconnect.de [79.202.35.146]) by bin-vsp-out-03.atm.binero.net (Halon) with ESMTPA id d7e63e55-a471-11ea-a73e-0050569116f7; Tue, 02 Jun 2020 03:39:05 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Jun 2020 03:39:09 +0200 Message-Id: <20200602013909.3170593-11-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> References: <20200602013909.3170593-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 10/10] libcamera: ipu3: Allow zero-copy RAW stream capture X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2020 01:39:35 -0000 With the refactored CIO2 interface it's now easy to add zero-copy for buffers in the RAW stream. Use the internally allocated buffers inside the CIO2Device if no buffer for the RAW stream is provided by the application, or use the application provided one if one is. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/cio2.cpp | 52 +++++++++++-------- src/libcamera/pipeline/ipu3/cio2.h | 6 +-- src/libcamera/pipeline/ipu3/ipu3.cpp | 76 +++++++++++----------------- 3 files changed, 62 insertions(+), 72 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 895848d2b3a1fba9..bbf18910c0e7abd4 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -205,31 +205,16 @@ int CIO2Device::exportBuffers(unsigned int count, return output_->exportBuffers(count, buffers); } -FrameBuffer *CIO2Device::getBuffer() -{ - if (availableBuffers_.empty()) { - LOG(IPU3, Error) << "CIO2 buffer underrun"; - return nullptr; - } - - FrameBuffer *buffer = availableBuffers_.front(); - - availableBuffers_.pop(); - - return buffer; -} - -void CIO2Device::putBuffer(FrameBuffer *buffer) -{ - availableBuffers_.push(buffer); -} - int CIO2Device::start() { - int ret = output_->allocateBuffers(CIO2_BUFFER_COUNT, &buffers_); + int ret = output_->exportBuffers(CIO2_BUFFER_COUNT, &buffers_); if (ret < 0) return ret; + ret = output_->importBuffers(CIO2_BUFFER_COUNT); + if (ret) + LOG(IPU3, Error) << "Failed to import CIO2 buffers"; + for (std::unique_ptr &buffer : buffers_) availableBuffers_.push(buffer.get()); @@ -253,11 +238,36 @@ int CIO2Device::stop() return ret; } -int CIO2Device::queueBuffer(FrameBuffer *buffer) +int CIO2Device::queueBuffer(Request *request, FrameBuffer *rawBuffer) { + FrameBuffer *buffer = rawBuffer; + + /* If no buffer provided in request, use an internal one. */ + if (!buffer) { + if (availableBuffers_.empty()) { + LOG(IPU3, Error) << "CIO2 buffer underrun"; + return -EINVAL; + } + + buffer = availableBuffers_.front(); + availableBuffers_.pop(); + } + + buffer->setRequest(request); + return output_->queueBuffer(buffer); } +void CIO2Device::tryReturnBuffer(FrameBuffer *buffer) +{ + for (const std::unique_ptr &buf : buffers_) { + if (buf.get() == buffer) { + availableBuffers_.push(buffer); + break; + } + } +} + V4L2PixelFormat CIO2Device::mediaBusToFormat(unsigned int code) { switch (code) { diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index 032b91c082889a63..1a9b700bf4e7e15c 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -47,9 +47,6 @@ public: int exportBuffers(unsigned int count, std::vector> *buffers); - FrameBuffer *getBuffer(); - void putBuffer(FrameBuffer *buffer); - int start(); int stop(); @@ -57,7 +54,8 @@ public: Size resolution() const { return sensor_->resolution(); } ControlList properties() const { return sensor_->properties(); } - int queueBuffer(FrameBuffer *buffer); + int queueBuffer(Request *request, FrameBuffer *rawBuffer); + void tryReturnBuffer(FrameBuffer *buffer); Signal bufferReady; private: diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 636c1c54627b5777..03e954d0f30d39cc 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -121,7 +121,6 @@ public: } void imguOutputBufferReady(FrameBuffer *buffer); - void imguInputBufferReady(FrameBuffer *buffer); void cio2BufferReady(FrameBuffer *buffer); CIO2Device cio2_; @@ -730,25 +729,31 @@ void PipelineHandlerIPU3::stop(Camera *camera) int PipelineHandlerIPU3::queueRequestDevice(Camera *camera, Request *request) { IPU3CameraData *data = cameraData(camera); - FrameBuffer *buffer; int error = 0; - /* Get a CIO2 buffer, associate it with the request and queue it. */ - buffer = data->cio2_.getBuffer(); - if (!buffer) - return -EINVAL; + /* Search for a RAW buffer in the request, if any. */ + FrameBuffer *rawBuffer = nullptr; + for (auto it : request->buffers()) { + IPU3Stream *stream = static_cast(it.first); + if (!stream->raw_) + continue; - buffer->setRequest(request); - data->cio2_.queueBuffer(buffer); + rawBuffer = it.second; + break; + } + /* Queue a buffer on the CIO2, the buffer may come from the request. */ + error = data->cio2_.queueBuffer(request, rawBuffer); + if (error) + return error; + + /* Queue all buffers from the request aimed for the ImgU. */ for (auto it : request->buffers()) { IPU3Stream *stream = static_cast(it.first); - buffer = it.second; - - /* Skip raw streams, they are copied from the CIO2 buffer. */ if (stream->raw_) continue; + FrameBuffer *buffer = it.second; int ret = stream->device_->dev->queueBuffer(buffer); if (ret < 0) error = ret; @@ -878,8 +883,8 @@ int PipelineHandlerIPU3::registerCameras() */ data->cio2_.bufferReady.connect(data.get(), &IPU3CameraData::cio2BufferReady); - data->imgu_->input_->bufferReady.connect(data.get(), - &IPU3CameraData::imguInputBufferReady); + data->imgu_->input_->bufferReady.connect(&data->cio2_, + &CIO2Device::tryReturnBuffer); data->imgu_->output_.dev->bufferReady.connect(data.get(), &IPU3CameraData::imguOutputBufferReady); data->imgu_->viewfinder_.dev->bufferReady.connect(data.get(), @@ -908,22 +913,6 @@ int PipelineHandlerIPU3::registerCameras() * Buffer Ready slots */ -/** - * \brief Handle buffers completion at the ImgU input - * \param[in] buffer The completed buffer - * - * Buffers completed from the ImgU input are immediately queued back to the - * CIO2 unit to continue frame capture. - */ -void IPU3CameraData::imguInputBufferReady(FrameBuffer *buffer) -{ - /* \todo Handle buffer failures when state is set to BufferError. */ - if (buffer->metadata().status == FrameMetadata::FrameCancelled) - return; - - cio2_.putBuffer(buffer); -} - /** * \brief Handle buffers completion at the ImgU output * \param[in] buffer The completed buffer @@ -956,27 +945,20 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) return; Request *request = buffer->request(); - FrameBuffer *raw = request->findBuffer(&rawStream_); - if (!raw) { - /* No RAW buffers present, just queue to IMGU. */ - imgu_->input_->queueBuffer(buffer); - return; - } - - /* RAW buffers present, special care is needed. */ - if (request->buffers().size() > 1) - imgu_->input_->queueBuffer(buffer); - - if (raw->copyFrom(buffer)) - LOG(IPU3, Debug) << "Copy of FrameBuffer failed"; - - pipe_->completeBuffer(camera_, request, raw); - - if (request->buffers().size() == 1) { - cio2_.putBuffer(buffer); + /* + * If the requests contains a buffer for the RAW stream and the buffer + * completed by the CIO2 satisfy all pending buffers in the request + * complete the request now, there is no need for this request to be + * processed by the ImGU. + */ + if (request->findBuffer(&rawStream_) && + pipe_->completeBuffer(camera_, request, buffer)) { pipe_->completeRequest(camera_, request); + return; } + + imgu_->input_->queueBuffer(buffer); } /* -----------------------------------------------------------------------------