From patchwork Mon Sep 14 14:21:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9593 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 663C0BF01C for ; Mon, 14 Sep 2020 14:22:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5C88462DFD; Mon, 14 Sep 2020 16:22:06 +0200 (CEST) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A344F62C8C for ; Mon, 14 Sep 2020 16:22:05 +0200 (CEST) X-Halon-ID: a96c795e-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id a96c795e-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:04 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:37 +0200 Message-Id: <20200914142149.63857-2-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 01/13] libcamera: pipeline: rkisp1: Set number of planes based on format X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The RkISP1 pipeline originally only supported NV formats which have 2 planes. When support for YUV formats was added the plane count on the output format was not made to reflect this. Instead of hard coding the plane count to 2 fetch the number of planes from the format information. Reported-by: Jacopo Mondi Fixes: 2b1a908b5222e263 ("libcamera: camera: Add a validation API to the CameraConfiguration class") Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index a6fc3b8e36f3b00a..009d190d3ec828f0 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -667,10 +667,11 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "Resizer output pad configured with " << format.toString(); + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); V4L2DeviceFormat outputFormat = {}; outputFormat.fourcc = video_->toV4L2PixelFormat(cfg.pixelFormat); outputFormat.size = cfg.size; - outputFormat.planesCount = 2; + outputFormat.planesCount = info.numPlanes(); ret = video_->setFormat(&outputFormat); if (ret) From patchwork Mon Sep 14 14:21:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9594 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A11B0BF01C for ; Mon, 14 Sep 2020 14:22:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7BBAB62E0A; Mon, 14 Sep 2020 16:22:08 +0200 (CEST) Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4956362C8C for ; Mon, 14 Sep 2020 16:22:07 +0200 (CEST) X-Halon-ID: aa2534ad-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id aa2534ad-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:06 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:38 +0200 Message-Id: <20200914142149.63857-3-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 02/13] libcamera: pipeline: rkisp1: Breakout mainpath size and format constraints X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Breakout the mainpath size and format constrains as it will be used in more places then just validate(). While at it use the new helpers to validate Size. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- * Changes since v1 - Use list initializers - Use constexpr instead of static const - Define constants in anonymoys namespace --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 34 +++++++++++++----------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 009d190d3ec828f0..86a3e3c63b17762b 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -37,6 +37,21 @@ namespace libcamera { LOG_DEFINE_CATEGORY(RkISP1) +namespace { + constexpr Size RKISP1_RSZ_MP_SRC_MIN { 32, 16 }; + constexpr Size RKISP1_RSZ_MP_SRC_MAX { 4416, 3312 }; + constexpr std::array RKISP1_RSZ_MP_FORMATS{ + formats::YUYV, + formats::YVYU, + formats::VYUY, + formats::NV16, + formats::NV61, + formats::NV21, + formats::NV12, + /* \todo Add support for 8-bit greyscale to DRM formats */ + }; +} + class PipelineHandlerRkISP1; class RkISP1ActionQueueBuffers; @@ -461,17 +476,6 @@ RkISP1CameraConfiguration::RkISP1CameraConfiguration(Camera *camera, CameraConfiguration::Status RkISP1CameraConfiguration::validate() { - static const std::array formats{ - formats::YUYV, - formats::YVYU, - formats::VYUY, - formats::NV16, - formats::NV61, - formats::NV21, - formats::NV12, - /* \todo Add support for 8-bit greyscale to DRM formats */ - }; - const CameraSensor *sensor = data_->sensor_; Status status = Valid; @@ -487,8 +491,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() StreamConfiguration &cfg = config_[0]; /* Adjust the pixel format. */ - if (std::find(formats.begin(), formats.end(), cfg.pixelFormat) == - formats.end()) { + if (std::find(RKISP1_RSZ_MP_FORMATS.begin(), RKISP1_RSZ_MP_FORMATS.end(), + cfg.pixelFormat) == RKISP1_RSZ_MP_FORMATS.end()) { LOG(RkISP1, Debug) << "Adjusting format to NV12"; cfg.pixelFormat = formats::NV12, status = Adjusted; @@ -525,8 +529,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() / sensorFormat_.size.width; } - cfg.size.width = std::max(32U, std::min(4416U, cfg.size.width)); - cfg.size.height = std::max(16U, std::min(3312U, cfg.size.height)); + cfg.size.boundTo(RKISP1_RSZ_MP_SRC_MAX); + cfg.size.expandTo(RKISP1_RSZ_MP_SRC_MIN); if (cfg.size != size) { LOG(RkISP1, Debug) From patchwork Mon Sep 14 14:21:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9595 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 082BCC3B5D for ; Mon, 14 Sep 2020 14:22:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A3D9362E0D; Mon, 14 Sep 2020 16:22:08 +0200 (CEST) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 02AD562C8C for ; Mon, 14 Sep 2020 16:22:07 +0200 (CEST) X-Halon-ID: ab15874c-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id ab15874c-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:07 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:39 +0200 Message-Id: <20200914142149.63857-4-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 03/13] libcamera: pipeline: rkisp1: Setup links as part of configuration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In preparation of supporting both the main and self path configure all the media graph links as a part of the configuration step. Before this change the link between ISP and DMA engine was setup at match time as the only supported path was the main path and only the link between sensor and ISP was updated at part of the configuration step. The main path is still the only path between ISP and DMA engine that is possible to enable. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- * Changes since v1 - Pass sensor directly to initLinks(). - Remove string variable form link selection. - Rewrite commit message. --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 79 +++++++++++++----------- 1 file changed, 42 insertions(+), 37 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 86a3e3c63b17762b..96046ec32c91cff3 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -213,7 +213,8 @@ private: friend RkISP1CameraData; friend RkISP1Frames; - int initLinks(); + int initLinks(const Camera *camera, const CameraSensor *sensor, + const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); void tryCompleteRequest(Request *request); void bufferReady(FrameBuffer *buffer); @@ -603,28 +604,9 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) CameraSensor *sensor = data->sensor_; int ret; - /* - * Configure the sensor links: enable the link corresponding to this - * camera and disable all the other sensor links. - */ - const MediaPad *pad = isp_->entity()->getPadByIndex(0); - - for (MediaLink *link : pad->links()) { - bool enable = link->source()->entity() == sensor->entity(); - - if (!!(link->flags() & MEDIA_LNK_FL_ENABLED) == enable) - continue; - - LOG(RkISP1, Debug) - << (enable ? "Enabling" : "Disabling") - << " link from sensor '" - << link->source()->entity()->name() - << "' to ISP"; - - ret = link->setEnabled(enable); - if (ret < 0) - return ret; - } + ret = initLinks(camera, sensor, *config); + if (ret) + return ret; /* * Configure the format on the sensor output and propagate it through @@ -927,22 +909,51 @@ int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) * Match and Setup */ -int PipelineHandlerRkISP1::initLinks() +int PipelineHandlerRkISP1::initLinks(const Camera *camera, + const CameraSensor *sensor, + const RkISP1CameraConfiguration &config) { - MediaLink *link; + RkISP1CameraData *data = cameraData(camera); int ret; ret = media_->disableLinks(); if (ret < 0) return ret; - link = media_->link("rkisp1_isp", 2, "rkisp1_resizer_mainpath", 0); - if (!link) - return -ENODEV; + /* + * Configure the sensor links: enable the link corresponding to this + * camera. + */ + const MediaPad *pad = isp_->entity()->getPadByIndex(0); + for (MediaLink *link : pad->links()) { + if (link->source()->entity() != sensor->entity()) + continue; - ret = link->setEnabled(true); - if (ret < 0) - return ret; + LOG(RkISP1, Debug) + << "Enabling link from sensor '" + << link->source()->entity()->name() + << "' to ISP"; + + ret = link->setEnabled(true); + if (ret < 0) + return ret; + } + + for (const StreamConfiguration &cfg : config) { + MediaLink *link = nullptr; + if (cfg.stream() == &data->stream_) + link = media_->link("rkisp1_isp", 2, + "rkisp1_resizer_mainpath", 0); + else + return -EINVAL; + + if (!link) + return -ENODEV; + + ret = link->setEnabled(true); + if (ret < 0) + return ret; + } return 0; } @@ -1024,12 +1035,6 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) stat_->bufferReady.connect(this, &PipelineHandlerRkISP1::statReady); param_->bufferReady.connect(this, &PipelineHandlerRkISP1::paramReady); - /* Configure default links. */ - if (initLinks() < 0) { - LOG(RkISP1, Error) << "Failed to setup links"; - return false; - } - /* * Enumerate all sensors connected to the ISP and create one * camera instance for each of them. From patchwork Mon Sep 14 14:21:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9596 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2D349BF01C for ; Mon, 14 Sep 2020 14:22:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 04CB862E09; Mon, 14 Sep 2020 16:22:11 +0200 (CEST) Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1629D62DEF for ; Mon, 14 Sep 2020 16:22:09 +0200 (CEST) X-Halon-ID: ab8a07e4-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id ab8a07e4-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:08 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:40 +0200 Message-Id: <20200914142149.63857-5-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 04/13] libcamera: pipeline: rkisp1: Prepare buffer ready handlers for multiple streams X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The buffer ready handlers are designed for a single application facing stream from the main path. To prepare for multiple application facing streams from main and/or self path the handlers need to be prepared. The data keeping tasks of frame number and advancing the timeline can be moved from the application facing buffer ready handler to the statistics handler. For each request processed there will always be a statistic buffer and as the ISP is inline and is the source of both main, self and statistic paths there is no lose in precision from this. The application facing handler no longer needs a special case for cancelled frames and can be made simpler. With this change the handlers are ready to deal with any combinations of application facing streams. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 96046ec32c91cff3..2023990013315e3f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1077,20 +1077,8 @@ void PipelineHandlerRkISP1::tryCompleteRequest(Request *request) void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) { ASSERT(activeCamera_); - RkISP1CameraData *data = cameraData(activeCamera_); Request *request = buffer->request(); - if (buffer->metadata().status == FrameMetadata::FrameCancelled) { - completeBuffer(activeCamera_, request, buffer); - completeRequest(activeCamera_, request); - return; - } - - data->timeline_.bufferReady(buffer); - - if (data->frame_ <= buffer->metadata().sequence) - data->frame_ = buffer->metadata().sequence + 1; - completeBuffer(activeCamera_, request, buffer); tryCompleteRequest(request); } @@ -1121,6 +1109,11 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) if (!info) return; + data->timeline_.bufferReady(buffer); + + if (data->frame_ <= buffer->metadata().sequence) + data->frame_ = buffer->metadata().sequence + 1; + IPAOperationData op; op.operation = RKISP1_IPA_EVENT_SIGNAL_STAT_BUFFER; op.data = { info->frame, info->statBuffer->cookie() }; From patchwork Mon Sep 14 14:21:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9597 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 89FBDBF01C for ; Mon, 14 Sep 2020 14:22:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5004162E12; Mon, 14 Sep 2020 16:22:11 +0200 (CEST) Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1112D62C8C for ; Mon, 14 Sep 2020 16:22:10 +0200 (CEST) X-Halon-ID: ac267308-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id ac267308-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:09 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:41 +0200 Message-Id: <20200914142149.63857-6-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 05/13] libcamera: pipeline: rkisp1: Create RkISP1Frames from camera data X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Create RkISP1Frames from camera data instead of picking information out from it. This is done to prepare for multi stream support where more information from the camera data will be needed. Signed-off-by: Niklas Söderlund Reviewed-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- * Changes since v1 - Grammar fixes in commit message. --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2023990013315e3f..e3802c76003268f6 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -54,6 +54,7 @@ namespace { class PipelineHandlerRkISP1; class RkISP1ActionQueueBuffers; +class RkISP1CameraData; enum RkISP1ActionType { SetSensor, @@ -79,7 +80,7 @@ class RkISP1Frames public: RkISP1Frames(PipelineHandler *pipe); - RkISP1FrameInfo *create(unsigned int frame, Request *request, Stream *stream); + RkISP1FrameInfo *create(const RkISP1CameraData *data, Request *request); int destroy(unsigned int frame); void clear(); @@ -244,8 +245,10 @@ RkISP1Frames::RkISP1Frames(PipelineHandler *pipe) { } -RkISP1FrameInfo *RkISP1Frames::create(unsigned int frame, Request *request, Stream *stream) +RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *request) { + unsigned int frame = data->frame_; + if (pipe_->availableParamBuffers_.empty()) { LOG(RkISP1, Error) << "Parameters buffer underrun"; return nullptr; @@ -258,7 +261,7 @@ RkISP1FrameInfo *RkISP1Frames::create(unsigned int frame, Request *request, Stre } FrameBuffer *statBuffer = pipe_->availableStatBuffers_.front(); - FrameBuffer *videoBuffer = request->findBuffer(stream); + FrameBuffer *videoBuffer = request->findBuffer(&data->stream_); if (!videoBuffer) { LOG(RkISP1, Error) << "Attempt to queue request with invalid stream"; @@ -883,10 +886,8 @@ void PipelineHandlerRkISP1::stop(Camera *camera) int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) { RkISP1CameraData *data = cameraData(camera); - Stream *stream = &data->stream_; - RkISP1FrameInfo *info = data->frameInfo_.create(data->frame_, request, - stream); + RkISP1FrameInfo *info = data->frameInfo_.create(data, request); if (!info) return -ENOENT; From patchwork Mon Sep 14 14:21:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9598 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E8B99C3B5D for ; Mon, 14 Sep 2020 14:22:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 884BF62E15; Mon, 14 Sep 2020 16:22:11 +0200 (CEST) Received: from bin-mail-out-06.binero.net (bin-mail-out-06.binero.net [195.74.38.229]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9F6A862DF9 for ; Mon, 14 Sep 2020 16:22:10 +0200 (CEST) X-Halon-ID: acc0ddfd-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id acc0ddfd-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:10 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:42 +0200 Message-Id: <20200914142149.63857-7-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 06/13] libcamera: pipeline: rkisp1: Export stream formats to applications X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The information about stream format is available but not exported to applications, fix this. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- * Changes since v1 - Fix s/applicaitons/applications/ in subject - Cap the max resolution reported to the sensor resolution. --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index e3802c76003268f6..a6cb16c0638f8d85 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -587,7 +587,13 @@ CameraConfiguration *PipelineHandlerRkISP1::generateConfiguration(Camera *camera if (roles.empty()) return config; - StreamConfiguration cfg{}; + std::map> streamFormats; + for (const PixelFormat &format : RKISP1_RSZ_MP_FORMATS) + streamFormats[format] = + { { RKISP1_RSZ_MP_SRC_MIN, data->sensor_->resolution() } }; + + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); cfg.pixelFormat = formats::NV12; cfg.size = data->sensor_->resolution(); From patchwork Mon Sep 14 14:21:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9599 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CE280BF01C for ; Mon, 14 Sep 2020 14:22:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A881262E0D; Mon, 14 Sep 2020 16:22:12 +0200 (CEST) Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 66AE062E08 for ; Mon, 14 Sep 2020 16:22:11 +0200 (CEST) X-Halon-ID: ad1ce7f8-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id ad1ce7f8-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:10 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:43 +0200 Message-Id: <20200914142149.63857-8-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 07/13] libcamera: pipeline: rkisp1: Set the crop rectangle X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Changing resolutions back and forth can provoke the crop rectangle to go out of sync, set it as port of format configuration. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index a6cb16c0638f8d85..45c5e40186df693d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -634,6 +634,11 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret < 0) return ret; + Rectangle rect(0, 0, format.size); + ret = isp_->setSelection(0, V4L2_SEL_TGT_CROP, &rect); + if (ret < 0) + return ret; + LOG(RkISP1, Debug) << "ISP input pad configured with " << format.toString(); /* YUYV8_2X8 is required on the ISP source path pad for YUV output. */ From patchwork Mon Sep 14 14:21:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9600 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 06726BF01C for ; Mon, 14 Sep 2020 14:22:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CE56962E09; Mon, 14 Sep 2020 16:22:14 +0200 (CEST) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 63A4062DFD for ; Mon, 14 Sep 2020 16:22:12 +0200 (CEST) X-Halon-ID: ad8fa69b-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id ad8fa69b-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:11 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:44 +0200 Message-Id: <20200914142149.63857-9-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 08/13] libcamera: pipeline: rkisp1: Prefix main path video and resizer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In preparation of supporting both the main and self path prefix the main path specific variables with mainPath. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 82 ++++++++++++------------ 1 file changed, 41 insertions(+), 41 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 45c5e40186df693d..4e1295486c184178 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -68,7 +68,7 @@ struct RkISP1FrameInfo { FrameBuffer *paramBuffer; FrameBuffer *statBuffer; - FrameBuffer *videoBuffer; + FrameBuffer *mainPathBuffer; bool paramFilled; bool paramDequeued; @@ -133,7 +133,7 @@ class RkISP1CameraData : public CameraData public: RkISP1CameraData(PipelineHandler *pipe, V4L2VideoDevice *video) : CameraData(pipe), sensor_(nullptr), frame_(0), - frameInfo_(pipe), video_(video) + frameInfo_(pipe), mainPathVideo_(video) { } @@ -144,14 +144,14 @@ public: int loadIPA(); - Stream stream_; + Stream mainPathStream_; CameraSensor *sensor_; unsigned int frame_; std::vector ipaBuffers_; RkISP1Frames frameInfo_; RkISP1Timeline timeline_; - V4L2VideoDevice *video_; + V4L2VideoDevice *mainPathVideo_; private: void queueFrameAction(unsigned int frame, @@ -227,8 +227,8 @@ private: MediaDevice *media_; V4L2Subdevice *isp_; - V4L2Subdevice *resizer_; - V4L2VideoDevice *video_; + V4L2Subdevice *mainPathResizer_; + V4L2VideoDevice *mainPathVideo_; V4L2VideoDevice *param_; V4L2VideoDevice *stat_; @@ -261,8 +261,8 @@ RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *req } FrameBuffer *statBuffer = pipe_->availableStatBuffers_.front(); - FrameBuffer *videoBuffer = request->findBuffer(&data->stream_); - if (!videoBuffer) { + FrameBuffer *mainPathBuffer = request->findBuffer(&data->mainPathStream_); + if (!mainPathBuffer) { LOG(RkISP1, Error) << "Attempt to queue request with invalid stream"; return nullptr; @@ -276,7 +276,7 @@ RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *req info->frame = frame; info->request = request; info->paramBuffer = paramBuffer; - info->videoBuffer = videoBuffer; + info->mainPathBuffer = mainPathBuffer; info->statBuffer = statBuffer; info->paramFilled = false; info->paramDequeued = false; @@ -335,7 +335,7 @@ RkISP1FrameInfo *RkISP1Frames::find(FrameBuffer *buffer) if (info->paramBuffer == buffer || info->statBuffer == buffer || - info->videoBuffer == buffer) + info->mainPathBuffer == buffer) return info; } @@ -407,7 +407,7 @@ protected: pipe_->param_->queueBuffer(info->paramBuffer); pipe_->stat_->queueBuffer(info->statBuffer); - pipe_->video_->queueBuffer(info->videoBuffer); + pipe_->mainPathVideo_->queueBuffer(info->mainPathBuffer); } private: @@ -546,10 +546,10 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() cfg.bufferCount = RKISP1_BUFFER_COUNT; V4L2DeviceFormat format = {}; - format.fourcc = data_->video_->toV4L2PixelFormat(cfg.pixelFormat); + format.fourcc = data_->mainPathVideo_->toV4L2PixelFormat(cfg.pixelFormat); format.size = cfg.size; - int ret = data_->video_->tryFormat(&format); + int ret = data_->mainPathVideo_->tryFormat(&format); if (ret) return Invalid; @@ -560,8 +560,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() } PipelineHandlerRkISP1::PipelineHandlerRkISP1(CameraManager *manager) - : PipelineHandler(manager), isp_(nullptr), resizer_(nullptr), - video_(nullptr), param_(nullptr), stat_(nullptr) + : PipelineHandler(manager), isp_(nullptr), mainPathResizer_(nullptr), + mainPathVideo_(nullptr), param_(nullptr), stat_(nullptr) { } @@ -569,8 +569,8 @@ PipelineHandlerRkISP1::~PipelineHandlerRkISP1() { delete param_; delete stat_; - delete video_; - delete resizer_; + delete mainPathVideo_; + delete mainPathResizer_; delete isp_; } @@ -651,7 +651,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "ISP output pad configured with " << format.toString(); - ret = resizer_->setFormat(0, &format); + ret = mainPathResizer_->setFormat(0, &format); if (ret < 0) return ret; @@ -661,7 +661,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "Configuring resizer output pad with " << format.toString(); - ret = resizer_->setFormat(1, &format); + ret = mainPathResizer_->setFormat(1, &format); if (ret < 0) return ret; @@ -669,16 +669,16 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); V4L2DeviceFormat outputFormat = {}; - outputFormat.fourcc = video_->toV4L2PixelFormat(cfg.pixelFormat); + outputFormat.fourcc = mainPathVideo_->toV4L2PixelFormat(cfg.pixelFormat); outputFormat.size = cfg.size; outputFormat.planesCount = info.numPlanes(); - ret = video_->setFormat(&outputFormat); + ret = mainPathVideo_->setFormat(&outputFormat); if (ret) return ret; if (outputFormat.size != cfg.size || - outputFormat.fourcc != video_->toV4L2PixelFormat(cfg.pixelFormat)) { + outputFormat.fourcc != mainPathVideo_->toV4L2PixelFormat(cfg.pixelFormat)) { LOG(RkISP1, Error) << "Unable to configure capture in " << cfg.toString(); return -EINVAL; @@ -696,7 +696,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; - cfg.setStream(&data->stream_); + cfg.setStream(&data->mainPathStream_); return 0; } @@ -705,17 +705,17 @@ int PipelineHandlerRkISP1::exportFrameBuffers([[maybe_unused]] Camera *camera, S std::vector> *buffers) { unsigned int count = stream->configuration().bufferCount; - return video_->exportBuffers(count, buffers); + return mainPathVideo_->exportBuffers(count, buffers); } int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) { RkISP1CameraData *data = cameraData(camera); - unsigned int count = data->stream_.configuration().bufferCount; + unsigned int count = data->mainPathStream_.configuration().bufferCount; unsigned int ipaBufferId = 1; int ret; - ret = video_->importBuffers(count); + ret = mainPathVideo_->importBuffers(count); if (ret < 0) goto error; @@ -748,7 +748,7 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) error: paramBuffers_.clear(); statBuffers_.clear(); - video_->releaseBuffers(); + mainPathVideo_->releaseBuffers(); return ret; } @@ -779,8 +779,8 @@ int PipelineHandlerRkISP1::freeBuffers(Camera *camera) if (stat_->releaseBuffers()) LOG(RkISP1, Error) << "Failed to release stat buffers"; - if (video_->releaseBuffers()) - LOG(RkISP1, Error) << "Failed to release video buffers"; + if (mainPathVideo_->releaseBuffers()) + LOG(RkISP1, Error) << "Failed to release main path buffers"; return 0; } @@ -824,7 +824,7 @@ int PipelineHandlerRkISP1::start(Camera *camera) return ret; } - ret = video_->streamOn(); + ret = mainPathVideo_->streamOn(); if (ret) { param_->streamOff(); stat_->streamOff(); @@ -849,8 +849,8 @@ int PipelineHandlerRkISP1::start(Camera *camera) std::map streamConfig; streamConfig[0] = { - .pixelFormat = data->stream_.configuration().pixelFormat, - .size = data->stream_.configuration().size, + .pixelFormat = data->mainPathStream_.configuration().pixelFormat, + .size = data->mainPathStream_.configuration().size, }; std::map entityControls; @@ -868,7 +868,7 @@ void PipelineHandlerRkISP1::stop(Camera *camera) RkISP1CameraData *data = cameraData(camera); int ret; - ret = video_->streamOff(); + ret = mainPathVideo_->streamOff(); if (ret) LOG(RkISP1, Warning) << "Failed to stop camera " << camera->id(); @@ -953,7 +953,7 @@ int PipelineHandlerRkISP1::initLinks(const Camera *camera, for (const StreamConfiguration &cfg : config) { MediaLink *link = nullptr; - if (cfg.stream() == &data->stream_) + if (cfg.stream() == &data->mainPathStream_) link = media_->link("rkisp1_isp", 2, "rkisp1_resizer_mainpath", 0); else @@ -975,7 +975,7 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) int ret; std::unique_ptr data = - std::make_unique(this, video_); + std::make_unique(this, mainPathVideo_); ControlInfoMap::Map ctrls; ctrls.emplace(std::piecewise_construct, @@ -996,7 +996,7 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) if (ret) return ret; - std::set streams{ &data->stream_ }; + std::set streams{ &data->mainPathStream_ }; std::shared_ptr camera = Camera::create(this, data->sensor_->id(), streams); registerCamera(std::move(camera), std::move(data)); @@ -1026,13 +1026,13 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) if (isp_->open() < 0) return false; - resizer_ = V4L2Subdevice::fromEntityName(media_, "rkisp1_resizer_mainpath"); - if (resizer_->open() < 0) + mainPathResizer_ = V4L2Subdevice::fromEntityName(media_, "rkisp1_resizer_mainpath"); + if (mainPathResizer_->open() < 0) return false; /* Locate and open the capture video node. */ - video_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_mainpath"); - if (video_->open() < 0) + mainPathVideo_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_mainpath"); + if (mainPathVideo_->open() < 0) return false; stat_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_stats"); @@ -1043,7 +1043,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) if (param_->open() < 0) return false; - video_->bufferReady.connect(this, &PipelineHandlerRkISP1::bufferReady); + mainPathVideo_->bufferReady.connect(this, &PipelineHandlerRkISP1::bufferReady); stat_->bufferReady.connect(this, &PipelineHandlerRkISP1::statReady); param_->bufferReady.connect(this, &PipelineHandlerRkISP1::paramReady); From patchwork Mon Sep 14 14:21:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9601 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 89DFCBF01C for ; Mon, 14 Sep 2020 14:22:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 45E0662E08; Mon, 14 Sep 2020 16:22:15 +0200 (CEST) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 292EA62DFD for ; Mon, 14 Sep 2020 16:22:13 +0200 (CEST) X-Halon-ID: ae29b803-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id ae29b803-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:12 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:45 +0200 Message-Id: <20200914142149.63857-10-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 09/13] libcamera: pipeline: rkisp1: Add self path devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the V4L2 device nodes needed to operate the self path. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 26 ++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 4e1295486c184178..5aa01f87afd71183 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -131,9 +131,11 @@ public: class RkISP1CameraData : public CameraData { public: - RkISP1CameraData(PipelineHandler *pipe, V4L2VideoDevice *video) + RkISP1CameraData(PipelineHandler *pipe, V4L2VideoDevice *mainPathVideo, + V4L2VideoDevice *selfPathVideo) : CameraData(pipe), sensor_(nullptr), frame_(0), - frameInfo_(pipe), mainPathVideo_(video) + frameInfo_(pipe), mainPathVideo_(mainPathVideo), + selfPathVideo_(selfPathVideo) { } @@ -152,6 +154,7 @@ public: RkISP1Timeline timeline_; V4L2VideoDevice *mainPathVideo_; + V4L2VideoDevice *selfPathVideo_; private: void queueFrameAction(unsigned int frame, @@ -228,7 +231,9 @@ private: MediaDevice *media_; V4L2Subdevice *isp_; V4L2Subdevice *mainPathResizer_; + V4L2Subdevice *selfPathResizer_; V4L2VideoDevice *mainPathVideo_; + V4L2VideoDevice *selfPathVideo_; V4L2VideoDevice *param_; V4L2VideoDevice *stat_; @@ -561,7 +566,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() PipelineHandlerRkISP1::PipelineHandlerRkISP1(CameraManager *manager) : PipelineHandler(manager), isp_(nullptr), mainPathResizer_(nullptr), - mainPathVideo_(nullptr), param_(nullptr), stat_(nullptr) + selfPathResizer_(nullptr), mainPathVideo_(nullptr), + selfPathVideo_(nullptr), param_(nullptr), stat_(nullptr) { } @@ -570,7 +576,9 @@ PipelineHandlerRkISP1::~PipelineHandlerRkISP1() delete param_; delete stat_; delete mainPathVideo_; + delete selfPathVideo_; delete mainPathResizer_; + delete selfPathResizer_; delete isp_; } @@ -975,7 +983,8 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) int ret; std::unique_ptr data = - std::make_unique(this, mainPathVideo_); + std::make_unique(this, mainPathVideo_, + selfPathVideo_); ControlInfoMap::Map ctrls; ctrls.emplace(std::piecewise_construct, @@ -1030,11 +1039,19 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) if (mainPathResizer_->open() < 0) return false; + selfPathResizer_ = V4L2Subdevice::fromEntityName(media_, "rkisp1_resizer_selfpath"); + if (selfPathResizer_->open() < 0) + return false; + /* Locate and open the capture video node. */ mainPathVideo_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_mainpath"); if (mainPathVideo_->open() < 0) return false; + selfPathVideo_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_selfpath"); + if (selfPathVideo_->open() < 0) + return false; + stat_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_stats"); if (stat_->open() < 0) return false; @@ -1044,6 +1061,7 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) return false; mainPathVideo_->bufferReady.connect(this, &PipelineHandlerRkISP1::bufferReady); + selfPathVideo_->bufferReady.connect(this, &PipelineHandlerRkISP1::bufferReady); stat_->bufferReady.connect(this, &PipelineHandlerRkISP1::statReady); param_->bufferReady.connect(this, &PipelineHandlerRkISP1::paramReady); From patchwork Mon Sep 14 14:21:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9602 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4B637BF01C for ; Mon, 14 Sep 2020 14:22:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 204BF62E0D; Mon, 14 Sep 2020 16:22:17 +0200 (CEST) Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2935162DF9 for ; Mon, 14 Sep 2020 16:22:14 +0200 (CEST) X-Halon-ID: ae99bf3c-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id ae99bf3c-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:13 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:46 +0200 Message-Id: <20200914142149.63857-11-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/13] libcamera: pipeline: rkisp1: Configure self path X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Allow for both the main and self path streams to be configured. This change adds the self path as an internal stream to the pipeline handler. It is not exposed as a Camera stream so it can not yet be used. Signed-off-by: Niklas Söderlund --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 200 ++++++++++++++++------- 1 file changed, 141 insertions(+), 59 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 5aa01f87afd71183..01cd461427e3a6dc 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -135,7 +135,8 @@ public: V4L2VideoDevice *selfPathVideo) : CameraData(pipe), sensor_(nullptr), frame_(0), frameInfo_(pipe), mainPathVideo_(mainPathVideo), - selfPathVideo_(selfPathVideo) + selfPathVideo_(selfPathVideo), mainPathActive_(false), + selfPathActive_(false) { } @@ -147,6 +148,7 @@ public: int loadIPA(); Stream mainPathStream_; + Stream selfPathStream_; CameraSensor *sensor_; unsigned int frame_; std::vector ipaBuffers_; @@ -156,6 +158,9 @@ public: V4L2VideoDevice *mainPathVideo_; V4L2VideoDevice *selfPathVideo_; + bool mainPathActive_; + bool selfPathActive_; + private: void queueFrameAction(unsigned int frame, const IPAOperationData &action); @@ -617,7 +622,6 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) RkISP1CameraConfiguration *config = static_cast(c); RkISP1CameraData *data = cameraData(camera); - StreamConfiguration &cfg = config->at(0); CameraSensor *sensor = data->sensor_; int ret; @@ -659,37 +663,55 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "ISP output pad configured with " << format.toString(); - ret = mainPathResizer_->setFormat(0, &format); - if (ret < 0) - return ret; - - LOG(RkISP1, Debug) << "Resizer input pad configured with " << format.toString(); - - format.size = cfg.size; - - LOG(RkISP1, Debug) << "Configuring resizer output pad with " << format.toString(); - - ret = mainPathResizer_->setFormat(1, &format); - if (ret < 0) - return ret; - - LOG(RkISP1, Debug) << "Resizer output pad configured with " << format.toString(); - - const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); - V4L2DeviceFormat outputFormat = {}; - outputFormat.fourcc = mainPathVideo_->toV4L2PixelFormat(cfg.pixelFormat); - outputFormat.size = cfg.size; - outputFormat.planesCount = info.numPlanes(); - - ret = mainPathVideo_->setFormat(&outputFormat); - if (ret) - return ret; - - if (outputFormat.size != cfg.size || - outputFormat.fourcc != mainPathVideo_->toV4L2PixelFormat(cfg.pixelFormat)) { - LOG(RkISP1, Error) - << "Unable to configure capture in " << cfg.toString(); - return -EINVAL; + data->mainPathActive_ = false; + data->selfPathActive_ = false; + for (const StreamConfiguration &cfg : *config) { + V4L2SubdeviceFormat ispFormat = format; + V4L2Subdevice *resizer; + V4L2VideoDevice *video; + + if (cfg.stream() == &data->mainPathStream_) { + resizer = mainPathResizer_; + video = mainPathVideo_; + data->mainPathActive_ = true; + } else { + resizer = selfPathResizer_; + video = selfPathVideo_; + data->selfPathActive_ = true; + } + + ret = resizer->setFormat(0, &ispFormat); + if (ret < 0) + return ret; + + LOG(RkISP1, Debug) << "Resizer input pad configured with " << ispFormat.toString(); + + ispFormat.size = cfg.size; + + LOG(RkISP1, Debug) << "Configuring resizer output pad with " << ispFormat.toString(); + + ret = resizer->setFormat(1, &ispFormat); + if (ret < 0) + return ret; + + LOG(RkISP1, Debug) << "Resizer output pad configured with " << ispFormat.toString(); + + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + V4L2DeviceFormat outputFormat = {}; + outputFormat.fourcc = video->toV4L2PixelFormat(cfg.pixelFormat); + outputFormat.size = cfg.size; + outputFormat.planesCount = info.numPlanes(); + + ret = video->setFormat(&outputFormat); + if (ret) + return ret; + + if (outputFormat.size != cfg.size || + outputFormat.fourcc != video->toV4L2PixelFormat(cfg.pixelFormat)) { + LOG(RkISP1, Error) + << "Unable to configure capture in " << cfg.toString(); + return -EINVAL; + } } V4L2DeviceFormat paramFormat = {}; @@ -704,28 +726,46 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; - cfg.setStream(&data->mainPathStream_); - return 0; } int PipelineHandlerRkISP1::exportFrameBuffers([[maybe_unused]] Camera *camera, Stream *stream, std::vector> *buffers) { + RkISP1CameraData *data = cameraData(camera); unsigned int count = stream->configuration().bufferCount; - return mainPathVideo_->exportBuffers(count, buffers); + + if (stream == &data->mainPathStream_) + return mainPathVideo_->exportBuffers(count, buffers); + + if (stream == &data->selfPathStream_) + return selfPathVideo_->exportBuffers(count, buffers); + + return -EINVAL; } int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) { RkISP1CameraData *data = cameraData(camera); - unsigned int count = data->mainPathStream_.configuration().bufferCount; unsigned int ipaBufferId = 1; int ret; - ret = mainPathVideo_->importBuffers(count); - if (ret < 0) - goto error; + unsigned int count = std::max({ + data->mainPathStream_.configuration().bufferCount, + data->selfPathStream_.configuration().bufferCount, + }); + + if (data->mainPathActive_) { + ret = mainPathVideo_->importBuffers(count); + if (ret < 0) + goto error; + } + + if (data->selfPathActive_) { + ret = selfPathVideo_->importBuffers(count); + if (ret < 0) + goto error; + } ret = param_->allocateBuffers(count, ¶mBuffers_); if (ret < 0) @@ -757,6 +797,7 @@ error: paramBuffers_.clear(); statBuffers_.clear(); mainPathVideo_->releaseBuffers(); + selfPathVideo_->releaseBuffers(); return ret; } @@ -790,6 +831,9 @@ int PipelineHandlerRkISP1::freeBuffers(Camera *camera) if (mainPathVideo_->releaseBuffers()) LOG(RkISP1, Error) << "Failed to release main path buffers"; + if (selfPathVideo_->releaseBuffers()) + LOG(RkISP1, Error) << "Failed to release self path buffers"; + return 0; } @@ -832,15 +876,47 @@ int PipelineHandlerRkISP1::start(Camera *camera) return ret; } - ret = mainPathVideo_->streamOn(); - if (ret) { - param_->streamOff(); - stat_->streamOff(); - data->ipa_->stop(); - freeBuffers(camera); - - LOG(RkISP1, Error) - << "Failed to start camera " << camera->id(); + std::map streamConfig; + + if (data->mainPathActive_) { + ret = mainPathVideo_->streamOn(); + if (ret) { + param_->streamOff(); + stat_->streamOff(); + data->ipa_->stop(); + freeBuffers(camera); + + LOG(RkISP1, Error) + << "Failed to start main path " << camera->id(); + return ret; + } + + streamConfig[0] = { + .pixelFormat = data->mainPathStream_.configuration().pixelFormat, + .size = data->mainPathStream_.configuration().size, + }; + } + + if (data->selfPathActive_) { + ret = selfPathVideo_->streamOn(); + if (ret) { + if (data->mainPathActive_) + mainPathVideo_->streamOff(); + + param_->streamOff(); + stat_->streamOff(); + data->ipa_->stop(); + freeBuffers(camera); + + LOG(RkISP1, Error) + << "Failed to start self path " << camera->id(); + return ret; + } + + streamConfig[1] = { + .pixelFormat = data->selfPathStream_.configuration().pixelFormat, + .size = data->selfPathStream_.configuration().size, + }; } activeCamera_ = camera; @@ -855,12 +931,6 @@ int PipelineHandlerRkISP1::start(Camera *camera) ret = 0; } - std::map streamConfig; - streamConfig[0] = { - .pixelFormat = data->mainPathStream_.configuration().pixelFormat, - .size = data->mainPathStream_.configuration().size, - }; - std::map entityControls; entityControls.emplace(0, data->sensor_->controls()); @@ -876,10 +946,19 @@ void PipelineHandlerRkISP1::stop(Camera *camera) RkISP1CameraData *data = cameraData(camera); int ret; - ret = mainPathVideo_->streamOff(); - if (ret) - LOG(RkISP1, Warning) - << "Failed to stop camera " << camera->id(); + if (data->selfPathActive_) { + ret = selfPathVideo_->streamOff(); + if (ret) + LOG(RkISP1, Warning) + << "Failed to stop self path " << camera->id(); + } + + if (data->mainPathActive_) { + ret = mainPathVideo_->streamOff(); + if (ret) + LOG(RkISP1, Warning) + << "Failed to stop main path " << camera->id(); + } ret = stat_->streamOff(); if (ret) @@ -964,6 +1043,9 @@ int PipelineHandlerRkISP1::initLinks(const Camera *camera, if (cfg.stream() == &data->mainPathStream_) link = media_->link("rkisp1_isp", 2, "rkisp1_resizer_mainpath", 0); + else if (cfg.stream() == &data->selfPathStream_) + link = media_->link("rkisp1_isp", 2, + "rkisp1_resizer_selfpath", 0); else return -EINVAL; From patchwork Mon Sep 14 14:21:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9603 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 91AE8C3B5D for ; Mon, 14 Sep 2020 14:22:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6237662E1C; Mon, 14 Sep 2020 16:22:17 +0200 (CEST) Received: from bin-mail-out-05.binero.net (bin-mail-out-05.binero.net [195.74.38.228]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BB7F262DF9 for ; Mon, 14 Sep 2020 16:22:14 +0200 (CEST) X-Halon-ID: af352800-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id af352800-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:14 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:47 +0200 Message-Id: <20200914142149.63857-12-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 11/13] libcamera: pipeline: rkisp1: Track buffers for self path X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In preparation of supporting both the main and self path extend RkISP1FrameInfo to track buffers from the self path stream. Signed-off-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 01cd461427e3a6dc..bc961f8e78f2c979 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -69,6 +69,7 @@ struct RkISP1FrameInfo { FrameBuffer *paramBuffer; FrameBuffer *statBuffer; FrameBuffer *mainPathBuffer; + FrameBuffer *selfPathBuffer; bool paramFilled; bool paramDequeued; @@ -272,7 +273,8 @@ RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *req FrameBuffer *statBuffer = pipe_->availableStatBuffers_.front(); FrameBuffer *mainPathBuffer = request->findBuffer(&data->mainPathStream_); - if (!mainPathBuffer) { + FrameBuffer *selfPathBuffer = request->findBuffer(&data->selfPathStream_); + if (!mainPathBuffer && !selfPathBuffer) { LOG(RkISP1, Error) << "Attempt to queue request with invalid stream"; return nullptr; @@ -287,6 +289,7 @@ RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *req info->request = request; info->paramBuffer = paramBuffer; info->mainPathBuffer = mainPathBuffer; + info->selfPathBuffer = selfPathBuffer; info->statBuffer = statBuffer; info->paramFilled = false; info->paramDequeued = false; @@ -345,7 +348,8 @@ RkISP1FrameInfo *RkISP1Frames::find(FrameBuffer *buffer) if (info->paramBuffer == buffer || info->statBuffer == buffer || - info->mainPathBuffer == buffer) + info->mainPathBuffer == buffer || + info->selfPathBuffer == buffer) return info; } @@ -417,7 +421,12 @@ protected: pipe_->param_->queueBuffer(info->paramBuffer); pipe_->stat_->queueBuffer(info->statBuffer); - pipe_->mainPathVideo_->queueBuffer(info->mainPathBuffer); + + if (info->mainPathBuffer) + pipe_->mainPathVideo_->queueBuffer(info->mainPathBuffer); + + if (info->selfPathBuffer) + pipe_->selfPathVideo_->queueBuffer(info->selfPathBuffer); } private: From patchwork Mon Sep 14 14:21:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9604 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E3C33BF01C for ; Mon, 14 Sep 2020 14:22:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BFCD162E1F; Mon, 14 Sep 2020 16:22:18 +0200 (CEST) Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B073D62E1B for ; Mon, 14 Sep 2020 16:22:15 +0200 (CEST) X-Halon-ID: af8e5a68-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id af8e5a68-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:14 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:48 +0200 Message-Id: <20200914142149.63857-13-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 12/13] libcamera: pipeline: rkisp1: Add format validation for self path X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Extend the format validation to work with both man and self path. The heuristics honors that the first stream in the configuration have the highest priority while still examining both streams for a best match. This change extends the formats supported by the Cameras backed by this pipeline to also include RGB888 and RGB656, that is only available on the self path. It is not possible to capture from the self path as the self stream is not yet exposed to applications. Signed-off-by: Niklas Söderlund --- * Changes since v1 - Store formats in std::vector instead of std::array to avoid template usage for validate function. --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 222 +++++++++++++++++------ 1 file changed, 171 insertions(+), 51 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index bc961f8e78f2c979..851ff68f138b98dd 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -9,6 +9,7 @@ #include #include #include +#include #include #include @@ -40,7 +41,7 @@ LOG_DEFINE_CATEGORY(RkISP1) namespace { constexpr Size RKISP1_RSZ_MP_SRC_MIN { 32, 16 }; constexpr Size RKISP1_RSZ_MP_SRC_MAX { 4416, 3312 }; - constexpr std::array RKISP1_RSZ_MP_FORMATS{ + const std::vector RKISP1_RSZ_MP_FORMATS{ formats::YUYV, formats::YVYU, formats::VYUY, @@ -50,7 +51,21 @@ namespace { formats::NV12, /* \todo Add support for 8-bit greyscale to DRM formats */ }; -} + + constexpr Size RKISP1_RSZ_SP_SRC_MIN { 32, 16 }; + constexpr Size RKISP1_RSZ_SP_SRC_MAX { 1920, 1920 }; + const std::vector RKISP1_RSZ_SP_FORMATS{ + formats::YUYV, + formats::YVYU, + formats::VYUY, + formats::NV16, + formats::NV61, + formats::NV21, + formats::NV12, + formats::BGR888, + formats::RGB565, + }; +}; class PipelineHandlerRkISP1; class RkISP1ActionQueueBuffers; @@ -181,13 +196,22 @@ public: private: static constexpr unsigned int RKISP1_BUFFER_COUNT = 4; + bool fitAnyPath(const StreamConfiguration &cfg); + + CameraConfiguration::Status validatePath(StreamConfiguration *cfg, + const std::vector &formats, + const Size &min, const Size &max, + V4L2VideoDevice *video); + CameraConfiguration::Status validateMainPath(StreamConfiguration *cfg); + CameraConfiguration::Status validateSelfPath(StreamConfiguration *cfg); + /* * The RkISP1CameraData instance is guaranteed to be valid as long as the * corresponding Camera instance is valid. In order to borrow a * reference to the camera data, store a new reference to the camera. */ std::shared_ptr camera_; - const RkISP1CameraData *data_; + RkISP1CameraData *data_; V4L2SubdeviceFormat sensorFormat_; }; @@ -497,6 +521,75 @@ RkISP1CameraConfiguration::RkISP1CameraConfiguration(Camera *camera, data_ = data; } +bool RkISP1CameraConfiguration::fitAnyPath(const StreamConfiguration &cfg) +{ + if (std::find(RKISP1_RSZ_MP_FORMATS.begin(), + RKISP1_RSZ_MP_FORMATS.end(), cfg.pixelFormat) == + RKISP1_RSZ_MP_FORMATS.end()) + return false; + + if (cfg.size < RKISP1_RSZ_MP_SRC_MIN || cfg.size > RKISP1_RSZ_MP_SRC_MAX) + return false; + + if (std::find(RKISP1_RSZ_SP_FORMATS.begin(), + RKISP1_RSZ_SP_FORMATS.end(), cfg.pixelFormat) == + RKISP1_RSZ_SP_FORMATS.end()) + return false; + + if (cfg.size < RKISP1_RSZ_SP_SRC_MIN || cfg.size > RKISP1_RSZ_SP_SRC_MAX) + return false; + + return true; +} + +CameraConfiguration::Status RkISP1CameraConfiguration::validatePath( + StreamConfiguration *cfg, const std::vector &formats, + const Size &min, const Size &max, V4L2VideoDevice *video) +{ + const StreamConfiguration reqCfg = *cfg; + Status status = Valid; + + if (std::find(formats.begin(), formats.end(), cfg->pixelFormat) == + formats.end()) + cfg->pixelFormat = formats::NV12; + + cfg->size.boundTo(max); + cfg->size.expandTo(min); + cfg->bufferCount = RKISP1_BUFFER_COUNT; + + V4L2DeviceFormat format = {}; + format.fourcc = video->toV4L2PixelFormat(cfg->pixelFormat); + format.size = cfg->size; + + int ret = video->tryFormat(&format); + if (ret) + return Invalid; + + cfg->stride = format.planes[0].bpl; + cfg->frameSize = format.planes[0].size; + + if (cfg->pixelFormat != reqCfg.pixelFormat || cfg->size != reqCfg.size) { + LOG(RkISP1, Debug) + << "Adjusting format from " << reqCfg.toString() + << " to " << cfg->toString(); + status = Adjusted; + } + + return status; +} + +CameraConfiguration::Status RkISP1CameraConfiguration::validateMainPath(StreamConfiguration *cfg) +{ + return validatePath(cfg, RKISP1_RSZ_MP_FORMATS, RKISP1_RSZ_MP_SRC_MIN, + RKISP1_RSZ_MP_SRC_MAX, data_->mainPathVideo_); +} + +CameraConfiguration::Status RkISP1CameraConfiguration::validateSelfPath(StreamConfiguration *cfg) +{ + return validatePath(cfg, RKISP1_RSZ_SP_FORMATS, RKISP1_RSZ_SP_SRC_MIN, + RKISP1_RSZ_SP_SRC_MAX, data_->selfPathVideo_); +} + CameraConfiguration::Status RkISP1CameraConfiguration::validate() { const CameraSensor *sensor = data_->sensor_; @@ -506,22 +599,86 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() return Invalid; /* Cap the number of entries to the available streams. */ - if (config_.size() > 1) { - config_.resize(1); + if (config_.size() > 2) { + config_.resize(2); status = Adjusted; } - StreamConfiguration &cfg = config_[0]; - - /* Adjust the pixel format. */ - if (std::find(RKISP1_RSZ_MP_FORMATS.begin(), RKISP1_RSZ_MP_FORMATS.end(), - cfg.pixelFormat) == RKISP1_RSZ_MP_FORMATS.end()) { - LOG(RkISP1, Debug) << "Adjusting format to NV12"; - cfg.pixelFormat = formats::NV12, - status = Adjusted; + /* + * If there are more then one stream in the configuration figure out the + * order to evaluate them streams. The first stream have the highest + * priority but if both main path and self path can satisfy it evaluate + * second stream first. + */ + std::vector order(config_.size()); + std::iota(order.begin(), order.end(), 0); + if (config_.size() == 2 && fitAnyPath(config_[0])) + std::reverse(order.begin(), order.end()); + + bool mainPathAvailable = true; + bool selfPathAvailable = true; + for (unsigned int index : order) { + StreamConfiguration &cfg = config_[index]; + + /* Try to match stream without adjusting configuration. */ + if (mainPathAvailable) { + StreamConfiguration tryCfg = cfg; + if (validateMainPath(&tryCfg) == Valid) { + mainPathAvailable = false; + cfg = tryCfg; + cfg.setStream(&data_->mainPathStream_); + LOG(RkISP1, Debug) << "Exact match main"; + continue; + } + } + + if (selfPathAvailable) { + StreamConfiguration tryCfg = cfg; + if (validateSelfPath(&tryCfg) == Valid) { + selfPathAvailable = false; + cfg = tryCfg; + cfg.setStream(&data_->selfPathStream_); + LOG(RkISP1, Debug) << "Exact match self"; + continue; + } + } + + /* Try to match stream allowing adjusting configuration. */ + if (mainPathAvailable) { + StreamConfiguration tryCfg = cfg; + if (validateMainPath(&tryCfg) == Adjusted) { + mainPathAvailable = false; + cfg = tryCfg; + cfg.setStream(&data_->mainPathStream_); + status = Adjusted; + LOG(RkISP1, Debug) << "Adjust match main"; + continue; + } + } + + if (selfPathAvailable) { + StreamConfiguration tryCfg = cfg; + if (validateSelfPath(&tryCfg) == Adjusted) { + selfPathAvailable = false; + cfg = tryCfg; + cfg.setStream(&data_->selfPathStream_); + status = Adjusted; + LOG(RkISP1, Debug) << "Adjust match self"; + continue; + } + } + + /* All paths rejected configuraiton. */ + LOG(RkISP1, Debug) << "Camera configuration not supported " + << cfg.toString(); + return Invalid; } /* Select the sensor format. */ + Size maxSize; + for (const StreamConfiguration &cfg : config_) + maxSize = std::max(maxSize, cfg.size); + sensorFormat_ = sensor->getFormat({ MEDIA_BUS_FMT_SBGGR12_1X12, MEDIA_BUS_FMT_SGBRG12_1X12, MEDIA_BUS_FMT_SGRBG12_1X12, @@ -534,47 +691,10 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() MEDIA_BUS_FMT_SGBRG8_1X8, MEDIA_BUS_FMT_SGRBG8_1X8, MEDIA_BUS_FMT_SRGGB8_1X8 }, - cfg.size); + maxSize); if (sensorFormat_.size.isNull()) sensorFormat_.size = sensor->resolution(); - /* - * Provide a suitable default that matches the sensor aspect - * ratio and clamp the size to the hardware bounds. - * - * \todo: Check the hardware alignment constraints. - */ - const Size size = cfg.size; - - if (cfg.size.isNull()) { - cfg.size.width = 1280; - cfg.size.height = 1280 * sensorFormat_.size.height - / sensorFormat_.size.width; - } - - cfg.size.boundTo(RKISP1_RSZ_MP_SRC_MAX); - cfg.size.expandTo(RKISP1_RSZ_MP_SRC_MIN); - - if (cfg.size != size) { - LOG(RkISP1, Debug) - << "Adjusting size from " << size.toString() - << " to " << cfg.size.toString(); - status = Adjusted; - } - - cfg.bufferCount = RKISP1_BUFFER_COUNT; - - V4L2DeviceFormat format = {}; - format.fourcc = data_->mainPathVideo_->toV4L2PixelFormat(cfg.pixelFormat); - format.size = cfg.size; - - int ret = data_->mainPathVideo_->tryFormat(&format); - if (ret) - return Invalid; - - cfg.stride = format.planes[0].bpl; - cfg.frameSize = format.planes[0].size; - return status; } From patchwork Mon Sep 14 14:21:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 9605 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 35246C3B5D for ; Mon, 14 Sep 2020 14:22:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 07F4162E0B; Mon, 14 Sep 2020 16:22:19 +0200 (CEST) Received: from vsp-unauthed02.binero.net (vsp-unauthed02.binero.net [195.74.38.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6A0C362E0B for ; Mon, 14 Sep 2020 16:22:16 +0200 (CEST) X-Halon-ID: b021a937-f695-11ea-a39b-005056917f90 Authorized-sender: niklas.soderlund@fsdn.se Received: from bismarck.berto.se (p54ac52a8.dip0.t-ipconnect.de [84.172.82.168]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id b021a937-f695-11ea-a39b-005056917f90; Mon, 14 Sep 2020 16:22:15 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: libcamera-devel@lists.libcamera.org Date: Mon, 14 Sep 2020 16:21:49 +0200 Message-Id: <20200914142149.63857-14-niklas.soderlund@ragnatech.se> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> References: <20200914142149.63857-1-niklas.soderlund@ragnatech.se> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 13/13] libcamera: pipeline: rkisp1: Expose self path stream X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Expose the self stream to applications and prefers it for the viewfinder and video roles as it can produce RGB. Keep preferring the main path for still capture as it could be extended to support RAW formats which makes most sens for still capture. With this change the self path becomes available to applications and a camera backed by this pipeline can produce two streams simultaneously. Signed-off-by: Niklas Söderlund --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 44 ++++++++++++++++++------ 1 file changed, 34 insertions(+), 10 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 851ff68f138b98dd..1260ed947d385ca9 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -729,17 +729,38 @@ CameraConfiguration *PipelineHandlerRkISP1::generateConfiguration(Camera *camera if (roles.empty()) return config; - std::map> streamFormats; - for (const PixelFormat &format : RKISP1_RSZ_MP_FORMATS) - streamFormats[format] = - { { RKISP1_RSZ_MP_SRC_MIN, data->sensor_->resolution() } }; + for (const StreamRole role : roles) { + std::map> streamFormats; - StreamFormats formats(streamFormats); - StreamConfiguration cfg(formats); - cfg.pixelFormat = formats::NV12; - cfg.size = data->sensor_->resolution(); + switch (role) { + case StreamRole::StillCapture: { + for (const PixelFormat &format : RKISP1_RSZ_MP_FORMATS) + streamFormats[format] = + { { RKISP1_RSZ_MP_SRC_MIN, data->sensor_->resolution() } }; + break; + } + case StreamRole::Viewfinder: + case StreamRole::VideoRecording: { + for (const PixelFormat &format : RKISP1_RSZ_SP_FORMATS) + streamFormats[format] = + { { RKISP1_RSZ_SP_SRC_MIN, data->sensor_->resolution() } }; + break; + } + default: + LOG(RkISP1, Warning) + << "Requested stream role not supported: " << role; + delete config; + return nullptr; + } - config->addConfiguration(cfg); + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); + cfg.pixelFormat = formats::NV12; + cfg.size = data->sensor_->resolution(); + cfg.bufferCount = 4; + + config->addConfiguration(cfg); + } config->validate(); @@ -1216,7 +1237,10 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) if (ret) return ret; - std::set streams{ &data->mainPathStream_ }; + std::set streams{ + &data->mainPathStream_, + &data->selfPathStream_, + }; std::shared_ptr camera = Camera::create(this, data->sensor_->id(), streams); registerCamera(std::move(camera), std::move(data));