From patchwork Thu Sep 21 16:55:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 19068 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A22FBC326C for ; Thu, 21 Sep 2023 16:56:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A887A62947; Thu, 21 Sep 2023 18:56:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1695315367; bh=qXGx6RyMquUh9GOu6AN6eszqs/7WzI1BqcrEed/s/+k=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=WWkYuKB+16ydkCkm19t9lz+VGlTczRF1sDSn8zpdK09CSCO+ZkBQmbbQLhyAB+yrJ 1/eR9sOtxPT2QaF5pAm6gfNp6uyya+AG27Jpwbqf8Yh05LpbjZv2D+Ks9G4QRnS+sy +Dgy///nyGwLpkxVEPAi6bjClAot2vjtab0CoeDZrXjuPpiqP6QMagEoCsuUxJbFdy TFTUUUWip/V4L7tlpJPmh8BwzYoo7GhDwwvVHsNtkVg+8MZP7XHyNR1jM+I6+2hbGj QiEn6b7QrmvXvt8Zjy0vYJ/2KHHgVnmYkkXd6ri8Z2XbORXQch4BxBRpRghbzhzoOt bkU6DXU0bPGWQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6665F628D8 for ; Thu, 21 Sep 2023 18:56:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="L7uHlzuV"; dkim-atps=neutral Received: from uno.lan (93-46-82-201.ip106.fastwebnet.it [93.46.82.201]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CCDDD8201; Thu, 21 Sep 2023 18:54:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1695315266; bh=qXGx6RyMquUh9GOu6AN6eszqs/7WzI1BqcrEed/s/+k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L7uHlzuVchT42s2S6oRuGCVlg1tAiizOoVBQ3uJcYGxEgGCUkfOrUxguyn3wgoz2t nf1shEEH6EgHQlbeN/EMDGPek4M1vRmalq8TRLv3+ZfVEoCc00A85VY7fovEI7JGIv 9d0LdZTqR28pBGwZPCSf/qzMGnImJ4J8NdFU31zk= To: libcamera-devel@lists.libcamera.org Date: Thu, 21 Sep 2023 18:55:41 +0200 Message-ID: <20230921165550.50956-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> References: <20230921165550.50956-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 04/13] libcamera: rpi: Handle SensorConfiguration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Handle the SensorConfiguration provided by the application in the pipeline validate() and configure() call chains. During validation, first make sure SensorConfiguration is valid, then handle it to compute the sensor format. For the VC4 platform where the RAW stream follows the sensor's configuration adjust the RAW stream configuration to match the sensor configuration. Reviewed-by: Naushir Patuck Reviewed-by: Kieran Bingham Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi --- .../pipeline/rpi/common/pipeline_base.cpp | 66 ++++++++++++++++--- .../pipeline/rpi/common/pipeline_base.h | 4 +- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 28 +++++++- 3 files changed, 84 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 51fa1bbf9aa9..c02ceb65cae8 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -180,6 +180,15 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + /* + * Make sure that if a sensor configuration has been requested it + * is valid. + */ + if (sensorConfig && !sensorConfig->valid()) { + LOG(RPI, Error) << "Invalid sensor configuration request"; + return Invalid; + } + status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); /* @@ -207,19 +216,43 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() std::sort(outStreams.begin(), outStreams.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - /* Compute the sensor configuration. */ - unsigned int bitDepth = defaultRawBitDepth; - if (!rawStreams.empty()) { + /* Compute the sensor's format then do any platform specific fixups. */ + unsigned int bitDepth; + Size sensorSize; + + if (sensorConfig) { + /* Use the application provided sensor configuration. */ + bitDepth = sensorConfig->bitDepth; + sensorSize = sensorConfig->outputSize; + } else if (!rawStreams.empty()) { + /* Use the RAW stream format and size. */ BayerFormat bayerFormat = BayerFormat::fromPixelFormat(rawStreams[0].cfg->pixelFormat); bitDepth = bayerFormat.bitDepth; + sensorSize = rawStreams[0].cfg->size; + } else { + bitDepth = defaultRawBitDepth; + sensorSize = outStreams[0].cfg->size; } - sensorFormat_ = data_->findBestFormat(rawStreams.empty() ? outStreams[0].cfg->size - : rawStreams[0].cfg->size, - bitDepth); + sensorFormat_ = data_->findBestFormat(sensorSize, bitDepth); + + /* + * If a sensor configuration has been requested, it should apply + * without modifications. + */ + if (sensorConfig) { + BayerFormat bayer = BayerFormat::fromMbusCode(sensorFormat_.mbus_code); + + if (bayer.bitDepth != sensorConfig->bitDepth || + sensorFormat_.size != sensorConfig->outputSize) { + LOG(RPI, Error) << "Invalid sensor configuration: " + << "bitDepth/size mismatch"; + return Invalid; + } + } /* Do any platform specific fixups. */ - status = data_->platformValidate(rawStreams, outStreams); + status = data_->platformValidate(this, rawStreams, outStreams); if (status == Invalid) return Invalid; @@ -467,12 +500,25 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) std::sort(ispStreams.begin(), ispStreams.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); - /* Apply the format on the sensor with any cached transform. */ + /* + * Apply the format on the sensor with any cached transform. + * + * If the application has provided a sensor configuration apply it + * instead of just applying a format. + */ const RPiCameraConfiguration *rpiConfig = static_cast(config); - V4L2SubdeviceFormat sensorFormat = rpiConfig->sensorFormat_; + V4L2SubdeviceFormat sensorFormat; - ret = data->sensor_->setFormat(&sensorFormat, rpiConfig->combinedTransform_); + if (rpiConfig->sensorConfig) { + ret = data->sensor_->applyConfiguration(*rpiConfig->sensorConfig, + rpiConfig->combinedTransform_, + &sensorFormat); + } else { + sensorFormat = rpiConfig->sensorFormat_; + ret = data->sensor_->setFormat(&sensorFormat, + rpiConfig->combinedTransform_); + } if (ret) return ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index dbabc61ea48c..81b2b7d2f4d1 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -42,6 +42,7 @@ namespace RPi { /* Map of mbus codes to supported sizes reported by the sensor. */ using SensorFormats = std::map>; +class RPiCameraConfiguration; class CameraData : public Camera::Private { public: @@ -72,7 +73,8 @@ public: V4L2VideoDevice *dev; }; - virtual CameraConfiguration::Status platformValidate(std::vector &rawStreams, + virtual CameraConfiguration::Status platformValidate(RPiCameraConfiguration *rpiConfig, + std::vector &rawStreams, std::vector &outStreams) const = 0; virtual int platformConfigure(const V4L2SubdeviceFormat &sensorFormat, std::optional packing, diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 018cf4881d0e..2670eb8c4bbc 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -65,7 +65,8 @@ public: { } - CameraConfiguration::Status platformValidate(std::vector &rawStreams, + CameraConfiguration::Status platformValidate(RPi::RPiCameraConfiguration *rpiConfig, + std::vector &rawStreams, std::vector &outStreams) const override; int platformPipelineConfigure(const std::unique_ptr &root) override; @@ -394,7 +395,8 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer return 0; } -CameraConfiguration::Status Vc4CameraData::platformValidate(std::vector &rawStreams, +CameraConfiguration::Status Vc4CameraData::platformValidate(RPi::RPiCameraConfiguration *rpiConfig, + std::vector &rawStreams, std::vector &outStreams) const { CameraConfiguration::Status status = CameraConfiguration::Status::Valid; @@ -405,9 +407,27 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(std::vectorsensorFormat_.mbus_code); + + /* Handle flips to make sure to match the RAW stream format. */ + if (flipsAlterBayerOrder_) + rawBayer = rawBayer.transform(rpiConfig->combinedTransform_); + PixelFormat rawFormat = rawBayer.toPixelFormat(); + + if (rawStream->pixelFormat != rawFormat || + rawStream->size != rpiConfig->sensorFormat_.size) { + rawStream->pixelFormat = rawFormat; + rawStream->size = rpiConfig->sensorFormat_.size; + + status = CameraConfiguration::Adjusted; + } + } + /* * For the two ISP outputs, one stream must be equal or smaller than the * other in all dimensions. @@ -417,6 +437,8 @@ CameraConfiguration::Status Vc4CameraData::platformValidate(std::vectorsize.width, outStreams[0].cfg->size.width); size.height = std::min(outStreams[i].cfg->size.height,