From patchwork Thu Dec 4 16:49:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 25359 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A491EC3257 for ; Thu, 4 Dec 2025 16:49:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 692EF61144; Thu, 4 Dec 2025 17:49:45 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="bpv6W2PS"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F163061130 for ; Thu, 4 Dec 2025 17:49:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1764866983; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uwfT+zTwUkI4Pir8xfW7YgNj500XlRunilsoc0dZ4xI=; b=bpv6W2PSn8MciixX/WsHhUND9k+B2Dd54RilYf2HyAokoaslauxj/M7+6GNzr6nwMj4Sm1 k8IA2pNzN5r2GvOyQHC6nCzhAJS4cAg6beURDiAn9P2fEzi3H/Lo9YeupezRJuDMIl5keL spNn969w+vl7ftfaKP6lnqre/wK5COM= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-491-H4mP85JoMc-_fm2UYMFb4A-1; Thu, 04 Dec 2025 11:49:40 -0500 X-MC-Unique: H4mP85JoMc-_fm2UYMFb4A-1 X-Mimecast-MFC-AGG-ID: H4mP85JoMc-_fm2UYMFb4A_1764866977 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8FC3D19560A5; Thu, 4 Dec 2025 16:49:37 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.71]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7FC8F300F965; Thu, 4 Dec 2025 16:49:34 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain , Pavel Machek Subject: [PATCH v17 3/7] libcamera: simple: Validate raw stream configurations Date: Thu, 4 Dec 2025 17:49:12 +0100 Message-ID: <20251204164918.83334-4-mzamazal@redhat.com> In-Reply-To: <20251204164918.83334-1-mzamazal@redhat.com> References: <20251204164918.83334-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: w4Gekg-e6G9EL99ZHDZIOOXLNFgqGFPs1Zj5QmBXLbo_1764866977 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. Raw streams are adjusted from the capture format and size. Configuration validation computes the maximum size of all the requested streams and compares it to the output sizes. When e.g. only a raw stream is requested then this may result in an invalid adjustment of its size. This is because the output sizes are computed for processed streams and may be smaller than capture sizes. If a raw stream with the capture size is requested, it may then be wrongly adjusted to a larger size because the output sizes, which are irrelevant for raw streams anyway, are smaller than the requested capture size. The problem is resolved by tracking raw and processed streams maximum sizes separately and comparing raw stream sizes against capture rather than output sizes. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. If the resolutions don't match mutually or don't match the available sizes, validation adjusts them. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Co-developed-by: Umang Jain Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Signed-off-by: Umang Jain --- src/libcamera/pipeline/simple/simple.cpp | 117 ++++++++++++++++------- 1 file changed, 82 insertions(+), 35 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 6b83254fb..aaafe0571 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -27,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -1138,29 +1140,57 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } - /* Find the largest stream size. */ - Size maxStreamSize; - for (const StreamConfiguration &cfg : config_) - maxStreamSize.expandTo(cfg.size); + /* Find the largest stream sizes. */ + Size maxProcessedStreamSize; + Size maxRawStreamSize; + for (const StreamConfiguration &cfg : config_) { + if (isRaw(cfg)) + maxRawStreamSize.expandTo(cfg.size); + else + maxProcessedStreamSize.expandTo(cfg.size); + } LOG(SimplePipeline, Debug) - << "Largest stream size is " << maxStreamSize; + << "Largest processed stream size is " << maxProcessedStreamSize; + LOG(SimplePipeline, Debug) + << "Largest raw stream size is " << maxRawStreamSize; + + /* Cap the number of raw stream configurations */ + unsigned int rawCount = 0; + PixelFormat requestedRawFormat; + for (const StreamConfiguration &cfg : config_) { + if (!isRaw(cfg)) + continue; + requestedRawFormat = cfg.pixelFormat; + rawCount++; + } + + if (rawCount > 1) { + LOG(SimplePipeline, Error) + << "Camera configuration with multiple raw streams not supported"; + return Invalid; + } /* * Find the best configuration for the pipeline using a heuristic. - * First select the pixel format based on the streams (which are - * considered ordered from highest to lowest priority). Default to the - * first pipeline configuration if no streams request a supported pixel - * format. + * First select the pixel format based on the raw streams followed by + * non-raw streams (which are considered ordered from highest to lowest + * priority). Default to the first pipeline configuration if no streams + * request a supported pixel format. */ const std::vector *configs = &data_->formats_.begin()->second; - for (const StreamConfiguration &cfg : config_) { - auto it = data_->formats_.find(cfg.pixelFormat); - if (it != data_->formats_.end()) { - configs = &it->second; - break; + auto rawIter = data_->formats_.find(requestedRawFormat); + if (rawIter != data_->formats_.end()) { + configs = &rawIter->second; + } else { + for (const StreamConfiguration &cfg : config_) { + auto it = data_->formats_.find(cfg.pixelFormat); + if (it != data_->formats_.end()) { + configs = &it->second; + break; + } } } @@ -1182,8 +1212,10 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() const Size &captureSize = pipeConfig->captureSize; const Size &maxOutputSize = pipeConfig->outputSizes.max; - if (maxOutputSize.width >= maxStreamSize.width && - maxOutputSize.height >= maxStreamSize.height) { + if (maxOutputSize.width >= maxProcessedStreamSize.width && + maxOutputSize.height >= maxProcessedStreamSize.height && + captureSize.width >= maxRawStreamSize.width && + captureSize.height >= maxRawStreamSize.height) { if (!pipeConfig_ || captureSize < pipeConfig_->captureSize) pipeConfig_ = pipeConfig; } @@ -1201,7 +1233,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() << V4L2SubdeviceFormat{ pipeConfig_->code, pipeConfig_->sensorSize, {} } << " -> " << pipeConfig_->captureSize << "-" << pipeConfig_->captureFormat - << " for max stream size " << maxStreamSize; + << " for max processed stream size " << maxProcessedStreamSize + << " and max raw stream size " << maxRawStreamSize; /* * Adjust the requested streams. @@ -1220,21 +1253,35 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + const bool raw = isRaw(cfg); /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); - - PixelFormat pixelFormat = *it; - if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) - << "Adjusting pixel format from " - << cfg.pixelFormat << " to " << pixelFormat; - cfg.pixelFormat = pixelFormat; - status = Adjusted; + if (raw) { + if (cfg.pixelFormat != pipeConfig_->captureFormat || + cfg.size != pipeConfig_->captureSize) { + cfg.pixelFormat = pipeConfig_->captureFormat; + cfg.size = pipeConfig_->captureSize; + + LOG(SimplePipeline, Debug) + << "Adjusting raw stream to " + << cfg.toString(); + status = Adjusted; + } + } else { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + + PixelFormat pixelFormat = *it; + if (cfg.pixelFormat != pixelFormat) { + LOG(SimplePipeline, Debug) + << "Adjusting processed pixel format from " + << cfg.pixelFormat << " to " << pixelFormat; + cfg.pixelFormat = pixelFormat; + status = Adjusted; + } } /* @@ -1245,7 +1292,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * case, perform the standard pixel format based color space adjustment. */ if (!cfg.colorSpace) { - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); switch (info.colourEncoding) { case PixelFormatInfo::ColourEncodingRGB: cfg.colorSpace = ColorSpace::Srgb; @@ -1265,9 +1312,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * adjusting a requested one, changes here shouldn't set the status * to Adjusted. */ - cfg.colorSpace->adjust(pixelFormat); + cfg.colorSpace->adjust(cfg.pixelFormat); } else { - if (cfg.colorSpace->adjust(pixelFormat)) { + if (cfg.colorSpace->adjust(cfg.pixelFormat)) { LOG(SimplePipeline, Debug) << "Color space adjusted to " << cfg.colorSpace.value().toString(); @@ -1275,7 +1322,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() } } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1298,7 +1345,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() needConversion_ = true; /* Set the stride and frameSize. */ - if (needConversion_) { + if (needConversion_ && !raw) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat,