From patchwork Wed Jul 23 18:08:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23913 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4F33BC3323 for ; Wed, 23 Jul 2025 18:09:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8F78369080; Wed, 23 Jul 2025 20:09:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Cef2WNRO"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DFE46907E for ; Wed, 23 Jul 2025 20:09:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753294137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZfcXv9hY4UJginfMu6aew12QKP2XdnssyilUoeqRXTQ=; b=Cef2WNROjz6iVIF76UVS1+HL31cC6VpM7WqCCmZvw7Hdg6AVP7PbNi04e4KcCtqqbTmP3g cg4TiiUrKUbgxQFuDCADvX0jk7bs/L+h9RGjgvzBR3zp+RPWsMxqUdcpRcN/JwnnqyK7wZ 2+4200OOcVMdXl+/wc1cDMqDWIZks78= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-646-1gJ4GdD1PyG7xjNcjni9MQ-1; Wed, 23 Jul 2025 14:08:49 -0400 X-MC-Unique: 1gJ4GdD1PyG7xjNcjni9MQ-1 X-Mimecast-MFC-AGG-ID: 1gJ4GdD1PyG7xjNcjni9MQ_1753294128 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0491F195FE16; Wed, 23 Jul 2025 18:08:48 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1E111195608D; Wed, 23 Jul 2025 18:08:44 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v11 5/8] libcamera: simple: Validate raw stream configurations Date: Wed, 23 Jul 2025 20:08:10 +0200 Message-ID: <20250723180815.82450-6-mzamazal@redhat.com> In-Reply-To: <20250723180815.82450-1-mzamazal@redhat.com> References: <20250723180815.82450-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: qo6H22Ou8yN_hpIcPpnY53s0hZAt3cxN9fmR1HWW09Q_1753294128 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. Raw streams are adjusted from the capture format and size. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. If the resolutions don't match mutually or available sizes, validation adjusts them. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 86 +++++++++++++++++------- 1 file changed, 62 insertions(+), 24 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 37abaa0e0..89ee699c3 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -27,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -1131,22 +1133,39 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() << "Largest stream size is " << maxStreamSize; /* - * Find the best configuration for the pipeline using a heuristic. - * First select the pixel format based on the streams (which are - * considered ordered from highest to lowest priority). Default to the - * first pipeline configuration if no streams request a supported pixel - * format. + * Find the best configuration for the pipeline using a heuristic. First + * select the pixel format based on the streams. If there is a raw stream, + * its format has precedence. If there is no raw stream, the streams are + * considered ordered from highest to lowest priority. Default to the first + * pipeline configuration if no streams request a supported pixel format. */ - const std::vector *configs = - &data_->formats_.begin()->second; + std::optional rawFormat; + for (const auto &cfg : config_) + if (cfg.colorSpace == ColorSpace::Raw) { + if (rawFormat) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return Invalid; + } + rawFormat = cfg.pixelFormat; + } - for (const StreamConfiguration &cfg : config_) { - auto it = data_->formats_.find(cfg.pixelFormat); - if (it != data_->formats_.end()) { + const std::vector *configs = nullptr; + if (rawFormat) { + auto it = data_->formats_.find(rawFormat.value()); + if (it != data_->formats_.end()) configs = &it->second; - break; - } } + if (!configs) + for (const StreamConfiguration &cfg : config_) { + auto it = data_->formats_.find(cfg.pixelFormat); + if (it != data_->formats_.end()) { + configs = &it->second; + break; + } + } + if (!configs) + configs = &data_->formats_.begin()->second; /* * \todo Pick the best sensor output media bus format when the @@ -1203,17 +1222,27 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + const bool raw = cfg.colorSpace == ColorSpace::Raw; - /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); + /* Adjust the pixel format, colour space and size. */ - PixelFormat pixelFormat = *it; + PixelFormat pixelFormat; + if (raw) { + pixelFormat = pipeConfig_->captureFormat; + } else { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + pixelFormat = *it; + } if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) << "Adjusting pixel format"; + LOG(SimplePipeline, Debug) + << "Adjusting pixel format of a " + << (raw ? "raw" : "processed") + << " stream from " << cfg.pixelFormat + << " to " << pixelFormat; cfg.pixelFormat = pixelFormat; /* * Do not touch the colour space for raw requested roles. @@ -1240,7 +1269,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1259,11 +1288,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* \todo Create a libcamera core class to group format and size */ if (cfg.pixelFormat != pipeConfig_->captureFormat || - cfg.size != pipeConfig_->captureSize) - needConversion_ = true; + cfg.size != pipeConfig_->captureSize) { + if (raw) { + cfg.pixelFormat = pipeConfig_->captureFormat; + cfg.size = pipeConfig_->captureSize; + LOG(SimplePipeline, Debug) + << "Adjusting raw configuration to " << cfg; + status = Adjusted; + } else { + needConversion_ = true; + } + } /* Set the stride, frameSize and bufferCount. */ - if (needConversion_) { + if (needConversion_ && !raw) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat,