From patchwork Fri Jul 11 17:53:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23784 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CA511C3237 for ; Fri, 11 Jul 2025 17:54:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8438E68F19; Fri, 11 Jul 2025 19:54:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="bGgD8EaY"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 430E268F22 for ; Fri, 11 Jul 2025 19:54:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752256460; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VyRxSJ1qar3geSLTPl3rizQhlAMu1q6r2Ds4+RIwx4o=; b=bGgD8EaYjA5VgX5HpjKCJ3cXvpniFtzjfrwM9DMwwPhGwqplffdlDTsl3g7KwWVgCNYhem bKVDWEXrh8jiRi4Hy+wA3wggdRiphYwPg1O55VoMBBF5uS+IoKpf33qvRO0tQcEyHFcgb0 buaweYhQWX14JSaRbkv0xZvkfapTKc4= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-426-bB2klvoiOrygNLlSkOV2xg-1; Fri, 11 Jul 2025 13:54:18 -0400 X-MC-Unique: bB2klvoiOrygNLlSkOV2xg-1 X-Mimecast-MFC-AGG-ID: bB2klvoiOrygNLlSkOV2xg_1752256457 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E4EB718600C4; Fri, 11 Jul 2025 17:54:13 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E3DC0180884B; Fri, 11 Jul 2025 17:54:10 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v10 5/8] libcamera: simple: Validate raw stream configurations Date: Fri, 11 Jul 2025 19:53:38 +0200 Message-ID: <20250711175345.90318-6-mzamazal@redhat.com> In-Reply-To: <20250711175345.90318-1-mzamazal@redhat.com> References: <20250711175345.90318-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: CWJ8nXu3Ft2esnVPtzgjGJvJDStzQwgBQMSl23NHqP8_1752256457 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. If only a processed stream is requested, nothing changes. If only a raw stream is requested, the pixel format and output size may not be adjusted. The patch adds checks for this. If both processed and raw streams are requested, things get more complicated. The raw stream is expected to be passed through intact and all the adjustments are made for the processed streams. We select a pipe configuration for the processed streams. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. It is the application responsibility to select the right parameters for the raw stream. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 119 ++++++++++++++++------- 1 file changed, 85 insertions(+), 34 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 37abaa0e0..9d87a7ffa 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -1186,6 +1187,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() << "-" << pipeConfig_->captureFormat << " for max stream size " << maxStreamSize; + bool rawRequested = false; + bool processedRequested = false; + for (const auto &cfg : config_) + if (cfg.colorSpace == ColorSpace::Raw) { + if (rawRequested) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return Invalid; + } + rawRequested = true; + } else { + processedRequested = true; + } + /* * Adjust the requested streams. * @@ -1204,43 +1219,70 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + /* + * If both processed and raw streams are requested, the pipe + * configuration is set up for the processed stream. The raw + * configuration needs to be compared against the capture format and + * size in such a case. + */ + const bool rawStream = cfg.colorSpace == ColorSpace::Raw; + const bool sideRawStream = rawStream && processedRequested; + /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); - - PixelFormat pixelFormat = *it; - if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) << "Adjusting pixel format"; - cfg.pixelFormat = pixelFormat; - /* - * Do not touch the colour space for raw requested roles. - * Even if the pixel format is non-raw (whatever it means), we - * shouldn't try to interpret the colour space of raw data. - */ - if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + + if (!sideRawStream) { + PixelFormat pixelFormat; + if (rawStream) { + pixelFormat = pipeConfig_->captureFormat; + } else { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + pixelFormat = *it; + } + + if (cfg.pixelFormat != pixelFormat) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw pixel format " + << cfg.pixelFormat + << " doesn't match any of the pipe output formats"; + return Invalid; + } + LOG(SimplePipeline, Debug) + << "Adjusting pixel format from " << cfg.pixelFormat + << " to " << pixelFormat; + cfg.pixelFormat = pixelFormat; + /* + * Do not touch the colour space for raw requested roles. + * Even if the pixel format is non-raw (whatever it means), we + * shouldn't try to interpret the colour space of raw data. + */ + if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + cfg.colorSpace->adjust(pixelFormat); + status = Adjusted; + } + + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } cfg.colorSpace->adjust(pixelFormat); - status = Adjusted; - } - if (!cfg.colorSpace) { - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); - switch (info.colourEncoding) { - case PixelFormatInfo::ColourEncodingRGB: - cfg.colorSpace = ColorSpace::Srgb; - break; - case libcamera::PixelFormatInfo::ColourEncodingYUV: - cfg.colorSpace = ColorSpace::Sycc; - break; - default: - cfg.colorSpace = ColorSpace::Raw; + status = Adjusted; } - cfg.colorSpace->adjust(pixelFormat); - status = Adjusted; } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!rawStream && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1259,11 +1301,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* \todo Create a libcamera core class to group format and size */ if (cfg.pixelFormat != pipeConfig_->captureFormat || - cfg.size != pipeConfig_->captureSize) + cfg.size != pipeConfig_->captureSize) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw output format " << cfg.pixelFormat + << " and size " << cfg.size + << " not matching pipe format " << pipeConfig_->captureFormat + << " and size " << pipeConfig_->captureSize; + return Invalid; + } needConversion_ = true; + } /* Set the stride, frameSize and bufferCount. */ - if (needConversion_) { + if (needConversion_ && !rawStream) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat,