{"id":25359,"url":"https://patchwork.libcamera.org/api/1.1/patches/25359/?format=json","web_url":"https://patchwork.libcamera.org/patch/25359/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20251204164918.83334-4-mzamazal@redhat.com>","date":"2025-12-04T16:49:12","name":"[v17,3/7] libcamera: simple: Validate raw stream configurations","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"381a36a16882972919ca1d86e0361636b676984f","submitter":{"id":177,"url":"https://patchwork.libcamera.org/api/1.1/people/177/?format=json","name":"Milan Zamazal","email":"mzamazal@redhat.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/25359/mbox/","series":[{"id":5637,"url":"https://patchwork.libcamera.org/api/1.1/series/5637/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5637","date":"2025-12-04T16:49:09","name":"Enable raw streams with software ISP","version":17,"mbox":"https://patchwork.libcamera.org/series/5637/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/25359/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/25359/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A491EC3257\n\tfor ;\n\tThu, 4 Dec 2025 16:49:45 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 692EF61144;\n\tThu, 4 Dec 2025 17:49:45 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id F163061130\n\tfor ;\n\tThu, 4 Dec 2025 17:49:43 +0100 (CET)","from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com\n\t(ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63])\n\tby relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3,\n\tcipher=TLS_AES_256_GCM_SHA384) id us-mta-491-H4mP85JoMc-_fm2UYMFb4A-1;\n\tThu, 04 Dec 2025 11:49:40 -0500","from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com\n\t(mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\tkey-exchange X25519 server-signature RSA-PSS (2048 bits)\n\tserver-digest SHA256) (No client certificate requested)\n\tby mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix)\n\twith ESMTPS id 8FC3D19560A5; Thu, 4 Dec 2025 16:49:37 +0000 (UTC)","from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.71])\n\tby mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix)\n\twith ESMTP id 7FC8F300F965; Thu, 4 Dec 2025 16:49:34 +0000 (UTC)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"bpv6W2PS\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1764866983;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tcontent-transfer-encoding:content-transfer-encoding:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=uwfT+zTwUkI4Pir8xfW7YgNj500XlRunilsoc0dZ4xI=;\n\tb=bpv6W2PSn8MciixX/WsHhUND9k+B2Dd54RilYf2HyAokoaslauxj/M7+6GNzr6nwMj4Sm1\n\tk8IA2pNzN5r2GvOyQHC6nCzhAJS4cAg6beURDiAn9P2fEzi3H/Lo9YeupezRJuDMIl5keL\n\tspNn969w+vl7ftfaKP6lnqre/wK5COM=","X-MC-Unique":"H4mP85JoMc-_fm2UYMFb4A-1","X-Mimecast-MFC-AGG-ID":"H4mP85JoMc-_fm2UYMFb4A_1764866977","From":"Milan Zamazal ","To":"libcamera-devel@lists.libcamera.org","Cc":"Milan Zamazal , Laurent Pinchart\n\t, Kieran Bingham\n\t, =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?=\n\t, Paul Elder\n\t, Umang Jain ,\n\tPavel Machek ","Subject":"[PATCH v17 3/7] libcamera: simple: Validate raw stream\n\tconfigurations","Date":"Thu, 4 Dec 2025 17:49:12 +0100","Message-ID":"<20251204164918.83334-4-mzamazal@redhat.com>","In-Reply-To":"<20251204164918.83334-1-mzamazal@redhat.com>","References":"<20251204164918.83334-1-mzamazal@redhat.com>","MIME-Version":"1.0","X-Scanned-By":"MIMEDefang 3.4.1 on 10.30.177.4","X-Mimecast-Spam-Score":"0","X-Mimecast-MFC-PROC-ID":"w4Gekg-e6G9EL99ZHDZIOOXLNFgqGFPs1Zj5QmBXLbo_1764866977","X-Mimecast-Originator":"redhat.com","Content-Transfer-Encoding":"8bit","content-type":"text/plain; charset=\"US-ASCII\"; x-default=true","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"SimpleCameraConfiguration::validate() looks for the best configuration.\nAs part of enabling raw stream support, the method must consider raw\nstreams in addition to the processed streams.\n\nRaw streams are adjusted from the capture format and size.\n\nConfiguration validation computes the maximum size of all the requested\nstreams and compares it to the output sizes. When e.g. only a raw\nstream is requested then this may result in an invalid adjustment of its\nsize. This is because the output sizes are computed for processed\nstreams and may be smaller than capture sizes. If a raw stream with the\ncapture size is requested, it may then be wrongly adjusted to a larger\nsize because the output sizes, which are irrelevant for raw streams\nanyway, are smaller than the requested capture size. The problem is\nresolved by tracking raw and processed streams maximum sizes separately\nand comparing raw stream sizes against capture rather than output sizes.\n\nNote that with both processed and raw streams, the requested sizes must\nbe mutually matching, including resizing due to debayer requirements.\nFor example, the following `cam' setup is valid for imx219\n\n cam -s role=viewfinder,width=1920,height=1080 \\\n -s role=raw,width=3280,height=2464\n\nrather than\n\n cam -s role=viewfinder,width=1920,height=1080 \\\n -s role=raw,width=1920,height=1080\n\ndue to the resolution of 1924x1080 actually selected for debayering to\n1920x1080. If the resolutions don't match mutually or don't match the\navailable sizes, validation adjusts them.\n\nSetting up the right configurations is still not enough to make the raw\nstreams working. Buffer handling must be changed in the simple\npipeline, which is addressed in followup patches.\n\nCo-developed-by: Umang Jain \nSigned-off-by: Milan Zamazal \n---\n src/libcamera/pipeline/simple/simple.cpp | 117 ++++++++++++++++-------\n 1 file changed, 82 insertions(+), 35 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\nindex 6b83254fb..aaafe0571 100644\n--- a/src/libcamera/pipeline/simple/simple.cpp\n+++ b/src/libcamera/pipeline/simple/simple.cpp\n@@ -11,6 +11,7 @@\n #include \n #include \n #include \n+#include \n #include \n #include \n #include \n@@ -27,6 +28,7 @@\n #include \n #include \n #include \n+#include \n #include \n #include \n #include \n@@ -1138,29 +1140,57 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n \t\tstatus = Adjusted;\n \t}\n \n-\t/* Find the largest stream size. */\n-\tSize maxStreamSize;\n-\tfor (const StreamConfiguration &cfg : config_)\n-\t\tmaxStreamSize.expandTo(cfg.size);\n+\t/* Find the largest stream sizes. */\n+\tSize maxProcessedStreamSize;\n+\tSize maxRawStreamSize;\n+\tfor (const StreamConfiguration &cfg : config_) {\n+\t\tif (isRaw(cfg))\n+\t\t\tmaxRawStreamSize.expandTo(cfg.size);\n+\t\telse\n+\t\t\tmaxProcessedStreamSize.expandTo(cfg.size);\n+\t}\n \n \tLOG(SimplePipeline, Debug)\n-\t\t<< \"Largest stream size is \" << maxStreamSize;\n+\t\t<< \"Largest processed stream size is \" << maxProcessedStreamSize;\n+\tLOG(SimplePipeline, Debug)\n+\t\t<< \"Largest raw stream size is \" << maxRawStreamSize;\n+\n+\t/* Cap the number of raw stream configurations */\n+\tunsigned int rawCount = 0;\n+\tPixelFormat requestedRawFormat;\n+\tfor (const StreamConfiguration &cfg : config_) {\n+\t\tif (!isRaw(cfg))\n+\t\t\tcontinue;\n+\t\trequestedRawFormat = cfg.pixelFormat;\n+\t\trawCount++;\n+\t}\n+\n+\tif (rawCount > 1) {\n+\t\tLOG(SimplePipeline, Error)\n+\t\t\t<< \"Camera configuration with multiple raw streams not supported\";\n+\t\treturn Invalid;\n+\t}\n \n \t/*\n \t * Find the best configuration for the pipeline using a heuristic.\n-\t * First select the pixel format based on the streams (which are\n-\t * considered ordered from highest to lowest priority). Default to the\n-\t * first pipeline configuration if no streams request a supported pixel\n-\t * format.\n+\t * First select the pixel format based on the raw streams followed by\n+\t * non-raw streams (which are considered ordered from highest to lowest\n+\t * priority). Default to the first pipeline configuration if no streams\n+\t * request a supported pixel format.\n \t */\n \tconst std::vector *configs =\n \t\t&data_->formats_.begin()->second;\n \n-\tfor (const StreamConfiguration &cfg : config_) {\n-\t\tauto it = data_->formats_.find(cfg.pixelFormat);\n-\t\tif (it != data_->formats_.end()) {\n-\t\t\tconfigs = &it->second;\n-\t\t\tbreak;\n+\tauto rawIter = data_->formats_.find(requestedRawFormat);\n+\tif (rawIter != data_->formats_.end()) {\n+\t\tconfigs = &rawIter->second;\n+\t} else {\n+\t\tfor (const StreamConfiguration &cfg : config_) {\n+\t\t\tauto it = data_->formats_.find(cfg.pixelFormat);\n+\t\t\tif (it != data_->formats_.end()) {\n+\t\t\t\tconfigs = &it->second;\n+\t\t\t\tbreak;\n+\t\t\t}\n \t\t}\n \t}\n \n@@ -1182,8 +1212,10 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n \t\tconst Size &captureSize = pipeConfig->captureSize;\n \t\tconst Size &maxOutputSize = pipeConfig->outputSizes.max;\n \n-\t\tif (maxOutputSize.width >= maxStreamSize.width &&\n-\t\t maxOutputSize.height >= maxStreamSize.height) {\n+\t\tif (maxOutputSize.width >= maxProcessedStreamSize.width &&\n+\t\t maxOutputSize.height >= maxProcessedStreamSize.height &&\n+\t\t captureSize.width >= maxRawStreamSize.width &&\n+\t\t captureSize.height >= maxRawStreamSize.height) {\n \t\t\tif (!pipeConfig_ || captureSize < pipeConfig_->captureSize)\n \t\t\t\tpipeConfig_ = pipeConfig;\n \t\t}\n@@ -1201,7 +1233,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n \t\t<< V4L2SubdeviceFormat{ pipeConfig_->code, pipeConfig_->sensorSize, {} }\n \t\t<< \" -> \" << pipeConfig_->captureSize\n \t\t<< \"-\" << pipeConfig_->captureFormat\n-\t\t<< \" for max stream size \" << maxStreamSize;\n+\t\t<< \" for max processed stream size \" << maxProcessedStreamSize\n+\t\t<< \" and max raw stream size \" << maxRawStreamSize;\n \n \t/*\n \t * Adjust the requested streams.\n@@ -1220,21 +1253,35 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n \n \tfor (unsigned int i = 0; i < config_.size(); ++i) {\n \t\tStreamConfiguration &cfg = config_[i];\n+\t\tconst bool raw = isRaw(cfg);\n \n \t\t/* Adjust the pixel format and size. */\n-\t\tauto it = std::find(pipeConfig_->outputFormats.begin(),\n-\t\t\t\t pipeConfig_->outputFormats.end(),\n-\t\t\t\t cfg.pixelFormat);\n-\t\tif (it == pipeConfig_->outputFormats.end())\n-\t\t\tit = pipeConfig_->outputFormats.begin();\n-\n-\t\tPixelFormat pixelFormat = *it;\n-\t\tif (cfg.pixelFormat != pixelFormat) {\n-\t\t\tLOG(SimplePipeline, Debug)\n-\t\t\t\t<< \"Adjusting pixel format from \"\n-\t\t\t\t<< cfg.pixelFormat << \" to \" << pixelFormat;\n-\t\t\tcfg.pixelFormat = pixelFormat;\n-\t\t\tstatus = Adjusted;\n+\t\tif (raw) {\n+\t\t\tif (cfg.pixelFormat != pipeConfig_->captureFormat ||\n+\t\t\t cfg.size != pipeConfig_->captureSize) {\n+\t\t\t\tcfg.pixelFormat = pipeConfig_->captureFormat;\n+\t\t\t\tcfg.size = pipeConfig_->captureSize;\n+\n+\t\t\t\tLOG(SimplePipeline, Debug)\n+\t\t\t\t\t<< \"Adjusting raw stream to \"\n+\t\t\t\t\t<< cfg.toString();\n+\t\t\t\tstatus = Adjusted;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tauto it = std::find(pipeConfig_->outputFormats.begin(),\n+\t\t\t\t\t pipeConfig_->outputFormats.end(),\n+\t\t\t\t\t cfg.pixelFormat);\n+\t\t\tif (it == pipeConfig_->outputFormats.end())\n+\t\t\t\tit = pipeConfig_->outputFormats.begin();\n+\n+\t\t\tPixelFormat pixelFormat = *it;\n+\t\t\tif (cfg.pixelFormat != pixelFormat) {\n+\t\t\t\tLOG(SimplePipeline, Debug)\n+\t\t\t\t\t<< \"Adjusting processed pixel format from \"\n+\t\t\t\t\t<< cfg.pixelFormat << \" to \" << pixelFormat;\n+\t\t\t\tcfg.pixelFormat = pixelFormat;\n+\t\t\t\tstatus = Adjusted;\n+\t\t\t}\n \t\t}\n \n \t\t/*\n@@ -1245,7 +1292,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n \t\t * case, perform the standard pixel format based color space adjustment.\n \t\t */\n \t\tif (!cfg.colorSpace) {\n-\t\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat);\n+\t\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat);\n \t\t\tswitch (info.colourEncoding) {\n \t\t\tcase PixelFormatInfo::ColourEncodingRGB:\n \t\t\t\tcfg.colorSpace = ColorSpace::Srgb;\n@@ -1265,9 +1312,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n \t\t\t * adjusting a requested one, changes here shouldn't set the status\n \t\t\t * to Adjusted.\n \t\t\t */\n-\t\t\tcfg.colorSpace->adjust(pixelFormat);\n+\t\t\tcfg.colorSpace->adjust(cfg.pixelFormat);\n \t\t} else {\n-\t\t\tif (cfg.colorSpace->adjust(pixelFormat)) {\n+\t\t\tif (cfg.colorSpace->adjust(cfg.pixelFormat)) {\n \t\t\t\tLOG(SimplePipeline, Debug)\n \t\t\t\t\t<< \"Color space adjusted to \"\n \t\t\t\t\t<< cfg.colorSpace.value().toString();\n@@ -1275,7 +1322,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n \t\t\t}\n \t\t}\n \n-\t\tif (!pipeConfig_->outputSizes.contains(cfg.size)) {\n+\t\tif (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) {\n \t\t\tSize adjustedSize = pipeConfig_->captureSize;\n \t\t\t/*\n \t\t\t * The converter (when present) may not be able to output\n@@ -1298,7 +1345,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate()\n \t\t\tneedConversion_ = true;\n \n \t\t/* Set the stride and frameSize. */\n-\t\tif (needConversion_) {\n+\t\tif (needConversion_ && !raw) {\n \t\t\tstd::tie(cfg.stride, cfg.frameSize) =\n \t\t\t\tdata_->converter_\n \t\t\t\t\t? data_->converter_->strideAndFrameSize(cfg.pixelFormat,\n","prefixes":["v17","3/7"]}