From patchwork Tue Nov 4 15:34:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24976 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A58BDC3241 for ; Tue, 4 Nov 2025 15:36:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4D9D260AAC; Tue, 4 Nov 2025 16:36:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="FrdWMEQg"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 368C360856 for ; Tue, 4 Nov 2025 16:36:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762270577; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l6+RgdCD3Vl1Dso89qIO/qt1d/DJtAjsv/fEzVqfX5M=; b=FrdWMEQgAHl78iUpfTT9Bk7rVofrxOT1IJx9YvV6OpNF1paVk5vJFjP3bvAjYBFtRdJ+8V LN1ChYJdaWXql67j4B++2cz+9ZRcb1ntvb4ifOZMBvuUQAOzjthKM9+vycGJDwFeH0suzS LpLp+WTaWWFxYI5WLC4QNfT6VpWl0Vg= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-633-kM04M440OTGGIjit2zr88Q-1; Tue, 04 Nov 2025 10:35:51 -0500 X-MC-Unique: kM04M440OTGGIjit2zr88Q-1 X-Mimecast-MFC-AGG-ID: kM04M440OTGGIjit2zr88Q_1762270519 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DB41B1801231; Tue, 4 Nov 2025 15:35:15 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.225.23]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 319D81956056; Tue, 4 Nov 2025 15:35:12 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v15 1/8] libcamera: software_isp: Assign colour spaces in configurations Date: Tue, 4 Nov 2025 16:34:52 +0100 Message-ID: <20251104153501.34362-2-mzamazal@redhat.com> In-Reply-To: <20251104153501.34362-1-mzamazal@redhat.com> References: <20251104153501.34362-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 58yqe3J0BC1F9bjLcmabFKHTOXoBtVpByR5akkR82aM_1762270519 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" StreamConfiguration's should have colorSpace set. This is not the case in the simple pipeline. Let's set it there. This also fixes a crash in `cam' due to accessing an unset colorSpace. We set the colour spaces according to the pixel format. This is not completely correct because pixel formats and colour spaces are different, although not completely independent, things. But for the lack of a better practical option to determine the colour space, we use this. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 33 ++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 91715b7f8..04446a3ef 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -36,6 +37,7 @@ #include "libcamera/internal/converter.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/global_configuration.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" @@ -1222,6 +1224,37 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } + /* + * Best effort to fix the color space. If the color space is not set, + * set it according to the pixel format, which may not be correct (pixel + * formats and color spaces are different things, although somewhat + * related) but we don't have a better option at the moment. Then in any + * case, perform the standard pixel format based color space adjustment. + */ + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } + LOG(SimplePipeline, Debug) + << "Unspecified color space set to " + << cfg.colorSpace.value().toString(); + status = Adjusted; + } + if (cfg.colorSpace->adjust(pixelFormat)) { + LOG(SimplePipeline, Debug) + << "Color space adjusted to " + << cfg.colorSpace.value().toString(); + status = Adjusted; + } + if (!pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* From patchwork Tue Nov 4 15:34:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24969 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 22B96C3241 for ; Tue, 4 Nov 2025 15:35:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D472E6069A; Tue, 4 Nov 2025 16:35:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Cj9ECZ/W"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F1F366069A for ; Tue, 4 Nov 2025 16:35:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762270524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fVMpFG1V1uCdiIcSsJX8xxdoZQZIRbLuYwk5oOZ3gtc=; b=Cj9ECZ/WDU1G+1msNQjuW2vCd3D3wJ/qX9vieCw2yvqUEnXcc4tOQlLCOOkbvCbVjnd63w PILSj6/eu03EYDgbY7bwoL0seKTH9F7Krh6bMrG3la21rrFXkmewVzKFc+c6vrVFev0HD9 xcENG9cNq+quJ+ddM3ho3d3Po0+6BRE= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-671-aIoWKDi1PRWTWNUXf4NBUw-1; Tue, 04 Nov 2025 10:35:21 -0500 X-MC-Unique: aIoWKDi1PRWTWNUXf4NBUw-1 X-Mimecast-MFC-AGG-ID: aIoWKDi1PRWTWNUXf4NBUw_1762270520 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 17B701956052; Tue, 4 Nov 2025 15:35:20 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.225.23]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5D95B1956056; Tue, 4 Nov 2025 15:35:16 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v15 2/8] libcamera: simple: Exclude raw configurations from output conversions Date: Tue, 4 Nov 2025 16:34:53 +0100 Message-ID: <20251104153501.34362-3-mzamazal@redhat.com> In-Reply-To: <20251104153501.34362-1-mzamazal@redhat.com> References: <20251104153501.34362-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: WsDXGA8CTxiGg5SwIVxV2qqhZtbBSVFOGKdVfJc7adI_1762270520 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to support raw streams, we need to add raw formats to software ISP configurations. In this preparatory patch, the raw formats are excluded from output configurations for conversions. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 04446a3ef..ba8a22ccc 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -264,6 +265,12 @@ static const SimplePipelineInfo supportedDevices[] = { { "sun6i-csi", {}, false }, }; +bool isRaw(const StreamConfiguration &cfg) +{ + return libcamera::PixelFormatInfo::info(cfg.pixelFormat).colourEncoding == + libcamera::PixelFormatInfo::ColourEncodingRAW; +} + } /* namespace */ class SimpleCameraData : public Camera::Private @@ -1446,7 +1453,7 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) cfg.setStream(&data->streams_[i]); - if (data->useConversion_) + if (data->useConversion_ && !isRaw(cfg)) outputCfgs.push_back(cfg); } From patchwork Tue Nov 4 15:34:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24970 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9D684C3241 for ; Tue, 4 Nov 2025 15:35:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6961A60A86; Tue, 4 Nov 2025 16:35:32 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="GTaAdkfg"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 07CBD60A7B for ; Tue, 4 Nov 2025 16:35:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762270529; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OToS9Qiclt6YWpdgTySmSkzYBfKmOsN0qjVaFw8jF9k=; b=GTaAdkfg6mQ8MltiGwTooQEb7ssSruM0G5VlZul8QkQ3wGaoBAik2arMdwdeqxuizBwLPB 1PW5SGXRAkU3W7Ak3unYXies4svG7TqdZx8gCgc2qraUSjqu+k08VYAZib0aaUXIe4kQrE xTepzHEks8MvWGU+ZFOWk4D8TIKWvJo= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-208-DPHnQ0DDNrmlRMW7_SBuVw-1; Tue, 04 Nov 2025 10:35:25 -0500 X-MC-Unique: DPHnQ0DDNrmlRMW7_SBuVw-1 X-Mimecast-MFC-AGG-ID: DPHnQ0DDNrmlRMW7_SBuVw_1762270524 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 68B2B19560AD; Tue, 4 Nov 2025 15:35:24 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.225.23]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A6B931956056; Tue, 4 Nov 2025 15:35:20 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v15 3/8] libcamera: simple: Handle processed and raw formats separately Date: Tue, 4 Nov 2025 16:34:54 +0100 Message-ID: <20251104153501.34362-4-mzamazal@redhat.com> In-Reply-To: <20251104153501.34362-1-mzamazal@redhat.com> References: <20251104153501.34362-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: BJajA2r056JzyhMf4qLs1usKoM0rFwnDquH8zhpSzjA_1762270524 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let's handle both processed and/or raw output configurations. In addition to the already handled processed formats and sizes, this patch adds handling of raw formats and sizes, which correspond to the capture formats and sizes. When creating stream configurations, raw or processed formats are selected according to the requested stream roles. This is another preparatory patch without making raw outputs working. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 79 +++++++++++++++++------- 1 file changed, 56 insertions(+), 23 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index ba8a22ccc..7961a9f02 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1344,42 +1344,75 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span> formats; + bool processedRequested = false; + bool rawRequested = false; + for (const auto &role : roles) + if (role == StreamRole::Raw) { + if (rawRequested) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return nullptr; + } + rawRequested = true; + } else { + processedRequested = true; + } + + /* Create the formats maps. */ + std::map> processedFormats; + std::map> rawFormats; for (const SimpleCameraData::Configuration &cfg : data->configs_) { + rawFormats[cfg.captureFormat].push_back(cfg.captureSize); for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); + processedFormats[format].push_back(cfg.outputSizes); } - /* Sort the sizes and merge any consecutive overlapping ranges. */ - for (auto &[format, sizes] : formats) { - std::sort(sizes.begin(), sizes.end(), - [](SizeRange &a, SizeRange &b) { - return a.min < b.min; - }); - - auto cur = sizes.begin(); - auto next = cur; - - while (++next != sizes.end()) { - if (cur->max.width >= next->min.width && - cur->max.height >= next->min.height) - cur->max = next->max; - else if (++cur != next) - *cur = *next; - } - - sizes.erase(++cur, sizes.end()); + if (processedRequested && processedFormats.empty()) { + LOG(SimplePipeline, Error) + << "Processed stream requested but no corresponding output configuration found"; + return nullptr; + } + if (rawRequested && rawFormats.empty()) { + LOG(SimplePipeline, Error) + << "Raw stream requested but no corresponding output configuration found"; + return nullptr; } + auto setUpFormatSizes = [](std::map> &formats) { + /* Sort the sizes and merge any consecutive overlapping ranges. */ + + for (auto &[format, sizes] : formats) { + std::sort(sizes.begin(), sizes.end(), + [](SizeRange &a, SizeRange &b) { + return a.min < b.min; + }); + + auto cur = sizes.begin(); + auto next = cur; + + while (++next != sizes.end()) { + if (cur->max.width >= next->min.width && + cur->max.height >= next->min.height) + cur->max = next->max; + else if (++cur != next) + *cur = *next; + } + + sizes.erase(++cur, sizes.end()); + } + }; + setUpFormatSizes(processedFormats); + setUpFormatSizes(rawFormats); + /* * Create the stream configurations. Take the first entry in the formats * map as the default, for lack of a better option. * * \todo Implement a better way to pick the default format */ - for ([[maybe_unused]] StreamRole role : roles) { + for (StreamRole role : roles) { + const auto &formats = (role == StreamRole::Raw ? rawFormats : processedFormats); StreamConfiguration cfg{ StreamFormats{ formats } }; cfg.pixelFormat = formats.begin()->first; cfg.size = formats.begin()->second[0].max; From patchwork Tue Nov 4 15:34:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24971 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4BD6DC3241 for ; Tue, 4 Nov 2025 15:35:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F024F60AA0; Tue, 4 Nov 2025 16:35:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="E2SF2AIg"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 25D5260A7B for ; Tue, 4 Nov 2025 16:35:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762270534; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mtKPlHH1fnav8HofrCbcoz1kzXHi2KHbTyYEyw4Mgus=; b=E2SF2AIgKUEWknfKt/g4RnlMhA5BxZI1/wmiaOAV3DkFROR5lO2maW/nHV3993sUiuTe2/ N0aaa9KTurNetvFfe07MPayx5PvxSQrkoGtgxXh4qXwl9LNq2t9jXLy8sg3e+3yOa1i6zz 6HE/t3gGbuIUQdZOcnVwCzOnfx95oPY= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-84-M7JbgyBbNbumnRNin-vtpA-1; Tue, 04 Nov 2025 10:35:29 -0500 X-MC-Unique: M7JbgyBbNbumnRNin-vtpA-1 X-Mimecast-MFC-AGG-ID: M7JbgyBbNbumnRNin-vtpA_1762270528 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D585E180035A; Tue, 4 Nov 2025 15:35:27 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.225.23]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id F01C019540DE; Tue, 4 Nov 2025 15:35:24 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v15 4/8] libcamera: simple: Validate raw stream configurations Date: Tue, 4 Nov 2025 16:34:55 +0100 Message-ID: <20251104153501.34362-5-mzamazal@redhat.com> In-Reply-To: <20251104153501.34362-1-mzamazal@redhat.com> References: <20251104153501.34362-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: gUANpa_dmYjyiOj-9dFz-M2LC_YJ4WpPA11g0y6AcDw_1762270528 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. Raw streams are adjusted from the capture format and size. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. If the resolutions don't match mutually or don't match the available sizes, validation adjusts them. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Signed-off-by: Milan Zamazal Co-developed-by: Umang Jain Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 89 +++++++++++++++++------- 1 file changed, 63 insertions(+), 26 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 7961a9f02..9dc862f89 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -27,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -1140,21 +1142,42 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() LOG(SimplePipeline, Debug) << "Largest stream size is " << maxStreamSize; + /* Cap the number of raw stream configuration */ + unsigned int rawCount = 0; + PixelFormat requestedRawFormat; + for (const StreamConfiguration &cfg : config_) { + if (!isRaw(cfg)) + continue; + requestedRawFormat = cfg.pixelFormat; + rawCount++; + } + + if (rawCount > 1) { + LOG(SimplePipeline, Error) + << "Camera configuration with multiple raw streams not supported"; + return Invalid; + } + /* * Find the best configuration for the pipeline using a heuristic. - * First select the pixel format based on the streams (which are - * considered ordered from highest to lowest priority). Default to the - * first pipeline configuration if no streams request a supported pixel - * format. + * First select the pixel format based on the raw streams followed by + * non-raw streams (which are considered ordered from highest to lowest + * priority). Default to the first pipeline configuration if no streams + * request a supported pixel format. */ const std::vector *configs = &data_->formats_.begin()->second; - for (const StreamConfiguration &cfg : config_) { - auto it = data_->formats_.find(cfg.pixelFormat); - if (it != data_->formats_.end()) { - configs = &it->second; - break; + auto rawIter = data_->formats_.find(requestedRawFormat); + if (rawIter != data_->formats_.end()) { + configs = &rawIter->second; + } else { + for (const StreamConfiguration &cfg : config_) { + auto it = data_->formats_.find(cfg.pixelFormat); + if (it != data_->formats_.end()) { + configs = &it->second; + break; + } } } @@ -1214,21 +1237,35 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + const bool raw = isRaw(cfg); /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); - - PixelFormat pixelFormat = *it; - if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) - << "Adjusting pixel format from " - << cfg.pixelFormat << " to " << pixelFormat; - cfg.pixelFormat = pixelFormat; - status = Adjusted; + if (raw) { + if (cfg.pixelFormat != pipeConfig_->captureFormat || + cfg.size != pipeConfig_->captureSize) { + cfg.pixelFormat = pipeConfig_->captureFormat; + cfg.size = pipeConfig_->captureSize; + + LOG(SimplePipeline, Debug) + << "Adjusting raw stream to " + << cfg.toString(); + status = Adjusted; + } + } else { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + + PixelFormat pixelFormat = *it; + if (cfg.pixelFormat != pixelFormat) { + LOG(SimplePipeline, Debug) + << "Adjusting processed pixel format from " + << cfg.pixelFormat << " to " << pixelFormat; + cfg.pixelFormat = pixelFormat; + status = Adjusted; + } } /* @@ -1239,7 +1276,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * case, perform the standard pixel format based color space adjustment. */ if (!cfg.colorSpace) { - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); switch (info.colourEncoding) { case PixelFormatInfo::ColourEncodingRGB: cfg.colorSpace = ColorSpace::Srgb; @@ -1255,14 +1292,14 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() << cfg.colorSpace.value().toString(); status = Adjusted; } - if (cfg.colorSpace->adjust(pixelFormat)) { + if (cfg.colorSpace->adjust(cfg.pixelFormat)) { LOG(SimplePipeline, Debug) << "Color space adjusted to " << cfg.colorSpace.value().toString(); status = Adjusted; } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1285,7 +1322,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() needConversion_ = true; /* Set the stride and frameSize. */ - if (needConversion_) { + if (needConversion_ && !raw) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat, From patchwork Tue Nov 4 15:34:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24972 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EC7C3C3241 for ; Tue, 4 Nov 2025 15:35:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A8A2160AA1; Tue, 4 Nov 2025 16:35:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Oxcxx944"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 12C9560AA7 for ; Tue, 4 Nov 2025 16:35:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762270535; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ah3Cm7rkhSaLT0qFsv3XmcH3q0HC+pZvo5305B5hRnw=; b=Oxcxx944dq6RXwcZOPUOEbeRqtSsbb4p+TCVJ5fYiMGJdLAIEZ1jyfdG6YmS4zH2pWqy6e R+zMoxgo8F9kITPimkcoWAuyTVAmLHG0g9Vd06JD/VrL7pdzdGvtNu2g1Qa4t/fHVMgWAg 5TB46jw+uGGLr1CZWC0sb1Kn0Fga1XU= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-690-rpRXkjTzMnacHrYg4Wuq5g-1; Tue, 04 Nov 2025 10:35:32 -0500 X-MC-Unique: rpRXkjTzMnacHrYg4Wuq5g-1 X-Mimecast-MFC-AGG-ID: rpRXkjTzMnacHrYg4Wuq5g_1762270531 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 572D118009C2; Tue, 4 Nov 2025 15:35:31 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.225.23]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 864431956056; Tue, 4 Nov 2025 15:35:28 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v15 5/8] libcamera: simple: Don't enforce conversion with an added raw stream Date: Tue, 4 Nov 2025 16:34:56 +0100 Message-ID: <20251104153501.34362-6-mzamazal@redhat.com> In-Reply-To: <20251104153501.34362-1-mzamazal@redhat.com> References: <20251104153501.34362-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: dNoYIrtdYdDLt4v6qyLo_ahqkPOsICPaQG_lSXAJaPk_1762270531 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, either alone or together with a processed stream, it can be produced without conversion. Let's amend the corresponding check on the number of configurations, so that the mere presence of a raw stream doesn't enforce conversion. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 9dc862f89..454921879 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1233,7 +1233,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * require any conversion, similar to raw capture use cases). This is * left as a future improvement. */ - needConversion_ = config_.size() > 1; + needConversion_ = config_.size() > 1 + rawCount; for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; From patchwork Tue Nov 4 15:34:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24973 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8E3C9C3241 for ; Tue, 4 Nov 2025 15:35:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 435C760A9D; Tue, 4 Nov 2025 16:35:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="c/R97OJX"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D2ACD60A86 for ; Tue, 4 Nov 2025 16:35:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762270539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dG66jIvadcBJ2nh6U7xriomSSJGJvU9eQ+YQXQu+j7c=; b=c/R97OJX+NHIHHMs8WoXKxoYned4fw8SRNY3fqZXtq0qo1pcOyGYD0PLgGtzaYCYXhztF/ IPoO6BN1l2CWTl+EDZZa6HqwvS3c91g3w+qt+FFtn77RISQ0TOpFrPNUTlCWA2zhzdIdQU M73LfGC0ZK6+Fl8TZiysjySBnue0s50= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-237-72iaolpAO8iGIB2YowoPpQ-1; Tue, 04 Nov 2025 10:35:36 -0500 X-MC-Unique: 72iaolpAO8iGIB2YowoPpQ-1 X-Mimecast-MFC-AGG-ID: 72iaolpAO8iGIB2YowoPpQ_1762270535 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9A60E18001E8; Tue, 4 Nov 2025 15:35:34 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.225.23]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CE32719540DD; Tue, 4 Nov 2025 15:35:31 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v15 6/8] libcamera: simple: Set the number of software ISP streams to 2 Date: Tue, 4 Nov 2025 16:34:57 +0100 Message-ID: <20251104153501.34362-7-mzamazal@redhat.com> In-Reply-To: <20251104153501.34362-1-mzamazal@redhat.com> References: <20251104153501.34362-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 2gdC775EJWALmulKMYmsa5SNtBT61MghVoczASdywPg_1762270535 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When software ISP is enabled, we want to be able to provide a raw stream in addition to the processed stream. For this purpose, we need two streams. If only the processed stream is requested, it doesn't harm to allocate two. The number of streams is determined as a camera property in the pipeline matching. To be able to produce both raw and processed output, two streams must be provided. The actual number of streams needed (one or two) is determined only in SimplePipelineHandler::validate(). In theory, software ISP could produce multiple processed streams but this is out of scope of this patch series. Hence two streams are sufficient at the moment. When software ISP is not enabled, the camera won't be able to produce multiple streams (assuming there's no hardware converter) and only single stream should be allocated as before. The simple pipeline handler assumes there's a linear pipeline from the camera sensor to a video capture device, and only supports a single stream. Branches in the hardware pipeline that would allow capturing multiple streams from the same camera sensor are not supported. We have no plan to change that, as a device that can produce multiple streams will likely be better supported by a dedicated pipeline handler. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 454921879..06123ac82 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1810,6 +1810,16 @@ bool SimplePipelineHandler::matchDevice(MediaDevice *media, } } + if (info.swIspEnabled) { + /* + * When the software ISP is enabled, the simple pipeline handler + * exposes the raw stream, giving a total of two streams. This + * is mutually exclusive with the presence of a converter. + */ + ASSERT(!converter_); + numStreams = 2; + } + swIspEnabled_ = info.swIspEnabled; const GlobalConfiguration &configuration = cameraManager()->_d()->configuration(); for (GlobalConfiguration::Configuration entry : From patchwork Tue Nov 4 15:34:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24974 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 37227C3241 for ; Tue, 4 Nov 2025 15:35:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E60F360AAC; Tue, 4 Nov 2025 16:35:44 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="J0EF9L5i"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BA30D60A86 for ; Tue, 4 Nov 2025 16:35:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762270542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b+e11SSyenBEG/lQ/Uk2u5ejPmXQBfRL1XVj2uvyZ6w=; b=J0EF9L5i/YkV1/g5TkcGQiXf2yiDTTAra4TOKoSyhSgk784NHk88MS3o5qVk3zQ1UB81cS X0QoZ1VhY3o5Xh7WuiUTk7j6idwk2YWGu5RPpgZsRXtm5VqndLpZ9+qCpcgguRKE/H665h FZ9MOPl/V0MjNU6AFXeNdeV7i0QX04I= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-608-GRSOMbLYPf2kAIj0nnxhIg-1; Tue, 04 Nov 2025 10:35:38 -0500 X-MC-Unique: GRSOMbLYPf2kAIj0nnxhIg-1 X-Mimecast-MFC-AGG-ID: GRSOMbLYPf2kAIj0nnxhIg_1762270537 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9340E1956073; Tue, 4 Nov 2025 15:35:37 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.225.23]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1D6381956056; Tue, 4 Nov 2025 15:35:34 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v15 7/8] libcamera: simple: Require metadata only when software ISP is used Date: Tue, 4 Nov 2025 16:34:58 +0100 Message-ID: <20251104153501.34362-8-mzamazal@redhat.com> In-Reply-To: <20251104153501.34362-1-mzamazal@redhat.com> References: <20251104153501.34362-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 6gw4gAmYslTlKrUZXMvmlSsmtmcR041GXRZnrexghTk_1762270537 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If software ISP is enabled then metadata is required in the simple pipeline. But this doesn't apply if the software ISP is not actually used, for example when only a raw stream is produced. Then the pipeline waits for metadata that never comes. This patch fixes the problem by requiring metadata only when software ISP is used. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 06123ac82..cb0b24f38 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1672,6 +1672,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) int ret; std::map buffers; + bool metadataRequired = false; for (auto &[stream, buffer] : request->buffers()) { /* @@ -1681,6 +1682,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) */ if (data->useConversion_) { buffers.emplace(stream, buffer); + metadataRequired = !!data->swIsp_; } else { ret = data->video_->queueBuffer(buffer); if (ret < 0) @@ -1688,7 +1690,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } - data->frameInfo_.create(request, !!data->swIsp_); + data->frameInfo_.create(request, metadataRequired); if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); if (data->swIsp_) From patchwork Tue Nov 4 15:34:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24975 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DC504C3241 for ; Tue, 4 Nov 2025 15:35:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8E85760AA4; Tue, 4 Nov 2025 16:35:46 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Tg8m3qY5"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 693F560A86 for ; Tue, 4 Nov 2025 16:35:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1762270543; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lYBXMC9zUmMe6bGofQCu3+cFT9jkjogV6Sr8JmDzYgQ=; b=Tg8m3qY5zdeRpoluBQTzdTat3gcXcNZqxBkeWgUjV8kZECEvGAk8GAlyyDUTMDQFQsuRio uXHfbqBj8Ww2lmVsPyJDXwLFVYe8w4mOS4P9+09EX9mKFhsWc6PInqRsvM8WX4PegOHOnI OPeM96q/lIAF4v9D6wFS42oY4FVqWFA= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-186-sOpmrLwCNG-Sf3pJTu5PCw-1; Tue, 04 Nov 2025 10:35:42 -0500 X-MC-Unique: sOpmrLwCNG-Sf3pJTu5PCw-1 X-Mimecast-MFC-AGG-ID: sOpmrLwCNG-Sf3pJTu5PCw_1762270540 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BDD08195422D; Tue, 4 Nov 2025 15:35:40 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.225.23]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2F10319540DE; Tue, 4 Nov 2025 15:35:37 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v15 8/8] libcamera: simple: Make raw streams working Date: Tue, 4 Nov 2025 16:34:59 +0100 Message-ID: <20251104153501.34362-9-mzamazal@redhat.com> In-Reply-To: <20251104153501.34362-1-mzamazal@redhat.com> References: <20251104153501.34362-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: cNcJRfJIaaUPPDCmM8Ks_neLukCRvoLmqp4GVmHmGjI_1762270540 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, whether alone or together with a processed stream, its buffers must be handled outside the software ISP machinery. They serve as output buffers, even when a processed stream is produced. At most one raw stream and at most one processed stream are supported and can be combined. An example of producing both raw and processed files using `cam' application: cam -c1 -C100 -Ffile# \ -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \ -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \ Note the difference in viewfinder and raw stream sizes due to the fact that debayering requires enlarging the image width, which enforces selecting a larger sensor resolution in this case. In order to track whether a raw stream is requested and which one it is, SimpleCameraData::rawStream_ member variable is introduced. This is the final step to make raw streams working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 41 +++++++++++++++++------- 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index cb0b24f38..dd32b70f1 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -333,6 +333,7 @@ public: }; std::vector streams_; + Stream *rawStream_; /* * All entities in the pipeline, from the camera sensor to the video @@ -467,7 +468,7 @@ private: SimpleCameraData::SimpleCameraData(SimplePipelineHandler *pipe, unsigned int numStreams, MediaEntity *sensor) - : Camera::Private(pipe), streams_(numStreams) + : Camera::Private(pipe), streams_(numStreams), rawStream_(nullptr) { /* * Find the shortest path from the camera sensor to a video capture @@ -878,10 +879,13 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * point converting an erroneous buffer. */ if (buffer->metadata().status != FrameMetadata::FrameSuccess) { - if (!useConversion_) { + if (!useConversion_ || rawStream_) { /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); + SimpleFrameInfo *info = frameInfo_.find(request->sequence()); + if (info) + info->metadataRequired = false; tryCompleteRequest(request); return; } @@ -940,7 +944,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) */ if (useConversion_) { if (conversionQueue_.empty()) { - video_->queueBuffer(buffer); + if (!rawStream_) + video_->queueBuffer(buffer); return; } @@ -956,6 +961,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) conversionQueue_.front().outputs); conversionQueue_.pop(); + if (rawStream_) + pipe->completeBuffer(request, buffer); return; } @@ -993,7 +1000,8 @@ void SimpleCameraData::tryCompleteRequest(Request *request) void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ - video_->queueBuffer(buffer); + if (!rawStream_) + video_->queueBuffer(buffer); } void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) @@ -1518,13 +1526,18 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) std::vector> outputCfgs; data->useConversion_ = config->needConversion(); + data->rawStream_ = nullptr; for (unsigned int i = 0; i < config->size(); ++i) { StreamConfiguration &cfg = config->at(i); + bool rawStream = isRaw(cfg); cfg.setStream(&data->streams_[i]); - if (data->useConversion_ && !isRaw(cfg)) + if (data->useConversion_ && !rawStream) outputCfgs.push_back(cfg); + + if (rawStream) + data->rawStream_ = &data->streams_[i]; } if (outputCfgs.empty()) @@ -1555,7 +1568,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ - if (data->useConversion_) + if (data->useConversion_ && stream != data->rawStream_) return data->converter_ ? data->converter_->exportBuffers(stream, count, buffers) : data->swIsp_->exportBuffers(stream, count, buffers); @@ -1578,7 +1591,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return -EBUSY; } - if (data->useConversion_) { + if (data->useConversion_ && !data->rawStream_) { /* * When using the converter allocate a fixed number of internal * buffers. @@ -1586,8 +1599,11 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL ret = video->allocateBuffers(kNumInternalBuffers, &data->conversionBuffers_); } else { - /* Otherwise, prepare for using buffers from the only stream. */ - Stream *stream = &data->streams_[0]; + /* + * Otherwise, prepare for using buffers from either the raw stream, if + * requested, or the only stream configured. + */ + Stream *stream = (data->rawStream_ ? data->rawStream_ : &data->streams_[0]); ret = video->importBuffers(stream->configuration().bufferCount); } if (ret < 0) { @@ -1628,8 +1644,9 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } /* Queue all internal buffers for capture. */ - for (std::unique_ptr &buffer : data->conversionBuffers_) - video->queueBuffer(buffer.get()); + if (!data->rawStream_) + for (std::unique_ptr &buffer : data->conversionBuffers_) + video->queueBuffer(buffer.get()); } return 0; @@ -1680,7 +1697,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * queue, it will be handed to the converter in the capture * completion handler. */ - if (data->useConversion_) { + if (data->useConversion_ && stream != data->rawStream_) { buffers.emplace(stream, buffer); metadataRequired = !!data->swIsp_; } else {