From patchwork Mon Aug 4 16:38:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24048 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 117F5BDCC1 for ; Mon, 4 Aug 2025 16:38:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BA67969217; Mon, 4 Aug 2025 18:38:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Cn0nk235"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2621669217 for ; Mon, 4 Aug 2025 18:38:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754325514; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2zzdDiFpTkEAZOr41i3alVO1cb4kj/y3CNtCSLhSdps=; b=Cn0nk235FJ0Rrq0OPZ65G8RbgWwAe6qVQ+S8IW0jOQgwIK1Vf7ke95lBW2pnBDY9buNEQ6 hO4Uk80KT2+evRqXYu+M//F/xpUvEcUcyOWzwhT/RkK3WyzTeXkfHjr7RNymzyxsat+a+G m7DmrAWbIXlBc4EHwG8SEKSQrrE9naU= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-617-Asg6FnUeM9Or6_-8NrgQbQ-1; Mon, 04 Aug 2025 12:38:31 -0400 X-MC-Unique: Asg6FnUeM9Or6_-8NrgQbQ-1 X-Mimecast-MFC-AGG-ID: Asg6FnUeM9Or6_-8NrgQbQ_1754325509 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8E1DF195DB9A; Mon, 4 Aug 2025 16:38:29 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.17]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9663E1800EF2; Mon, 4 Aug 2025 16:38:21 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v12 1/8] libcamera: software_isp: Assign colour spaces in configurations Date: Mon, 4 Aug 2025 18:38:04 +0200 Message-ID: <20250804163812.126022-2-mzamazal@redhat.com> In-Reply-To: <20250804163812.126022-1-mzamazal@redhat.com> References: <20250804163812.126022-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: fEYx1_nQM3pFMHpiRnTil6KOvY1GPr2T_UZ_RR67rXc_1754325509 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" StreamConfiguration's should have colorSpace set. This is not the case in the simple pipeline. Let's set it there. This also fixes a crash in `cam' due to accessing an unset colorSpace. We set the colour spaces according to the pixel format. This is not completely correct because pixel formats and colour spaces are different, although not completely independent, things. But for the lack of a better practical option to determine the colour space, we use this. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 33 ++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 4323472e1..50a7e65fb 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -35,6 +36,7 @@ #include "libcamera/internal/converter.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/software_isp/software_isp.h" @@ -1212,6 +1214,37 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } + /* + * Best effort to fix the color space. If the color space is not set, + * set it according to the pixel format, which may not be correct (pixel + * formats and color spaces are different things, although somewhat + * related) but we don't have a better option at the moment. Then in any + * case, perform the standard pixel format based color space adjustment. + */ + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } + LOG(SimplePipeline, Debug) + << "Unspecified color space set to " + << cfg.colorSpace.value().toString(); + status = Adjusted; + } + if (cfg.colorSpace->adjust(pixelFormat)) { + LOG(SimplePipeline, Debug) + << "Color space adjusted to " + << cfg.colorSpace.value().toString(); + status = Adjusted; + } + if (!pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* From patchwork Mon Aug 4 16:38:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24049 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 153CBBDCC1 for ; Mon, 4 Aug 2025 16:38:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4747D69224; Mon, 4 Aug 2025 18:38:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Z24z4ioz"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 05B7A69217 for ; Mon, 4 Aug 2025 18:38:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754325515; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NL0enjGHLSxlH4XHz1Eu6jgkBqL/nZ1qcNeL4kWD0RI=; b=Z24z4ioz4ujazLMRW6n6Fqg/hAuGepiSH4xOfrEu56WnwBlIfRqZBRhim0Zi0RuGCpyQMS mAriqBay5jpyhTokX9veVgp+SPhv2QdtOpNhS8XCye6RSTbJN7EFQZnKudm1VpXFDCr6cK dR+wtrH5bOXyivGipIkTeJ10gKNheTA= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-295-VPJc3bxzMxOjLwX_IgbmOA-1; Mon, 04 Aug 2025 12:38:33 -0400 X-MC-Unique: VPJc3bxzMxOjLwX_IgbmOA-1 X-Mimecast-MFC-AGG-ID: VPJc3bxzMxOjLwX_IgbmOA_1754325512 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2B62F19773E8; Mon, 4 Aug 2025 16:38:32 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.17]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 11A941800D8A; Mon, 4 Aug 2025 16:38:29 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v12 2/8] libcamera: simple: Exclude raw configurations from output conversions Date: Mon, 4 Aug 2025 18:38:05 +0200 Message-ID: <20250804163812.126022-3-mzamazal@redhat.com> In-Reply-To: <20250804163812.126022-1-mzamazal@redhat.com> References: <20250804163812.126022-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Jqh9oXEh-h_PRuOyaqjTqsfq5d2lBMzX7jCrgiuTtNs_1754325512 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to support raw streams, we need to add raw formats to software ISP configurations. In this preparatory patch, the raw formats are excluded from output configurations for conversions. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 50a7e65fb..5fba94170 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -261,6 +262,12 @@ static const SimplePipelineInfo supportedDevices[] = { { "sun6i-csi", {}, false }, }; +bool isRaw(const StreamConfiguration &cfg) +{ + return libcamera::PixelFormatInfo::info(cfg.pixelFormat).colourEncoding == + libcamera::PixelFormatInfo::ColourEncodingRAW; +} + } /* namespace */ class SimpleCameraData : public Camera::Private @@ -1424,7 +1431,7 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) cfg.setStream(&data->streams_[i]); - if (data->useConversion_) + if (data->useConversion_ && !isRaw(cfg)) outputCfgs.push_back(cfg); } From patchwork Mon Aug 4 16:38:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24050 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2597CBDCC1 for ; Mon, 4 Aug 2025 16:38:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D15DF6921D; Mon, 4 Aug 2025 18:38:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="TY3LxD1C"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D908C69222 for ; Mon, 4 Aug 2025 18:38:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754325517; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t2eAK69fV8Rhan1y/6XqS9mKSQ3cDd64kJTWrZsD6Kw=; b=TY3LxD1CD9n1tz+/Mesly5tG2rbzIBV44A+iT5x4RZMCYy8QH9IIsSLeeVNl2PbKUwY1Y7 DQUYaVzhveSM4Uu6A/CGIQBR4qyT8bFtAeffsTeGyFXtpeVDdF+vyilbuN9ZRptJoEETCC 3AH68ugHl5JAmWmIeftycGcDMoXMttI= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-294-lxhA4xuFPhyYYit0Lpm1ow-1; Mon, 04 Aug 2025 12:38:36 -0400 X-MC-Unique: lxhA4xuFPhyYYit0Lpm1ow-1 X-Mimecast-MFC-AGG-ID: lxhA4xuFPhyYYit0Lpm1ow_1754325515 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E0A601936D82; Mon, 4 Aug 2025 16:38:34 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.17]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9A2771800EF0; Mon, 4 Aug 2025 16:38:32 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v12 3/8] libcamera: simple: Handle processed and raw formats separately Date: Mon, 4 Aug 2025 18:38:06 +0200 Message-ID: <20250804163812.126022-4-mzamazal@redhat.com> In-Reply-To: <20250804163812.126022-1-mzamazal@redhat.com> References: <20250804163812.126022-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: gjnNsilODTBP4KiRWubttVKFcUk72wOQ7mEStcXCgSA_1754325515 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let's handle both processed and/or raw output configurations. In addition to the already handled processed formats and sizes, this patch adds handling of raw formats and sizes, which correspond to the capture formats and sizes. When creating stream configurations, raw or processed formats are selected according to the requested stream roles. This is another preparatory patch without making raw outputs working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 79 +++++++++++++++++------- 1 file changed, 56 insertions(+), 23 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 5fba94170..bc4d7bd41 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1322,42 +1322,75 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span> formats; + bool processedRequested = false; + bool rawRequested = false; + for (const auto &role : roles) + if (role == StreamRole::Raw) { + if (rawRequested) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return nullptr; + } + rawRequested = true; + } else { + processedRequested = true; + } + + /* Create the formats maps. */ + std::map> processedFormats; + std::map> rawFormats; for (const SimpleCameraData::Configuration &cfg : data->configs_) { + rawFormats[cfg.captureFormat].push_back(cfg.captureSize); for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); + processedFormats[format].push_back(cfg.outputSizes); } - /* Sort the sizes and merge any consecutive overlapping ranges. */ - for (auto &[format, sizes] : formats) { - std::sort(sizes.begin(), sizes.end(), - [](SizeRange &a, SizeRange &b) { - return a.min < b.min; - }); - - auto cur = sizes.begin(); - auto next = cur; - - while (++next != sizes.end()) { - if (cur->max.width >= next->min.width && - cur->max.height >= next->min.height) - cur->max = next->max; - else if (++cur != next) - *cur = *next; - } - - sizes.erase(++cur, sizes.end()); + if (processedRequested && processedFormats.empty()) { + LOG(SimplePipeline, Error) + << "Processed stream requsted but no corresponding output configuration found"; + return nullptr; + } + if (rawRequested && rawFormats.empty()) { + LOG(SimplePipeline, Error) + << "Raw stream requsted but no corresponding output configuration found"; + return nullptr; } + auto setUpFormatSizes = [](std::map> &formats) { + /* Sort the sizes and merge any consecutive overlapping ranges. */ + + for (auto &[format, sizes] : formats) { + std::sort(sizes.begin(), sizes.end(), + [](SizeRange &a, SizeRange &b) { + return a.min < b.min; + }); + + auto cur = sizes.begin(); + auto next = cur; + + while (++next != sizes.end()) { + if (cur->max.width >= next->min.width && + cur->max.height >= next->min.height) + cur->max = next->max; + else if (++cur != next) + *cur = *next; + } + + sizes.erase(++cur, sizes.end()); + } + }; + setUpFormatSizes(processedFormats); + setUpFormatSizes(rawFormats); + /* * Create the stream configurations. Take the first entry in the formats * map as the default, for lack of a better option. * * \todo Implement a better way to pick the default format */ - for ([[maybe_unused]] StreamRole role : roles) { + for (StreamRole role : roles) { + const auto &formats = (role == StreamRole::Raw ? rawFormats : processedFormats); StreamConfiguration cfg{ StreamFormats{ formats } }; cfg.pixelFormat = formats.begin()->first; cfg.size = formats.begin()->second[0].max; From patchwork Mon Aug 4 16:38:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24051 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BE06EBDCC1 for ; Mon, 4 Aug 2025 16:38:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7BC5B69222; Mon, 4 Aug 2025 18:38:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="eMg3vOil"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D665A6921A for ; Mon, 4 Aug 2025 18:38:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754325524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4g9Ccmh7piaYCtomDUi/3Vc8f39//YnbWrG7uEkgyPk=; b=eMg3vOilLvN/Zl+zgeAs0mJWeZOGg9rnGrDwaFYF3WjkYFwc0GB7POZ7yYfcicEIKr4Ot6 lQUmSKb5/4qAzMWyGHukJ0sRajdZ0hIctVSn4PVK9Rfccc0ymhxOWoCmJR2GXqy3NmqNpe HBNTNh450qV5T8bF/MQgkeEloIJNdzw= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-443-KKs8NfYpNk2g6R41s6vgzg-1; Mon, 04 Aug 2025 12:38:39 -0400 X-MC-Unique: KKs8NfYpNk2g6R41s6vgzg-1 X-Mimecast-MFC-AGG-ID: KKs8NfYpNk2g6R41s6vgzg_1754325518 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1442019774D9; Mon, 4 Aug 2025 16:38:38 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.17]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 60E261800D8A; Mon, 4 Aug 2025 16:38:35 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v12 4/8] libcamera: simple: Validate raw stream configurations Date: Mon, 4 Aug 2025 18:38:07 +0200 Message-ID: <20250804163812.126022-5-mzamazal@redhat.com> In-Reply-To: <20250804163812.126022-1-mzamazal@redhat.com> References: <20250804163812.126022-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: tJGyQF2ZPunryOC0fVdReTnC46f5GFgGMcgwGB5DZ58_1754325518 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. Raw streams are adjusted from the capture format and size. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. If the resolutions don't match mutually or don't match the available sizes, validation adjusts them. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 86 +++++++++++++++++------- 1 file changed, 62 insertions(+), 24 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index bc4d7bd41..1ad982a4c 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -27,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -1134,22 +1136,39 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() << "Largest stream size is " << maxStreamSize; /* - * Find the best configuration for the pipeline using a heuristic. - * First select the pixel format based on the streams (which are - * considered ordered from highest to lowest priority). Default to the - * first pipeline configuration if no streams request a supported pixel - * format. + * Find the best configuration for the pipeline using a heuristic. First + * select the pixel format based on the streams. If there is a raw stream, + * its format has precedence. If there is no raw stream, the streams are + * considered ordered from highest to lowest priority. Default to the first + * pipeline configuration if no streams request a supported pixel format. */ - const std::vector *configs = - &data_->formats_.begin()->second; + std::optional rawFormat; + for (const auto &cfg : config_) + if (isRaw(cfg)) { + if (rawFormat) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return Invalid; + } + rawFormat = cfg.pixelFormat; + } - for (const StreamConfiguration &cfg : config_) { - auto it = data_->formats_.find(cfg.pixelFormat); - if (it != data_->formats_.end()) { + const std::vector *configs = nullptr; + if (rawFormat) { + auto it = data_->formats_.find(rawFormat.value()); + if (it != data_->formats_.end()) configs = &it->second; - break; - } } + if (!configs) + for (const StreamConfiguration &cfg : config_) { + auto it = data_->formats_.find(cfg.pixelFormat); + if (it != data_->formats_.end()) { + configs = &it->second; + break; + } + } + if (!configs) + configs = &data_->formats_.begin()->second; /* * \todo Pick the best sensor output media bus format when the @@ -1206,17 +1225,27 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + const bool raw = isRaw(cfg); - /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); + /* Adjust the pixel format, colour space and size. */ - PixelFormat pixelFormat = *it; + PixelFormat pixelFormat; + if (raw) { + pixelFormat = pipeConfig_->captureFormat; + } else { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + pixelFormat = *it; + } if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) << "Adjusting pixel format"; + LOG(SimplePipeline, Debug) + << "Adjusting pixel format of a " + << (raw ? "raw" : "processed") + << " stream from " << cfg.pixelFormat + << " to " << pixelFormat; cfg.pixelFormat = pixelFormat; status = Adjusted; } @@ -1252,7 +1281,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1271,11 +1300,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* \todo Create a libcamera core class to group format and size */ if (cfg.pixelFormat != pipeConfig_->captureFormat || - cfg.size != pipeConfig_->captureSize) - needConversion_ = true; + cfg.size != pipeConfig_->captureSize) { + if (raw) { + cfg.pixelFormat = pipeConfig_->captureFormat; + cfg.size = pipeConfig_->captureSize; + LOG(SimplePipeline, Debug) + << "Adjusting raw configuration to " << cfg; + status = Adjusted; + } else { + needConversion_ = true; + } + } /* Set the stride, frameSize and bufferCount. */ - if (needConversion_) { + if (needConversion_ && !raw) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat, From patchwork Mon Aug 4 16:38:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24053 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4A2DDBDCC1 for ; Mon, 4 Aug 2025 16:38:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F30C269224; Mon, 4 Aug 2025 18:38:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="W4BE4gvI"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C005F6921A for ; Mon, 4 Aug 2025 18:38:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754325530; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4kvyvMpbKewLq6FHdRUGhcdP5trXS8J8g89vavGk9SY=; b=W4BE4gvI/DkSoZJmwLeJ7haAm8354oeXK68qLKS+9G2oc8gaQ5Qwf6ki8cm/LsyiuQqHDh BlfzXMcX3RUfCx1u5C4Ydlf6S3ePV63u5Avy/sjBlkNvcg5FC1czyO9s2hJw/rLMmmYCIm d3jsJBD4doAZsOAB1LYNc/IFoV6xPls= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-490-VvDj7aXyOXSl8eop3BpXYQ-1; Mon, 04 Aug 2025 12:38:47 -0400 X-MC-Unique: VvDj7aXyOXSl8eop3BpXYQ-1 X-Mimecast-MFC-AGG-ID: VvDj7aXyOXSl8eop3BpXYQ_1754325526 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 577ED195D02A; Mon, 4 Aug 2025 16:38:41 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.17]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 988521800D8A; Mon, 4 Aug 2025 16:38:38 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v12 5/8] libcamera: simple: Don't enforce conversion with an added raw stream Date: Mon, 4 Aug 2025 18:38:08 +0200 Message-ID: <20250804163812.126022-6-mzamazal@redhat.com> In-Reply-To: <20250804163812.126022-1-mzamazal@redhat.com> References: <20250804163812.126022-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: QjUHhahE43tVo8go2aY5Z8WPQoN5guhp1Xufh-7rX4w_1754325526 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, either alone or together with a processed stream, it can be produced without conversion. Let's amend the corresponding check on the number of configurations, so that the mere presence of a raw stream doesn't enforce conversion. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 1ad982a4c..12fbe533e 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1221,7 +1221,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * require any conversion, similar to raw capture use cases). This is * left as a future improvement. */ - needConversion_ = config_.size() > 1; + needConversion_ = config_.size() > 1 + (rawFormat ? 1 : 0); for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; From patchwork Mon Aug 4 16:38:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24052 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 682E2BDCC1 for ; Mon, 4 Aug 2025 16:38:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1CABC6922D; Mon, 4 Aug 2025 18:38:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="NlKeFTNS"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5F4086921A for ; Mon, 4 Aug 2025 18:38:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754325528; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HHNzS8/IhpAzKwaxcf4Ii3lYeHiGhz0nVlAH3F+G9JU=; b=NlKeFTNSl0Wods6A4BLUHEiAGXnjrhXb0pBCVZuOhJnamPqm2i+qSTHyfAM3yidsg0CNGa oqNXA5W2fmFuK8OnUEB/RuI1s54oAy0VnefuMM3mTvvcOA0S4NhnsibErjBuTC6fH29nFg heP0V2sQpk0HCTFkrCzwhmsCu36GdvA= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-360-6omnnU-_MSWMosPh_ATETA-1; Mon, 04 Aug 2025 12:38:45 -0400 X-MC-Unique: 6omnnU-_MSWMosPh_ATETA-1 X-Mimecast-MFC-AGG-ID: 6omnnU-_MSWMosPh_ATETA_1754325524 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 83C2D19373E5; Mon, 4 Aug 2025 16:38:44 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.17]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id F013A1800EEC; Mon, 4 Aug 2025 16:38:41 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v12 6/8] libcamera: simple: Set the number of software ISP streams to 2 Date: Mon, 4 Aug 2025 18:38:09 +0200 Message-ID: <20250804163812.126022-7-mzamazal@redhat.com> In-Reply-To: <20250804163812.126022-1-mzamazal@redhat.com> References: <20250804163812.126022-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: L-yk3oMRFs3G20BqXHTK788IbLIHe-XT8hAfVFDHHbQ_1754325524 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When software ISP is enabled, we want to be able to provide a raw stream in addition to the processed stream. For this purpose, we need two streams. If only the processed stream is requested, it doesn't harm to allocate two. The number of streams is determined as a camera property in the pipeline matching. To be able to produce both raw and processed output, two streams must be provided. The actual number of streams needed (one or two) is determined only when examining roles in SimplePipelineHandler::generateConfiguration. In theory, software ISP could produce multiple processed streams but this is out of scope of this patch series. Hence two streams are sufficient at the moment. When software ISP is not enabled, the camera won't be able to produce multiple streams (assuming there's no hardware converter) and only single stream should be allocated as before. The simple pipeline handler assumes there's a linear pipeline from the camera sensor to a video capture device, and only supports a single stream. Branches in the hardware pipeline that would allow capturing multiple streams from the same camera sensor are not supported. We have no plan to change that, as a device that can produce multiple streams will likely be better supported by a dedicated pipeline handler. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain --- src/libcamera/pipeline/simple/simple.cpp | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 12fbe533e..46c316fd7 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1789,6 +1789,16 @@ bool SimplePipelineHandler::matchDevice(MediaDevice *media, } } + if (info.swIspEnabled) { + /* + * When the software ISP is enabled, the simple pipeline handler + * exposes the raw stream, giving a total of two streams. This + * is mutally exclusive with the presence of a converter. + */ + ASSERT(!converter_); + numStreams = 2; + } + swIspEnabled_ = info.swIspEnabled; /* Locate the sensors. */ From patchwork Mon Aug 4 16:38:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24055 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0773BBDCC1 for ; Mon, 4 Aug 2025 16:38:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ADCA569234; Mon, 4 Aug 2025 18:38:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="hJaDgXiu"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BB6026921A for ; Mon, 4 Aug 2025 18:38:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754325534; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eNRmcjFhC17bYyg+7w+oGTSxXnmYBLUutGTO0akIbew=; b=hJaDgXiumUjKWduC1j+fQPMrvSJ8uNCoHKLIDLJbJl7W1RBkEHl+YUr84bi0k8enEBeKoD 4qq1SjalVkou2iXMPSCAWNv/RbLoOC+G6f7DIdyFcGhAQYcHb4uyD3HxOrOf95GxZYAC7u X/T5gumVE3T7ZMHIOQrljoy8niBJZFA= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-224-b7mFQepBOjKAOWT3bRFEQQ-1; Mon, 04 Aug 2025 12:38:49 -0400 X-MC-Unique: b7mFQepBOjKAOWT3bRFEQQ-1 X-Mimecast-MFC-AGG-ID: b7mFQepBOjKAOWT3bRFEQQ_1754325528 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7AA811977408; Mon, 4 Aug 2025 16:38:47 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.17]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1E90B1800EF0; Mon, 4 Aug 2025 16:38:44 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v12 7/8] libcamera: simple: Require metadata only when software ISP is used Date: Mon, 4 Aug 2025 18:38:10 +0200 Message-ID: <20250804163812.126022-8-mzamazal@redhat.com> In-Reply-To: <20250804163812.126022-1-mzamazal@redhat.com> References: <20250804163812.126022-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: piFUaPx0HFdmcHrkqe-DyRGU_mZIUTxqKU7HqAIhELk_1754325528 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If software ISP is enabled then metadata is required in the simple pipeline. But this doesn't apply if the software ISP is not actually used, for example when only a raw stream is produced. Then the pipeline waits for metadata that never comes. This patch fixes the problem by requiring metadata only when software ISP is used. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 46c316fd7..e5449cd2e 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1651,6 +1651,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) int ret; std::map buffers; + bool metadataRequired = false; for (auto &[stream, buffer] : request->buffers()) { /* @@ -1660,6 +1661,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) */ if (data->useConversion_) { buffers.emplace(stream, buffer); + metadataRequired = !!data->swIsp_; } else { ret = data->video_->queueBuffer(buffer); if (ret < 0) @@ -1667,7 +1669,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } - data->frameInfo_.create(request, !!data->swIsp_); + data->frameInfo_.create(request, metadataRequired); if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); if (data->swIsp_) From patchwork Mon Aug 4 16:38:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 24054 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 50E1CBDCC1 for ; Mon, 4 Aug 2025 16:38:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0C2EC69222; Mon, 4 Aug 2025 18:38:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="TtNkPtYF"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EEF9E6921A for ; Mon, 4 Aug 2025 18:38:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754325535; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LbGkShpbs2PeI8uJ7JktmSapC2GLxkOsYGL3zgnHrvw=; b=TtNkPtYF/WGuITD3w6Y/MCWdpvVEPrlPlVoe35bDtaInCtfpoXoqk+AooK3MyuRozlqZSe FokxTdouAbSXk0qSxqCNb3SiR22PMhVU35Lny3JGYRpwFoMZSVWIiVuuzhAa9wrP6CT6Fy Z0rqrtmZfCCj4T/Q7h1kIWjM/2YE+Tg= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-318-NQtbNPNjNLWT_0bJB-NhfQ-1; Mon, 04 Aug 2025 12:38:51 -0400 X-MC-Unique: NQtbNPNjNLWT_0bJB-NhfQ-1 X-Mimecast-MFC-AGG-ID: NQtbNPNjNLWT_0bJB-NhfQ_1754325530 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8CB2819560B4; Mon, 4 Aug 2025 16:38:50 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.17]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 16EA01800EF4; Mon, 4 Aug 2025 16:38:47 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v12 8/8] libcamera: simple: Make raw streams working Date: Mon, 4 Aug 2025 18:38:11 +0200 Message-ID: <20250804163812.126022-9-mzamazal@redhat.com> In-Reply-To: <20250804163812.126022-1-mzamazal@redhat.com> References: <20250804163812.126022-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 7fBpVyOScgIDCIMwM5pI8wrXnKLelgkBaJSBUbeLkbg_1754325530 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, whether alone or together with a processed stream, its buffers must be handled outside the software ISP machinery. They serve as output buffers, even when a processed stream is produced. At most one raw stream and at most one processed stream are supported and can be combined. An example of producing both raw and processed files using `cam' application: cam -c1 -C100 -Ffile# \ -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \ -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \ Note the difference in viewfinder and raw stream sizes due to the fact that debayering requires enlarging the image width, which enforces selecting a larger sensor resolution in this case. In order to track whether a raw stream is requested and which one it is, SimpleCameraData::rawStream_ member variable is introduced. This is the final step to make raw streams working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 55 ++++++++++++++++++------ 1 file changed, 41 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index e5449cd2e..d854d6a72 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -330,6 +330,7 @@ public: }; std::vector streams_; + Stream *rawStream_; /* * All entities in the pipeline, from the camera sensor to the video @@ -368,6 +369,11 @@ private: void ispStatsReady(uint32_t frame, uint32_t bufferId); void metadataReady(uint32_t frame, const ControlList &metadata); void setSensorControls(const ControlList &sensorControls); + + bool processedRequested() const + { + return streams_.size() - (rawStream_ ? 1 : 0) > 0; + } }; class SimpleCameraConfiguration : public CameraConfiguration @@ -460,7 +466,7 @@ private: SimpleCameraData::SimpleCameraData(SimplePipelineHandler *pipe, unsigned int numStreams, MediaEntity *sensor) - : Camera::Private(pipe), streams_(numStreams) + : Camera::Private(pipe), streams_(numStreams), rawStream_(nullptr) { /* * Find the shortest path from the camera sensor to a video capture @@ -871,10 +877,13 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * point converting an erroneous buffer. */ if (buffer->metadata().status != FrameMetadata::FrameSuccess) { - if (!useConversion_) { + if (rawStream_) { /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); + SimpleFrameInfo *info = frameInfo_.find(request->sequence()); + if (info) + info->metadataRequired = false; tryCompleteRequest(request); return; } @@ -884,7 +893,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * buffer for capture (unless the stream is being stopped), and * complete the request with all the user-facing buffers. */ - if (buffer->metadata().status != FrameMetadata::FrameCancelled) + if (buffer->metadata().status != FrameMetadata::FrameCancelled && + !rawStream_) video_->queueBuffer(buffer); if (conversionQueue_.empty()) @@ -933,13 +943,14 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) */ if (useConversion_) { if (conversionQueue_.empty()) { - video_->queueBuffer(buffer); + if (!rawStream_) + video_->queueBuffer(buffer); return; } if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front().outputs); - else + else if (processedRequested()) /* * request->sequence() cannot be retrieved from `buffer' inside * queueBuffers because unique_ptr's make buffer->request() invalid @@ -949,6 +960,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) conversionQueue_.front().outputs); conversionQueue_.pop(); + if (rawStream_) + pipe->completeBuffer(request, buffer); return; } @@ -986,7 +999,8 @@ void SimpleCameraData::tryCompleteRequest(Request *request) void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ - video_->queueBuffer(buffer); + if (!rawStream_) + video_->queueBuffer(buffer); } void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) @@ -1499,11 +1513,20 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) for (unsigned int i = 0; i < config->size(); ++i) { StreamConfiguration &cfg = config->at(i); + bool rawStream = isRaw(cfg); cfg.setStream(&data->streams_[i]); - if (data->useConversion_ && !isRaw(cfg)) + if (data->useConversion_ && !rawStream) outputCfgs.push_back(cfg); + + if (rawStream) { + if (data->rawStream_) { + LOG(SimplePipeline, Error) << "Multiple raw streams not supported"; + return -EINVAL; + } + data->rawStream_ = &data->streams_[i]; + } } if (outputCfgs.empty()) @@ -1534,7 +1557,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ - if (data->useConversion_) + if (data->useConversion_ && stream != data->rawStream_) return data->converter_ ? data->converter_->exportBuffers(stream, count, buffers) : data->swIsp_->exportBuffers(stream, count, buffers); @@ -1557,7 +1580,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return -EBUSY; } - if (data->useConversion_) { + if (data->useConversion_ && !data->rawStream_) { /* * When using the converter allocate a fixed number of internal * buffers. @@ -1565,8 +1588,11 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL ret = video->allocateBuffers(kNumInternalBuffers, &data->conversionBuffers_); } else { - /* Otherwise, prepare for using buffers from the only stream. */ - Stream *stream = &data->streams_[0]; + /* + * Otherwise, prepare for using buffers from either the raw stream, if + * requested, or the only stream configured. + */ + Stream *stream = (data->rawStream_ ? data->rawStream_ : &data->streams_[0]); ret = video->importBuffers(stream->configuration().bufferCount); } if (ret < 0) { @@ -1607,8 +1633,9 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } /* Queue all internal buffers for capture. */ - for (std::unique_ptr &buffer : data->conversionBuffers_) - video->queueBuffer(buffer.get()); + if (!data->rawStream_) + for (std::unique_ptr &buffer : data->conversionBuffers_) + video->queueBuffer(buffer.get()); } return 0; @@ -1659,7 +1686,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * queue, it will be handed to the converter in the capture * completion handler. */ - if (data->useConversion_) { + if (data->useConversion_ && stream != data->rawStream_) { buffers.emplace(stream, buffer); metadataRequired = !!data->swIsp_; } else {