From patchwork Wed Jul 23 18:08:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23907 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DDF74C3237 for ; Wed, 23 Jul 2025 18:08:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 553D96906C; Wed, 23 Jul 2025 20:08:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ZDMGRdSI"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 52CBD69069 for ; Wed, 23 Jul 2025 20:08:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753294118; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TZqyNKO9CDLYZpC3z5nN6ndJwcsqu3fO8xRaefcfGfc=; b=ZDMGRdSIREVCiDE0R9HkaWHAR7i2RBuUNAovB2G5EelUyxRG/xNVXAkqF5EINOd/JYzULs WCd4JedwDLcLIZ0Q8hwUu1uQ+K7H0ErfkH43aEf+MRKDUUmR/qw7giSi25CKE3C9JPy4MI MiRas8n/nK6O22m7mhKNrSzeL5W3h9Q= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-641-RtI3BS6QObeBBwSSD3CaSA-1; Wed, 23 Jul 2025 14:08:36 -0400 X-MC-Unique: RtI3BS6QObeBBwSSD3CaSA-1 X-Mimecast-MFC-AGG-ID: RtI3BS6QObeBBwSSD3CaSA_1753294115 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1C7C81800359; Wed, 23 Jul 2025 18:08:35 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1CB1D195608D; Wed, 23 Jul 2025 18:08:31 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v11 1/8] libcamera: software_isp: Assign colour spaces in configurations Date: Wed, 23 Jul 2025 20:08:06 +0200 Message-ID: <20250723180815.82450-2-mzamazal@redhat.com> In-Reply-To: <20250723180815.82450-1-mzamazal@redhat.com> References: <20250723180815.82450-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: jnznUI8YH77ykEsH3Oy6Ng9Q7R8EDkbkgpp1uLg4_Vo_1753294115 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" StreamConfiguration's should have colorSpace set. This is not the case in the simple pipeline. Let's set it there. This also fixes a crash in `cam' due to accessing an unset colorSpace. The colour space is assigned as follows: - If raw role is requested, then the colour space must be raw, regardless of the stream format. - Otherwise, if software ISP is used, the default colour space is set to Srgb (YcbcrEncoding::None, Range::Full). - Otherwise, if a converter is used, the default colour space is left unset. - Then in configuration validation, if the pixel format is changed, the colour space is set according to the new pixel format. - If the colour space is still unset, it is set according to the resulting pixel format. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 38 +++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index efb07051b..d45480fe7 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -35,6 +36,7 @@ #include "libcamera/internal/converter.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/software_isp/software_isp.h" @@ -1206,6 +1208,28 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() if (cfg.pixelFormat != pixelFormat) { LOG(SimplePipeline, Debug) << "Adjusting pixel format"; cfg.pixelFormat = pixelFormat; + /* + * Do not touch the colour space for raw requested roles. + * Even if the pixel format is non-raw (whatever it means), we + * shouldn't try to interpret the colour space of raw data. + */ + if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + cfg.colorSpace->adjust(pixelFormat); + status = Adjusted; + } + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } + cfg.colorSpace->adjust(pixelFormat); status = Adjusted; } @@ -1314,11 +1338,23 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Spanfirst; cfg.size = formats.begin()->second[0].max; + if (role == StreamRole::Raw) { + /* Enforce raw colour space for raw roles. */ + cfg.colorSpace = ColorSpace::Raw; + } else if (data->swIsp_) { + /* + * This is what software ISP currently produces. It may be arguable + * whether this applies also when CCM is not used but even then it's + * still likely to be a reasonable setting. + */ + cfg.colorSpace = ColorSpace::Srgb; + } + config->addConfiguration(cfg); } From patchwork Wed Jul 23 18:08:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23908 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9E956C3323 for ; Wed, 23 Jul 2025 18:08:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3B3766906A; Wed, 23 Jul 2025 20:08:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="d2nhNnU8"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DFE0469064 for ; Wed, 23 Jul 2025 20:08:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753294122; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8wFXUaYoFNERZep7XD3mebdsvA8+dekpozWqRvYhk7k=; b=d2nhNnU8+qeBA2Bp6hjCLfDB8mWgiEJGERdblIPICGM/jjnzLoCVNydUVHMRdh9dB4q4wT aVV5EkmZJhQ8CYOfdYNWJkvNPLkDHkEefxtVRLjDgVNu0a8BKQnaYLYRn+nsXKRVB17JcC v0EJjejlGMtfvj3XeQs8qYUF2V6f4MA= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-282-yQ2SIAmJOta6uIrtd_0ebg-1; Wed, 23 Jul 2025 14:08:39 -0400 X-MC-Unique: yQ2SIAmJOta6uIrtd_0ebg-1 X-Mimecast-MFC-AGG-ID: yQ2SIAmJOta6uIrtd_0ebg_1753294118 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 481D81800286; Wed, 23 Jul 2025 18:08:38 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9B37A1955F4A; Wed, 23 Jul 2025 18:08:35 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v11 2/8] libcamera: simple: Set the number of software ISP streams to 2 Date: Wed, 23 Jul 2025 20:08:07 +0200 Message-ID: <20250723180815.82450-3-mzamazal@redhat.com> In-Reply-To: <20250723180815.82450-1-mzamazal@redhat.com> References: <20250723180815.82450-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: bN-_y70V4cdCcKJJezC8pZ0n5Frd0Sb3g2zdjQB5Af0_1753294118 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When software ISP is enabled, we want to be able to provide a raw stream in addition to the processed stream. For this purpose, we need two streams. If only the processed stream is requested, it doesn't harm to allocate two. This is a hack for the lack of a better easy option. The number of streams is determined as a camera property in the pipeline matching. The actual number of streams needed (one or two) is determined only when examining roles in SimplePipelineHandler::generateConfiguration. In theory, software ISP could produce multiple processed streams but this is out of scope of this patch series. Hence two streams are sufficient at the moment. When software ISP is not enabled, the camera won't be able to produce multiple streams (assuming there's no hardware converter) and only single stream should be allocated as before. The simple pipeline handler assumes there's a linear pipeline from the camera sensor to a video capture device, and only supports a single stream. Branches in the hardware pipeline that would allow capturing multiple streams from the same camera sensor are not supported. We have no plan to change that, as a device that can produce multiple streams will likely be better supported by a dedicated pipeline handler. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index d45480fe7..06532ed26 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1723,7 +1723,18 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) } } - swIspEnabled_ = info->swIspEnabled; + if (info->swIspEnabled) { + /* + * When the software ISP is enabled, the simple pipeline handler + * exposes the raw stream, giving a total of two streams. This + * is mutally exclusive with the presence of a converter. + */ + ASSERT(!converter_); + numStreams = 2; + swIspEnabled_ = true; + } else { + swIspEnabled_ = false; + } /* Locate the sensors. */ std::vector sensors = locateSensors(media); From patchwork Wed Jul 23 18:08:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23909 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5E31FC3237 for ; Wed, 23 Jul 2025 18:08:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1507169072; Wed, 23 Jul 2025 20:08:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="HBS07e+S"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 078FF69067 for ; Wed, 23 Jul 2025 20:08:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753294128; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jZ/zy4WtB63rHdRrggPxOHDVNqPWrIKk4uMYmwMKk5A=; b=HBS07e+SYw2idqhuKSEW5fBo7w50uuQu3HDR7fvLgl7TbOQR/JxN1zmGCnmy0ooOFlydw/ H8CK5FVKxxiRZb6MwHOJbtAtqM8xoxer0baWDcpU27ahrV/zaQaUT0huUu+gbzsJWs84U0 Mi3CxIHkkhm4/oe5fycK+0gEmWqEnQ8= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-68-8yqwe3Q1NQaMpOo5OU2hIA-1; Wed, 23 Jul 2025 14:08:43 -0400 X-MC-Unique: 8yqwe3Q1NQaMpOo5OU2hIA-1 X-Mimecast-MFC-AGG-ID: 8yqwe3Q1NQaMpOo5OU2hIA_1753294121 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8A2DB1800C3F; Wed, 23 Jul 2025 18:08:41 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DE3BB195608D; Wed, 23 Jul 2025 18:08:38 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v11 3/8] libcamera: simple: Exclude raw configurations from output conversions Date: Wed, 23 Jul 2025 20:08:08 +0200 Message-ID: <20250723180815.82450-4-mzamazal@redhat.com> In-Reply-To: <20250723180815.82450-1-mzamazal@redhat.com> References: <20250723180815.82450-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: SUnFdcpUyLKjaWg0DqBOuNNwJh3CWGhMUBeicthSX6M_1753294121 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to support raw streams, we need to add raw formats to software ISP configurations. In this preparatory patch, the raw formats are excluded from output configurations for conversions. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain --- src/libcamera/pipeline/simple/simple.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 06532ed26..87b26d4a2 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -261,6 +262,12 @@ static const SimplePipelineInfo supportedDevices[] = { { "sun6i-csi", {}, false }, }; +bool isFormatRaw(const libcamera::PixelFormat &pixFmt) +{ + return libcamera::PixelFormatInfo::info(pixFmt).colourEncoding == + libcamera::PixelFormatInfo::ColourEncodingRAW; +} + } /* namespace */ class SimpleCameraData : public Camera::Private @@ -1424,7 +1431,7 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) cfg.setStream(&data->streams_[i]); - if (data->useConversion_) + if (data->useConversion_ && !isFormatRaw(cfg.pixelFormat)) outputCfgs.push_back(cfg); } From patchwork Wed Jul 23 18:08:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23910 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 195D7C3237 for ; Wed, 23 Jul 2025 18:08:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B634A69076; Wed, 23 Jul 2025 20:08:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="dkWhdhsj"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EF11B69067 for ; Wed, 23 Jul 2025 20:08:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753294130; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jl9xfCAU03T6MQZe9LMyGoUEEt/fHSLHEoM66+PGXiM=; b=dkWhdhsjBep09H8umiYUKoV8Di7Cyhl8GpTjIURVf6TLW9uqzfGYjSgKjuB8nwZOqvCBKc wvQpzWKycYtD4S8PUZ4L10lVFtKbHJW+VnSQBnGhUF4AepkXqXMkOgtwp1oJVsx7Lixp0V oW9zamzYyb/v+whQLg3wPVcjXfh5Zh0= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-606-Yjox-6U3NQ-dVYI3etO32A-1; Wed, 23 Jul 2025 14:08:46 -0400 X-MC-Unique: Yjox-6U3NQ-dVYI3etO32A-1 X-Mimecast-MFC-AGG-ID: Yjox-6U3NQ-dVYI3etO32A_1753294124 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8D4F619560BC; Wed, 23 Jul 2025 18:08:44 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0C976195608D; Wed, 23 Jul 2025 18:08:41 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v11 4/8] libcamera: simple: Handle processed and raw formats separately Date: Wed, 23 Jul 2025 20:08:09 +0200 Message-ID: <20250723180815.82450-5-mzamazal@redhat.com> In-Reply-To: <20250723180815.82450-1-mzamazal@redhat.com> References: <20250723180815.82450-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Tp3GMOvil2Bn9NVbNwrPMOGo4u8GcLQ8gAMDcLQ0bEo_1753294124 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let's handle both processed and/or raw output configurations, depending on the requested stream roles. In addition to the already handled processed formats and sizes, this patch adds handling of raw formats and sizes, which correspond to the capture formats and sizes. Raw and processed formats are handled separately. The intention is that in case both raw and processed formats are requested then the raw formats should be left intact to the extent possible and not be influenced by the processed formats (implying that raw parameters compatible with the processed output requirements must be requested by the application). Accompanying checks for invalid role specifications are introduced. This is another preparatory patch without making raw outputs working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 80 +++++++++++++++++------- 1 file changed, 57 insertions(+), 23 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 87b26d4a2..37abaa0e0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1310,35 +1310,67 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span> formats; + bool processedRequested = false; + bool rawRequested = false; + for (const auto &role : roles) + if (role == StreamRole::Raw) { + if (rawRequested) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return nullptr; + } + rawRequested = true; + } else { + processedRequested = true; + } + + /* Create the formats maps. */ + std::map> processedFormats; + std::map> rawFormats; for (const SimpleCameraData::Configuration &cfg : data->configs_) { + rawFormats[cfg.captureFormat].push_back(cfg.captureSize); for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); + processedFormats[format].push_back(cfg.outputSizes); } - /* Sort the sizes and merge any consecutive overlapping ranges. */ - for (auto &[format, sizes] : formats) { - std::sort(sizes.begin(), sizes.end(), - [](SizeRange &a, SizeRange &b) { - return a.min < b.min; - }); - - auto cur = sizes.begin(); - auto next = cur; - - while (++next != sizes.end()) { - if (cur->max.width >= next->min.width && - cur->max.height >= next->min.height) - cur->max = next->max; - else if (++cur != next) - *cur = *next; - } - - sizes.erase(++cur, sizes.end()); + if (processedRequested && processedFormats.empty()) { + LOG(SimplePipeline, Error) + << "Processed stream requsted but no corresponding output configuration found"; + return nullptr; + } + if (rawRequested && rawFormats.empty()) { + LOG(SimplePipeline, Error) + << "Raw stream requsted but no corresponding output configuration found"; + return nullptr; } + auto setUpFormatSizes = [](std::map> &formats) { + /* Sort the sizes and merge any consecutive overlapping ranges. */ + + for (auto &[format, sizes] : formats) { + std::sort(sizes.begin(), sizes.end(), + [](SizeRange &a, SizeRange &b) { + return a.min < b.min; + }); + + auto cur = sizes.begin(); + auto next = cur; + + while (++next != sizes.end()) { + if (cur->max.width >= next->min.width && + cur->max.height >= next->min.height) + cur->max = next->max; + else if (++cur != next) + *cur = *next; + } + + sizes.erase(++cur, sizes.end()); + } + }; + setUpFormatSizes(processedFormats); + setUpFormatSizes(rawFormats); + /* * Create the stream configurations. Take the first entry in the formats * map as the default, for lack of a better option. @@ -1346,11 +1378,13 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Spanfirst; cfg.size = formats.begin()->second[0].max; - if (role == StreamRole::Raw) { + if (raw) { /* Enforce raw colour space for raw roles. */ cfg.colorSpace = ColorSpace::Raw; } else if (data->swIsp_) { From patchwork Wed Jul 23 18:08:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23913 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4F33BC3323 for ; Wed, 23 Jul 2025 18:09:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8F78369080; Wed, 23 Jul 2025 20:09:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Cef2WNRO"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3DFE46907E for ; Wed, 23 Jul 2025 20:09:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753294137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZfcXv9hY4UJginfMu6aew12QKP2XdnssyilUoeqRXTQ=; b=Cef2WNROjz6iVIF76UVS1+HL31cC6VpM7WqCCmZvw7Hdg6AVP7PbNi04e4KcCtqqbTmP3g cg4TiiUrKUbgxQFuDCADvX0jk7bs/L+h9RGjgvzBR3zp+RPWsMxqUdcpRcN/JwnnqyK7wZ 2+4200OOcVMdXl+/wc1cDMqDWIZks78= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-646-1gJ4GdD1PyG7xjNcjni9MQ-1; Wed, 23 Jul 2025 14:08:49 -0400 X-MC-Unique: 1gJ4GdD1PyG7xjNcjni9MQ-1 X-Mimecast-MFC-AGG-ID: 1gJ4GdD1PyG7xjNcjni9MQ_1753294128 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0491F195FE16; Wed, 23 Jul 2025 18:08:48 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1E111195608D; Wed, 23 Jul 2025 18:08:44 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v11 5/8] libcamera: simple: Validate raw stream configurations Date: Wed, 23 Jul 2025 20:08:10 +0200 Message-ID: <20250723180815.82450-6-mzamazal@redhat.com> In-Reply-To: <20250723180815.82450-1-mzamazal@redhat.com> References: <20250723180815.82450-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: qo6H22Ou8yN_hpIcPpnY53s0hZAt3cxN9fmR1HWW09Q_1753294128 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. Raw streams are adjusted from the capture format and size. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. If the resolutions don't match mutually or available sizes, validation adjusts them. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 86 +++++++++++++++++------- 1 file changed, 62 insertions(+), 24 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 37abaa0e0..89ee699c3 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -27,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -1131,22 +1133,39 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() << "Largest stream size is " << maxStreamSize; /* - * Find the best configuration for the pipeline using a heuristic. - * First select the pixel format based on the streams (which are - * considered ordered from highest to lowest priority). Default to the - * first pipeline configuration if no streams request a supported pixel - * format. + * Find the best configuration for the pipeline using a heuristic. First + * select the pixel format based on the streams. If there is a raw stream, + * its format has precedence. If there is no raw stream, the streams are + * considered ordered from highest to lowest priority. Default to the first + * pipeline configuration if no streams request a supported pixel format. */ - const std::vector *configs = - &data_->formats_.begin()->second; + std::optional rawFormat; + for (const auto &cfg : config_) + if (cfg.colorSpace == ColorSpace::Raw) { + if (rawFormat) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return Invalid; + } + rawFormat = cfg.pixelFormat; + } - for (const StreamConfiguration &cfg : config_) { - auto it = data_->formats_.find(cfg.pixelFormat); - if (it != data_->formats_.end()) { + const std::vector *configs = nullptr; + if (rawFormat) { + auto it = data_->formats_.find(rawFormat.value()); + if (it != data_->formats_.end()) configs = &it->second; - break; - } } + if (!configs) + for (const StreamConfiguration &cfg : config_) { + auto it = data_->formats_.find(cfg.pixelFormat); + if (it != data_->formats_.end()) { + configs = &it->second; + break; + } + } + if (!configs) + configs = &data_->formats_.begin()->second; /* * \todo Pick the best sensor output media bus format when the @@ -1203,17 +1222,27 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + const bool raw = cfg.colorSpace == ColorSpace::Raw; - /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); + /* Adjust the pixel format, colour space and size. */ - PixelFormat pixelFormat = *it; + PixelFormat pixelFormat; + if (raw) { + pixelFormat = pipeConfig_->captureFormat; + } else { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + pixelFormat = *it; + } if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) << "Adjusting pixel format"; + LOG(SimplePipeline, Debug) + << "Adjusting pixel format of a " + << (raw ? "raw" : "processed") + << " stream from " << cfg.pixelFormat + << " to " << pixelFormat; cfg.pixelFormat = pixelFormat; /* * Do not touch the colour space for raw requested roles. @@ -1240,7 +1269,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() status = Adjusted; } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!raw && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1259,11 +1288,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* \todo Create a libcamera core class to group format and size */ if (cfg.pixelFormat != pipeConfig_->captureFormat || - cfg.size != pipeConfig_->captureSize) - needConversion_ = true; + cfg.size != pipeConfig_->captureSize) { + if (raw) { + cfg.pixelFormat = pipeConfig_->captureFormat; + cfg.size = pipeConfig_->captureSize; + LOG(SimplePipeline, Debug) + << "Adjusting raw configuration to " << cfg; + status = Adjusted; + } else { + needConversion_ = true; + } + } /* Set the stride, frameSize and bufferCount. */ - if (needConversion_) { + if (needConversion_ && !raw) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat, From patchwork Wed Jul 23 18:08:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23911 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DB2B7C3237 for ; Wed, 23 Jul 2025 18:08:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 90A4369076; Wed, 23 Jul 2025 20:08:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="GD/7XOgN"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8511E69067 for ; Wed, 23 Jul 2025 20:08:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753294134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pw9IK4pwNBlCgex5t3JTbCKf8GchWZu8U8uGzctP9e0=; b=GD/7XOgNUsMcxbhJ0eYG83raIpV1bXnALbx2vvNhdc3+N/Mji0oilj6OtJZ6g/S+zuZDve ZFPX0A+1hLzWdxN5CprkZUo16eramENZWsMZN9mq0hxaS+MEnQ/5GuwgcMBNm/0lUm8c6s SGupshofrv0jTM0/uKJeF88Pf/xM+iI= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-122-E-uHoIx1MCSXQRu5dwwBGQ-1; Wed, 23 Jul 2025 14:08:52 -0400 X-MC-Unique: E-uHoIx1MCSXQRu5dwwBGQ-1 X-Mimecast-MFC-AGG-ID: E-uHoIx1MCSXQRu5dwwBGQ_1753294131 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 484351800C38; Wed, 23 Jul 2025 18:08:51 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9B638195608D; Wed, 23 Jul 2025 18:08:48 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v11 6/8] libcamera: simple: Don't enforce conversion with an added raw stream Date: Wed, 23 Jul 2025 20:08:11 +0200 Message-ID: <20250723180815.82450-7-mzamazal@redhat.com> In-Reply-To: <20250723180815.82450-1-mzamazal@redhat.com> References: <20250723180815.82450-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: mMd9dAeCdrxWzOW2FLz1zj00gPS6tY_MlVai31ON91E_1753294131 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, either alone or together with a processed stream, it can be produced without conversion. Let's amend the corresponding check on the number of configurations, so that the mere presence of a raw stream doesn't enforce conversion. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 89ee699c3..068ac5f96 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1218,7 +1218,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * require any conversion, similar to raw capture use cases). This is * left as a future improvement. */ - needConversion_ = config_.size() > 1; + needConversion_ = config_.size() > 1 + (rawFormat ? 1 : 0); for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; From patchwork Wed Jul 23 18:08:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23912 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 94AE9C3237 for ; Wed, 23 Jul 2025 18:09:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3724D6907D; Wed, 23 Jul 2025 20:09:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="P70y9gvQ"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2BA7069067 for ; Wed, 23 Jul 2025 20:09:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753294139; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4gxUgda7MWQtm9SfPqzzb53FEL3jUkaiTVOBYpZnAzI=; b=P70y9gvQfOCNZscHuNkzBRclaygqPcHb+2PRYaPA6vdl7ERC/I0bCNiIzxoF7F9PV/+9G2 11BI5btKJsg0OEWHHDtRklqJxVoyNeusMZR8j/icWX5nXXLj/g0mgAlBkAQq6DLd5WPyWL UXTd0WzGyyXsyFKhQBVf3VNW46NkbTQ= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-310-gruseSd2MuG3U-06dI_bkQ-1; Wed, 23 Jul 2025 14:08:55 -0400 X-MC-Unique: gruseSd2MuG3U-06dI_bkQ-1 X-Mimecast-MFC-AGG-ID: gruseSd2MuG3U-06dI_bkQ_1753294134 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9513418009BE; Wed, 23 Jul 2025 18:08:54 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D7355195608D; Wed, 23 Jul 2025 18:08:51 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v11 7/8] libcamera: simple: Require metadata only when software ISP is used Date: Wed, 23 Jul 2025 20:08:12 +0200 Message-ID: <20250723180815.82450-8-mzamazal@redhat.com> In-Reply-To: <20250723180815.82450-1-mzamazal@redhat.com> References: <20250723180815.82450-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 7qfrpNmX7YSLZLbHJ1c4kujDtjpTE2cT83hb03TbUK8_1753294134 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If software ISP is enabled then metadata is required in the simple pipeline. But this doesn't apply if the software ISP is not actually used, for example when only a raw stream is produced. Then the pipeline waits for metadata that never comes. This patch fixes the problem by requiring metadata only when software ISP is used. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 068ac5f96..3de982e67 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1652,6 +1652,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) int ret; std::map buffers; + bool metadataRequired = false; for (auto &[stream, buffer] : request->buffers()) { /* @@ -1661,6 +1662,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) */ if (data->useConversion_) { buffers.emplace(stream, buffer); + metadataRequired = !!data->swIsp_; } else { ret = data->video_->queueBuffer(buffer); if (ret < 0) @@ -1668,7 +1670,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } - data->frameInfo_.create(request, !!data->swIsp_); + data->frameInfo_.create(request, metadataRequired); if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); if (data->swIsp_) From patchwork Wed Jul 23 18:08:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23914 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0F344C3237 for ; Wed, 23 Jul 2025 18:09:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AD9FD69080; Wed, 23 Jul 2025 20:09:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="YqSL69pl"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EF78D6906A for ; Wed, 23 Jul 2025 20:09:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753294144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LKNI83jWOA51/H/VRVuF1//ouIwINH/8OPFDO3k43gg=; b=YqSL69pl9zq1rCs3q9dpnuH/a96+UxMuqD7ACik5gqcZ1rK9HI5bgx0vs5Uq5crzeMR4sL fwKRkZPLGymvMvMX57cYU3YwDLw/BEqfuskg+bd/8VNdmowsj9ZjZrPRmtJJiGQ0kyeuoV FZSKl23rDYBv8RbvFRxZw7cAKDx14AM= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-256-MXf_3cUJNK6V42goCwqFTg-1; Wed, 23 Jul 2025 14:08:59 -0400 X-MC-Unique: MXf_3cUJNK6V42goCwqFTg-1 X-Mimecast-MFC-AGG-ID: MXf_3cUJNK6V42goCwqFTg_1753294138 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4947F19560BE; Wed, 23 Jul 2025 18:08:58 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.15]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 168011955F4A; Wed, 23 Jul 2025 18:08:54 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v11 8/8] libcamera: simple: Make raw streams working Date: Wed, 23 Jul 2025 20:08:13 +0200 Message-ID: <20250723180815.82450-9-mzamazal@redhat.com> In-Reply-To: <20250723180815.82450-1-mzamazal@redhat.com> References: <20250723180815.82450-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 62NKKkALTpjrmusPHwu_PwVzr6xnUi1Gsm1xK4kwIys_1753294138 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, whether alone or together with a processed stream, its buffers must be handled outside the software ISP machinery. They serve as output buffers, even when a processed stream is produced. At most one raw stream and at most one processed stream are supported and can be combined. An example of producing both raw and processed files using `cam' application: cam -c1 -C100 -Ffile# \ -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \ -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \ Note the difference in viewfinder and raw stream sizes due to the fact that debayering requires enlarging the image width, which enforces selecting a larger sensor resolution in this case. In order to track whether a raw stream is requested and which one it is, SimpleCameraData::rawStream_ member variable is introduced. This is the final step to make raw streams working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 55 ++++++++++++++++++------ 1 file changed, 41 insertions(+), 14 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 3de982e67..0a61ed518 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -330,6 +330,7 @@ public: }; std::vector streams_; + Stream *rawStream_; /* * All entities in the pipeline, from the camera sensor to the video @@ -368,6 +369,11 @@ private: void ispStatsReady(uint32_t frame, uint32_t bufferId); void metadataReady(uint32_t frame, const ControlList &metadata); void setSensorControls(const ControlList &sensorControls); + + bool processedRequested() const + { + return streams_.size() - (rawStream_ ? 1 : 0) > 0; + } }; class SimpleCameraConfiguration : public CameraConfiguration @@ -457,7 +463,7 @@ private: SimpleCameraData::SimpleCameraData(SimplePipelineHandler *pipe, unsigned int numStreams, MediaEntity *sensor) - : Camera::Private(pipe), streams_(numStreams) + : Camera::Private(pipe), streams_(numStreams), rawStream_(nullptr) { /* * Find the shortest path from the camera sensor to a video capture @@ -868,10 +874,13 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * point converting an erroneous buffer. */ if (buffer->metadata().status != FrameMetadata::FrameSuccess) { - if (!useConversion_) { + if (rawStream_) { /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); + SimpleFrameInfo *info = frameInfo_.find(request->sequence()); + if (info) + info->metadataRequired = false; tryCompleteRequest(request); return; } @@ -881,7 +890,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * buffer for capture (unless the stream is being stopped), and * complete the request with all the user-facing buffers. */ - if (buffer->metadata().status != FrameMetadata::FrameCancelled) + if (buffer->metadata().status != FrameMetadata::FrameCancelled && + !rawStream_) video_->queueBuffer(buffer); if (conversionQueue_.empty()) @@ -930,13 +940,14 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) */ if (useConversion_) { if (conversionQueue_.empty()) { - video_->queueBuffer(buffer); + if (!rawStream_) + video_->queueBuffer(buffer); return; } if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front().outputs); - else + else if (processedRequested()) /* * request->sequence() cannot be retrieved from `buffer' inside * queueBuffers because unique_ptr's make buffer->request() invalid @@ -946,6 +957,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) conversionQueue_.front().outputs); conversionQueue_.pop(); + if (rawStream_) + pipe->completeBuffer(request, buffer); return; } @@ -983,7 +996,8 @@ void SimpleCameraData::tryCompleteRequest(Request *request) void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ - video_->queueBuffer(buffer); + if (!rawStream_) + video_->queueBuffer(buffer); } void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) @@ -1500,11 +1514,20 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) for (unsigned int i = 0; i < config->size(); ++i) { StreamConfiguration &cfg = config->at(i); + bool rawStream = isFormatRaw(cfg.pixelFormat); cfg.setStream(&data->streams_[i]); - if (data->useConversion_ && !isFormatRaw(cfg.pixelFormat)) + if (data->useConversion_ && !rawStream) outputCfgs.push_back(cfg); + + if (rawStream) { + if (data->rawStream_) { + LOG(SimplePipeline, Error) << "Multiple raw streams not supported"; + return -EINVAL; + } + data->rawStream_ = &data->streams_[i]; + } } if (outputCfgs.empty()) @@ -1535,7 +1558,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ - if (data->useConversion_) + if (data->useConversion_ && stream != data->rawStream_) return data->converter_ ? data->converter_->exportBuffers(stream, count, buffers) : data->swIsp_->exportBuffers(stream, count, buffers); @@ -1558,7 +1581,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return -EBUSY; } - if (data->useConversion_) { + if (data->useConversion_ && !data->rawStream_) { /* * When using the converter allocate a fixed number of internal * buffers. @@ -1566,8 +1589,11 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL ret = video->allocateBuffers(kNumInternalBuffers, &data->conversionBuffers_); } else { - /* Otherwise, prepare for using buffers from the only stream. */ - Stream *stream = &data->streams_[0]; + /* + * Otherwise, prepare for using buffers from either the raw stream, if + * requested, or the only stream configured. + */ + Stream *stream = (data->rawStream_ ? data->rawStream_ : &data->streams_[0]); ret = video->importBuffers(stream->configuration().bufferCount); } if (ret < 0) { @@ -1608,8 +1634,9 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } /* Queue all internal buffers for capture. */ - for (std::unique_ptr &buffer : data->conversionBuffers_) - video->queueBuffer(buffer.get()); + if (!data->rawStream_) + for (std::unique_ptr &buffer : data->conversionBuffers_) + video->queueBuffer(buffer.get()); } return 0; @@ -1660,7 +1687,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * queue, it will be handed to the converter in the capture * completion handler. */ - if (data->useConversion_) { + if (data->useConversion_ && stream != data->rawStream_) { buffers.emplace(stream, buffer); metadataRequired = !!data->swIsp_; } else {