From patchwork Wed Jul 2 12:09:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23706 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 79AB8BDCBF for ; Wed, 2 Jul 2025 12:10:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 33A4B68E34; Wed, 2 Jul 2025 14:10:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="CKoySFai"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7B3DF68E05 for ; Wed, 2 Jul 2025 14:10:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458240; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Czy+xXD1fOnM3D+B6VeIahV3hQ2Y/vFBOyTsQSQmbHY=; b=CKoySFaiLaRXrnHTijLi/siTMbczE1fzE2CpZzJh9tG3ufBYTquEKlw9/PuwH1sCnsknno 4TAd/JOIMCDq1CE5KU0Twyx+Fd82i98+J94lfDrURFbrhgDprNz9EHcrLikR9qS+fdS3wG PZ6dJnC1u3w9yCKK6XdnHjwYDz8Clt0= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-312-_zALCPIfOdeNGVtTuofmgg-1; Wed, 02 Jul 2025 08:10:37 -0400 X-MC-Unique: _zALCPIfOdeNGVtTuofmgg-1 X-Mimecast-MFC-AGG-ID: _zALCPIfOdeNGVtTuofmgg_1751458234 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8576318DA5C2; Wed, 2 Jul 2025 12:10:34 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0DE521800287; Wed, 2 Jul 2025 12:10:30 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 01/10] libcamera: software_isp: Assign colour spaces in configurations Date: Wed, 2 Jul 2025 14:09:57 +0200 Message-ID: <20250702121024.38728-2-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: CM-wKTUNENL0QfRQL5xRJpQQmkQu5Y_Wz7so20OX2Og_1751458234 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" StreamConfiguration's should have colorSpace set. This is not the case in the simple pipeline. Let's set it there. This also fixes a crash in `cam' due to accessing an unset colorSpace. The colour space is assigned as follows: - If raw role is requested, then the colour space must be raw, regardless of the stream format. - Otherwise, if software ISP is used, the default colour space is set to Srgb (YcbcrEncoding::None, Range::Full). - Otherwise, if a converter is used, the default colour space is left unset. - Then in configuration validation, if the pixel format is changed, the colour space is set according to the new pixel format. - If the colour space is still unset, it is set according to the resulting pixel format. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 38 +++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index efb07051b..d45480fe7 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -35,6 +36,7 @@ #include "libcamera/internal/converter.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/software_isp/software_isp.h" @@ -1206,6 +1208,28 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() if (cfg.pixelFormat != pixelFormat) { LOG(SimplePipeline, Debug) << "Adjusting pixel format"; cfg.pixelFormat = pixelFormat; + /* + * Do not touch the colour space for raw requested roles. + * Even if the pixel format is non-raw (whatever it means), we + * shouldn't try to interpret the colour space of raw data. + */ + if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + cfg.colorSpace->adjust(pixelFormat); + status = Adjusted; + } + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } + cfg.colorSpace->adjust(pixelFormat); status = Adjusted; } @@ -1314,11 +1338,23 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Spanfirst; cfg.size = formats.begin()->second[0].max; + if (role == StreamRole::Raw) { + /* Enforce raw colour space for raw roles. */ + cfg.colorSpace = ColorSpace::Raw; + } else if (data->swIsp_) { + /* + * This is what software ISP currently produces. It may be arguable + * whether this applies also when CCM is not used but even then it's + * still likely to be a reasonable setting. + */ + cfg.colorSpace = ColorSpace::Srgb; + } + config->addConfiguration(cfg); } From patchwork Wed Jul 2 12:09:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23707 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 22A52BDCBF for ; Wed, 2 Jul 2025 12:10:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CA21868E38; Wed, 2 Jul 2025 14:10:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ZbJ4Kkoo"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B4EDA68E32 for ; Wed, 2 Jul 2025 14:10:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458242; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WV5TobAHr8uumxWgXpvDFUT5+QWiq9S1NdaDg/nBheI=; b=ZbJ4KkooiE8TfAnDVgLUOJCHN4Aq0KSGFo8yoAsWELOt6njNJtFDA8eTDvcr3H70DlGzwZ J55cxe2zCpz1fFeCB+jpXQLKOu0v2UaaFoH3+0uLNhgkyk+VdipqsqGb9QoBiRZvzO7j6l 9YumzdTcrpF1I455WZ3Iu0EX4DpLouE= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-25-V4PFyCP6O1ifjlAikK9dkA-1; Wed, 02 Jul 2025 08:10:39 -0400 X-MC-Unique: V4PFyCP6O1ifjlAikK9dkA-1 X-Mimecast-MFC-AGG-ID: V4PFyCP6O1ifjlAikK9dkA_1751458238 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2092B18001D6; Wed, 2 Jul 2025 12:10:38 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 209FC18003FC; Wed, 2 Jul 2025 12:10:34 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 02/10] libcamera: simple: Set the number of software ISP streams to 2 Date: Wed, 2 Jul 2025 14:09:58 +0200 Message-ID: <20250702121024.38728-3-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: gO5vd6vkiCO4flXfcJPcqZCu9IxLTHaeefKm2O_ShjY_1751458238 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When software ISP is enabled, we want to be able to provide a raw stream in addition to the processed stream. For this purpose, we need two streams. If only the processed stream is requested, it doesn't harm to allocate two. This is a hack for the lack of a better easy option. The number of streams is determined as a camera property in the pipeline matching. The actual number of streams needed (one or two) is determined only when examining roles in SimplePipelineHandler::generateConfiguration. In theory, software ISP could produce multiple processed streams but this is out of scope of this patch series. Hence two streams are sufficient at the moment. When software ISP is not enabled, the camera won't be able to produce multiple streams (assuming there's no hardware converter) and only single stream should be allocated as before. The simple pipeline handler assumes there's a linear pipeline from the camera sensor to a video capture device, and only supports a single stream. Branches in the hardware pipeline that would allow capturing multiple streams from the same camera sensor are not supported. We have no plan to change that, as a device that can produce multiple streams will likely be better supported by a dedicated pipeline handler. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index d45480fe7..a900918db 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1723,7 +1723,16 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) } } - swIspEnabled_ = info->swIspEnabled; + if (info->swIspEnabled) { + /* + * When the software ISP is enabled, the simple pipeline handler + * exposes the raw stream, giving a total of two streams. This + * is mutally exclusive with the presence of a converter. + */ + ASSERT(!converter_); + numStreams = 2; + swIspEnabled_ = true; + } /* Locate the sensors. */ std::vector sensors = locateSensors(media); From patchwork Wed Jul 2 12:09:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23708 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E672BBDCBF for ; Wed, 2 Jul 2025 12:10:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9BF3B68E38; Wed, 2 Jul 2025 14:10:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="fTPQhbwD"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4F81768E05 for ; Wed, 2 Jul 2025 14:10:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458249; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xLCuLW7Uop0AndI3k18crvoNXTO1uklduUF48s9AhIo=; b=fTPQhbwD80vikeHxDLUXR0hnoRkycTr8OBf7lqCo4nYlcNlQ7iF17N04IjH6Y1o0YfPGhG JA73DaFN8aGCEWZv5JEB0WXREb9kK8ji/NeXyqG2Czt7q+EzUfmvR74Sdg/Q6F1BzL2g+S iLbRLQCe4jNJGap+7sjjSpCVO/9aF4A= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-159-Wlj3-3mEOqK1Urg6EhVXZQ-1; Wed, 02 Jul 2025 08:10:42 -0400 X-MC-Unique: Wlj3-3mEOqK1Urg6EhVXZQ-1 X-Mimecast-MFC-AGG-ID: Wlj3-3mEOqK1Urg6EhVXZQ_1751458241 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 846311800290; Wed, 2 Jul 2025 12:10:41 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 866C2180035C; Wed, 2 Jul 2025 12:10:38 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 03/10] libcamera: simple: Don't use raw output formats with conversions Date: Wed, 2 Jul 2025 14:09:59 +0200 Message-ID: <20250702121024.38728-4-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: W-dyHhb_4FzQosWTX1crtJCitk4XUVyvrhMwkB34oEI_1751458241 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to support raw streams, we need to add raw formats to software ISP configurations. In this preparatory patch, the raw formats are excluded from output configurations for conversions. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index a900918db..368bf3020 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -261,6 +262,12 @@ static const SimplePipelineInfo supportedDevices[] = { { "sun6i-csi", {}, false }, }; +bool isFormatRaw(const libcamera::PixelFormat &pixFmt) +{ + return libcamera::PixelFormatInfo::info(pixFmt).colourEncoding == + libcamera::PixelFormatInfo::ColourEncodingRAW; +} + } /* namespace */ class SimpleCameraData : public Camera::Private @@ -1424,7 +1431,7 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) cfg.setStream(&data->streams_[i]); - if (data->useConversion_) + if (data->useConversion_ && !isFormatRaw(cfg.pixelFormat)) outputCfgs.push_back(cfg); } From patchwork Wed Jul 2 12:10:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23709 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 08877BDCBF for ; Wed, 2 Jul 2025 12:10:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B47A368E3E; Wed, 2 Jul 2025 14:10:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ZnNkKRkz"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D3BC268E34 for ; Wed, 2 Jul 2025 14:10:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458256; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EE/nS2yfYz+RHkARxEoYq5PWWd9a/9m4d7lzlq8oVEM=; b=ZnNkKRkzhX8DrRuTfhQCdCEHs5f0jNawP1DnWk30tV+MOXxvbGZRF9FNxPXicMEbKm2lWc zfVXo7JuSOXh1EvsO9K1CyHFAxhqOdbjX7N4rvsuZZaVlde7fU5I4NxZmHlWABELAJdoAH Uu+FDVENqIZUQYw4eAtGQacfF0ec6Bc= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-279-ytz74NfVN3ma93ooUtsBkw-1; Wed, 02 Jul 2025 08:10:50 -0400 X-MC-Unique: ytz74NfVN3ma93ooUtsBkw-1 X-Mimecast-MFC-AGG-ID: ytz74NfVN3ma93ooUtsBkw_1751458249 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 809B41955F3E; Wed, 2 Jul 2025 12:10:49 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BC139180035C; Wed, 2 Jul 2025 12:10:41 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 04/10] libcamera: simple: Add plain output configurations Date: Wed, 2 Jul 2025 14:10:00 +0200 Message-ID: <20250702121024.38728-5-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 5Chw8QIClzXmRBRhrsXB6IFVc1vTtrscuvvSflA07_I_1751458249 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Output configurations in simple pipeline are added depending on whether a converter, software ISP or none of them are used. If a converter or software ISP is used, no raw output configurations are added. Unless only raw configurations are available, in which case we add them to avoid ... In order to support raw output at least with software ISP, let's always add raw output configurations. A flag is added to SimpleCameraData::Configuration indicating whether it's for a raw or a converted output. We later filter formats and output sizes for particular stream configurations according to the new configuration flag. This is just preparation and raw output is still not supported. At the moment, we simply filter out raw configurations unconditionally to keep the current code working; this will be changed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 34 +++++++++++++++--------- 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 368bf3020..1a7474228 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -325,6 +325,7 @@ public: Size captureSize; std::vector outputFormats; SizeRange outputSizes; + bool raw; }; std::vector streams_; @@ -714,27 +715,33 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) continue; Configuration config; + + /* Raw configuration */ config.code = code; config.sensorSize = size; config.captureFormat = pixelFormat; config.captureSize = format.size; + config.outputFormats = { pixelFormat }; + config.outputSizes = config.captureSize; + config.raw = true; + configs_.push_back(config); + /* Modified, non-raw, configuration */ + config.raw = false; if (converter_) { config.outputFormats = converter_->formats(pixelFormat); config.outputSizes = converter_->sizes(format.size); } else if (swIsp_) { - config.outputFormats = swIsp_->formats(pixelFormat); - config.outputSizes = swIsp_->sizes(pixelFormat, format.size); - if (config.outputFormats.empty()) { + std::vector outputFormats = swIsp_->formats(pixelFormat); + if (outputFormats.empty()) { /* Do not use swIsp for unsupported pixelFormat's. */ - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; + continue; } + config.outputFormats = outputFormats; + config.outputSizes = swIsp_->sizes(pixelFormat, format.size); } else { - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; + continue; } - configs_.push_back(config); } } @@ -1313,10 +1320,13 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span> formats; - for (const SimpleCameraData::Configuration &cfg : data->configs_) { - for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); - } + const bool rawOnly = std::all_of(data->configs_.cbegin(), + data->configs_.cend(), + [](const auto &c) { return c.raw; }); + for (const SimpleCameraData::Configuration &cfg : data->configs_) + if (!cfg.raw || rawOnly) + for (PixelFormat format : cfg.outputFormats) + formats[format].push_back(cfg.outputSizes); /* Sort the sizes and merge any consecutive overlapping ranges. */ for (auto &[format, sizes] : formats) { From patchwork Wed Jul 2 12:10:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23710 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9F84EBDCBF for ; Wed, 2 Jul 2025 12:11:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5098C68E44; Wed, 2 Jul 2025 14:11:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="DagpUOb7"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 488E368E3D for ; Wed, 2 Jul 2025 14:10:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wkLCpDjPaE3TkzwBZo7++wGiGgE9FFwIXF5WEAnVchI=; b=DagpUOb7il+UgcRAfmKzEWK4E/dO2DHmDpHCTNZbFiZ24nkFL+M6A/N8X09qIQrGvHkOsi rvSP9J8x+6gzQb4bKnA0Lo8hr+i/bpTKvtFxt6cAF4SscQ8CdU3eWbJKe2E3vhEz2qChCw cEUgOQLteeV13ZkG3sh8SB+5XN3sziA= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-623-s6-aWNwDNhCB08fEPTfOjQ-1; Wed, 02 Jul 2025 08:10:53 -0400 X-MC-Unique: s6-aWNwDNhCB08fEPTfOjQ-1 X-Mimecast-MFC-AGG-ID: s6-aWNwDNhCB08fEPTfOjQ_1751458252 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 911BC180120E; Wed, 2 Jul 2025 12:10:52 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2A17D1800288; Wed, 2 Jul 2025 12:10:49 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 05/10] libcamera: simple: Identify requested stream roles Date: Wed, 2 Jul 2025 14:10:01 +0200 Message-ID: <20250702121024.38728-6-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: rx27zYytPvL-47ZhFxzpIrizmy-hTHgtaeDS4SXWWG0_1751458252 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently, raw streams don't work in the simple pipeline and the requested stream roles are ignored. In order to support raw streams, we now track in SimpleCameraConfiguration whether raw and/or processed streams are requested. We also check that at most one raw stream is requested, there is no reason to have more. That information will be used in the followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 1a7474228..6330afeb5 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -384,6 +384,9 @@ public: bool needConversion() const { return needConversion_; } const Transform &combinedTransform() const { return combinedTransform_; } + bool processedRequested_; + bool rawRequested_; + private: /* * The SimpleCameraData instance is guaranteed to be valid as long as @@ -1311,12 +1314,27 @@ std::unique_ptr SimplePipelineHandler::generateConfiguration(Camera *camera, Span roles) { SimpleCameraData *data = cameraData(camera); - std::unique_ptr config = + std::unique_ptr config = std::make_unique(camera, data); + config->processedRequested_ = false; + config->rawRequested_ = false; + if (roles.empty()) return config; + for (const auto &role : roles) + if (role == StreamRole::Raw) { + if (config->rawRequested_) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return nullptr; + } + config->rawRequested_ = true; + } else { + config->processedRequested_ = true; + } + /* Create the formats map. */ std::map> formats; From patchwork Wed Jul 2 12:10:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23711 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C3D12C3293 for ; Wed, 2 Jul 2025 12:11:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4A07368E42; Wed, 2 Jul 2025 14:11:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="U2mxl6+j"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 59D2A68E41 for ; Wed, 2 Jul 2025 14:10:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458258; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1PjdueQBrDfnTJomz/GbTKagvIwesOLmmebne9AABo4=; b=U2mxl6+jnEtosg5I3RxMl/whIa4nI+u+Eygm+xmp8UJxnXEBFwR+X87YQJpnA7tAphdr1M nUj+Zbd2i+sXa0ejhCsLVY7ZXpeekW+T8dIATVs1TQjLrZdv+RLI4f1sjMG2PcvoyLbeHx ekgcxwkiMT1fCwqvP4/sG1ogi77G/Pk= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-556-L6CQIFM5PlG_dXQM0q_r1w-1; Wed, 02 Jul 2025 08:10:56 -0400 X-MC-Unique: L6CQIFM5PlG_dXQM0q_r1w-1 X-Mimecast-MFC-AGG-ID: L6CQIFM5PlG_dXQM0q_r1w_1751458255 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B251418001D1; Wed, 2 Jul 2025 12:10:55 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 16C52180035C; Wed, 2 Jul 2025 12:10:52 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 06/10] libcamera: simple: Consider raw output configurations Date: Wed, 2 Jul 2025 14:10:02 +0200 Message-ID: <20250702121024.38728-7-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: sdC54_Y8gPKfhM-Yy1-_tBbj8cBwe1IEA7mx9TE9UvA_1751458255 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When generating simple pipeline configuration, raw output configurations are generated but later filtered out. Let's include processed and/or raw output configurations depending on the requested stream roles. Raw and processed formats are handled separately. The intention is that in case both raw and processed formats are requested then the raw formats should be left intact to the extent possible and not be influenced by the processed formats (implying that raw parameters compatible with the processed output requirements must be requested by the application). Raw output configurations are identified by colorSpace being set to ColorSpace::Raw. This is another preparatory patch without making raw outputs working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 71 ++++++++++++++---------- 1 file changed, 43 insertions(+), 28 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 6330afeb5..27425e867 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1335,38 +1335,51 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, SpanprocessedRequested_ = true; } - /* Create the formats map. */ - std::map> formats; + /* Create the formats maps. */ + std::map> processedFormats; + std::map> rawFormats; - const bool rawOnly = std::all_of(data->configs_.cbegin(), - data->configs_.cend(), - [](const auto &c) { return c.raw; }); for (const SimpleCameraData::Configuration &cfg : data->configs_) - if (!cfg.raw || rawOnly) - for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); - - /* Sort the sizes and merge any consecutive overlapping ranges. */ - for (auto &[format, sizes] : formats) { - std::sort(sizes.begin(), sizes.end(), - [](SizeRange &a, SizeRange &b) { - return a.min < b.min; - }); - - auto cur = sizes.begin(); - auto next = cur; - - while (++next != sizes.end()) { - if (cur->max.width >= next->min.width && - cur->max.height >= next->min.height) - cur->max = next->max; - else if (++cur != next) - *cur = *next; - } + for (PixelFormat format : cfg.outputFormats) + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes); - sizes.erase(++cur, sizes.end()); + if (config->processedRequested_ && processedFormats.empty()) { + LOG(SimplePipeline, Error) + << "Processed stream requsted but no corresponding output configuration found"; + return nullptr; + } + if (config->rawRequested_ && rawFormats.empty()) { + LOG(SimplePipeline, Error) + << "Raw stream requsted but no corresponding output configuration found"; + return nullptr; } + auto setUpFormatSizes = [](std::map> &formats) { + /* Sort the sizes and merge any consecutive overlapping ranges. */ + + for (auto &[format, sizes] : formats) { + std::sort(sizes.begin(), sizes.end(), + [](SizeRange &a, SizeRange &b) { + return a.min < b.min; + }); + + auto cur = sizes.begin(); + auto next = cur; + + while (++next != sizes.end()) { + if (cur->max.width >= next->min.width && + cur->max.height >= next->min.height) + cur->max = next->max; + else if (++cur != next) + *cur = *next; + } + + sizes.erase(++cur, sizes.end()); + } + }; + setUpFormatSizes(processedFormats); + setUpFormatSizes(rawFormats); + /* * Create the stream configurations. Take the first entry in the formats * map as the default, for lack of a better option. @@ -1374,11 +1387,13 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Spanfirst; cfg.size = formats.begin()->second[0].max; - if (role == StreamRole::Raw) { + if (raw) { /* Enforce raw colour space for raw roles. */ cfg.colorSpace = ColorSpace::Raw; } else if (data->swIsp_) { From patchwork Wed Jul 2 12:10:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23712 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 90159BDCBF for ; Wed, 2 Jul 2025 12:11:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2D60B68E48; Wed, 2 Jul 2025 14:11:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Mnd+1qZt"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6E69768E34 for ; Wed, 2 Jul 2025 14:11:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yIu9Pt3J1zjhjbzz07VRYDL9JQGU3odEhg/Yj+lZmR0=; b=Mnd+1qZtoN/5dODR9p/sYEMw7+5cAhHi66gE+QAyS2J6KewdopH8vpt4P+m7898RVkPv9K aNLGlt63uBLvJjgU3M16qxv+Qmp0k0BWq1MfgD8RPtynmemtK5jCdz1SWbpfMUvdFw+w5l CnZmnVY/jdb57LxvWVUcnaz3BkvEyHQ= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-178-zlID6zLzPgm054SwXozpgg-1; Wed, 02 Jul 2025 08:11:00 -0400 X-MC-Unique: zlID6zLzPgm054SwXozpgg-1 X-Mimecast-MFC-AGG-ID: zlID6zLzPgm054SwXozpgg_1751458259 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DE22419560A1; Wed, 2 Jul 2025 12:10:58 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3322918003FC; Wed, 2 Jul 2025 12:10:55 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 07/10] libcamera: simple: Validate raw stream configurations Date: Wed, 2 Jul 2025 14:10:03 +0200 Message-ID: <20250702121024.38728-8-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: w5vpq_oM1iSbmTyaS_yf9kkmZpZcfVH9koKgJ9tYW_o_1751458259 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. If only a processed stream is requested, nothing changes. If only a raw stream is requested, the pixel format and output size may not be adjusted. The patch adds checks for this. If both processed and raw streams are requested, things get more complicated. The raw stream is expected to be passed through intact and all the adjustments are made for the processed streams. We select a pipe configuration for the processed streams. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. It is the application responsibility to select the right parameters for the raw stream. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 127 ++++++++++++++++------- 1 file changed, 92 insertions(+), 35 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 27425e867..b1766c907 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -1173,6 +1174,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() pipeConfig_ = nullptr; for (const SimpleCameraData::Configuration *pipeConfig : *configs) { + if (processedRequested_ && pipeConfig->raw) + continue; + const Size &size = pipeConfig->captureSize; if (size.width >= maxStreamSize.width && @@ -1196,6 +1200,18 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() << "-" << pipeConfig_->captureFormat << " for max stream size " << maxStreamSize; + /* + * Update raw/processed flags. If e.g. an application calls + * generateConfiguration() with an empty list of roles and adds + * configurations by calling addConfiguration(), the flags must be updated + * according to those configurations. + */ + for (const auto &cfg : config_) + if (cfg.colorSpace == ColorSpace::Raw) + rawRequested_ = true; + else + processedRequested_ = true; + /* * Adjust the requested streams. * @@ -1214,43 +1230,66 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + /* + * If both processed and raw streams are requested, the pipe + * configuration is set up for the processed stream. The raw + * configuration needs to be compared against the capture format and + * size in such a case. + */ + const bool rawStream = cfg.colorSpace == ColorSpace::Raw; + const bool sideRawStream = rawStream && processedRequested_; + /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); - - PixelFormat pixelFormat = *it; - if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) << "Adjusting pixel format"; - cfg.pixelFormat = pixelFormat; - /* - * Do not touch the colour space for raw requested roles. - * Even if the pixel format is non-raw (whatever it means), we - * shouldn't try to interpret the colour space of raw data. - */ - if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + + if (!sideRawStream) { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + + PixelFormat pixelFormat = *it; + + if (cfg.pixelFormat != pixelFormat) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw pixel format " + << cfg.pixelFormat + << " doesn't match any of the pipe output formats"; + return Invalid; + } + LOG(SimplePipeline, Debug) + << "Adjusting pixel format from " << cfg.pixelFormat + << " to " << pixelFormat; + cfg.pixelFormat = pixelFormat; + /* + * Do not touch the colour space for raw requested roles. + * Even if the pixel format is non-raw (whatever it means), we + * shouldn't try to interpret the colour space of raw data. + */ + if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + cfg.colorSpace->adjust(pixelFormat); + status = Adjusted; + } + + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } cfg.colorSpace->adjust(pixelFormat); - status = Adjusted; - } - if (!cfg.colorSpace) { - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); - switch (info.colourEncoding) { - case PixelFormatInfo::ColourEncodingRGB: - cfg.colorSpace = ColorSpace::Srgb; - break; - case libcamera::PixelFormatInfo::ColourEncodingYUV: - cfg.colorSpace = ColorSpace::Sycc; - break; - default: - cfg.colorSpace = ColorSpace::Raw; + status = Adjusted; } - cfg.colorSpace->adjust(pixelFormat); - status = Adjusted; } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!sideRawStream && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1258,8 +1297,17 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * not guaranteed to be a valid output size. In such cases, use * the smaller valid output size closest to the requested. */ - if (!pipeConfig_->outputSizes.contains(adjustedSize)) + if (!pipeConfig_->outputSizes.contains(adjustedSize)) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw output size " + << cfg.size + << " doesn't match any of the pipe output sizes: " + << pipeConfig_->outputSizes; + return Invalid; + } adjustedSize = adjustSize(cfg.size, pipeConfig_->outputSizes); + } LOG(SimplePipeline, Debug) << "Adjusting size from " << cfg.size << " to " << adjustedSize; @@ -1269,11 +1317,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* \todo Create a libcamera core class to group format and size */ if (cfg.pixelFormat != pipeConfig_->captureFormat || - cfg.size != pipeConfig_->captureSize) + cfg.size != pipeConfig_->captureSize) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw output format " << cfg.pixelFormat + << " and size " << cfg.size + << " not matching pipe format " << pipeConfig_->captureFormat + << " and size " << pipeConfig_->captureSize; + return Invalid; + } needConversion_ = true; + } /* Set the stride, frameSize and bufferCount. */ - if (needConversion_) { + if (needConversion_ && !rawStream) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat, From patchwork Wed Jul 2 12:10:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23713 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4AD3CBDCBF for ; Wed, 2 Jul 2025 12:11:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F16B768E4A; Wed, 2 Jul 2025 14:11:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="gEeUOAtm"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ED73368E42 for ; Wed, 2 Jul 2025 14:11:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eqVaYIUoPfVLYjTay1s76z9DVkCiT/KCn1hwWgLuMgs=; b=gEeUOAtm2EtS6vdP//8cLz8P5aEkUloJj9LZ1WQdntDT7wpP3236K3SqypBQss8wElZGcL SXbF0hO2JIP2EP/eqGbBkmu1DBxUcaES0czzDYm0NNJ/gkyVQebHUYn40z+Lm/K6AqUzQs V2rAQ5YywnRiUBqkHdcTJVXgEFn81/U= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-394-XOf6HnAuPE2bdroJujVURw-1; Wed, 02 Jul 2025 08:11:03 -0400 X-MC-Unique: XOf6HnAuPE2bdroJujVURw-1 X-Mimecast-MFC-AGG-ID: XOf6HnAuPE2bdroJujVURw_1751458262 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7A96219560BE; Wed, 2 Jul 2025 12:11:02 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7B00218003FC; Wed, 2 Jul 2025 12:10:59 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 08/10] libcamera: simple: Don't enforce conversion with an added raw stream Date: Wed, 2 Jul 2025 14:10:04 +0200 Message-ID: <20250702121024.38728-9-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 7_AvEus4iyRpc48uYrPKHuEWM5dpAvlWzgaS-suuAJA_1751458262 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, either alone or together with a processed stream, it can be produced without conversion. Let's amend the corresponding check on the number of configurations, so that the mere presence of a raw stream doesn't enforce conversion. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index b1766c907..a85e8e4b4 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1225,7 +1225,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * require any conversion, similar to raw capture use cases). This is * left as a future improvement. */ - needConversion_ = config_.size() > 1; + needConversion_ = config_.size() > 1 + (rawRequested_ ? 1 : 0); for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; From patchwork Wed Jul 2 12:10:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23714 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F177DBDCBF for ; Wed, 2 Jul 2025 12:11:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9BA5468E4E; Wed, 2 Jul 2025 14:11:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="FtW63zul"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6409168E42 for ; Wed, 2 Jul 2025 14:11:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458270; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pcIiOwCnLt6AGpeEiZMTjRIqapnU1o7NfUJ4tpQpANc=; b=FtW63zulaJitbvPvJRL9DJJaic5yDbRbYHDQ7AOx2MdcUNOPAOfo1lIA8Zid/zKSOnxdY/ wxp3RZcLIJJXYjufvQ8/a3diCBO7vFSVUpWpElyL09t/Fot6vGLU22MfNVU6xIV/5qMyBW bUos2Aiq9EIkMa7zvU6J5ehpvBwlYdw= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-68-fLE9noVENoi7LAUfbq8yDQ-1; Wed, 02 Jul 2025 08:11:06 -0400 X-MC-Unique: fLE9noVENoi7LAUfbq8yDQ-1 X-Mimecast-MFC-AGG-ID: fLE9noVENoi7LAUfbq8yDQ_1751458265 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5FD4B1800368; Wed, 2 Jul 2025 12:11:05 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 062C918003FC; Wed, 2 Jul 2025 12:11:02 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 09/10] libcamera: simple: Require metadata only when software ISP is used Date: Wed, 2 Jul 2025 14:10:05 +0200 Message-ID: <20250702121024.38728-10-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: EYxddbBbp_M0TEpSWNnTlwI-tmUFJ4bOHIlRglZGelU_1751458265 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If software ISP is enabled then metadata is required in the simple pipeline. But this doesn't apply if the software ISP is not actually used, for example when only a raw stream is produced. Then the pipeline waits for metadata that never comes. This patch fixes the problem by requiring metadata only when software ISP is used. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index a85e8e4b4..aceb0d572 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1680,6 +1680,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) int ret; std::map buffers; + bool metadataRequired = false; for (auto &[stream, buffer] : request->buffers()) { /* @@ -1689,6 +1690,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) */ if (data->useConversion_) { buffers.emplace(stream, buffer); + metadataRequired = !!data->swIsp_; } else { ret = data->video_->queueBuffer(buffer); if (ret < 0) @@ -1696,7 +1698,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } - data->frameInfo_.create(request, !!data->swIsp_); + data->frameInfo_.create(request, metadataRequired); if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); if (data->swIsp_) From patchwork Wed Jul 2 12:10:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23715 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CF1F4BDCBF for ; Wed, 2 Jul 2025 12:11:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7D99168E52; Wed, 2 Jul 2025 14:11:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="dvzQnV+M"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 352A768E48 for ; Wed, 2 Jul 2025 14:11:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751458274; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+w5N4yPbsWaqNYpfnLqmWGpp34yHeomGnWWtM4uN3Hg=; b=dvzQnV+MwlQmRlE3t8fnC+DMj4jFoj6/jSorpkD+qaONeP3cIQmNrgZ9VswNduyQKB/NP8 rkt1TIa3/xOrHSZuHUcszSGptpCPOd31H8Ak1t9ZyogDFtg2TBJehway8jojaVCabhRdG4 IVbdSvtYLPW1gc2HrL0iK+5RtSFMM3Q= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-82-MTL_LElWNyOZE6S5REqRmA-1; Wed, 02 Jul 2025 08:11:10 -0400 X-MC-Unique: MTL_LElWNyOZE6S5REqRmA-1 X-Mimecast-MFC-AGG-ID: MTL_LElWNyOZE6S5REqRmA_1751458268 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 40F9919560AE; Wed, 2 Jul 2025 12:11:08 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.33.1]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id F1DE518003FC; Wed, 2 Jul 2025 12:11:05 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v8 10/10] libcamera: simple: Make raw streams working Date: Wed, 2 Jul 2025 14:10:06 +0200 Message-ID: <20250702121024.38728-11-mzamazal@redhat.com> In-Reply-To: <20250702121024.38728-1-mzamazal@redhat.com> References: <20250702121024.38728-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 5vwaiP8gt1f9jWX311Ie1EBu7rD748qWIMtIjnUuVhQ_1751458268 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, whether alone or together with a processed stream, its buffers must be handled outside the software ISP machinery. They serve as output buffers, even when a processed stream is produced. At most one raw stream and at most one processed stream are supported and can be combined. An example of producing both raw and processed files using `cam' application: cam -c1 -C100 -Ffile# \ -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \ -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \ Note the difference in viewfinder and raw stream sizes due to the fact that debayering requires enlarging the image width, which enforces selecting a larger sensor resolution in this case. In order to distinguish between different cases, processedRequested_ and rawRequested_ flags are copied from SimpleCameraConfiguration to SimpleCameraData after successful configure(), similarly to useConversion_ flag. This is the final step to make raw streams working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 35 ++++++++++++++++-------- 1 file changed, 24 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index aceb0d572..5b57f6eb4 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -352,6 +352,8 @@ public: }; std::queue conversionQueue_; bool useConversion_; + bool processedRequested_; + bool rawRequested_; std::unique_ptr converter_; std::unique_ptr swIsp_; @@ -877,10 +879,13 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * point converting an erroneous buffer. */ if (buffer->metadata().status != FrameMetadata::FrameSuccess) { - if (!useConversion_) { + if (rawRequested_) { /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); + SimpleFrameInfo *info = frameInfo_.find(request->sequence()); + if (info) + info->metadataRequired = false; tryCompleteRequest(request); return; } @@ -890,7 +895,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * buffer for capture (unless the stream is being stopped), and * complete the request with all the user-facing buffers. */ - if (buffer->metadata().status != FrameMetadata::FrameCancelled) + if (buffer->metadata().status != FrameMetadata::FrameCancelled && + !rawRequested_) video_->queueBuffer(buffer); if (conversionQueue_.empty()) @@ -939,13 +945,14 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) */ if (useConversion_) { if (conversionQueue_.empty()) { - video_->queueBuffer(buffer); + if (!rawRequested_) + video_->queueBuffer(buffer); return; } if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front().outputs); - else + else if (processedRequested_) /* * request->sequence() cannot be retrieved from `buffer' inside * queueBuffers because unique_ptr's make buffer->request() invalid @@ -955,6 +962,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) conversionQueue_.front().outputs); conversionQueue_.pop(); + if (rawRequested_) + pipe->completeBuffer(request, buffer); return; } @@ -992,7 +1001,8 @@ void SimpleCameraData::tryCompleteRequest(Request *request) void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ - video_->queueBuffer(buffer); + if (!rawRequested_) + video_->queueBuffer(buffer); } void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) @@ -1525,6 +1535,8 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) /* Configure the converter if needed. */ std::vector> outputCfgs; data->useConversion_ = config->needConversion(); + data->processedRequested_ = config->processedRequested_; + data->rawRequested_ = config->rawRequested_; for (unsigned int i = 0; i < config->size(); ++i) { StreamConfiguration &cfg = config->at(i); @@ -1563,7 +1575,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ - if (data->useConversion_) + if (data->useConversion_ && !data->rawRequested_) return data->converter_ ? data->converter_->exportBuffers(stream, count, buffers) : data->swIsp_->exportBuffers(stream, count, buffers); @@ -1586,7 +1598,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return -EBUSY; } - if (data->useConversion_) { + if (data->useConversion_ && !data->rawRequested_) { /* * When using the converter allocate a fixed number of internal * buffers. @@ -1594,7 +1606,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL ret = video->allocateBuffers(kNumInternalBuffers, &data->conversionBuffers_); } else { - /* Otherwise, prepare for using buffers from the only stream. */ + /* Otherwise, prepare for using buffers from the only or raw stream. */ Stream *stream = &data->streams_[0]; ret = video->importBuffers(stream->configuration().bufferCount); } @@ -1636,8 +1648,9 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } /* Queue all internal buffers for capture. */ - for (std::unique_ptr &buffer : data->conversionBuffers_) - video->queueBuffer(buffer.get()); + if (!data->rawRequested_) + for (std::unique_ptr &buffer : data->conversionBuffers_) + video->queueBuffer(buffer.get()); } return 0; @@ -1688,7 +1701,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * queue, it will be handed to the converter in the capture * completion handler. */ - if (data->useConversion_) { + if (data->useConversion_ && !isFormatRaw(stream->configuration().pixelFormat)) { buffers.emplace(stream, buffer); metadataRequired = !!data->swIsp_; } else {