From patchwork Fri Jul 11 17:53:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23781 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 90566C3237 for ; Fri, 11 Jul 2025 17:54:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 42A1868F16; Fri, 11 Jul 2025 19:54:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="GSOZcGIE"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BD98468F13 for ; Fri, 11 Jul 2025 19:54:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752256446; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TZqyNKO9CDLYZpC3z5nN6ndJwcsqu3fO8xRaefcfGfc=; b=GSOZcGIELcSkctLa4T5PHO9x1DkQWn3p0uZl/logANj/WT9fOo1BTzEFICAwWrk2w6T+zy H4jvAeXVm3EPxfiVEQix+dquTjp0en7QKHPR6OuyntHtWW2IxCPce0csbOM6i92p1sypSy xWScjzU7TD0nWxaxdFAIXZNjHuYuays= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-586-RU4HqYraOeGdH1HiFWseig-1; Fri, 11 Jul 2025 13:54:03 -0400 X-MC-Unique: RU4HqYraOeGdH1HiFWseig-1 X-Mimecast-MFC-AGG-ID: RU4HqYraOeGdH1HiFWseig_1752256442 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E3ED61800366; Fri, 11 Jul 2025 17:54:00 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4FCD018002B5; Fri, 11 Jul 2025 17:53:56 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v10 1/8] libcamera: software_isp: Assign colour spaces in configurations Date: Fri, 11 Jul 2025 19:53:34 +0200 Message-ID: <20250711175345.90318-2-mzamazal@redhat.com> In-Reply-To: <20250711175345.90318-1-mzamazal@redhat.com> References: <20250711175345.90318-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: phOOScnESiEnEVq3LPBIKsurgDgF0eoVt_CRlIDrjqE_1752256442 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" StreamConfiguration's should have colorSpace set. This is not the case in the simple pipeline. Let's set it there. This also fixes a crash in `cam' due to accessing an unset colorSpace. The colour space is assigned as follows: - If raw role is requested, then the colour space must be raw, regardless of the stream format. - Otherwise, if software ISP is used, the default colour space is set to Srgb (YcbcrEncoding::None, Range::Full). - Otherwise, if a converter is used, the default colour space is left unset. - Then in configuration validation, if the pixel format is changed, the colour space is set according to the new pixel format. - If the colour space is still unset, it is set according to the resulting pixel format. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 38 +++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index efb07051b..d45480fe7 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -35,6 +36,7 @@ #include "libcamera/internal/converter.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/software_isp/software_isp.h" @@ -1206,6 +1208,28 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() if (cfg.pixelFormat != pixelFormat) { LOG(SimplePipeline, Debug) << "Adjusting pixel format"; cfg.pixelFormat = pixelFormat; + /* + * Do not touch the colour space for raw requested roles. + * Even if the pixel format is non-raw (whatever it means), we + * shouldn't try to interpret the colour space of raw data. + */ + if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + cfg.colorSpace->adjust(pixelFormat); + status = Adjusted; + } + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } + cfg.colorSpace->adjust(pixelFormat); status = Adjusted; } @@ -1314,11 +1338,23 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Spanfirst; cfg.size = formats.begin()->second[0].max; + if (role == StreamRole::Raw) { + /* Enforce raw colour space for raw roles. */ + cfg.colorSpace = ColorSpace::Raw; + } else if (data->swIsp_) { + /* + * This is what software ISP currently produces. It may be arguable + * whether this applies also when CCM is not used but even then it's + * still likely to be a reasonable setting. + */ + cfg.colorSpace = ColorSpace::Srgb; + } + config->addConfiguration(cfg); } From patchwork Fri Jul 11 17:53:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23788 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6D34EBE175 for ; Fri, 11 Jul 2025 18:04:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8269A68F1E; Fri, 11 Jul 2025 20:04:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="HRun26Co"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 983B068E30 for ; Fri, 11 Jul 2025 20:04:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752257057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8wFXUaYoFNERZep7XD3mebdsvA8+dekpozWqRvYhk7k=; b=HRun26CoqHBqhcel4opK6rzdi1TuRHOvh6fR/dviLriHi1HCMdMsgk0jZKNlMPv0C2xqtF SVOOnMVPEiVPwzhWrsj4qF6wNwaAMTedIbQHE5hIshALiesaJB0ET4hzzVS6a7+5ABchKR aryxpcjgmDuZe9Vga1PmYO0MaQyyp04= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-664-nI5d_mddPRGD9DUJrlDmYA-1; Fri, 11 Jul 2025 13:54:05 -0400 X-MC-Unique: nI5d_mddPRGD9DUJrlDmYA-1 X-Mimecast-MFC-AGG-ID: nI5d_mddPRGD9DUJrlDmYA_1752256444 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 224401801208; Fri, 11 Jul 2025 17:54:04 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 863A01803AF2; Fri, 11 Jul 2025 17:54:01 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v10 2/8] libcamera: simple: Set the number of software ISP streams to 2 Date: Fri, 11 Jul 2025 19:53:35 +0200 Message-ID: <20250711175345.90318-3-mzamazal@redhat.com> In-Reply-To: <20250711175345.90318-1-mzamazal@redhat.com> References: <20250711175345.90318-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: u3fP4Iaira1GxZTqVovIv3N-Bxyrf0s18iLdXrlC7dM_1752256444 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When software ISP is enabled, we want to be able to provide a raw stream in addition to the processed stream. For this purpose, we need two streams. If only the processed stream is requested, it doesn't harm to allocate two. This is a hack for the lack of a better easy option. The number of streams is determined as a camera property in the pipeline matching. The actual number of streams needed (one or two) is determined only when examining roles in SimplePipelineHandler::generateConfiguration. In theory, software ISP could produce multiple processed streams but this is out of scope of this patch series. Hence two streams are sufficient at the moment. When software ISP is not enabled, the camera won't be able to produce multiple streams (assuming there's no hardware converter) and only single stream should be allocated as before. The simple pipeline handler assumes there's a linear pipeline from the camera sensor to a video capture device, and only supports a single stream. Branches in the hardware pipeline that would allow capturing multiple streams from the same camera sensor are not supported. We have no plan to change that, as a device that can produce multiple streams will likely be better supported by a dedicated pipeline handler. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index d45480fe7..06532ed26 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1723,7 +1723,18 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) } } - swIspEnabled_ = info->swIspEnabled; + if (info->swIspEnabled) { + /* + * When the software ISP is enabled, the simple pipeline handler + * exposes the raw stream, giving a total of two streams. This + * is mutally exclusive with the presence of a converter. + */ + ASSERT(!converter_); + numStreams = 2; + swIspEnabled_ = true; + } else { + swIspEnabled_ = false; + } /* Locate the sensors. */ std::vector sensors = locateSensors(media); From patchwork Fri Jul 11 17:53:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23782 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 99032C3237 for ; Fri, 11 Jul 2025 17:54:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4CD6268F21; Fri, 11 Jul 2025 19:54:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="TjspY3AP"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EB12B68E30 for ; Fri, 11 Jul 2025 19:54:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752256455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4fmLtovBn3u4MmSbj2hKS5JbrPSnPRQ381HKDRVtSNU=; b=TjspY3APcPHUds54jt+GbBtDbR844lrzq3JoEO7c2g2cz6VyeoAeW4HNZGdkc6iVnrZ0Op pokJPgjCMjd6lbhZUMIx2D94/N50mVxTjwCfHXRaPk0X0fTvgzUU7lkjnQRAerRVsl5L50 UCLjNLlp/wJZY9dXv5/HWbFrIcxzkF8= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-674-sbh6PA_QNDy0HGbBELFArA-1; Fri, 11 Jul 2025 13:54:08 -0400 X-MC-Unique: sbh6PA_QNDy0HGbBELFArA-1 X-Mimecast-MFC-AGG-ID: sbh6PA_QNDy0HGbBELFArA_1752256447 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2C54F19560AD; Fri, 11 Jul 2025 17:54:07 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B31F618002B5; Fri, 11 Jul 2025 17:54:04 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v10 3/8] libcamera: simple: Exclude raw configurations from output conversions Date: Fri, 11 Jul 2025 19:53:36 +0200 Message-ID: <20250711175345.90318-4-mzamazal@redhat.com> In-Reply-To: <20250711175345.90318-1-mzamazal@redhat.com> References: <20250711175345.90318-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 59BKo-KMHBnJVZvHBO_Zck8d3c4NicarcmAJGMyB_VA_1752256447 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to support raw streams, we need to add raw formats to software ISP configurations. In this preparatory patch, the raw formats are excluded from output configurations for conversions. Signed-off-by: Milan Zamazal Reviewed-by: Umang Jain Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 06532ed26..87b26d4a2 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -261,6 +262,12 @@ static const SimplePipelineInfo supportedDevices[] = { { "sun6i-csi", {}, false }, }; +bool isFormatRaw(const libcamera::PixelFormat &pixFmt) +{ + return libcamera::PixelFormatInfo::info(pixFmt).colourEncoding == + libcamera::PixelFormatInfo::ColourEncodingRAW; +} + } /* namespace */ class SimpleCameraData : public Camera::Private @@ -1424,7 +1431,7 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) cfg.setStream(&data->streams_[i]); - if (data->useConversion_) + if (data->useConversion_ && !isFormatRaw(cfg.pixelFormat)) outputCfgs.push_back(cfg); } From patchwork Fri Jul 11 17:53:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23783 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2C522C3237 for ; Fri, 11 Jul 2025 17:54:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D7B5568F19; Fri, 11 Jul 2025 19:54:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KLW0bcoS"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9B40968F22 for ; Fri, 11 Jul 2025 19:54:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752256456; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jl9xfCAU03T6MQZe9LMyGoUEEt/fHSLHEoM66+PGXiM=; b=KLW0bcoS0OU2EXYc2Tnw+8KJtiCHyfEt4vbwTKJwzcN/RTHr1oDk9wAfCwZLbjFvWgj2Pk AF/BkfN8E23dyR7qbUPdLzn7iAfj7KMUPXJwO35YetzX5XFX5Wr+/3Y1nF5DRM6o3FF95i OqUawbnezGMvlY32qWhQlB4OalGB3aE= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-622-goDA_i2BOuC5Z1M9L2iqVA-1; Fri, 11 Jul 2025 13:54:13 -0400 X-MC-Unique: goDA_i2BOuC5Z1M9L2iqVA-1 X-Mimecast-MFC-AGG-ID: goDA_i2BOuC5Z1M9L2iqVA_1752256452 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 717151809CBC; Fri, 11 Jul 2025 17:54:10 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BC20E18002B5; Fri, 11 Jul 2025 17:54:07 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v10 4/8] libcamera: simple: Handle processed and raw formats separately Date: Fri, 11 Jul 2025 19:53:37 +0200 Message-ID: <20250711175345.90318-5-mzamazal@redhat.com> In-Reply-To: <20250711175345.90318-1-mzamazal@redhat.com> References: <20250711175345.90318-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 4-pIfiH185pwcMNHM_1WO1Sx-RNDrq2Ytmdl9xWhWbE_1752256452 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Let's handle both processed and/or raw output configurations, depending on the requested stream roles. In addition to the already handled processed formats and sizes, this patch adds handling of raw formats and sizes, which correspond to the capture formats and sizes. Raw and processed formats are handled separately. The intention is that in case both raw and processed formats are requested then the raw formats should be left intact to the extent possible and not be influenced by the processed formats (implying that raw parameters compatible with the processed output requirements must be requested by the application). Accompanying checks for invalid role specifications are introduced. This is another preparatory patch without making raw outputs working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 80 +++++++++++++++++------- 1 file changed, 57 insertions(+), 23 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 87b26d4a2..37abaa0e0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1310,35 +1310,67 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span> formats; + bool processedRequested = false; + bool rawRequested = false; + for (const auto &role : roles) + if (role == StreamRole::Raw) { + if (rawRequested) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return nullptr; + } + rawRequested = true; + } else { + processedRequested = true; + } + + /* Create the formats maps. */ + std::map> processedFormats; + std::map> rawFormats; for (const SimpleCameraData::Configuration &cfg : data->configs_) { + rawFormats[cfg.captureFormat].push_back(cfg.captureSize); for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); + processedFormats[format].push_back(cfg.outputSizes); } - /* Sort the sizes and merge any consecutive overlapping ranges. */ - for (auto &[format, sizes] : formats) { - std::sort(sizes.begin(), sizes.end(), - [](SizeRange &a, SizeRange &b) { - return a.min < b.min; - }); - - auto cur = sizes.begin(); - auto next = cur; - - while (++next != sizes.end()) { - if (cur->max.width >= next->min.width && - cur->max.height >= next->min.height) - cur->max = next->max; - else if (++cur != next) - *cur = *next; - } - - sizes.erase(++cur, sizes.end()); + if (processedRequested && processedFormats.empty()) { + LOG(SimplePipeline, Error) + << "Processed stream requsted but no corresponding output configuration found"; + return nullptr; + } + if (rawRequested && rawFormats.empty()) { + LOG(SimplePipeline, Error) + << "Raw stream requsted but no corresponding output configuration found"; + return nullptr; } + auto setUpFormatSizes = [](std::map> &formats) { + /* Sort the sizes and merge any consecutive overlapping ranges. */ + + for (auto &[format, sizes] : formats) { + std::sort(sizes.begin(), sizes.end(), + [](SizeRange &a, SizeRange &b) { + return a.min < b.min; + }); + + auto cur = sizes.begin(); + auto next = cur; + + while (++next != sizes.end()) { + if (cur->max.width >= next->min.width && + cur->max.height >= next->min.height) + cur->max = next->max; + else if (++cur != next) + *cur = *next; + } + + sizes.erase(++cur, sizes.end()); + } + }; + setUpFormatSizes(processedFormats); + setUpFormatSizes(rawFormats); + /* * Create the stream configurations. Take the first entry in the formats * map as the default, for lack of a better option. @@ -1346,11 +1378,13 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Spanfirst; cfg.size = formats.begin()->second[0].max; - if (role == StreamRole::Raw) { + if (raw) { /* Enforce raw colour space for raw roles. */ cfg.colorSpace = ColorSpace::Raw; } else if (data->swIsp_) { From patchwork Fri Jul 11 17:53:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23784 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CA511C3237 for ; Fri, 11 Jul 2025 17:54:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8438E68F19; Fri, 11 Jul 2025 19:54:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="bGgD8EaY"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 430E268F22 for ; Fri, 11 Jul 2025 19:54:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752256460; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VyRxSJ1qar3geSLTPl3rizQhlAMu1q6r2Ds4+RIwx4o=; b=bGgD8EaYjA5VgX5HpjKCJ3cXvpniFtzjfrwM9DMwwPhGwqplffdlDTsl3g7KwWVgCNYhem bKVDWEXrh8jiRi4Hy+wA3wggdRiphYwPg1O55VoMBBF5uS+IoKpf33qvRO0tQcEyHFcgb0 buaweYhQWX14JSaRbkv0xZvkfapTKc4= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-426-bB2klvoiOrygNLlSkOV2xg-1; Fri, 11 Jul 2025 13:54:18 -0400 X-MC-Unique: bB2klvoiOrygNLlSkOV2xg-1 X-Mimecast-MFC-AGG-ID: bB2klvoiOrygNLlSkOV2xg_1752256457 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E4EB718600C4; Fri, 11 Jul 2025 17:54:13 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E3DC0180884B; Fri, 11 Jul 2025 17:54:10 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v10 5/8] libcamera: simple: Validate raw stream configurations Date: Fri, 11 Jul 2025 19:53:38 +0200 Message-ID: <20250711175345.90318-6-mzamazal@redhat.com> In-Reply-To: <20250711175345.90318-1-mzamazal@redhat.com> References: <20250711175345.90318-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: CWJ8nXu3Ft2esnVPtzgjGJvJDStzQwgBQMSl23NHqP8_1752256457 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. If only a processed stream is requested, nothing changes. If only a raw stream is requested, the pixel format and output size may not be adjusted. The patch adds checks for this. If both processed and raw streams are requested, things get more complicated. The raw stream is expected to be passed through intact and all the adjustments are made for the processed streams. We select a pipe configuration for the processed streams. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. It is the application responsibility to select the right parameters for the raw stream. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 119 ++++++++++++++++------- 1 file changed, 85 insertions(+), 34 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 37abaa0e0..9d87a7ffa 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -1186,6 +1187,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() << "-" << pipeConfig_->captureFormat << " for max stream size " << maxStreamSize; + bool rawRequested = false; + bool processedRequested = false; + for (const auto &cfg : config_) + if (cfg.colorSpace == ColorSpace::Raw) { + if (rawRequested) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return Invalid; + } + rawRequested = true; + } else { + processedRequested = true; + } + /* * Adjust the requested streams. * @@ -1204,43 +1219,70 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + /* + * If both processed and raw streams are requested, the pipe + * configuration is set up for the processed stream. The raw + * configuration needs to be compared against the capture format and + * size in such a case. + */ + const bool rawStream = cfg.colorSpace == ColorSpace::Raw; + const bool sideRawStream = rawStream && processedRequested; + /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); - - PixelFormat pixelFormat = *it; - if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) << "Adjusting pixel format"; - cfg.pixelFormat = pixelFormat; - /* - * Do not touch the colour space for raw requested roles. - * Even if the pixel format is non-raw (whatever it means), we - * shouldn't try to interpret the colour space of raw data. - */ - if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + + if (!sideRawStream) { + PixelFormat pixelFormat; + if (rawStream) { + pixelFormat = pipeConfig_->captureFormat; + } else { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + pixelFormat = *it; + } + + if (cfg.pixelFormat != pixelFormat) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw pixel format " + << cfg.pixelFormat + << " doesn't match any of the pipe output formats"; + return Invalid; + } + LOG(SimplePipeline, Debug) + << "Adjusting pixel format from " << cfg.pixelFormat + << " to " << pixelFormat; + cfg.pixelFormat = pixelFormat; + /* + * Do not touch the colour space for raw requested roles. + * Even if the pixel format is non-raw (whatever it means), we + * shouldn't try to interpret the colour space of raw data. + */ + if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + cfg.colorSpace->adjust(pixelFormat); + status = Adjusted; + } + + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } cfg.colorSpace->adjust(pixelFormat); - status = Adjusted; - } - if (!cfg.colorSpace) { - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); - switch (info.colourEncoding) { - case PixelFormatInfo::ColourEncodingRGB: - cfg.colorSpace = ColorSpace::Srgb; - break; - case libcamera::PixelFormatInfo::ColourEncodingYUV: - cfg.colorSpace = ColorSpace::Sycc; - break; - default: - cfg.colorSpace = ColorSpace::Raw; + status = Adjusted; } - cfg.colorSpace->adjust(pixelFormat); - status = Adjusted; } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!rawStream && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1259,11 +1301,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* \todo Create a libcamera core class to group format and size */ if (cfg.pixelFormat != pipeConfig_->captureFormat || - cfg.size != pipeConfig_->captureSize) + cfg.size != pipeConfig_->captureSize) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw output format " << cfg.pixelFormat + << " and size " << cfg.size + << " not matching pipe format " << pipeConfig_->captureFormat + << " and size " << pipeConfig_->captureSize; + return Invalid; + } needConversion_ = true; + } /* Set the stride, frameSize and bufferCount. */ - if (needConversion_) { + if (needConversion_ && !rawStream) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat, From patchwork Fri Jul 11 17:53:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23786 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5B445C3237 for ; Fri, 11 Jul 2025 17:54:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1398E68F2E; Fri, 11 Jul 2025 19:54:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="HMnKzz4P"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2DA9D68F19 for ; Fri, 11 Jul 2025 19:54:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752256468; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7TafP4kG0ikMs4iD7uEczFQmQGH+TgLqY7myaejW+CE=; b=HMnKzz4PJ5qsSIxPA7rKE4QfdvYJPTtf6+shmuOQx3D94uULbzAaMr7eYCNedlRLBTLqHB VFgrloplM1Te4JY6jdAU0/SCE1rggxBvdOfrUX86z0E6Jipr63BFJc+SE72pc0ppUAATPa H+DbqwYzZuO/T4CpjsrEORPQZQx7oMw= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-587-cESq6necNhSz3H7TtqHZBA-1; Fri, 11 Jul 2025 13:54:22 -0400 X-MC-Unique: cESq6necNhSz3H7TtqHZBA-1 X-Mimecast-MFC-AGG-ID: cESq6necNhSz3H7TtqHZBA_1752256461 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1BD2D1801223; Fri, 11 Jul 2025 17:54:17 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9611A1803AF2; Fri, 11 Jul 2025 17:54:14 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v10 6/8] libcamera: simple: Don't enforce conversion with an added raw stream Date: Fri, 11 Jul 2025 19:53:39 +0200 Message-ID: <20250711175345.90318-7-mzamazal@redhat.com> In-Reply-To: <20250711175345.90318-1-mzamazal@redhat.com> References: <20250711175345.90318-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: ovt12I7VsAlgFVEY9t-km5BJG_pZ8i1r3sU4wLU3JZQ_1752256461 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, either alone or together with a processed stream, it can be produced without conversion. Let's amend the corresponding check on the number of configurations, so that the mere presence of a raw stream doesn't enforce conversion. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 9d87a7ffa..e7cfcedd7 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1214,7 +1214,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * require any conversion, similar to raw capture use cases). This is * left as a future improvement. */ - needConversion_ = config_.size() > 1; + needConversion_ = config_.size() > 1 + (rawRequested ? 1 : 0); for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; From patchwork Fri Jul 11 17:53:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23787 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0F16EC32C8 for ; Fri, 11 Jul 2025 17:54:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B95E468F2D; Fri, 11 Jul 2025 19:54:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="QOy9yJIg"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B386768F19 for ; Fri, 11 Jul 2025 19:54:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752256469; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jXLMmSBUgBROEjm/i7gfuuTwCb6UsRpO8fVBwaAxXVI=; b=QOy9yJIgwbJDS1HrgXgPzdUzbUJ16K+mgIfRFYoylp/xOeQ5v4CsDLr/FHf+kqJmD/smEM 3XABbCtf/tj1wr9X1tHHUFdTH4/ReFRxlstq97+u1l1iUt0uEQIMWf5Tk7Gh0HDxAE3o+R rG3+VTJbbdsmRwXLQ0D6nvXxDGfAnH4= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-563-a325YLAgOmmU_ymkFPHWwg-1; Fri, 11 Jul 2025 13:54:26 -0400 X-MC-Unique: a325YLAgOmmU_ymkFPHWwg-1 X-Mimecast-MFC-AGG-ID: a325YLAgOmmU_ymkFPHWwg_1752256465 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EE225180AC45; Fri, 11 Jul 2025 17:54:19 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 92A971803AF2; Fri, 11 Jul 2025 17:54:17 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v10 7/8] libcamera: simple: Require metadata only when software ISP is used Date: Fri, 11 Jul 2025 19:53:40 +0200 Message-ID: <20250711175345.90318-8-mzamazal@redhat.com> In-Reply-To: <20250711175345.90318-1-mzamazal@redhat.com> References: <20250711175345.90318-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: ZMPt5wNQPZcKb0OUAaih7X7oJlxUbUzmw7cC28fCBr8_1752256465 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If software ISP is enabled then metadata is required in the simple pipeline. But this doesn't apply if the software ISP is not actually used, for example when only a raw stream is produced. Then the pipeline waits for metadata that never comes. This patch fixes the problem by requiring metadata only when software ISP is used. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index e7cfcedd7..01b25fb20 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1665,6 +1665,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) int ret; std::map buffers; + bool metadataRequired = false; for (auto &[stream, buffer] : request->buffers()) { /* @@ -1674,6 +1675,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) */ if (data->useConversion_) { buffers.emplace(stream, buffer); + metadataRequired = !!data->swIsp_; } else { ret = data->video_->queueBuffer(buffer); if (ret < 0) @@ -1681,7 +1683,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } - data->frameInfo_.create(request, !!data->swIsp_); + data->frameInfo_.create(request, metadataRequired); if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); if (data->swIsp_) From patchwork Fri Jul 11 17:53:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23785 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B7D2CC3237 for ; Fri, 11 Jul 2025 17:54:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 59AA368F2B; Fri, 11 Jul 2025 19:54:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="c4WAplZ3"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54A3368F16 for ; Fri, 11 Jul 2025 19:54:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1752256467; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/S61lpbEt2Cp0S+zIPTx2ICqBOHzAyKS/hBYAvmJlhk=; b=c4WAplZ3L99zc9AhMKmyxdk1NYzwIBBbeQvpnOBA+ND4hGQwpPXZTvomnAg6tEgrSdPG7O 25IKrUKcq99oBtD2ZdzhptGwzM0MayEqW34dchwOuuFW3C0A+iw3jojnQNu1021Aik48eG 5dLZkkMeCQTrSNpKcDyIut7KcXA1sDk= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-550-1VPyciNNNpCIc-ouB-d1-w-1; Fri, 11 Jul 2025 13:54:25 -0400 X-MC-Unique: 1VPyciNNNpCIc-ouB-d1-w-1 X-Mimecast-MFC-AGG-ID: 1VPyciNNNpCIc-ouB-d1-w_1752256464 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 17CDF180029E; Fri, 11 Jul 2025 17:54:23 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.49]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8B35918046C7; Fri, 11 Jul 2025 17:54:20 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Paul Elder , Umang Jain Subject: [PATCH v10 8/8] libcamera: simple: Make raw streams working Date: Fri, 11 Jul 2025 19:53:41 +0200 Message-ID: <20250711175345.90318-9-mzamazal@redhat.com> In-Reply-To: <20250711175345.90318-1-mzamazal@redhat.com> References: <20250711175345.90318-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: j18_c-OWvn42VlJu9b15sQAZSpT_RY0yUpYPS5TwfT0_1752256464 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, whether alone or together with a processed stream, its buffers must be handled outside the software ISP machinery. They serve as output buffers, even when a processed stream is produced. At most one raw stream and at most one processed stream are supported and can be combined. An example of producing both raw and processed files using `cam' application: cam -c1 -C100 -Ffile# \ -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \ -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \ Note the difference in viewfinder and raw stream sizes due to the fact that debayering requires enlarging the image width, which enforces selecting a larger sensor resolution in this case. In order to distinguish between different cases, processedRequested_ and rawRequested_ flags are copied from SimpleCameraConfiguration to SimpleCameraData after successful configure(), similarly to useConversion_ flag. This is the final step to make raw streams working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 41 +++++++++++++++++------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 01b25fb20..c3db5c068 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -351,6 +351,8 @@ public: }; std::queue conversionQueue_; bool useConversion_; + bool processedRequested_; + bool rawRequested_; std::unique_ptr converter_; std::unique_ptr swIsp_; @@ -384,6 +386,9 @@ public: bool needConversion() const { return needConversion_; } const Transform &combinedTransform() const { return combinedTransform_; } + bool processedRequested_; + bool rawRequested_; + private: /* * The SimpleCameraData instance is guaranteed to be valid as long as @@ -867,10 +872,13 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * point converting an erroneous buffer. */ if (buffer->metadata().status != FrameMetadata::FrameSuccess) { - if (!useConversion_) { + if (rawRequested_) { /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); + SimpleFrameInfo *info = frameInfo_.find(request->sequence()); + if (info) + info->metadataRequired = false; tryCompleteRequest(request); return; } @@ -880,7 +888,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * buffer for capture (unless the stream is being stopped), and * complete the request with all the user-facing buffers. */ - if (buffer->metadata().status != FrameMetadata::FrameCancelled) + if (buffer->metadata().status != FrameMetadata::FrameCancelled && + !rawRequested_) video_->queueBuffer(buffer); if (conversionQueue_.empty()) @@ -929,13 +938,14 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) */ if (useConversion_) { if (conversionQueue_.empty()) { - video_->queueBuffer(buffer); + if (!rawRequested_) + video_->queueBuffer(buffer); return; } if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front().outputs); - else + else if (processedRequested_) /* * request->sequence() cannot be retrieved from `buffer' inside * queueBuffers because unique_ptr's make buffer->request() invalid @@ -945,6 +955,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) conversionQueue_.front().outputs); conversionQueue_.pop(); + if (rawRequested_) + pipe->completeBuffer(request, buffer); return; } @@ -982,7 +994,8 @@ void SimpleCameraData::tryCompleteRequest(Request *request) void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ - video_->queueBuffer(buffer); + if (!rawRequested_) + video_->queueBuffer(buffer); } void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) @@ -1201,6 +1214,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() processedRequested = true; } + rawRequested_ = rawRequested; + processedRequested_ = processedRequested; + /* * Adjust the requested streams. * @@ -1510,6 +1526,8 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) /* Configure the converter if needed. */ std::vector> outputCfgs; data->useConversion_ = config->needConversion(); + data->processedRequested_ = config->processedRequested_; + data->rawRequested_ = config->rawRequested_; for (unsigned int i = 0; i < config->size(); ++i) { StreamConfiguration &cfg = config->at(i); @@ -1548,7 +1566,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ - if (data->useConversion_) + if (data->useConversion_ && !isFormatRaw(stream->configuration().pixelFormat)) return data->converter_ ? data->converter_->exportBuffers(stream, count, buffers) : data->swIsp_->exportBuffers(stream, count, buffers); @@ -1571,7 +1589,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return -EBUSY; } - if (data->useConversion_) { + if (data->useConversion_ && !data->rawRequested_) { /* * When using the converter allocate a fixed number of internal * buffers. @@ -1579,7 +1597,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL ret = video->allocateBuffers(kNumInternalBuffers, &data->conversionBuffers_); } else { - /* Otherwise, prepare for using buffers from the only stream. */ + /* Otherwise, prepare for using buffers from the only or raw stream. */ Stream *stream = &data->streams_[0]; ret = video->importBuffers(stream->configuration().bufferCount); } @@ -1621,8 +1639,9 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } /* Queue all internal buffers for capture. */ - for (std::unique_ptr &buffer : data->conversionBuffers_) - video->queueBuffer(buffer.get()); + if (!data->rawRequested_) + for (std::unique_ptr &buffer : data->conversionBuffers_) + video->queueBuffer(buffer.get()); } return 0; @@ -1673,7 +1692,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * queue, it will be handed to the converter in the capture * completion handler. */ - if (data->useConversion_) { + if (data->useConversion_ && !isFormatRaw(stream->configuration().pixelFormat)) { buffers.emplace(stream, buffer); metadataRequired = !!data->swIsp_; } else {