From patchwork Wed Mar 5 19:26:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22932 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3BFA0C0F1B for ; Wed, 5 Mar 2025 19:26:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E78F568823; Wed, 5 Mar 2025 20:26:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="SowhJ3p3"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B6311687DD for ; Wed, 5 Mar 2025 20:26:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741202796; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BkpqGcdaYYpxTu+vGuccbFgeqK+eZ4P+mUghp1l1Br0=; b=SowhJ3p3fRQWXr9tG2D3yAUH/7P2kn67g5oiw/9DxqUiSMZTz06W6JUQ4ei0zN1xdUpoqZ /IHGber9/JJG0hig2zUShKLK2Gg2tnZWgHLOaB4ly/erv7GXVwLqdF1Le1u45HdW+e8uaf 8nEjCULW4+P9Mlk04Htx5d+2HYSkBhM= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-661-PSl-49W7OeyS_ovUNbJUlQ-1; Wed, 05 Mar 2025 14:26:30 -0500 X-MC-Unique: PSl-49W7OeyS_ovUNbJUlQ-1 X-Mimecast-MFC-AGG-ID: PSl-49W7OeyS_ovUNbJUlQ_1741202789 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 554331800DB3; Wed, 5 Mar 2025 19:26:29 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.87]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BE861180049D; Wed, 5 Mar 2025 19:26:27 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v3 1/8] libcamera: simple: Set the number of software ISP streams to 2 Date: Wed, 5 Mar 2025 20:26:08 +0100 Message-ID: <20250305192617.18361-2-mzamazal@redhat.com> In-Reply-To: <20250305192617.18361-1-mzamazal@redhat.com> References: <20250305192617.18361-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Upb0gTixabdyYyZGGr9bmDCKEDzxuJtMkBnpXZD6Fpc_1741202789 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When software ISP is enabled, we want to be able to provide a raw stream in addition to the processed stream. For this purpose, we need two streams. If only the processed stream is requested, it doesn't harm to allocate two. This is a hack for the lack of a better easy option. The number of streams is determined as a camera property in the pipeline matching. The actual number of streams needed (one or two) is determined only when examining roles in SimplePipelineHandler::generateConfiguration. In theory, software ISP could produce multiple processed streams but this is out of scope of this patch series. Hence two streams are sufficient at the moment. When software ISP is not enabled, the camera won't be able to produce multiple streams (assuming there's no hardware converter) and only single stream should be allocated as before. The simple pipeline handler assumes there's a linear pipeline from the camera sensor to a video capture device, and only supports a single stream. Branches in the hardware pipeline that would allow capturing multiple streams from the same camera sensor are not supported. We have no plan to change that, as a device that can produce multiple streams will likely be better supported by a dedicated pipeline handler. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 6e039bf3..ab3a68ff 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1559,7 +1559,16 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) } } - swIspEnabled_ = info->swIspEnabled; + if (info->swIspEnabled) { + /* + * When the software ISP is enabled, the simple pipeline handler + * exposes the raw stream, giving a total of two streams. This + * is mutally exclusive with the presence of a converter. + */ + ASSERT(!converter_); + numStreams = 2; + swIspEnabled_ = true; + } /* Locate the sensors. */ std::vector sensors = locateSensors(media); From patchwork Wed Mar 5 19:26:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22937 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1E785C0F1B for ; Wed, 5 Mar 2025 19:26:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AC2CC68889; Wed, 5 Mar 2025 20:26:51 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="LTk4Y9jO"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2F3D7688AE for ; Wed, 5 Mar 2025 20:26:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741202808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hHFfDlMShFt1q2vxroGbFlHHj+Q+oFXQag/PUlVPzwQ=; b=LTk4Y9jOzrxabxpsOkExtq2Oy6HXpn6juv0VA5MARDsJItaB1wDzLosZzI9E+dsWVjlY92 KaP8dza1DNfofXpTew5elAQ4pySDOSdRHCJ6EcN8MEJigt2XeHztvd5w9kg07gcCHBvyds 1jpt7mjz1udSMvywq8+0zT4hAefQUEs= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-544-h6aK16UZNcqBTnUSY3xONQ-1; Wed, 05 Mar 2025 14:26:32 -0500 X-MC-Unique: h6aK16UZNcqBTnUSY3xONQ-1 X-Mimecast-MFC-AGG-ID: h6aK16UZNcqBTnUSY3xONQ_1741202791 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 50AEE1800EC5; Wed, 5 Mar 2025 19:26:31 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.87]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C71B1180049D; Wed, 5 Mar 2025 19:26:29 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v3 2/8] libcamera: simple: Don't use raw output formats with conversions Date: Wed, 5 Mar 2025 20:26:09 +0100 Message-ID: <20250305192617.18361-3-mzamazal@redhat.com> In-Reply-To: <20250305192617.18361-1-mzamazal@redhat.com> References: <20250305192617.18361-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: qN_7iQ1WtM8mwhT4Snvo1bJfTh7UC3pZ-t7oLK8Zm0c_1741202791 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to support raw streams, we need to add raw formats to software ISP configurations. In this preparatory patch, the raw formats are excluded from output configurations for conversions. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index ab3a68ff..4af5926f 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -26,6 +26,7 @@ #include #include +#include #include #include @@ -209,6 +210,12 @@ static const SimplePipelineInfo supportedDevices[] = { { "sun6i-csi", {}, false }, }; +bool isRawFormat(const PixelFormat &format) +{ + return libcamera::PixelFormatInfo::info(format).colourEncoding == + libcamera::PixelFormatInfo::ColourEncodingRAW; +} + } /* namespace */ class SimpleCameraData : public Camera::Private @@ -1270,7 +1277,7 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) cfg.setStream(&data->streams_[i]); - if (data->useConversion_) + if (data->useConversion_ && !isRawFormat(cfg.pixelFormat)) outputCfgs.push_back(cfg); } From patchwork Wed Mar 5 19:26:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22933 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0AEEC0F1B for ; Wed, 5 Mar 2025 19:26:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 91C5C6888A; Wed, 5 Mar 2025 20:26:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="cZlNpFMs"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E105687DD for ; Wed, 5 Mar 2025 20:26:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741202799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8xavB+t1/eE69OcBO60jpOzoOfwp/dEbrHtAoTGzyqw=; b=cZlNpFMs/xtshS3MvSBSbWuxNm5RagUkSCK2d8FTRuSJeUljoN3kV2yunbyy8nttiv+65R 9lLJ/CvU8eXqtYIsgvgm01zNG6PCYpSRJP9Q+d+rzTXRxe6PougjvwG2jgAHPBfW9MF5+J HK61YYmHq55ieG5MEckNwKQPAIPvrC8= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-287-d8IwSaC8M-eyvFCtX0N8sg-1; Wed, 05 Mar 2025 14:26:34 -0500 X-MC-Unique: d8IwSaC8M-eyvFCtX0N8sg-1 X-Mimecast-MFC-AGG-ID: d8IwSaC8M-eyvFCtX0N8sg_1741202793 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 601761955BCC; Wed, 5 Mar 2025 19:26:33 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.87]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E2EEF1800255; Wed, 5 Mar 2025 19:26:31 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v3 3/8] libcamera: simple: Add plain output configurations Date: Wed, 5 Mar 2025 20:26:10 +0100 Message-ID: <20250305192617.18361-4-mzamazal@redhat.com> In-Reply-To: <20250305192617.18361-1-mzamazal@redhat.com> References: <20250305192617.18361-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 2vdbE4B3tbKav5bSYTki1kaO-Wm1KDxEs3sNUbGn4TY_1741202793 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Output configurations in simple pipeline are added depending on whether a converter, software ISP or none of them are used. If a converter or software ISP is used, no raw output configurations are added. In order to support raw output at least with software ISP, let's always add raw output configurations. A flag is added to SimpleCameraData::Configuration indicating whether it's for a raw or a converted output. We later filter formats and output sizes for particular stream configurations according to the new configuration flag. This is just preparation and raw output is still not supported. At the moment, we simply filter out raw configurations unconditionally to keep the current code working; this will be changed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 31 +++++++++++++++--------- 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 4af5926f..c805639c 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -273,6 +273,7 @@ public: Size captureSize; std::vector outputFormats; SizeRange outputSizes; + bool raw; }; std::vector streams_; @@ -636,27 +637,33 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) continue; Configuration config; + + /* Raw configuration */ config.code = code; config.sensorSize = size; config.captureFormat = pixelFormat; config.captureSize = format.size; + config.outputFormats = { pixelFormat }; + config.outputSizes = config.captureSize; + config.raw = true; + configs_.push_back(config); + /* Modified, non-raw, configuration */ + config.raw = false; if (converter_) { config.outputFormats = converter_->formats(pixelFormat); config.outputSizes = converter_->sizes(format.size); } else if (swIsp_) { - config.outputFormats = swIsp_->formats(pixelFormat); - config.outputSizes = swIsp_->sizes(pixelFormat, format.size); - if (config.outputFormats.empty()) { + std::vector outputFormats = swIsp_->formats(pixelFormat); + if (outputFormats.empty()) { /* Do not use swIsp for unsupported pixelFormat's. */ - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; + continue; } + config.outputFormats = outputFormats; + config.outputSizes = swIsp_->sizes(pixelFormat, format.size); } else { - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; + continue; } - configs_.push_back(config); } } @@ -1171,10 +1178,10 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span> formats; - for (const SimpleCameraData::Configuration &cfg : data->configs_) { - for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); - } + for (const SimpleCameraData::Configuration &cfg : data->configs_) + if (!cfg.raw) + for (PixelFormat format : cfg.outputFormats) + formats[format].push_back(cfg.outputSizes); /* Sort the sizes and merge any consecutive overlapping ranges. */ for (auto &[format, sizes] : formats) { From patchwork Wed Mar 5 19:26:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22934 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 26474C0F1B for ; Wed, 5 Mar 2025 19:26:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0BEEE6884F; Wed, 5 Mar 2025 20:26:44 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="EfiGMSO8"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B5E26687F0 for ; Wed, 5 Mar 2025 20:26:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741202799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rrm9felxLn7sVT95Ltk4Vfnef0t/fVQ0OcXe6eQ8wAM=; b=EfiGMSO8/KINIlvVos5d7JQFwtM4fcy4SxcM7dEVvrtUAeusXGGLzeCibKY/djp1W+9Rh8 WkbMXDpAYQfrNuuBBhvmDW2tIV56yLD6UoNPcrNK2sEQSjHJBlCF9ItoHgavGb+nZ38TUW KDWmKJeLCsh/WNRAIJmVy3TYfsreTPY= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-581-5HJDmbHLNPy1BWB0Bo1x_w-1; Wed, 05 Mar 2025 14:26:36 -0500 X-MC-Unique: 5HJDmbHLNPy1BWB0Bo1x_w-1 X-Mimecast-MFC-AGG-ID: 5HJDmbHLNPy1BWB0Bo1x_w_1741202795 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 55F521956053; Wed, 5 Mar 2025 19:26:35 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.87]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D5752180049D; Wed, 5 Mar 2025 19:26:33 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v3 4/8] libcamera: simple: Identify requested stream roles Date: Wed, 5 Mar 2025 20:26:11 +0100 Message-ID: <20250305192617.18361-5-mzamazal@redhat.com> In-Reply-To: <20250305192617.18361-1-mzamazal@redhat.com> References: <20250305192617.18361-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: IZlbxNTElwki3l1kcm6qolY1gHqDjNjH1hl5pAtycIM_1741202795 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently, raw streams don't work in the simple pipeline and the requested stream roles are ignored. In order to support raw streams, we now track in SimpleCameraData whether raw and/or processed streams are requested. We also check that at most one raw stream is requested, there is no reason to have more. That information will be used in the followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index c805639c..0c6e0f44 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -298,6 +298,8 @@ public: }; std::queue conversionQueue_; bool useConversion_; + bool processedRequested_; + bool rawRequested_; std::unique_ptr converter_; std::unique_ptr swIsp_; @@ -1175,6 +1177,20 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, SpanprocessedRequested_ = false; + data->rawRequested_ = false; + for (auto &role : roles) + if (role == StreamRole::Raw) { + if (data->rawRequested_) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return nullptr; + } + data->rawRequested_ = true; + } else { + data->processedRequested_ = true; + } + /* Create the formats map. */ std::map> formats; From patchwork Wed Mar 5 19:26:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22936 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 205A5C32DE for ; Wed, 5 Mar 2025 19:26:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B0B7368889; Wed, 5 Mar 2025 20:26:47 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="eb2KJGJO"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 73708687DD for ; Wed, 5 Mar 2025 20:26:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741202801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3pShpBl3wCOhbeIIAAFixnlX6htwzs6sJYz+O1jphrs=; b=eb2KJGJOIyL9KEL4328w1SarQybWFp0RCctuya2iHag5lIVNGjI5TtE+24b7nAavjh0dkD jGD9plhYm6XEzgQNR1je2SEbbFpYvG+SM2y7ew76J6St0nQ7XaMG4rykndF5DdtD0DV1k3 k25cI153iNFAo+jxmYfZsbR3gV276PE= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-447-X3pJA1i1OFaOgAXY5FzYCg-1; Wed, 05 Mar 2025 14:26:38 -0500 X-MC-Unique: X3pJA1i1OFaOgAXY5FzYCg-1 X-Mimecast-MFC-AGG-ID: X3pJA1i1OFaOgAXY5FzYCg_1741202797 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 33A53180AB1C; Wed, 5 Mar 2025 19:26:37 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.87]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E14B9180049D; Wed, 5 Mar 2025 19:26:35 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v3 5/8] libcamera: simple: Consider raw output configurations Date: Wed, 5 Mar 2025 20:26:12 +0100 Message-ID: <20250305192617.18361-6-mzamazal@redhat.com> In-Reply-To: <20250305192617.18361-1-mzamazal@redhat.com> References: <20250305192617.18361-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: S1UMB0GpXyjjTa3NRhgylC8w_JwNmrKwPPzyFseVWqw_1741202797 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When generating simple pipeline configuration, raw output configurations are generated but later filtered out. Let's include processed and/or raw output configurations depending on the requested stream roles. Raw and processed formats are handled separately. The intention is that in case both raw and processed formats are requested then the raw formats should be left intact to the extent possible and not be influenced by the processed formats (implying that raw parameters compatible with the processed output requirements must be requested by the application). Raw output configurations are marked by setting their colorSpace to ColorSpace::Raw, which is the only place there suitable for the purpose. This is another preparatory patch without making raw outputs working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++-------- 1 file changed, 49 insertions(+), 23 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 0c6e0f44..92b9608b 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -389,6 +390,8 @@ private: const MediaPad *acquirePipeline(SimpleCameraData *data); void releasePipeline(SimpleCameraData *data); + void setUpFormatSizes(std::map> formats); + std::map entities_; MediaDevice *converter_; @@ -1191,15 +1194,56 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, SpanprocessedRequested_ = true; } - /* Create the formats map. */ - std::map> formats; + /* Create the formats maps. */ + std::map> processedFormats; + std::map> rawFormats; for (const SimpleCameraData::Configuration &cfg : data->configs_) - if (!cfg.raw) - for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); + for (PixelFormat format : cfg.outputFormats) + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes); + + if (data->processedRequested_ && processedFormats.empty()) { + LOG(SimplePipeline, Error) + << "Processed stream requsted but no corresponding output configuration found"; + return nullptr; + } + if (data->rawRequested_ && rawFormats.empty()) { + LOG(SimplePipeline, Error) + << "Raw stream requsted but no corresponding output configuration found"; + return nullptr; + } + + setUpFormatSizes(processedFormats); + setUpFormatSizes(rawFormats); + + /* + * Create the stream configurations. Take the first entry in the formats + * map as the default, for lack of a better option. + * + * \todo Implement a better way to pick the default format + */ + for (StreamRole role : roles) { + bool raw = (role == StreamRole::Raw); + auto formats = (raw ? rawFormats : processedFormats); + StreamConfiguration cfg{ StreamFormats{ formats } }; + cfg.pixelFormat = formats.begin()->first; + cfg.size = formats.begin()->second[0].max; + if (raw) + cfg.colorSpace = ColorSpace::Raw; + + config->addConfiguration(cfg); + } + + config->validate(); + + return config; +} +void SimplePipelineHandler::setUpFormatSizes( + std::map> formats) +{ /* Sort the sizes and merge any consecutive overlapping ranges. */ + for (auto &[format, sizes] : formats) { std::sort(sizes.begin(), sizes.end(), [](SizeRange &a, SizeRange &b) { @@ -1219,24 +1263,6 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Spanfirst; - cfg.size = formats.begin()->second[0].max; - - config->addConfiguration(cfg); - } - - config->validate(); - - return config; } int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) From patchwork Wed Mar 5 19:26:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22935 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D2A07C0F1B for ; Wed, 5 Mar 2025 19:26:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6F8416890C; Wed, 5 Mar 2025 20:26:46 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="R43E6kZj"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8BD1468889 for ; Wed, 5 Mar 2025 20:26:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741202801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9UX6UlSZXYfWGPrxA9N7nkutyQM+q7iohk7UROC+j7U=; b=R43E6kZjOrRJZKXz4q1T7+KCDP1eXXqQuRepRKKUSZdUKdhfUzZZn70PjGU3GQJuQxzJO6 fcuDL+fGDHCHYbe8ZOiaD3XTDvHJiyI7g4Yczs/CMq7lrUhub9afPaPt3p1f33R/LDC58X UjLHwg9y6fbzfCAQ/jRVMPAsLs0O20Q= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-630-ZKzcP9xFMBeNgQk8Fzv8pQ-1; Wed, 05 Mar 2025 14:26:40 -0500 X-MC-Unique: ZKzcP9xFMBeNgQk8Fzv8pQ-1 X-Mimecast-MFC-AGG-ID: ZKzcP9xFMBeNgQk8Fzv8pQ_1741202799 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4005B180899B; Wed, 5 Mar 2025 19:26:39 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.87]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BB03C180035E; Wed, 5 Mar 2025 19:26:37 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v3 6/8] libcamera: simple: Validate raw stream configurations Date: Wed, 5 Mar 2025 20:26:13 +0100 Message-ID: <20250305192617.18361-7-mzamazal@redhat.com> In-Reply-To: <20250305192617.18361-1-mzamazal@redhat.com> References: <20250305192617.18361-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 9d-y95TryXgBraD08ECa0wTncxkyLTHI0ZteKulg5gk_1741202799 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. If only a processed stream is requested, nothing changes. If only a raw stream is requested, the pixel format and output size may not be adjusted. The patch adds checks for this. If both processed and raw streams are requested, things get more complicated. The raw stream is expected to be passed through intact and all the adjustments are made for the processed streams. We select a pipe configuration for the processed streams. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. It is the application responsibility to select the right parameters for the raw stream. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 74 +++++++++++++++++++----- 1 file changed, 59 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 92b9608b..a387d34b 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -1055,6 +1056,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() pipeConfig_ = nullptr; for (const SimpleCameraData::Configuration *pipeConfig : *configs) { + if (data_->processedRequested_ && pipeConfig->raw) + continue; + const Size &size = pipeConfig->captureSize; if (size.width >= maxStreamSize.width && @@ -1096,21 +1100,43 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + /* + * If both processed and raw streams are requested, the pipe + * configuration is set up for the processed stream. The raw + * configuration needs to be compared against the capture format and + * size in such a case. + */ + const bool rawStream = cfg.colorSpace == ColorSpace::Raw; + const bool sideRawStream = rawStream && data_->processedRequested_; + /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); - - PixelFormat pixelFormat = *it; - if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) << "Adjusting pixel format"; - cfg.pixelFormat = pixelFormat; - status = Adjusted; + + if (!sideRawStream) { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + + PixelFormat pixelFormat = *it; + + if (cfg.pixelFormat != pixelFormat) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw pixel format " + << cfg.pixelFormat + << " doesn't match any of the pipe output formats"; + return Invalid; + } + LOG(SimplePipeline, Debug) + << "Adjusting pixel format from " << cfg.pixelFormat + << " to " << pixelFormat; + cfg.pixelFormat = pixelFormat; + status = Adjusted; + } } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!sideRawStream && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1118,8 +1144,17 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * not guaranteed to be a valid output size. In such cases, use * the smaller valid output size closest to the requested. */ - if (!pipeConfig_->outputSizes.contains(adjustedSize)) + if (!pipeConfig_->outputSizes.contains(adjustedSize)) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw output size " + << cfg.size + << " doesn't match any of the pipe output sizes: " + << pipeConfig_->outputSizes; + return Invalid; + } adjustedSize = adjustSize(cfg.size, pipeConfig_->outputSizes); + } LOG(SimplePipeline, Debug) << "Adjusting size from " << cfg.size << " to " << adjustedSize; @@ -1129,11 +1164,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* \todo Create a libcamera core class to group format and size */ if (cfg.pixelFormat != pipeConfig_->captureFormat || - cfg.size != pipeConfig_->captureSize) + cfg.size != pipeConfig_->captureSize) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw output format " << cfg.pixelFormat + << " and size " << cfg.size + << " not matching pipe format " << pipeConfig_->captureFormat + << " and size " << pipeConfig_->captureSize; + return Invalid; + } needConversion_ = true; + } /* Set the stride, frameSize and bufferCount. */ - if (needConversion_) { + if (needConversion_ && !rawStream) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat, From patchwork Wed Mar 5 19:26:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22938 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09F4AC32DE for ; Wed, 5 Mar 2025 19:26:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8A91068932; Wed, 5 Mar 2025 20:26:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="bVomtkqN"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9A3D1687DD for ; Wed, 5 Mar 2025 20:26:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741202808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+pN8McLpo/xrCArS8EZWpjXiRkYG9sGufHpLr7IF38s=; b=bVomtkqNqIObIOHHanq3YMKDJjw5dVdRDAEmo2F2LlRr+qk4x/KrWDTXqpwUHmDhtGZEv9 KuOTbJwe9oXozV8peITOWhyNjCFNDm5NXd1Vp4ly/MYTl+Vt0mYKsOQRfclzQf8rJJpptW UnWfsWT4KUQ/lEbs+KcVmw+/OIKcutI= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-647-5XO2Tre8PlW9RKfhgoidcQ-1; Wed, 05 Mar 2025 14:26:42 -0500 X-MC-Unique: 5XO2Tre8PlW9RKfhgoidcQ-1 X-Mimecast-MFC-AGG-ID: 5XO2Tre8PlW9RKfhgoidcQ_1741202801 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 42676180025E; Wed, 5 Mar 2025 19:26:41 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.87]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C6EED180049D; Wed, 5 Mar 2025 19:26:39 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v3 7/8] libcamera: simple: Don't enforce conversion with an added raw stream Date: Wed, 5 Mar 2025 20:26:14 +0100 Message-ID: <20250305192617.18361-8-mzamazal@redhat.com> In-Reply-To: <20250305192617.18361-1-mzamazal@redhat.com> References: <20250305192617.18361-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 9jsV8Q5TE8GwaSdcWN4bdX6P3d5D_94XYOyJC9pB_QM_1741202801 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, either alone or together with a processed stream, it can be produced without conversion. Let's amend the corresponding check on the number of configurations, so that the mere presence of a raw stream doesn't enforce conversion. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index a387d34b..5b68d2e0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1095,7 +1095,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * require any conversion, similar to raw capture use cases). This is * left as a future improvement. */ - needConversion_ = config_.size() > 1; + needConversion_ = config_.size() > 1 + (data_->rawRequested_ ? 1 : 0); for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; From patchwork Wed Mar 5 19:26:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22939 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 88EDCC0F1B for ; Wed, 5 Mar 2025 19:26:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 07A3E68889; Wed, 5 Mar 2025 20:26:54 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="G9/s1v/G"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F14C468920 for ; Wed, 5 Mar 2025 20:26:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741202808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HM68KFARCrM2vr8lHNnRbM6+SZstacHyV0HWzeI+xT0=; b=G9/s1v/GlN9T/0a1EYia/CJT2qdO0eZ0DN5q4MuB1/lP1Z44/CBlFOiLtubelleeQLX+bm PjXBv7iiYdNeOQjJccgDJokkFuVi5xQcuZD6CQPtOEjoArQlwtB49FqB7N8BDvvjUMWeTM xqM29UWuYIStR4d9QevPTKquadEXM8s= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-688-xyxaOlWbMhKEZfwB05vZVg-1; Wed, 05 Mar 2025 14:26:44 -0500 X-MC-Unique: xyxaOlWbMhKEZfwB05vZVg-1 X-Mimecast-MFC-AGG-ID: xyxaOlWbMhKEZfwB05vZVg_1741202803 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4E76619560B5; Wed, 5 Mar 2025 19:26:43 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.44.32.87]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D29B7180049D; Wed, 5 Mar 2025 19:26:41 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart Subject: [PATCH v3 8/8] libcamera: simple: Make raw streams working Date: Wed, 5 Mar 2025 20:26:15 +0100 Message-ID: <20250305192617.18361-9-mzamazal@redhat.com> In-Reply-To: <20250305192617.18361-1-mzamazal@redhat.com> References: <20250305192617.18361-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: bwq6rB_wmsi2AqOXCpi-4PlAr8IhKUz4jl3rC-CMZkQ_1741202803 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, whether alone or together with a processed stream, its buffers must be handled outside the software ISP machinery. They serve as output buffers, even when a processed stream is produced. At most one raw stream and at most one processed stream are supported and can be combined. An example of producing both raw and processed files using `cam' application: cam -c1 -C100 -Ffile# \ -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \ -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \ Note the difference in viewfinder and raw stream sizes due to the fact that debayering requires enlarging the image width, which enforces selecting a larger sensor resolution in this case. This is the final step to make raw streams working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 26 +++++++++++++++--------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 5b68d2e0..8eee9d8a 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -814,7 +814,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * buffer for capture (unless the stream is being stopped), and * complete the request with all the user-facing buffers. */ - if (buffer->metadata().status != FrameMetadata::FrameCancelled) + if (buffer->metadata().status != FrameMetadata::FrameCancelled && + !rawRequested_) video_->queueBuffer(buffer); if (conversionQueue_.empty()) @@ -860,13 +861,14 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) */ if (useConversion_) { if (conversionQueue_.empty()) { - video_->queueBuffer(buffer); + if (!rawRequested_) + video_->queueBuffer(buffer); return; } if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front().outputs); - else + else if (processedRequested_) /* * request->sequence() cannot be retrieved from `buffer' inside * queueBuffers because unique_ptr's make buffer->request() invalid @@ -876,6 +878,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) conversionQueue_.front().outputs); conversionQueue_.pop(); + if (rawRequested_) + pipe->completeBuffer(request, buffer); return; } @@ -895,7 +899,8 @@ void SimpleCameraData::clearIncompleteRequests() void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ - video_->queueBuffer(buffer); + if (!rawRequested_) + video_->queueBuffer(buffer); } void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) @@ -1413,7 +1418,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ - if (data->useConversion_) + if (data->useConversion_ && !data->rawRequested_) return data->converter_ ? data->converter_->exportBuffers(stream, count, buffers) : data->swIsp_->exportBuffers(stream, count, buffers); @@ -1435,7 +1440,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return -EBUSY; } - if (data->useConversion_) { + if (data->useConversion_ && !data->rawRequested_) { /* * When using the converter allocate a fixed number of internal * buffers. @@ -1443,7 +1448,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL ret = video->allocateBuffers(kNumInternalBuffers, &data->conversionBuffers_); } else { - /* Otherwise, prepare for using buffers from the only stream. */ + /* Otherwise, prepare for using buffers from the only or raw stream. */ Stream *stream = &data->streams_[0]; ret = video->importBuffers(stream->configuration().bufferCount); } @@ -1474,8 +1479,9 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } /* Queue all internal buffers for capture. */ - for (std::unique_ptr &buffer : data->conversionBuffers_) - video->queueBuffer(buffer.get()); + if (!data->rawRequested_) + for (std::unique_ptr &buffer : data->conversionBuffers_) + video->queueBuffer(buffer.get()); } return 0; @@ -1517,7 +1523,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * queue, it will be handed to the converter in the capture * completion handler. */ - if (data->useConversion_) { + if (data->useConversion_ && !isRawFormat(stream->configuration().pixelFormat)) { buffers.emplace(stream, buffer); } else { ret = data->video_->queueBuffer(buffer);