From patchwork Mon Apr 7 08:56:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23143 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E3422C327D for ; Mon, 7 Apr 2025 08:57:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 63DB4689AE; Mon, 7 Apr 2025 10:57:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="N8Amu/Ar"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 18E6B6899C for ; Mon, 7 Apr 2025 10:57:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016221; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZxhoIVFUnGagjaR0uKU2r7ivbSFW8Fp8ERXiPE6LF8o=; b=N8Amu/ArFIoJ3PPgzpVHwzgJB4yM+wPDfoOukabgP3q+FD6ibe4vv/d5CU+/cQNWM8z+am Sqth2jvxNGI6PwhvY0MtSPgGTSijZxiKrYxhgdA5VGqBIiA+eX5zuDs4U29hHb+zK5GJxq zu0yT9Irv1olFMq0sy82HPCVMuhkqP0= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-113-kM0q1YebMPGulu4-lgpcOw-1; Mon, 07 Apr 2025 04:56:59 -0400 X-MC-Unique: kM0q1YebMPGulu4-lgpcOw-1 X-Mimecast-MFC-AGG-ID: kM0q1YebMPGulu4-lgpcOw_1744016218 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2B2E4195608F; Mon, 7 Apr 2025 08:56:58 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 865A2180176A; Mon, 7 Apr 2025 08:56:56 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 01/11] libcamera: software_isp: Assign colour spaces in configurations Date: Mon, 7 Apr 2025 10:56:27 +0200 Message-ID: <20250407085639.16180-2-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: _TeX47QrGdnAs2_lRmklgbgKntNe8_QAkkFvycQrbsQ_1744016218 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" StreamConfiguration's should have colorSpace set. This is not the case in the simple pipeline. Let's set it there. This also fixes a crash in `cam' due to accessing an unset colorSpace. The colour space is assigned as follows: - If raw role is requested, then the colour space must be raw. - Otherwise, if software ISP is used, the default colour space is set to Srgb (YcbcrEncoding::None, Range::Full). - Otherwise, if a converter is used, the default colour space is left unset. - Then in configuration validation, if the pixel format is changed, the colour space is set according to the new pixel format. - If the colour space is still unset, it is set according to the resulting pixel format. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 25 +++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index efb07051..a008b13d 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -35,6 +36,7 @@ #include "libcamera/internal/converter.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/software_isp/software_isp.h" @@ -1206,8 +1208,24 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() if (cfg.pixelFormat != pixelFormat) { LOG(SimplePipeline, Debug) << "Adjusting pixel format"; cfg.pixelFormat = pixelFormat; + if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + cfg.colorSpace->adjust(pixelFormat); status = Adjusted; } + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } + cfg.colorSpace->adjust(pixelFormat); + } if (!pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; @@ -1314,11 +1332,16 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Spanfirst; cfg.size = formats.begin()->second[0].max; + if (role == StreamRole::Raw) + cfg.colorSpace = ColorSpace::Raw; + else if (data->swIsp_) + cfg.colorSpace = ColorSpace::Srgb; + config->addConfiguration(cfg); } From patchwork Mon Apr 7 08:56:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23144 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DE937C327D for ; Mon, 7 Apr 2025 08:57:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8B6196899C; Mon, 7 Apr 2025 10:57:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="eYpVSRzK"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9245C6899C for ; Mon, 7 Apr 2025 10:57:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016223; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1tU2HGv41f23gChmBxR+7odxjJJ+0pgAxlVL2oYPliM=; b=eYpVSRzKO0YYxFqqgoBMQ3shGO+KYbX0UMKshK5KcQkBDSx1JLDmVElY4jWX5oYM8DT7wP kMf/8IqC8TSXMZvkuJPxE96S/5FA89JwvunknIpyWSYVH6HsuDicdgzqStnfe6Zh4O/R1e wc9bGsz65uklo24AS4IWhsBWjSoosq4= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-440-z3NiAb1-MSWzJdvEFX3eVg-1; Mon, 07 Apr 2025 04:57:02 -0400 X-MC-Unique: z3NiAb1-MSWzJdvEFX3eVg-1 X-Mimecast-MFC-AGG-ID: z3NiAb1-MSWzJdvEFX3eVg_1744016221 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2CF9C195608F; Mon, 7 Apr 2025 08:57:01 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id AA39C180176A; Mon, 7 Apr 2025 08:56:58 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 02/11] libcamera: simple: Set the number of software ISP streams to 2 Date: Mon, 7 Apr 2025 10:56:28 +0200 Message-ID: <20250407085639.16180-3-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: fHQIW74oMqMDjgrm5RfmX-C5pPvRyqZWljoWL0dbVXc_1744016221 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When software ISP is enabled, we want to be able to provide a raw stream in addition to the processed stream. For this purpose, we need two streams. If only the processed stream is requested, it doesn't harm to allocate two. This is a hack for the lack of a better easy option. The number of streams is determined as a camera property in the pipeline matching. The actual number of streams needed (one or two) is determined only when examining roles in SimplePipelineHandler::generateConfiguration. In theory, software ISP could produce multiple processed streams but this is out of scope of this patch series. Hence two streams are sufficient at the moment. When software ISP is not enabled, the camera won't be able to produce multiple streams (assuming there's no hardware converter) and only single stream should be allocated as before. The simple pipeline handler assumes there's a linear pipeline from the camera sensor to a video capture device, and only supports a single stream. Branches in the hardware pipeline that would allow capturing multiple streams from the same camera sensor are not supported. We have no plan to change that, as a device that can produce multiple streams will likely be better supported by a dedicated pipeline handler. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index a008b13d..ab19149e 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1710,7 +1710,16 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) } } - swIspEnabled_ = info->swIspEnabled; + if (info->swIspEnabled) { + /* + * When the software ISP is enabled, the simple pipeline handler + * exposes the raw stream, giving a total of two streams. This + * is mutally exclusive with the presence of a converter. + */ + ASSERT(!converter_); + numStreams = 2; + swIspEnabled_ = true; + } /* Locate the sensors. */ std::vector sensors = locateSensors(media); From patchwork Mon Apr 7 08:56:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23145 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B29BEC327D for ; Mon, 7 Apr 2025 08:57:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 54E61689D6; Mon, 7 Apr 2025 10:57:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Dc1ZnmqI"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54C6A689AF for ; Mon, 7 Apr 2025 10:57:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016226; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M4x4PGSMBUnUj+MsYLutJaqwRZrCI/3+EtS+JkDC/UA=; b=Dc1ZnmqIwEIKIpbwsOLQm3rVh8HLfdWC81NqQtj0j4DKM5qhjX+MxfNu872389L0+WfYdo X/vgd2fVVOYbiyBhOhjHtqid+hNlej0eImUnf25vm5HVzV+dKDWTnLqCfnKh8Gn+hWIU3R qjdh2DQydqYlgZ9Vk7Lj57b7RXvWHEk= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-563-fsTtIr0hNz-2xxLmqZxzTg-1; Mon, 07 Apr 2025 04:57:04 -0400 X-MC-Unique: fsTtIr0hNz-2xxLmqZxzTg-1 X-Mimecast-MFC-AGG-ID: fsTtIr0hNz-2xxLmqZxzTg_1744016223 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9CBED180AF59; Mon, 7 Apr 2025 08:57:03 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BBDCA180176A; Mon, 7 Apr 2025 08:57:01 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 03/11] libcamera: formats: Add a helper to check for a raw pixel format Date: Mon, 7 Apr 2025 10:56:29 +0200 Message-ID: <20250407085639.16180-4-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: -7SqI_uamaSmSU_JP1-fWGpnJuhDRLeXOASC5UJWQLg_1744016223 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There are several places with the same pattern to check whether a given pixel format is a raw format: return libcamera::PixelFormatInfo::info(pixFmt).colourEncoding == libcamera::PixelFormatInfo::ColourEncodingRAW; Let's move the corresponding isFormatRaw helper from mali-c55.cpp to formats.cpp and use it where applicable. This also avoids a need to introduce the same helper (or the same pattern) in the followup patches. Signed-off-by: Milan Zamazal --- include/libcamera/internal/formats.h | 2 ++ src/libcamera/formats.cpp | 11 +++++++++++ src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 4 ++-- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 10 ---------- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 7 ++----- 5 files changed, 17 insertions(+), 17 deletions(-) diff --git a/include/libcamera/internal/formats.h b/include/libcamera/internal/formats.h index 6a3e9c16..bc4417d0 100644 --- a/include/libcamera/internal/formats.h +++ b/include/libcamera/internal/formats.h @@ -62,4 +62,6 @@ public: std::array planes; }; +bool isFormatRaw(const libcamera::PixelFormat &pixFmt); + } /* namespace libcamera */ diff --git a/src/libcamera/formats.cpp b/src/libcamera/formats.cpp index bfcdfc08..c6e0a262 100644 --- a/src/libcamera/formats.cpp +++ b/src/libcamera/formats.cpp @@ -1215,4 +1215,15 @@ unsigned int PixelFormatInfo::numPlanes() const return count; } +/** + * \brief Return whether the given pixel format is a raw format + * \param[in] pixFmt The pixel format to examine + * \return True iff the given format is a raw format + */ +bool isFormatRaw(const libcamera::PixelFormat &pixFmt) +{ + return libcamera::PixelFormatInfo::info(pixFmt).colourEncoding == + libcamera::PixelFormatInfo::ColourEncodingRAW; +} + } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp index 4e66b336..9ff11a41 100644 --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp @@ -24,6 +24,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/v4l2_subdevice.h" @@ -312,8 +313,7 @@ unsigned int ISICameraData::getYuvMediaBusFormat(const PixelFormat &pixelFormat) unsigned int ISICameraData::getMediaBusFormat(PixelFormat *pixelFormat) const { - if (PixelFormatInfo::info(*pixelFormat).colourEncoding == - PixelFormatInfo::ColourEncodingRAW) + if (isFormatRaw(*pixelFormat)) return getRawMediaBusFormat(pixelFormat); return getYuvMediaBusFormat(*pixelFormat); diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index a05e11fc..3721fb30 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -42,16 +42,6 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" -namespace { - -bool isFormatRaw(const libcamera::PixelFormat &pixFmt) -{ - return libcamera::PixelFormatInfo::info(pixFmt).colourEncoding == - libcamera::PixelFormatInfo::ColourEncodingRAW; -} - -} /* namespace */ - namespace libcamera { LOG_DEFINE_CATEGORY(MaliC55) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 52633fe3..a5b613bb 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -536,8 +536,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() */ if (config_.size() > 1) { for (const auto &cfg : config_) { - if (PixelFormatInfo::info(cfg.pixelFormat).colourEncoding == - PixelFormatInfo::ColourEncodingRAW) { + if (isFormatRaw(cfg.pixelFormat)) { config_.resize(1); status = Adjusted; break; @@ -551,9 +550,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() * Platforms with dewarper support, such as i.MX8MP, support * only a single stream. We can inspect config_[0] only here. */ - bool isRaw = PixelFormatInfo::info(config_[0].pixelFormat).colourEncoding == - PixelFormatInfo::ColourEncodingRAW; - if (!isRaw) + if (!isFormatRaw(config_[0].pixelFormat)) useDewarper = true; } From patchwork Mon Apr 7 08:56:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23147 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F0750C3284 for ; Mon, 7 Apr 2025 08:57:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8BA4468A54; Mon, 7 Apr 2025 10:57:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="HaXcGFX7"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 08534689B0 for ; Mon, 7 Apr 2025 10:57:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016230; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d64K+K3+8Rva9LCM+PVwdvUH7J/vc6/ZROsw7RbwsY4=; b=HaXcGFX7e4x6Yx4qi0ZN//TrNGlDV/pSvI9Ge+5SnHFmCOrZR6XVa4wXpbgW+ywxWqIV2A 3O2osJ4fvjCJw79xCueYToRNyWD+MiVTTPKJj2QNzHe1qGjnyA7UFGMOE5nLNEORgY7ZSa aLPX1ahKnNruQ8Yec2l8dFtKSXLfwHc= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-314-UCudccfRMCaoi7lw5775sg-1; Mon, 07 Apr 2025 04:57:07 -0400 X-MC-Unique: UCudccfRMCaoi7lw5775sg-1 X-Mimecast-MFC-AGG-ID: UCudccfRMCaoi7lw5775sg_1744016226 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 68966180AF57; Mon, 7 Apr 2025 08:57:06 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2E968180176A; Mon, 7 Apr 2025 08:57:03 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 04/11] libcamera: simple: Don't use raw output formats with conversions Date: Mon, 7 Apr 2025 10:56:30 +0200 Message-ID: <20250407085639.16180-5-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Mk_yD_9ctPGNLZDRzN28jDYj0PM8HagcQHyiOU9e43M_1744016226 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to support raw streams, we need to add raw formats to software ISP configurations. In this preparatory patch, the raw formats are excluded from output configurations for conversions. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index ab19149e..60b7e1b1 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -1411,7 +1412,7 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) cfg.setStream(&data->streams_[i]); - if (data->useConversion_) + if (data->useConversion_ && !isFormatRaw(cfg.pixelFormat)) outputCfgs.push_back(cfg); } From patchwork Mon Apr 7 08:56:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23146 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EF009C327D for ; Mon, 7 Apr 2025 08:57:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 771D4689F5; Mon, 7 Apr 2025 10:57:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="JtAG+T1U"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ECC5F689AF for ; Mon, 7 Apr 2025 10:57:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016231; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ScCUY5CxECfN4e/E9w0iTDuE0ehmVV2f3aFjLZsYxg0=; b=JtAG+T1UMAdQzvtr2wLuFhd7xQKzLy/GV+zM17MqbPhBw41R5k/5nxFaqYQ45UeMhs/kj/ mPOVAxrRyXRu61E5FzNYtwDUNGBE2iSY3IU/cnRX/04Z/BBPM5HQVvUbLKX8err8wL1gj7 WuAnBIlHBc95kXjeXmI4WHLwQVlDpDI= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-658-PstUlyaPN96RMEN18hDuNA-1; Mon, 07 Apr 2025 04:57:09 -0400 X-MC-Unique: PstUlyaPN96RMEN18hDuNA-1 X-Mimecast-MFC-AGG-ID: PstUlyaPN96RMEN18hDuNA_1744016228 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 85CE41944D17; Mon, 7 Apr 2025 08:57:08 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9ADF21809BE8; Mon, 7 Apr 2025 08:57:06 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 05/11] libcamera: simple: Add plain output configurations Date: Mon, 7 Apr 2025 10:56:31 +0200 Message-ID: <20250407085639.16180-6-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: piVMn5WhHJDXmBJV5XnglfAyud2gBzup9mTnC6Asyt0_1744016228 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Output configurations in simple pipeline are added depending on whether a converter, software ISP or none of them are used. If a converter or software ISP is used, no raw output configurations are added. In order to support raw output at least with software ISP, let's always add raw output configurations. A flag is added to SimpleCameraData::Configuration indicating whether it's for a raw or a converted output. We later filter formats and output sizes for particular stream configurations according to the new configuration flag. This is just preparation and raw output is still not supported. At the moment, we simply filter out raw configurations unconditionally to keep the current code working; this will be changed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 31 +++++++++++++++--------- 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 60b7e1b1..82d7a9a5 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -319,6 +319,7 @@ public: Size captureSize; std::vector outputFormats; SizeRange outputSizes; + bool raw; }; std::vector streams_; @@ -708,27 +709,33 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) continue; Configuration config; + + /* Raw configuration */ config.code = code; config.sensorSize = size; config.captureFormat = pixelFormat; config.captureSize = format.size; + config.outputFormats = { pixelFormat }; + config.outputSizes = config.captureSize; + config.raw = true; + configs_.push_back(config); + /* Modified, non-raw, configuration */ + config.raw = false; if (converter_) { config.outputFormats = converter_->formats(pixelFormat); config.outputSizes = converter_->sizes(format.size); } else if (swIsp_) { - config.outputFormats = swIsp_->formats(pixelFormat); - config.outputSizes = swIsp_->sizes(pixelFormat, format.size); - if (config.outputFormats.empty()) { + std::vector outputFormats = swIsp_->formats(pixelFormat); + if (outputFormats.empty()) { /* Do not use swIsp for unsupported pixelFormat's. */ - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; + continue; } + config.outputFormats = outputFormats; + config.outputSizes = swIsp_->sizes(pixelFormat, format.size); } else { - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; + continue; } - configs_.push_back(config); } } @@ -1301,10 +1308,10 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span> formats; - for (const SimpleCameraData::Configuration &cfg : data->configs_) { - for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); - } + for (const SimpleCameraData::Configuration &cfg : data->configs_) + if (!cfg.raw) + for (PixelFormat format : cfg.outputFormats) + formats[format].push_back(cfg.outputSizes); /* Sort the sizes and merge any consecutive overlapping ranges. */ for (auto &[format, sizes] : formats) { From patchwork Mon Apr 7 08:56:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23148 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ECA23C327D for ; Mon, 7 Apr 2025 08:57:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 939C968A38; Mon, 7 Apr 2025 10:57:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="U2zAbQig"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E6F0F689B0 for ; Mon, 7 Apr 2025 10:57:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016236; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wG/ccQkA3BDA3J/cs8wWQgGndHKdAZFPifITzkKoG1w=; b=U2zAbQigudw+s+zOAlXdFUpHj3U4oUEXc3Gr8ZD8pfqgAE8900gwbQhZZ+3bBHaSMj6IwN 6a+mRSBAAtwUKoryWlJP9a3BsLbQdxFkiT7o/Z8C/++C7qzzz6unqQLMJ6JmfMgCTcpIxh dPjZrL0vHnCpb09bpaNoBEWy7kRcwUA= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-611-uQ9ZdXi0O2uwYvtbKQyZBw-1; Mon, 07 Apr 2025 04:57:12 -0400 X-MC-Unique: uQ9ZdXi0O2uwYvtbKQyZBw-1 X-Mimecast-MFC-AGG-ID: uQ9ZdXi0O2uwYvtbKQyZBw_1744016232 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C513D18001E3; Mon, 7 Apr 2025 08:57:10 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 07CEF1809BED; Mon, 7 Apr 2025 08:57:08 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 06/11] libcamera: simple: Identify requested stream roles Date: Mon, 7 Apr 2025 10:56:32 +0200 Message-ID: <20250407085639.16180-7-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: H9JE4xlwPcNGS_pa0p1gKNagzq1luDXGjQ6cDDrX3xw_1744016232 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently, raw streams don't work in the simple pipeline and the requested stream roles are ignored. In order to support raw streams, we now track in SimpleCameraData whether raw and/or processed streams are requested. We also check that at most one raw stream is requested, there is no reason to have more. That information will be used in the followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 82d7a9a5..bf9d75f4 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -345,6 +345,8 @@ public: }; std::queue conversionQueue_; bool useConversion_; + bool processedRequested_; + bool rawRequested_; std::unique_ptr converter_; std::unique_ptr swIsp_; @@ -1305,6 +1307,20 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, SpanprocessedRequested_ = false; + data->rawRequested_ = false; + for (auto &role : roles) + if (role == StreamRole::Raw) { + if (data->rawRequested_) { + LOG(SimplePipeline, Error) + << "Can't capture multiple raw streams"; + return nullptr; + } + data->rawRequested_ = true; + } else { + data->processedRequested_ = true; + } + /* Create the formats map. */ std::map> formats; From patchwork Mon Apr 7 08:56:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23149 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F13A8C327D for ; Mon, 7 Apr 2025 08:57:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 40A1F68AA2; Mon, 7 Apr 2025 10:57:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="iMheN1d3"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8A65168A38 for ; Mon, 7 Apr 2025 10:57:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016238; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sBiwVCtgCTIevGTo3qi7ilO14tzDFZqNESeGhEi7BLc=; b=iMheN1d3RqO2BMKd18h+QdETkGip7MTHZTbfY41UMR1+tYKgH4jpm+0COs/gkf2i/ClJjQ HORs+X6s3gaV5PKW086L5ib/Mz7Gltyjtm2kQI1OKWIK65RVp7ThKReqqcPHF0OMSQ9tGM +wEqY68qoRLIFUGuK2x3iCYvpUKVYNU= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-35-z_xgi4BtNuitPtN6YCjN3g-1; Mon, 07 Apr 2025 04:57:14 -0400 X-MC-Unique: z_xgi4BtNuitPtN6YCjN3g-1 X-Mimecast-MFC-AGG-ID: z_xgi4BtNuitPtN6YCjN3g_1744016233 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 160321956080; Mon, 7 Apr 2025 08:57:13 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3F40D180176A; Mon, 7 Apr 2025 08:57:10 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 07/11] libcamera: simple: Consider raw output configurations Date: Mon, 7 Apr 2025 10:56:33 +0200 Message-ID: <20250407085639.16180-8-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: D0hXxZdBrsLc7LZKlfHP2luaJDh_TotzVpoZTG7QmaQ_1744016233 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When generating simple pipeline configuration, raw output configurations are generated but later filtered out. Let's include processed and/or raw output configurations depending on the requested stream roles. Raw and processed formats are handled separately. The intention is that in case both raw and processed formats are requested then the raw formats should be left intact to the extent possible and not be influenced by the processed formats (implying that raw parameters compatible with the processed output requirements must be requested by the application). Raw output configurations are identified by colorSpace being set to ColorSpace::Raw. This is another preparatory patch without making raw outputs working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 73 ++++++++++++++++-------- 1 file changed, 48 insertions(+), 25 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index bf9d75f4..6b65d79f 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -439,6 +439,8 @@ private: const MediaPad *acquirePipeline(SimpleCameraData *data); void releasePipeline(SimpleCameraData *data); + void setUpFormatSizes(std::map> &formats); + std::map entities_; MediaDevice *converter_; @@ -1321,35 +1323,28 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, SpanprocessedRequested_ = true; } - /* Create the formats map. */ - std::map> formats; + /* Create the formats maps. */ + std::map> processedFormats; + std::map> rawFormats; for (const SimpleCameraData::Configuration &cfg : data->configs_) - if (!cfg.raw) - for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); + for (PixelFormat format : cfg.outputFormats) + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes); - /* Sort the sizes and merge any consecutive overlapping ranges. */ - for (auto &[format, sizes] : formats) { - std::sort(sizes.begin(), sizes.end(), - [](SizeRange &a, SizeRange &b) { - return a.min < b.min; - }); - - auto cur = sizes.begin(); - auto next = cur; - - while (++next != sizes.end()) { - if (cur->max.width >= next->min.width && - cur->max.height >= next->min.height) - cur->max = next->max; - else if (++cur != next) - *cur = *next; - } - - sizes.erase(++cur, sizes.end()); + if (data->processedRequested_ && processedFormats.empty()) { + LOG(SimplePipeline, Error) + << "Processed stream requsted but no corresponding output configuration found"; + return nullptr; + } + if (data->rawRequested_ && rawFormats.empty()) { + LOG(SimplePipeline, Error) + << "Raw stream requsted but no corresponding output configuration found"; + return nullptr; } + setUpFormatSizes(processedFormats); + setUpFormatSizes(rawFormats); + /* * Create the stream configurations. Take the first entry in the formats * map as the default, for lack of a better option. @@ -1357,11 +1352,13 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Spanfirst; cfg.size = formats.begin()->second[0].max; - if (role == StreamRole::Raw) + if (raw) cfg.colorSpace = ColorSpace::Raw; else if (data->swIsp_) cfg.colorSpace = ColorSpace::Srgb; @@ -1374,6 +1371,32 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span> &formats) +{ + /* Sort the sizes and merge any consecutive overlapping ranges. */ + + for (auto &[format, sizes] : formats) { + std::sort(sizes.begin(), sizes.end(), + [](SizeRange &a, SizeRange &b) { + return a.min < b.min; + }); + + auto cur = sizes.begin(); + auto next = cur; + + while (++next != sizes.end()) { + if (cur->max.width >= next->min.width && + cur->max.height >= next->min.height) + cur->max = next->max; + else if (++cur != next) + *cur = *next; + } + + sizes.erase(++cur, sizes.end()); + } +} + int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) { SimpleCameraConfiguration *config = From patchwork Mon Apr 7 08:56:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23150 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A956BC327D for ; Mon, 7 Apr 2025 08:57:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1148368AA8; Mon, 7 Apr 2025 10:57:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="IXh4RNBF"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B3947689B0 for ; Mon, 7 Apr 2025 10:57:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016239; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iohyZK7bFcUjLR/ILg8wI58ibo2i+lJCI2jpw05JGBg=; b=IXh4RNBFjMr/CQFjRcxxVdyS4Jnvx/Dzv6h+f3KSyGsFx5f0emWsFy7OxEP82xtOdXhoEN M/aaApmU5AEplrJUoPR/FlYZmC/mC9w5BX9Z2eYrwL0Yldgc8QxNP3b9fqAVlZzesnh+Ip wswvCcDC/ObOZCTAf34wjEYU7M6+GtA= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-120-dt559nngPquZ0fnj1N0Urw-1; Mon, 07 Apr 2025 04:57:16 -0400 X-MC-Unique: dt559nngPquZ0fnj1N0Urw-1 X-Mimecast-MFC-AGG-ID: dt559nngPquZ0fnj1N0Urw_1744016235 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4BD901801A1A; Mon, 7 Apr 2025 08:57:15 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9E3C71801A6D; Mon, 7 Apr 2025 08:57:13 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 08/11] libcamera: simple: Validate raw stream configurations Date: Mon, 7 Apr 2025 10:56:34 +0200 Message-ID: <20250407085639.16180-9-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: _seDtr4c4zPkiS1s7CHb4lhpZ40PWPpaYhMg0D6uC1o_1744016235 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SimpleCameraConfiguration::validate() looks for the best configuration. As part of enabling raw stream support, the method must consider raw streams in addition to the processed streams. If only a processed stream is requested, nothing changes. If only a raw stream is requested, the pixel format and output size may not be adjusted. The patch adds checks for this. If both processed and raw streams are requested, things get more complicated. The raw stream is expected to be passed through intact and all the adjustments are made for the processed streams. We select a pipe configuration for the processed streams. Note that with both processed and raw streams, the requested sizes must be mutually matching, including resizing due to debayer requirements. For example, the following `cam' setup is valid for imx219 cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=3280,height=2464 rather than cam -s role=viewfinder,width=1920,height=1080 \ -s role=raw,width=1920,height=1080 due to the resolution of 1924x1080 actually selected for debayering to 1920x1080. It is the application responsibility to select the right parameters for the raw stream. Setting up the right configurations is still not enough to make the raw streams working. Buffer handling must be changed in the simple pipeline, which is addressed in followup patches. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 103 ++++++++++++++++------- 1 file changed, 74 insertions(+), 29 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 6b65d79f..313c454c 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -1168,6 +1169,9 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() pipeConfig_ = nullptr; for (const SimpleCameraData::Configuration *pipeConfig : *configs) { + if (data_->processedRequested_ && pipeConfig->raw) + continue; + const Size &size = pipeConfig->captureSize; if (size.width >= maxStreamSize.width && @@ -1209,37 +1213,60 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; + /* + * If both processed and raw streams are requested, the pipe + * configuration is set up for the processed stream. The raw + * configuration needs to be compared against the capture format and + * size in such a case. + */ + const bool rawStream = cfg.colorSpace == ColorSpace::Raw; + const bool sideRawStream = rawStream && data_->processedRequested_; + /* Adjust the pixel format and size. */ - auto it = std::find(pipeConfig_->outputFormats.begin(), - pipeConfig_->outputFormats.end(), - cfg.pixelFormat); - if (it == pipeConfig_->outputFormats.end()) - it = pipeConfig_->outputFormats.begin(); - - PixelFormat pixelFormat = *it; - if (cfg.pixelFormat != pixelFormat) { - LOG(SimplePipeline, Debug) << "Adjusting pixel format"; - cfg.pixelFormat = pixelFormat; - if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + + if (!sideRawStream) { + auto it = std::find(pipeConfig_->outputFormats.begin(), + pipeConfig_->outputFormats.end(), + cfg.pixelFormat); + if (it == pipeConfig_->outputFormats.end()) + it = pipeConfig_->outputFormats.begin(); + + PixelFormat pixelFormat = *it; + + if (cfg.pixelFormat != pixelFormat) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw pixel format " + << cfg.pixelFormat + << " doesn't match any of the pipe output formats"; + return Invalid; + } + LOG(SimplePipeline, Debug) + << "Adjusting pixel format from " << cfg.pixelFormat + << " to " << pixelFormat; + cfg.pixelFormat = pixelFormat; + if (cfg.colorSpace && cfg.colorSpace != ColorSpace::Raw) + cfg.colorSpace->adjust(pixelFormat); + status = Adjusted; + } + + if (!cfg.colorSpace) { + const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + cfg.colorSpace = ColorSpace::Srgb; + break; + case libcamera::PixelFormatInfo::ColourEncodingYUV: + cfg.colorSpace = ColorSpace::Sycc; + break; + default: + cfg.colorSpace = ColorSpace::Raw; + } cfg.colorSpace->adjust(pixelFormat); - status = Adjusted; - } - if (!cfg.colorSpace) { - const PixelFormatInfo &info = PixelFormatInfo::info(pixelFormat); - switch (info.colourEncoding) { - case PixelFormatInfo::ColourEncodingRGB: - cfg.colorSpace = ColorSpace::Srgb; - break; - case libcamera::PixelFormatInfo::ColourEncodingYUV: - cfg.colorSpace = ColorSpace::Sycc; - break; - default: - cfg.colorSpace = ColorSpace::Raw; } - cfg.colorSpace->adjust(pixelFormat); } - if (!pipeConfig_->outputSizes.contains(cfg.size)) { + if (!sideRawStream && !pipeConfig_->outputSizes.contains(cfg.size)) { Size adjustedSize = pipeConfig_->captureSize; /* * The converter (when present) may not be able to output @@ -1247,8 +1274,17 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * not guaranteed to be a valid output size. In such cases, use * the smaller valid output size closest to the requested. */ - if (!pipeConfig_->outputSizes.contains(adjustedSize)) + if (!pipeConfig_->outputSizes.contains(adjustedSize)) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw output size " + << cfg.size + << " doesn't match any of the pipe output sizes: " + << pipeConfig_->outputSizes; + return Invalid; + } adjustedSize = adjustSize(cfg.size, pipeConfig_->outputSizes); + } LOG(SimplePipeline, Debug) << "Adjusting size from " << cfg.size << " to " << adjustedSize; @@ -1258,11 +1294,20 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() /* \todo Create a libcamera core class to group format and size */ if (cfg.pixelFormat != pipeConfig_->captureFormat || - cfg.size != pipeConfig_->captureSize) + cfg.size != pipeConfig_->captureSize) { + if (rawStream) { + LOG(SimplePipeline, Info) + << "Raw output format " << cfg.pixelFormat + << " and size " << cfg.size + << " not matching pipe format " << pipeConfig_->captureFormat + << " and size " << pipeConfig_->captureSize; + return Invalid; + } needConversion_ = true; + } /* Set the stride, frameSize and bufferCount. */ - if (needConversion_) { + if (needConversion_ && !rawStream) { std::tie(cfg.stride, cfg.frameSize) = data_->converter_ ? data_->converter_->strideAndFrameSize(cfg.pixelFormat, From patchwork Mon Apr 7 08:56:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23151 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C475DC327D for ; Mon, 7 Apr 2025 08:57:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6119C68AA9; Mon, 7 Apr 2025 10:57:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="dVPIH326"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3E0AC689F5 for ; Mon, 7 Apr 2025 10:57:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016242; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yY42ccmKJug69GPMTGjCWBTt8f3GLCD1Oq9x8zvN8Ps=; b=dVPIH326Zm/McMNpQFseAXd+JPoee9712XJvJquv8xjreTQFPyBYTp4w78Pqai2RScnD6e 23yJ3DJWXuRi+wm81GbzybOgf+EQ7HdfeeG/XF503gMylrhrK5PUwJFtd6ajEB1hGwpOrW FGTZ5vFSSco+rBjdWTVVtJKycC6qH50= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-540-Idv8GO0-NnqKcYxX760W8A-1; Mon, 07 Apr 2025 04:57:18 -0400 X-MC-Unique: Idv8GO0-NnqKcYxX760W8A-1 X-Mimecast-MFC-AGG-ID: Idv8GO0-NnqKcYxX760W8A_1744016237 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BC49619560A1; Mon, 7 Apr 2025 08:57:17 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DB724180176A; Mon, 7 Apr 2025 08:57:15 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 09/11] libcamera: simple: Don't enforce conversion with an added raw stream Date: Mon, 7 Apr 2025 10:56:35 +0200 Message-ID: <20250407085639.16180-10-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 2ZgLXTM6t721WXzxxlccSoe-8DtCsueRok8qft_j09Q_1744016237 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, either alone or together with a processed stream, it can be produced without conversion. Let's amend the corresponding check on the number of configurations, so that the mere presence of a raw stream doesn't enforce conversion. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 313c454c..2932fb45 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1208,7 +1208,7 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * require any conversion, similar to raw capture use cases). This is * left as a future improvement. */ - needConversion_ = config_.size() > 1; + needConversion_ = config_.size() > 1 + (data_->rawRequested_ ? 1 : 0); for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; From patchwork Mon Apr 7 08:56:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23152 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BCEA9C327D for ; Mon, 7 Apr 2025 08:57:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 747C268A9F; Mon, 7 Apr 2025 10:57:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="f2fbHZ0w"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6421868A99 for ; Mon, 7 Apr 2025 10:57:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016244; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fCVfrwjZK03ZhYsAj2Ty034zDPh4Z3FThXQlUTfE8zs=; b=f2fbHZ0wFubfv+ry1yvyw9wJ7bQb8WnghzmpnnvhZ5w7dl0Hy7uly6VYtI/4ywe8EkdJIl oNjL1VA3mBQMJYda0l6f075H3LpkdnuVnWeNYKNLGMmAVed9yEZhNs2hKN+R4fEWFJ8BVK ELzE0x5gW4IUgAdFSmNJU2hH/pvj4oQ= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-586-a3abDm-FO9WXl-3lnIFbgg-1; Mon, 07 Apr 2025 04:57:21 -0400 X-MC-Unique: a3abDm-FO9WXl-3lnIFbgg-1 X-Mimecast-MFC-AGG-ID: a3abDm-FO9WXl-3lnIFbgg_1744016240 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 061521956087; Mon, 7 Apr 2025 08:57:20 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5FF5A1801A6D; Mon, 7 Apr 2025 08:57:18 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 10/11] libcamera: simple: Require metadata only when software ISP is used Date: Mon, 7 Apr 2025 10:56:36 +0200 Message-ID: <20250407085639.16180-11-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 5z0vN0EtlKQct_eu_43DFWNU_7BOMoSIFSjR9zpXh6k_1744016240 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If software ISP is enabled then metadata is required in the simple pipeline. But this doesn't apply if the software ISP is not actually used, for example when only a raw stream is produced. Then the pipeline waits for metadata that never comes. This patch fixes the problem by requiring metadata only when software ISP is used. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 2932fb45..ff41e772 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -1652,6 +1652,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) int ret; std::map buffers; + bool metadataRequired = false; for (auto &[stream, buffer] : request->buffers()) { /* @@ -1661,6 +1662,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) */ if (data->useConversion_) { buffers.emplace(stream, buffer); + metadataRequired = !!data->swIsp_; } else { ret = data->video_->queueBuffer(buffer); if (ret < 0) @@ -1668,7 +1670,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) } } - data->frameInfo_.create(request, !!data->swIsp_); + data->frameInfo_.create(request, metadataRequired); if (data->useConversion_) { data->conversionQueue_.push({ request, std::move(buffers) }); if (data->swIsp_) From patchwork Mon Apr 7 08:56:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 23153 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9369AC3284 for ; Mon, 7 Apr 2025 08:57:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 28A8E68AA4; Mon, 7 Apr 2025 10:57:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="i8bUL1Ut"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4FF1868A9F for ; Mon, 7 Apr 2025 10:57:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1744016246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RJUIFwS5JPKBaosXNE667ChyEQN7CBHNwisRQ/rW6Ls=; b=i8bUL1UtdKF1GQu34QYLDPlbHnoxfYTqQlalJG+WOPeeYUMIqUxvhSWyWzwUfH4ChFRvH6 WZJgIuEzr0vTcyyJDByqte2w43XRWqKGxeFenC6bCmVBskvVbLKFZiJUkmRZniGNH7vBUP m9fNCS4TnfUO12iMSetyPThNVZot0Hg= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-614-2TVwX-DVMv6eWw752O-3bg-1; Mon, 07 Apr 2025 04:57:23 -0400 X-MC-Unique: 2TVwX-DVMv6eWw752O-3bg-1 X-Mimecast-MFC-AGG-ID: 2TVwX-DVMv6eWw752O-3bg_1744016242 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3A9871954B37; Mon, 7 Apr 2025 08:57:22 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.tpbc.com (unknown [10.45.224.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 951F0180176A; Mon, 7 Apr 2025 08:57:20 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Laurent Pinchart , Kieran Bingham Subject: [PATCH v4 11/11] libcamera: simple: Make raw streams working Date: Mon, 7 Apr 2025 10:56:37 +0200 Message-ID: <20250407085639.16180-12-mzamazal@redhat.com> In-Reply-To: <20250407085639.16180-1-mzamazal@redhat.com> References: <20250407085639.16180-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: -CE6amUlKtB3lKr6KT3MuNwxXPzFtpSscGkiy7EL8Zs_1744016242 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a raw stream is requested, whether alone or together with a processed stream, its buffers must be handled outside the software ISP machinery. They serve as output buffers, even when a processed stream is produced. At most one raw stream and at most one processed stream are supported and can be combined. An example of producing both raw and processed files using `cam' application: cam -c1 -C100 -Ffile# \ -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \ -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \ Note the difference in viewfinder and raw stream sizes due to the fact that debayering requires enlarging the image width, which enforces selecting a larger sensor resolution in this case. This is the final step to make raw streams working. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 31 +++++++++++++++--------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index ff41e772..260d2d6c 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -872,10 +872,13 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * point converting an erroneous buffer. */ if (buffer->metadata().status != FrameMetadata::FrameSuccess) { - if (!useConversion_) { + if (rawRequested_) { /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); + SimpleFrameInfo *info = frameInfo_.find(request->sequence()); + if (info) + info->metadataRequired = false; tryCompleteRequest(request); return; } @@ -885,7 +888,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * buffer for capture (unless the stream is being stopped), and * complete the request with all the user-facing buffers. */ - if (buffer->metadata().status != FrameMetadata::FrameCancelled) + if (buffer->metadata().status != FrameMetadata::FrameCancelled && + !rawRequested_) video_->queueBuffer(buffer); if (conversionQueue_.empty()) @@ -934,13 +938,14 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) */ if (useConversion_) { if (conversionQueue_.empty()) { - video_->queueBuffer(buffer); + if (!rawRequested_) + video_->queueBuffer(buffer); return; } if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front().outputs); - else + else if (processedRequested_) /* * request->sequence() cannot be retrieved from `buffer' inside * queueBuffers because unique_ptr's make buffer->request() invalid @@ -950,6 +955,8 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) conversionQueue_.front().outputs); conversionQueue_.pop(); + if (rawRequested_) + pipe->completeBuffer(request, buffer); return; } @@ -987,7 +994,8 @@ void SimpleCameraData::tryCompleteRequest(Request *request) void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { /* Queue the input buffer back for capture. */ - video_->queueBuffer(buffer); + if (!rawRequested_) + video_->queueBuffer(buffer); } void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) @@ -1535,7 +1543,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ - if (data->useConversion_) + if (data->useConversion_ && !data->rawRequested_) return data->converter_ ? data->converter_->exportBuffers(stream, count, buffers) : data->swIsp_->exportBuffers(stream, count, buffers); @@ -1558,7 +1566,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL return -EBUSY; } - if (data->useConversion_) { + if (data->useConversion_ && !data->rawRequested_) { /* * When using the converter allocate a fixed number of internal * buffers. @@ -1566,7 +1574,7 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL ret = video->allocateBuffers(kNumInternalBuffers, &data->conversionBuffers_); } else { - /* Otherwise, prepare for using buffers from the only stream. */ + /* Otherwise, prepare for using buffers from the only or raw stream. */ Stream *stream = &data->streams_[0]; ret = video->importBuffers(stream->configuration().bufferCount); } @@ -1608,8 +1616,9 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL } /* Queue all internal buffers for capture. */ - for (std::unique_ptr &buffer : data->conversionBuffers_) - video->queueBuffer(buffer.get()); + if (!data->rawRequested_) + for (std::unique_ptr &buffer : data->conversionBuffers_) + video->queueBuffer(buffer.get()); } return 0; @@ -1660,7 +1669,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * queue, it will be handed to the converter in the capture * completion handler. */ - if (data->useConversion_) { + if (data->useConversion_ && !isFormatRaw(stream->configuration().pixelFormat)) { buffers.emplace(stream, buffer); metadataRequired = !!data->swIsp_; } else {