From patchwork Fri Nov 22 20:13:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 22058 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4A2E7C3301 for ; Fri, 22 Nov 2024 20:13:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4AB9365FE3; Fri, 22 Nov 2024 21:13:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="TE1gx9e8"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 77A7B65F51 for ; Fri, 22 Nov 2024 21:13:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732306397; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=osah4s1ASiDIZtvXJGEHzca7u6OxP5pS/RkoFMRcUwI=; b=TE1gx9e8xR2/95NxfhUO12XUw1ln571aJQLg6O77ptXlTibc3gu3hE7BS6Zq4Q01OIf8/g b1FfaWT1ZDQibnNnvyp38GiYs5jckUo3mhIWNOff196WZ1PSmo0OKgCbzqyU0fe9ePck91 ouMF4FxhDOqaNZFKQGDrA9vCCZ399qU= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-455-Lrxd60ntO8G7WNz21XXy2w-1; Fri, 22 Nov 2024 15:13:16 -0500 X-MC-Unique: Lrxd60ntO8G7WNz21XXy2w-1 X-Mimecast-MFC-AGG-ID: Lrxd60ntO8G7WNz21XXy2w Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1E5B019560AF; Fri, 22 Nov 2024 20:13:15 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.2]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 90A121955F43; Fri, 22 Nov 2024 20:13:13 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , bryan.odonoghue@linaro.org Subject: [RFC PATCH 1/1] libcamera: simple: Fix raw output Date: Fri, 22 Nov 2024 21:13:04 +0100 Message-ID: <20241122201305.1668098-2-mzamazal@redhat.com> In-Reply-To: <20241122201305.1668098-1-mzamazal@redhat.com> References: <20241122201305.1668098-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: cGyjKETU95VWCvTLPHxJLP4Iu2seLtQ3n6nMyW6tW-8_1732306395 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Simple pipeline was used to introduce software ISP. If software ISP is enabled, the pipeline always debayers the input and doesn't produce raw output anymore, even when requested. This patch fixes the problem and allows producing raw output if it is requested, by specifying raw stream role. When raw output is requested, only single stream (raw or processed) is allowed. This is not necessarily an inherent property, it just simplifies the implementation and is sufficient for the current needs. If raw stream is requested, it means raw as it is and no pixel format or size conversions are permitted. This is not only logical but also simplifying the changes because we don't have to check for raw when conversion is used. The patch must deal with the fact that software ISP and its output configurations are arranged before we know whether a raw or a processed stream will be requested. This means that if software ISP is enabled then it is always initialized and we store raw output configurations together with software ISP ones. We filter output configurations later as needed. The raw output may not be useful without exposure and gain adjustments, it can be all black (or just noisy). This will be handled in a separate patch introducing manual gain and exposure controls. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 66 +++++++++++++++++++----- 1 file changed, 54 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 41fdf84cc..ec7cd7123 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -265,6 +265,7 @@ public: Size captureSize; std::vector outputFormats; SizeRange outputSizes; + bool swisp; }; std::vector streams_; @@ -289,6 +290,7 @@ public: }; std::queue conversionQueue_; bool useConversion_; + bool isRaw_; std::unique_ptr converter_; std::unique_ptr swIsp_; @@ -646,19 +648,24 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) config.sensorSize = size; config.captureFormat = pixelFormat; config.captureSize = format.size; + config.swisp = false; if (converter_) { config.outputFormats = converter_->formats(pixelFormat); config.outputSizes = converter_->sizes(format.size); - } else if (swIsp_) { - config.outputFormats = swIsp_->formats(pixelFormat); - config.outputSizes = swIsp_->sizes(pixelFormat, format.size); - if (config.outputFormats.empty()) { - /* Do not use swIsp for unsupported pixelFormat's. */ - config.outputFormats = { pixelFormat }; - config.outputSizes = config.captureSize; - } } else { + if (swIsp_) { + Configuration swispConfig = config; + swispConfig.outputFormats = swIsp_->formats(pixelFormat); + swispConfig.outputSizes = swIsp_->sizes(pixelFormat, format.size); + if (swispConfig.outputFormats.empty()) { + /* Do not use swIsp for unsupported pixelFormat's. */ + swispConfig.outputFormats = { pixelFormat }; + swispConfig.outputSizes = swispConfig.captureSize; + } + swispConfig.swisp = true; + configs_.push_back(swispConfig); + } config.outputFormats = { pixelFormat }; config.outputSizes = config.captureSize; } @@ -859,7 +866,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front().outputs); - else + else if (!isRaw_) { /* * request->sequence() cannot be retrieved from `buffer' inside * queueBuffers because unique_ptr's make buffer->request() invalid @@ -867,6 +874,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) */ swIsp_->queueBuffers(request->sequence(), buffer, conversionQueue_.front().outputs); + } conversionQueue_.pop(); return; @@ -1049,6 +1057,8 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() pipeConfig_ = nullptr; for (const SimpleCameraData::Configuration *pipeConfig : *configs) { + if (pipeConfig->swisp == data_->isRaw_) + continue; const Size &size = pipeConfig->captureSize; if (size.width >= maxStreamSize.width && @@ -1099,6 +1109,13 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() PixelFormat pixelFormat = *it; if (cfg.pixelFormat != pixelFormat) { + if (data_->isRaw_) { + LOG(SimplePipeline, Error) + << "Cannot convert pixel format with raw output (from " + << cfg.pixelFormat << " to " + << pixelFormat << ")"; + return Invalid; + } LOG(SimplePipeline, Debug) << "Adjusting pixel format"; cfg.pixelFormat = pixelFormat; status = Adjusted; @@ -1112,8 +1129,16 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * not guaranteed to be a valid output size. In such cases, use * the smaller valid output size closest to the requested. */ - if (!pipeConfig_->outputSizes.contains(adjustedSize)) + if (!pipeConfig_->outputSizes.contains(adjustedSize)) { + if (data_->isRaw_) { + LOG(SimplePipeline, Error) + << "Cannot adjust output size with raw output (from " + << cfg.pixelFormat << " to " + << pixelFormat << ")"; + return Invalid; + } adjustedSize = adjustSize(cfg.size, pipeConfig_->outputSizes); + } LOG(SimplePipeline, Debug) << "Adjusting size from " << cfg.size << " to " << adjustedSize; @@ -1174,12 +1199,29 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span 1) { + LOG(SimplePipeline, Error) + << "Can't capture multiple streams with a raw stream"; + return nullptr; + } + data->isRaw_ = raw; + LOG(SimplePipeline, Debug) << "Raw stream requested: " << raw; + /* Create the formats map. */ std::map> formats; for (const SimpleCameraData::Configuration &cfg : data->configs_) { - for (PixelFormat format : cfg.outputFormats) - formats[format].push_back(cfg.outputSizes); + if (raw != cfg.swisp) { + for (PixelFormat format : cfg.outputFormats) + formats[format].push_back(cfg.outputSizes); + } } /* Sort the sizes and merge any consecutive overlapping ranges. */