From patchwork Thu Nov 7 10:58:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21834 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 36864BE173 for ; Thu, 7 Nov 2024 10:59:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4A4CF6547A; Thu, 7 Nov 2024 11:58:56 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ZhnqoFvF"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1227665466 for ; Thu, 7 Nov 2024 11:58:54 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1EC8E792; Thu, 7 Nov 2024 11:58:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977125; bh=Tgo1+go4F2Ac1dNvJhPZYUg4DRFXhEp89K5a7BvzrwQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZhnqoFvF8Tq7gtZdCpAvq37GeCkoc9nlIqY5Cl2rvPENe0EXxn0rwYyuZEIEk8LXT AFFA0v8G0q2aB1BVPOzyLuKSsSg9VSspFYdNr0/AIYQQ5sbISqMLb6hxIeR9zEmHBM oBpcLqv7t1HYv151QCLrCFze+ScV4odqQslzfKM0= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain , Kieran Bingham Subject: [PATCH v5 01/13] libcamera: mali-c55: Limit max size to sensor resolution Date: Thu, 7 Nov 2024 10:58:34 +0000 Message-Id: <20241107105846.52287-2-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The Mali C55 ISP cannot upscale. The maximum allowed output size is the sensor's resolution. For RAW streams this is already handled in adjustRawSizes(), while for processed streams the maximum allowed resolution was wrongly set to the ISP maximum output size (8192x8192). Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 45c71c1d..1c1fef23 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -342,7 +342,11 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() rawConfig = &config; } - Size maxSize = kMaliC55MaxSize; + /* + * The C55 can not upscale. Limit the configuration to the ISP + * capabilities and the sensor resolution. + */ + Size maxSize = kMaliC55MaxSize.boundedTo(data_->resolution()); if (rawConfig) { /* * \todo Take into account the Bayer components ordering once From patchwork Thu Nov 7 10:58:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21835 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 509BBBE173 for ; Thu, 7 Nov 2024 10:59:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C04246546C; Thu, 7 Nov 2024 11:58:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FAOWEn0u"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6707F65458 for ; Thu, 7 Nov 2024 11:58:54 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7C4A55B3; Thu, 7 Nov 2024 11:58:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977125; bh=WbWnNWdBwyCQNoAK7/feTj4rGMxCn4Jml6OiQRFzkPg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FAOWEn0uSkfsFERpZO4vsujbvZeOTfWO24wrGiJ3zrfLp1s9hcUG0KoAsfjlB3WGc LhZiEtXseq7FJ4fkJmSodGJ/erUkcvcmtcC7QpOSI9l1UZD1i3JdVtKhFznPu+DyMA s85yjifLCspqNbCiJtxINQv2KsGb19N4NlS43FNk= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v5 02/13] libcamera: mali-c55: Limit ISP input size Date: Thu, 7 Nov 2024 10:58:35 +0000 Message-Id: <20241107105846.52287-3-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The Mali-C55 ISP has a minimum input size limit of 640x480. Filter out resolutions smaller than this when selecting the sensor format. While at it, rename 'maxYuvSize' to a more appropriate 'minSensorSize'. Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 38 ++++++++++++-------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 1c1fef23..d1ae1a61 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -79,6 +79,7 @@ const std::map maliC55FmtToCode = { { formats::SGRBG16, MEDIA_BUS_FMT_SGRBG16_1X16 }, }; +constexpr Size kMaliC55MinInputSize = { 640, 480 }; constexpr Size kMaliC55MinSize = { 128, 128 }; constexpr Size kMaliC55MaxSize = { 8192, 8192 }; constexpr unsigned int kMaliC55ISPInternalFormat = MEDIA_BUS_FMT_RGB121212_1X36; @@ -265,13 +266,16 @@ PixelFormat MaliC55CameraData::adjustRawFormat(const PixelFormat &rawFmt) const return rawFmt; } -Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &rawSize) const +Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &size) const { /* Just make sure the format is supported. */ auto it = maliC55FmtToCode.find(rawFmt); if (it == maliC55FmtToCode.end()) return {}; + /* Expand the RAW size to the minimum ISP input size. */ + Size rawSize = size.expandedTo(kMaliC55MinInputSize); + /* Check if the size is natively supported. */ unsigned int rawCode = it->second; const auto rawSizes = sizes(rawCode); @@ -282,14 +286,14 @@ Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &ra /* Or adjust it to the closest supported size. */ uint16_t distance = std::numeric_limits::max(); Size bestSize; - for (const Size &size : rawSizes) { + for (const Size &sz : rawSizes) { uint16_t dist = std::abs(static_cast(rawSize.width) - - static_cast(size.width)) + + static_cast(sz.width)) + std::abs(static_cast(rawSize.height) - - static_cast(size.height)); + static_cast(sz.height)); if (dist < distance) { dist = distance; - bestSize = size; + bestSize = sz; } } @@ -376,8 +380,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() frPipeAvailable = false; } - /* Adjust processed streams. */ - Size maxYuvSize; + /* + * Adjust processed streams. + * + * Compute the minimum sensor size to be later used to select the + * sensor configuration. + */ + Size minSensorSize = kMaliC55MinInputSize; for (StreamConfiguration &config : config_) { if (isFormatRaw(config.pixelFormat)) continue; @@ -399,8 +408,8 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() status = Adjusted; } - if (maxYuvSize < size) - maxYuvSize = size; + if (minSensorSize < size) + minSensorSize = size; if (frPipeAvailable) { config.setStream(const_cast(&data_->frStream_)); @@ -416,7 +425,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() if (rawConfig) { const auto it = maliC55FmtToCode.find(rawConfig->pixelFormat); sensorFormat_.code = it->second; - sensorFormat_.size = rawConfig->size; + sensorFormat_.size = rawConfig->size.expandedTo(minSensorSize); return status; } @@ -430,14 +439,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() const auto sizes = data_->sizes(it->second); Size bestSize; for (const auto &size : sizes) { - /* Skip sensor sizes that are smaller than the max YUV size. */ - if (maxYuvSize.width > size.width || - maxYuvSize.height > size.height) + if (minSensorSize.width > size.width || + minSensorSize.height > size.height) continue; - uint16_t dist = std::abs(static_cast(maxYuvSize.width) - + uint16_t dist = std::abs(static_cast(minSensorSize.width) - static_cast(size.width)) + - std::abs(static_cast(maxYuvSize.height) - + std::abs(static_cast(minSensorSize.height) - static_cast(size.height)); if (dist < distance) { dist = distance; From patchwork Thu Nov 7 10:58:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21836 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E215DBE173 for ; Thu, 7 Nov 2024 10:59:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6895665466; Thu, 7 Nov 2024 11:58:58 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ua6G3QJl"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C52666546F for ; Thu, 7 Nov 2024 11:58:54 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D16519FF; Thu, 7 Nov 2024 11:58:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977126; bh=mvqwLnfycSikUqNwIF1bgP1TqjfUKa/Ao6De+Nr10S4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ua6G3QJlhYDHBbftDaR/I4soMgvQZtpR6hFsKN9hpnaT5G/U1GRF5Uajv0/ptCcwH YA6JG7gTaA6ok0oKRo2jrQyEfhJTilZa7xLtnQAfKH+0P4NLOf6ZQUHjMhAU7o5/SX CFEfzR7S7a6s45PIcn7DikBKWyMOkJX3bq3oP7C8= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Umang Jain , Kieran Bingham , Jacopo Mondi Subject: [PATCH v5 03/13] libcamera: mali-c55: Init camera properties Date: Thu, 7 Nov 2024 10:58:36 +0000 Message-Id: <20241107105846.52287-4-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Initialise the camera properties using the CameraSensor properties. Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index d1ae1a61..bffd842c 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -965,7 +965,7 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) if (data->init()) return false; - /* \todo: Init properties and controls. */ + data->properties_ = data->sensor_->properties(); registerMaliCamera(std::move(data), sensor->name()); } From patchwork Thu Nov 7 10:58:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21837 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 862F9BE173 for ; Thu, 7 Nov 2024 10:59:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5FE6B65480; Thu, 7 Nov 2024 11:58:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Lqf34TAw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 277466546A for ; Thu, 7 Nov 2024 11:58:55 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 361E7A47; Thu, 7 Nov 2024 11:58:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977126; bh=TZXN9wNPr2IgJJm1mjh+s8JFocnhUPvisAH+/FsdMo4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Lqf34TAwcxYcaYzo9069XJvhAIIYLhSK3XgWOLwSZqBfC3Tx5Viop1YOT4jqRcKfv k2MW6aUgicQT5iD8chPNhm+ekhE1JD45JIu04dpyh6ovelBN1peFDJu4UDZm1rM8GH wURMJ0aSfsGHeCWxx80h/hsxPOXydr1HG9I3H7p8= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham , Laurent Pinchart Subject: [PATCH v5 04/13] libcamera: mali-c55: Simplify bufferReady() Date: Thu, 7 Nov 2024 10:58:37 +0000 Message-Id: <20241107105846.52287-5-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The PipelineHandler::completeBuffer() base class function returns a boolean to indicate if there still are pending buffers in the Request. Simplify the bufferReady() function in the Mali-C55 pipeline handler using the completeBuffer() return value. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index bffd842c..12e32f23 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -894,12 +894,8 @@ void PipelineHandlerMaliC55::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); - completeBuffer(request, buffer); - - if (request->hasPendingBuffers()) - return; - - completeRequest(request); + if (completeBuffer(request, buffer)) + completeRequest(request); } void PipelineHandlerMaliC55::registerMaliCamera(std::unique_ptr data, From patchwork Thu Nov 7 10:58:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21838 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D3C4BC324E for ; Thu, 7 Nov 2024 10:59:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3125465481; Thu, 7 Nov 2024 11:59:00 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nv0k89xI"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7B4256546F for ; Thu, 7 Nov 2024 11:58:55 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9217CBDB; Thu, 7 Nov 2024 11:58:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977126; bh=xpRucEToqc68lR5BVm1Jls8ePX0OSJoB8zcQwPaYUWA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nv0k89xI85ONW05O+VEbtz4dqUnuImZ74n1dC2Y3QYP3Vv2QH9EnAipyRAnr/LD/1 si4YFRhw0SnYooR5TA6v8NQYROeoC99zO2fPbzFTVbHIXr+BgqBnQF0SmIIuIdXa7l 1U+2oFSYe2GRbk5pJTXJui26jX3QkahKgseaHam8= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham , Laurent Pinchart Subject: [PATCH v5 05/13] libcamera: mali-c55: Remove MaliC55CameraData::mbusCodes() Date: Thu, 7 Nov 2024 10:58:38 +0000 Message-Id: <20241107105846.52287-6-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The MaliC55CameraData::mbusCodes() function is unused. Remove it. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 9 --------- 1 file changed, 9 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 12e32f23..877a156c 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -95,7 +95,6 @@ public: int init(); /* Deflect these functionalities to either TPG or CameraSensor. */ - const std::vector mbusCodes() const; const std::vector sizes(unsigned int mbusCode) const; const Size resolution() const; @@ -178,14 +177,6 @@ void MaliC55CameraData::initTPGData() tpgResolution_ = tpgSizes_.back(); } -const std::vector MaliC55CameraData::mbusCodes() const -{ - if (sensor_) - return sensor_->mbusCodes(); - - return tpgCodes_; -} - const std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const { if (sensor_) From patchwork Thu Nov 7 10:58:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21839 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 25CDCBE173 for ; Thu, 7 Nov 2024 10:59:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2C2766546E; Thu, 7 Nov 2024 11:59:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Hjn4FsHh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D142D65476 for ; Thu, 7 Nov 2024 11:58:55 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E8537F88; Thu, 7 Nov 2024 11:58:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977127; bh=KAiE/Dgl3oWDSyV/BXFyzqAc3ukv4LCM//njqjaXIgw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hjn4FsHhjdwzVLNDaKj1Gtf5xjDNesdPKRRpGmlNDUFptbg1/nZGlchUS6+5YA2aw Am6b8VslN+LHo49ERlEV8t7JRTtaIQI24Tnn40jUjBpOnnxob84uwi6SOoktY6SEmg bT7d5r+y6xzJ9tNIhXiTuSCilqeAuFIRw7DH32ec= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham , Umang Jain Subject: [PATCH v5 06/13] libcamera: mali-c55: Add stride and size to rawConfig Date: Thu, 7 Nov 2024 10:58:39 +0000 Message-Id: <20241107105846.52287-7-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Complete the RAW StreamConfiguration by populating the frame stride and the frame size. Set the minimum required alignment to 4 bytes as the Mali C55 ISP output expands RAW output to 16 bits and a RAW Bayer macro-pixel requires two samples to be complete. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 877a156c..28ad0172 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -367,6 +367,10 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() maxSize = rawSize; + const PixelFormatInfo &info = PixelFormatInfo::info(rawConfig->pixelFormat); + rawConfig->stride = info.stride(rawConfig->size.width, 0, 4); + rawConfig->frameSize = info.frameSize(rawConfig->size, 4); + rawConfig->setStream(const_cast(&data_->frStream_)); frPipeAvailable = false; } From patchwork Thu Nov 7 10:58:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21840 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 24A80C324E for ; Thu, 7 Nov 2024 10:59:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4E05765489; Thu, 7 Nov 2024 11:59:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Zgld5dRF"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3617465478 for ; Thu, 7 Nov 2024 11:58:56 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4C5C51648; Thu, 7 Nov 2024 11:58:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977127; bh=kdhRssPPJNomY0jh8UIoGmydOcqN4Hvx+Uf+CTcZ/+A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Zgld5dRFe5vqUBak6YalMfP3mkpc1yyJ7N/5BaHHLtQdWOiEawLw8U08DQli3W7P8 tUeJnBA5L23I/FMN4z94EK7iyUFh8f22cpCD3X7Omm8d1nq6qOBsEcPZnc3LujlBNC e4CPnsUS0bMS/mJKKHU/G+J2LVDS024i4DAxlZ5Y= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v5 07/13] include: media-bus-formats: Add Mali-C55 mbus codes Date: Thu, 7 Nov 2024 10:58:40 +0000 Message-Id: <20241107105846.52287-8-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add media bus codes introduced by the Mali C55 ISP support to describe the 20-bit input formats supported by the ISP. Add the following formats - MEDIA_BUS_FMT_RGB202020_1X60 for processed input formats https://lore.kernel.org/linux-media/20240529152858.183799-2-dan.scally@ideasonboard.com/ - MEDIA_BUS_FMT_SBGGR20_1X20 MEDIA_BUS_FMT_SGBRG20_1X20 MEDIA_BUS_FMT_SGRBG20_1X20 MEDIA_BUS_FMT_SRGGB20_1X20 for the RAW bayer input format https://lore.kernel.org/linux-media/20240529152858.183799-3-dan.scally@ideasonboard.com/ Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Laurent Pinchart --- Changes in v5: - None include/linux/media-bus-format.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/media-bus-format.h b/include/linux/media-bus-format.h index d4c1d991..b6acf8c8 100644 --- a/include/linux/media-bus-format.h +++ b/include/linux/media-bus-format.h @@ -34,7 +34,7 @@ #define MEDIA_BUS_FMT_FIXED 0x0001 -/* RGB - next is 0x1026 */ +/* RGB - next is 0x1027 */ #define MEDIA_BUS_FMT_RGB444_1X12 0x1016 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_BE 0x1001 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_LE 0x1002 @@ -72,6 +72,7 @@ #define MEDIA_BUS_FMT_RGB888_1X36_CPADLO 0x1021 #define MEDIA_BUS_FMT_RGB121212_1X36 0x1019 #define MEDIA_BUS_FMT_RGB161616_1X48 0x101a +#define MEDIA_BUS_FMT_RGB202020_1X60 0x1026 /* YUV (including grey) - next is 0x202f */ #define MEDIA_BUS_FMT_Y8_1X8 0x2001 @@ -121,7 +122,7 @@ #define MEDIA_BUS_FMT_YUV16_1X48 0x202a #define MEDIA_BUS_FMT_UYYVYY16_0_5X48 0x202b -/* Bayer - next is 0x3021 */ +/* Bayer - next is 0x3025 */ #define MEDIA_BUS_FMT_SBGGR8_1X8 0x3001 #define MEDIA_BUS_FMT_SGBRG8_1X8 0x3013 #define MEDIA_BUS_FMT_SGRBG8_1X8 0x3002 @@ -154,6 +155,10 @@ #define MEDIA_BUS_FMT_SGBRG16_1X16 0x301e #define MEDIA_BUS_FMT_SGRBG16_1X16 0x301f #define MEDIA_BUS_FMT_SRGGB16_1X16 0x3020 +#define MEDIA_BUS_FMT_SBGGR20_1X20 0x3021 +#define MEDIA_BUS_FMT_SGBRG20_1X20 0x3022 +#define MEDIA_BUS_FMT_SGRBG20_1X20 0x3023 +#define MEDIA_BUS_FMT_SRGGB20_1X20 0x3024 /* JPEG compressed formats - next is 0x4002 */ #define MEDIA_BUS_FMT_JPEG_1X8 0x4001 From patchwork Thu Nov 7 10:58:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21841 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0F684BE173 for ; Thu, 7 Nov 2024 10:59:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 83DB36546C; Thu, 7 Nov 2024 11:59:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oZVez6Th"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 93FEA6547B for ; Thu, 7 Nov 2024 11:58:56 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A08911858; Thu, 7 Nov 2024 11:58:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977127; bh=rraBUnUOv+vIq7n8dgbvDB/CjdJ8TzYjxBckOdUggmg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oZVez6Th/GBija9rGU4oQpzbM87GoeykGVmM6EyVT98mxJJumLPEbyvLcm0X0grn5 wRa+1XTFA6H40HBxu4/RxQOLczHTefkli2i+60dRWR5WYj7aSEMVHk9l53T1vbfMMO s3s6E/Tu6n8Um2a+jiMesfbrFcJEX7UMX6Mi5OIk= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v5 08/13] libcamera: v4l2-subdevice: Add Mali C55 media bus formats Date: Thu, 7 Nov 2024 10:58:41 +0000 Message-Id: <20241107105846.52287-9-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add support in the mediaBusFormatInfo map in the v4l2-subdevice.c file for the media bus formats used by the Mali C55 ISP. Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - None src/libcamera/v4l2_subdevice.cpp | 42 ++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 9f2ec479..3a0d075f 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -188,6 +188,20 @@ const std::map mediaBusFormatInfo{ .bitsPerPixel = 24, .colourEncoding = PixelFormatInfo::ColourEncodingRGB, } }, + { MEDIA_BUS_FMT_RGB121212_1X36, { + .name = "RGB121212_1X36", + .code = MEDIA_BUS_FMT_RGB121212_1X36, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 36, + .colourEncoding = PixelFormatInfo::ColourEncodingRGB, + } }, + { MEDIA_BUS_FMT_RGB202020_1X60, { + .name = "RGB202020_1X60", + .code = MEDIA_BUS_FMT_RGB202020_1X60, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 60, + .colourEncoding = PixelFormatInfo::ColourEncodingRGB, + } }, { MEDIA_BUS_FMT_ARGB8888_1X32, { .name = "ARGB8888_1X32", .code = MEDIA_BUS_FMT_ARGB8888_1X32, @@ -678,6 +692,34 @@ const std::map mediaBusFormatInfo{ .bitsPerPixel = 16, .colourEncoding = PixelFormatInfo::ColourEncodingRAW } }, + { MEDIA_BUS_FMT_SBGGR20_1X20, { + .name = "SBGGR20_1X20", + .code = MEDIA_BUS_FMT_SBGGR20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, + { MEDIA_BUS_FMT_SGBRG20_1X20, { + .name = "SGBRG20_1X20", + .code = MEDIA_BUS_FMT_SGBRG20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, + { MEDIA_BUS_FMT_SGRBG20_1X20, { + .name = "SGRBG20_1X20", + .code = MEDIA_BUS_FMT_SGRBG20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, + { MEDIA_BUS_FMT_SRGGB20_1X20, { + .name = "SRGGB20_1X20", + .code = MEDIA_BUS_FMT_SRGGB20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, /* \todo Clarify colour encoding for HSV formats */ { MEDIA_BUS_FMT_AHSV8888_1X32, { .name = "AHSV8888_1X32", From patchwork Thu Nov 7 10:58:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21842 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D2A3AC32CE for ; Thu, 7 Nov 2024 10:59:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2935E65497; Thu, 7 Nov 2024 11:59:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RqcPauF6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EFACF6546E for ; Thu, 7 Nov 2024 11:58:56 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 06DED5B3; Thu, 7 Nov 2024 11:58:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977128; bh=0ycJPOLaeKCRjeRdYS01ObExY/Y2R7GR/l7TPhxFvyk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RqcPauF6nGcUyugvMh056iIaxdaO+Qc0vzq4scQJguXfjknnlYvBxvCsLCs83CH6D W1z8Di+u+UpiUyA47kKhT7vmnqL8GalbbIEy/f4VpM1gm0SWUKjqnDVi5aTlMLfBud RZhgr1pW99Ttql5nUKndyVzgqYpLjmmx1C9lpez4= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v5 09/13] libcamera: mali-c55: Propagate CSI-2 format to ISP Date: Thu, 7 Nov 2024 10:58:42 +0000 Message-Id: <20241107105846.52287-10-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The latest version of the Mali C55 driver has changed the format accepted by the ISP sink pad to be the 20-bit wide. The CSI-2 receiver handles the format expansion internally by propagating the sensor produced format from its sink to the 20-bit expanded version on its source pad. Instead of re-applying the sensor format to the CSI-2 receiver source pad (which is now an invalid operation) read from there the format propagated by the driver internally and further propagate it to the ISP subdevice. Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 28ad0172..156560c1 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -768,7 +768,7 @@ int PipelineHandlerMaliC55::configure(Camera *camera, if (ret) return ret; - ret = data->csi_->setFormat(1, &subdevFormat); + ret = data->csi_->getFormat(1, &subdevFormat); if (ret) return ret; } From patchwork Thu Nov 7 10:58:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21843 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DDBD4C324E for ; Thu, 7 Nov 2024 10:59:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7895E65495; Thu, 7 Nov 2024 11:59:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TnKzcJ+q"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5439E6547D for ; Thu, 7 Nov 2024 11:58:57 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 602B11870; Thu, 7 Nov 2024 11:58:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977128; bh=5TIRCs1LJFZRvKUfyRoNHkRuFY2ZpNhQ2ASRspN45OI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TnKzcJ+qRGBpsKX9wS1YPg8Nwhe4XSCwZf7jCrEeoHGKbwf/hdVUjcbOh3qOOJmTR Apk8GAXNwq9NC1eACeqQ7A+JKzDqpkV5b0nYPIQtFUCZ7VhzulDHWI03sU/S6gULr0 Kn3QQPsRDFN87auOzOQdg2OhJLPUkqPNFKRRhasY= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Kieran Bingham Subject: [PATCH v5 10/13] libcamera: mali-c55: Correct input/output format representation Date: Thu, 7 Nov 2024 10:58:43 +0000 Message-Id: <20241107105846.52287-11-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" At present we configure raw streams by looping through the pixel formats we support and finding one with an associated media bus format code that the sensor can produce. In the new representation of raw data from the kernel driver this will not work - the sensor could produce 8, 10, 12, 14 or 16 bit data and the ISP will force it to RAW16, which is the only actually supported output. To fix the issue move to simply finding a pixel format with a bayer order that matches that of the media bus format produced by the sensor. If the sensor can produce multiple formats with the same bayer order use the one with the largest bitdepth. Finally, remove the claim to support RAW formats of less than 16 bits. Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v5: - Dropped the Fatal log. src/libcamera/pipeline/mali-c55/mali-c55.cpp | 128 ++++++++++++------- 1 file changed, 80 insertions(+), 48 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 156560c1..97827abd 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -57,22 +57,6 @@ const std::map maliC55FmtToCode = { { formats::NV21, MEDIA_BUS_FMT_YUV10_1X30 }, /* RAW formats, FR pipe only. */ - { formats::SGBRG8, MEDIA_BUS_FMT_SGBRG8_1X8 }, - { formats::SRGGB8, MEDIA_BUS_FMT_SRGGB8_1X8 }, - { formats::SBGGR8, MEDIA_BUS_FMT_SBGGR8_1X8 }, - { formats::SGRBG8, MEDIA_BUS_FMT_SGRBG8_1X8 }, - { formats::SGBRG10, MEDIA_BUS_FMT_SGBRG10_1X10 }, - { formats::SRGGB10, MEDIA_BUS_FMT_SRGGB10_1X10 }, - { formats::SBGGR10, MEDIA_BUS_FMT_SBGGR10_1X10 }, - { formats::SGRBG10, MEDIA_BUS_FMT_SGRBG10_1X10 }, - { formats::SGBRG12, MEDIA_BUS_FMT_SGBRG12_1X12 }, - { formats::SRGGB12, MEDIA_BUS_FMT_SRGGB12_1X12 }, - { formats::SBGGR12, MEDIA_BUS_FMT_SBGGR12_1X12 }, - { formats::SGRBG12, MEDIA_BUS_FMT_SGRBG12_1X12 }, - { formats::SGBRG14, MEDIA_BUS_FMT_SGBRG14_1X14 }, - { formats::SRGGB14, MEDIA_BUS_FMT_SRGGB14_1X14 }, - { formats::SBGGR14, MEDIA_BUS_FMT_SBGGR14_1X14 }, - { formats::SGRBG14, MEDIA_BUS_FMT_SGRBG14_1X14 }, { formats::SGBRG16, MEDIA_BUS_FMT_SGBRG16_1X16 }, { formats::SRGGB16, MEDIA_BUS_FMT_SRGGB16_1X16 }, { formats::SBGGR16, MEDIA_BUS_FMT_SBGGR16_1X16 }, @@ -98,7 +82,8 @@ public: const std::vector sizes(unsigned int mbusCode) const; const Size resolution() const; - PixelFormat bestRawFormat() const; + int pixfmtToMbusCode(const PixelFormat &pixFmt) const; + const PixelFormat &bestRawFormat() const; PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; @@ -208,33 +193,75 @@ const Size MaliC55CameraData::resolution() const return tpgResolution_; } -PixelFormat MaliC55CameraData::bestRawFormat() const +/* + * The Mali C55 ISP can only produce 16-bit RAW output in bypass modes, but the + * sensors connected to it might produce 8/10/12/16 bits. We simply search the + * sensor's supported formats for the one with a matching bayer order and the + * greatest bitdepth. + */ +int MaliC55CameraData::pixfmtToMbusCode(const PixelFormat &pixFmt) const { + auto it = maliC55FmtToCode.find(pixFmt); + if (it == maliC55FmtToCode.end()) + return -EINVAL; + + BayerFormat bayerFormat = BayerFormat::fromMbusCode(it->second); + if (!bayerFormat.isValid()) + return -EINVAL; + + V4L2Subdevice::Formats formats = sd_->formats(0); + unsigned int sensorMbusCode = 0; unsigned int bitDepth = 0; - PixelFormat rawFormat; - /* - * Iterate over all the supported PixelFormat and find the one - * supported by the camera with the largest bitdepth. - */ - for (const auto &maliFormat : maliC55FmtToCode) { - PixelFormat pixFmt = maliFormat.first; - if (!isFormatRaw(pixFmt)) + for (const auto &[code, sizes] : formats) { + BayerFormat sdBayerFormat = BayerFormat::fromMbusCode(code); + if (!sdBayerFormat.isValid()) continue; - unsigned int rawCode = maliFormat.second; - const auto rawSizes = sizes(rawCode); - if (rawSizes.empty()) + if (sdBayerFormat.order != bayerFormat.order) continue; - BayerFormat bayer = BayerFormat::fromMbusCode(rawCode); - if (bayer.bitDepth > bitDepth) { - bitDepth = bayer.bitDepth; - rawFormat = pixFmt; + if (sdBayerFormat.bitDepth > bitDepth) { + bitDepth = sdBayerFormat.bitDepth; + sensorMbusCode = code; + } + } + + if (!sensorMbusCode) + return -EINVAL; + + return sensorMbusCode; +} + +/* + * Find a RAW PixelFormat supported by both the ISP and the sensor. + * + * The situation is mildly complicated by the fact that we expect the sensor to + * output something like RAW8/10/12/16, but the ISP can only accept as input + * RAW20 and can only produce as output RAW16. The one constant in that is the + * bayer order of the data, so we'll simply check that the sensor produces a + * format with a bayer order that matches that of one of the formats we support, + * and select that. + */ +const PixelFormat &MaliC55CameraData::bestRawFormat() const +{ + static const PixelFormat invalidPixFmt = {}; + + for (const auto &fmt : sd_->formats(0)) { + BayerFormat sensorBayer = BayerFormat::fromMbusCode(fmt.first); + + for (const auto &[pixFmt, rawCode] : maliC55FmtToCode) { + if (!isFormatRaw(pixFmt)) + continue; + + BayerFormat bayer = BayerFormat::fromMbusCode(rawCode); + if (bayer.order == sensorBayer.order) + return pixFmt; } } - return rawFormat; + LOG(MaliC55, Error) << "Sensor doesn't provide a compatible format"; + return invalidPixFmt; } /* @@ -243,13 +270,11 @@ PixelFormat MaliC55CameraData::bestRawFormat() const */ PixelFormat MaliC55CameraData::adjustRawFormat(const PixelFormat &rawFmt) const { - /* Make sure the provided raw format is supported by the pipeline. */ - auto it = maliC55FmtToCode.find(rawFmt); - if (it == maliC55FmtToCode.end()) + /* Make sure the RAW mbus code is supported by the image source. */ + int rawCode = pixfmtToMbusCode(rawFmt); + if (rawCode < 0) return bestRawFormat(); - /* Now make sure the RAW mbus code is supported by the image source. */ - unsigned int rawCode = it->second; const auto rawSizes = sizes(rawCode); if (rawSizes.empty()) return bestRawFormat(); @@ -259,16 +284,14 @@ PixelFormat MaliC55CameraData::adjustRawFormat(const PixelFormat &rawFmt) const Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &size) const { - /* Just make sure the format is supported. */ - auto it = maliC55FmtToCode.find(rawFmt); - if (it == maliC55FmtToCode.end()) - return {}; - /* Expand the RAW size to the minimum ISP input size. */ Size rawSize = size.expandedTo(kMaliC55MinInputSize); /* Check if the size is natively supported. */ - unsigned int rawCode = it->second; + int rawCode = pixfmtToMbusCode(rawFmt); + if (rawCode < 0) + return {}; + const auto rawSizes = sizes(rawCode); auto sizeIt = std::find(rawSizes.begin(), rawSizes.end(), rawSize); if (sizeIt != rawSizes.end()) @@ -349,6 +372,10 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() */ PixelFormat rawFormat = data_->adjustRawFormat(rawConfig->pixelFormat); + + if (!rawFormat.isValid()) + return Invalid; + if (rawFormat != rawConfig->pixelFormat) { LOG(MaliC55, Debug) << "RAW format adjusted to " << rawFormat; @@ -418,8 +445,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() /* If there's a RAW config, sensor configuration follows it. */ if (rawConfig) { - const auto it = maliC55FmtToCode.find(rawConfig->pixelFormat); - sensorFormat_.code = it->second; + sensorFormat_.code = data_->pixfmtToMbusCode(rawConfig->pixelFormat); sensorFormat_.size = rawConfig->size.expandedTo(minSensorSize); return status; @@ -427,6 +453,9 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() /* If there's no RAW config, compute the sensor configuration here. */ PixelFormat rawFormat = data_->bestRawFormat(); + if (!rawFormat.isValid()) + return Invalid; + const auto it = maliC55FmtToCode.find(rawFormat); sensorFormat_.code = it->second; @@ -614,7 +643,10 @@ PipelineHandlerMaliC55::generateConfiguration(Camera *camera, if (isRaw) { /* Make sure the mbus code is supported. */ - unsigned int rawCode = maliFormat.second; + int rawCode = data->pixfmtToMbusCode(pixFmt); + if (rawCode < 0) + continue; + const auto sizes = data->sizes(rawCode); if (sizes.empty()) continue; From patchwork Thu Nov 7 10:58:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21844 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 80982C330B for ; Thu, 7 Nov 2024 10:59:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F404765485; Thu, 7 Nov 2024 11:59:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MjuiAsLx"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E58E76547F for ; Thu, 7 Nov 2024 11:58:57 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BA7651890; Thu, 7 Nov 2024 11:58:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977128; bh=1BWGNjNH+ArkioShSA1zbZVIsnJfSG1BVW5CkmaxX6Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MjuiAsLxZzwbA4yagS/EfDfpmr7c1NVDe+Xdlu2Kz+vEJAYrDqAsOe9EsKMbqgCYM Vcozw74f6CE0NITbfR0pzLaJauxhMe2ZvVut2bI2Zxw17VUm3HenweaY4wSXqyM25M 4r4t4VqX52K9u7V2aADBLW/KtvJtNCTt9jO9DGVc= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Kieran Bingham Subject: [PATCH v5 11/13] libcamera: mali-c55: Enable links between resizer and video node Date: Thu, 7 Nov 2024 10:58:44 +0000 Message-Id: <20241107105846.52287-12-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The mali-c55 driver now expects links to video devices to be enabled in order for those devices to be streamed from / to. Enable the media link between the resizers and their associated video device to fulfil the requirement. Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally Reviewed-by: Jacopo Mondi --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 97827abd..b00cbce5 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -834,6 +834,17 @@ int PipelineHandlerMaliC55::configure(Camera *camera, Stream *stream = streamConfig.stream(); MaliC55Pipe *pipe = pipeFromStream(data, stream); + /* + * Enable the media link between the pipe's resizer and the + * capture video device + */ + const MediaEntity *rszEntity = pipe->resizer->entity(); + ret = rszEntity->getPadByIndex(1)->links()[0]->setEnabled(true); + if (ret) { + LOG(MaliC55, Error) << "Couldn't enable resizer's link"; + return ret; + } + if (isFormatRaw(streamConfig.pixelFormat)) ret = configureRawStream(data, streamConfig, subdevFormat); else From patchwork Thu Nov 7 10:58:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21845 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 137A6BE173 for ; Thu, 7 Nov 2024 10:59:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0494A65496; Thu, 7 Nov 2024 11:59:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VS3X+avp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1EAB065467 for ; Thu, 7 Nov 2024 11:58:58 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0F9A01961; Thu, 7 Nov 2024 11:58:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977129; bh=7SSFt3cRRenEMq/rYfWTHKQbMMY5D517oVdMwLRjYIo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VS3X+avptDdlLZawEvFZ+wNR7ymyCQ5gDKHBGo+XWCAtd4ctjhXgGqhP02F0JXZgR SXkX2qZGfHsx++QxGb834hMNCk/DjQrYurhci2I7Bca25UkJb0iqncN2Vkv6wir7kC S3d8ajM2IgGr487i2/CGM+vgj4R6Lwx3WdPKBSUY= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham , Daniel Scally Subject: [PATCH v5 12/13] libcamera: mali-c55: Enable usage of scaler Date: Thu, 7 Nov 2024 10:58:45 +0000 Message-Id: <20241107105846.52287-13-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The Mali C55 ISP has a resizing pipeline that allows to crop and scale images. So far the mali-c55 pipeline has only supported cropping without using the scaling functionalities. Now that the kernel has gained support for the scaling operations, make the libcamera pipeline use it by combining it with a first cropping step to align the input and output images FOV ratio, and then scale to the desired output size. Reviewed-by: Kieran Bingham Reviewed-by: Daniel Scally Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index b00cbce5..2d4d5950 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -754,16 +754,28 @@ int PipelineHandlerMaliC55::configureProcessedStream(MaliC55CameraData *data, if (ret) return ret; - /* \todo Configure the resizer crop/compose rectangles. */ - Rectangle ispCrop = { 0, 0, config.size }; + /* + * Compute the scaler-in to scaler-out ratio: first center-crop to align + * the FOV to the desired resolution, then scale to the desired size. + */ + Size scalerIn = subdevFormat.size.boundedToAspectRatio(config.size); + int xCrop = (subdevFormat.size.width - scalerIn.width) / 2; + int yCrop = (subdevFormat.size.height - scalerIn.height) / 2; + Rectangle ispCrop = { xCrop, yCrop, scalerIn }; ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_CROP, &ispCrop); if (ret) return ret; - ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_COMPOSE, &ispCrop); + Rectangle ispCompose = { 0, 0, config.size }; + ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_COMPOSE, &ispCompose); if (ret) return ret; + /* + * The source pad format size comes directly from the sink + * compose rectangle. + */ + subdevFormat.size = ispCompose.size(); subdevFormat.code = maliC55FmtToCode.find(config.pixelFormat)->second; return pipe->resizer->setFormat(1, &subdevFormat); } From patchwork Thu Nov 7 10:58:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 21846 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 88F40C330C for ; Thu, 7 Nov 2024 10:59:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0CB3765489; Thu, 7 Nov 2024 11:59:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="r/LZOcvp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54ACF65483 for ; Thu, 7 Nov 2024 11:58:58 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5F298198D; Thu, 7 Nov 2024 11:58:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730977129; bh=qasxcKGQzZer6/VbehIgGBsk7XYNF7w1L1HDkII1UY8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r/LZOcvpX3EU797Lbi3xHnzOe6tjPF9f0ooP2TOuWaYJkSv3SReQm4Ps1gVkaUafW 4jXRslz7QCghrmyAlyImSW2asbgaJnfyvWz3Hzhb3Fwo+c7QZN8uE0dP950UoCSbJa eCwoRzRKGn96ptjMqQEBnzUfmPJJ1vBM2r/IorWc= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham , Daniel Scally Subject: [PATCH v5 13/13] libcamera: mali-c55: implement support for ScalerCrop Date: Thu, 7 Nov 2024 10:58:46 +0000 Message-Id: <20241107105846.52287-14-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241107105846.52287-1-dan.scally@ideasonboard.com> References: <20241107105846.52287-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Implement support for the ScalerCrop control that allows to apply a digital zoom to the captured streams. Initialize the camera controls at camera registration time and update them at configure time as the sensor's analogue crop size might change depending on the desired Camera configuration. Reviewed-by: Kieran Bingham Reviewed-by: Daniel Scally Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 133 +++++++++++++++++++ 1 file changed, 133 insertions(+) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 2d4d5950..0845a077 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -85,6 +85,8 @@ public: int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; + void updateControls(); + PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; @@ -264,6 +266,27 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const return invalidPixFmt; } +void MaliC55CameraData::updateControls() +{ + if (!sensor_) + return; + + IPACameraSensorInfo sensorInfo; + int ret = sensor_->sensorInfo(&sensorInfo); + if (ret) { + LOG(MaliC55, Error) << "Failed to retrieve sensor info"; + return; + } + + ControlInfoMap::Map controls; + Rectangle ispMinCrop{ 0, 0, 640, 480 }; + controls[&controls::ScalerCrop] = + ControlInfo(ispMinCrop, sensorInfo.analogCrop, + sensorInfo.analogCrop); + + controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); +} + /* * Make sure the provided raw pixel format is supported and adjust it to * one of the supported ones if it's not. @@ -551,6 +574,8 @@ private: const StreamConfiguration &config, V4L2SubdeviceFormat &subdevFormat); + void applyScalerCrop(Camera *camera, const ControlList &controls); + void registerMaliCamera(std::unique_ptr data, const std::string &name); bool registerTPGCamera(MediaLink *link); @@ -878,6 +903,8 @@ int PipelineHandlerMaliC55::configure(Camera *camera, pipe->stream = stream; } + data->updateControls(); + return 0; } @@ -925,6 +952,102 @@ void PipelineHandlerMaliC55::stopDevice([[maybe_unused]] Camera *camera) } } +void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, + const ControlList &controls) +{ + MaliC55CameraData *data = cameraData(camera); + + const auto &scalerCrop = controls.get(controls::ScalerCrop); + if (!scalerCrop) + return; + + if (!data->sensor_) { + LOG(MaliC55, Error) << "ScalerCrop not supported for TPG"; + return; + } + + Rectangle nativeCrop = *scalerCrop; + + IPACameraSensorInfo sensorInfo; + int ret = data->sensor_->sensorInfo(&sensorInfo); + if (ret) { + LOG(MaliC55, Error) << "Failed to retrieve sensor info"; + return; + } + + /* + * The ScalerCrop rectangle re-scaling in the ISP crop rectangle + * comes straight from the RPi pipeline handler. + * + * Create a version of the crop rectangle aligned to the analogue crop + * rectangle top-left coordinates and scaled in the [analogue crop to + * output frame] ratio to take into account binning/skipping on the + * sensor. + */ + Rectangle ispCrop = nativeCrop.translatedBy(-sensorInfo.analogCrop + .topLeft()); + ispCrop.scaleBy(sensorInfo.outputSize, sensorInfo.analogCrop.size()); + + /* + * The crop rectangle should be: + * 1. At least as big as ispMinCropSize_, once that's been + * enlarged to the same aspect ratio. + * 2. With the same mid-point, if possible. + * 3. But it can't go outside the sensor area. + */ + Rectangle ispMinCrop{ 0, 0, 640, 480 }; + Size minSize = ispMinCrop.size().expandedToAspectRatio(nativeCrop.size()); + Size size = ispCrop.size().expandedTo(minSize); + ispCrop = size.centeredTo(ispCrop.center()) + .enclosedIn(Rectangle(sensorInfo.outputSize)); + + /* + * As the resizer can't upscale, the crop rectangle has to be larger + * than the larger stream output size. + */ + Size maxYuvSize; + for (MaliC55Pipe &pipe : pipes_) { + if (!pipe.stream) + continue; + + const StreamConfiguration &config = pipe.stream->configuration(); + if (isFormatRaw(config.pixelFormat)) { + LOG(MaliC55, Debug) << "Cannot crop with a RAW stream"; + return; + } + + Size streamSize = config.size; + if (streamSize.width > maxYuvSize.width) + maxYuvSize.width = streamSize.width; + if (streamSize.height > maxYuvSize.height) + maxYuvSize.height = streamSize.height; + } + + ispCrop.size().expandTo(maxYuvSize); + + /* + * Now apply the scaler crop to each enabled output. This overrides the + * crop configuration performed at configure() time and can cause + * square pixels if the crop rectangle and scaler output FOV ratio are + * different. + */ + for (MaliC55Pipe &pipe : pipes_) { + if (!pipe.stream) + continue; + + /* Create a copy to avoid setSelection() to modify ispCrop. */ + Rectangle pipeCrop = ispCrop; + ret = pipe.resizer->setSelection(0, V4L2_SEL_TGT_CROP, &pipeCrop); + if (ret) { + LOG(MaliC55, Error) + << "Failed to apply crop to " + << (pipe.stream == &data->frStream_ ? + "FR" : "DS") << " pipe"; + return; + } + } +} + int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { int ret; @@ -937,6 +1060,15 @@ int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) return ret; } + /* + * Some controls need to be applied immediately, as in example, + * the ScalerCrop one. + * + * \todo Move it buffer queue time (likely after the IPA has filled in + * the parameters buffer) once we have plumbed the IPA loop in. + */ + applyScalerCrop(camera, request->controls()); + return 0; } @@ -1012,6 +1144,7 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return false; data->properties_ = data->sensor_->properties(); + data->updateControls(); registerMaliCamera(std::move(data), sensor->name()); }