From patchwork Tue Jul 9 14:39:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20618 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 67A5CBDB1C for ; Tue, 9 Jul 2024 14:39:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 059536337F; Tue, 9 Jul 2024 16:39:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FdChxek9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0E249619C1 for ; Tue, 9 Jul 2024 16:39:20 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6C1A716C3; Tue, 9 Jul 2024 16:38:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535927; bh=kGF5kMqdHRI9qnD49DVIAMqqroOG26NPu6xcaqD1H5E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FdChxek9JCWFH66xIMWZiWINkGjQAxeQnUh+ds5kdQtuxRXOtwP+ZmDTY6az8BAFC FqgiU8kEhxnk6aIqdDZVUVpdZTYf5G/UwQmNTbvQr5OYqDLE9a5dyxJidLGvOm3CQC Av3bIC8KypZjj5AqBOOV0H1th2tOD2TPMpmMQ1Zk= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain , Kieran Bingham Subject: [PATCH v4 01/13] libcamera: mali-c55: Limit max size to sensor resolution Date: Tue, 9 Jul 2024 15:39:01 +0100 Message-Id: <20240709143913.3276983-2-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The Mali C55 ISP cannot upscale. The maximum allowed output size is the sensor's resolution. For RAW streams this is already handled in adjustRawSizes(), while for processed streams the maximum allowed resolution was wrongly set to the ISP maximum output size (8192x8192). Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 45c71c1d..1c1fef23 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -342,7 +342,11 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() rawConfig = &config; } - Size maxSize = kMaliC55MaxSize; + /* + * The C55 can not upscale. Limit the configuration to the ISP + * capabilities and the sensor resolution. + */ + Size maxSize = kMaliC55MaxSize.boundedTo(data_->resolution()); if (rawConfig) { /* * \todo Take into account the Bayer components ordering once From patchwork Tue Jul 9 14:39:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20619 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5B365BDB1C for ; Tue, 9 Jul 2024 14:39:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BB98C63382; Tue, 9 Jul 2024 16:39:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="huuTBniH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 46F8A62C95 for ; Tue, 9 Jul 2024 16:39:20 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BA0731894; Tue, 9 Jul 2024 16:38:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535927; bh=l7e6nx5pOwro+KRDgIiXF5yhuQcefoeJiIVM5rQrKnc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=huuTBniHncLibujjCw/Of6xaW61Ltpzn79aNnjwBbudnfVFnZyd3jmCnVussgmruL vsmpT8UKGIcG+9bNHacJlPZXCnm4SlsKRyrejZzxAufMXlpOehBr6euFAR1bhYj/0L SEJeUWpHX6GobgkXsxsEdV0yWrybEDJZLfF7KRmk= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v4 02/13] libcamera: mali-c55: Limit ISP input size Date: Tue, 9 Jul 2024 15:39:02 +0100 Message-Id: <20240709143913.3276983-3-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The Mali-C55 ISP has a minimum input size limit of 640x480. Filter out resolutions smaller than this when selecting the sensor format. While at it, rename 'maxYuvSize' to a more appropriate 'minSensorSize'. Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 38 ++++++++++++-------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 1c1fef23..d1ae1a61 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -79,6 +79,7 @@ const std::map maliC55FmtToCode = { { formats::SGRBG16, MEDIA_BUS_FMT_SGRBG16_1X16 }, }; +constexpr Size kMaliC55MinInputSize = { 640, 480 }; constexpr Size kMaliC55MinSize = { 128, 128 }; constexpr Size kMaliC55MaxSize = { 8192, 8192 }; constexpr unsigned int kMaliC55ISPInternalFormat = MEDIA_BUS_FMT_RGB121212_1X36; @@ -265,13 +266,16 @@ PixelFormat MaliC55CameraData::adjustRawFormat(const PixelFormat &rawFmt) const return rawFmt; } -Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &rawSize) const +Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &size) const { /* Just make sure the format is supported. */ auto it = maliC55FmtToCode.find(rawFmt); if (it == maliC55FmtToCode.end()) return {}; + /* Expand the RAW size to the minimum ISP input size. */ + Size rawSize = size.expandedTo(kMaliC55MinInputSize); + /* Check if the size is natively supported. */ unsigned int rawCode = it->second; const auto rawSizes = sizes(rawCode); @@ -282,14 +286,14 @@ Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &ra /* Or adjust it to the closest supported size. */ uint16_t distance = std::numeric_limits::max(); Size bestSize; - for (const Size &size : rawSizes) { + for (const Size &sz : rawSizes) { uint16_t dist = std::abs(static_cast(rawSize.width) - - static_cast(size.width)) + + static_cast(sz.width)) + std::abs(static_cast(rawSize.height) - - static_cast(size.height)); + static_cast(sz.height)); if (dist < distance) { dist = distance; - bestSize = size; + bestSize = sz; } } @@ -376,8 +380,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() frPipeAvailable = false; } - /* Adjust processed streams. */ - Size maxYuvSize; + /* + * Adjust processed streams. + * + * Compute the minimum sensor size to be later used to select the + * sensor configuration. + */ + Size minSensorSize = kMaliC55MinInputSize; for (StreamConfiguration &config : config_) { if (isFormatRaw(config.pixelFormat)) continue; @@ -399,8 +408,8 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() status = Adjusted; } - if (maxYuvSize < size) - maxYuvSize = size; + if (minSensorSize < size) + minSensorSize = size; if (frPipeAvailable) { config.setStream(const_cast(&data_->frStream_)); @@ -416,7 +425,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() if (rawConfig) { const auto it = maliC55FmtToCode.find(rawConfig->pixelFormat); sensorFormat_.code = it->second; - sensorFormat_.size = rawConfig->size; + sensorFormat_.size = rawConfig->size.expandedTo(minSensorSize); return status; } @@ -430,14 +439,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() const auto sizes = data_->sizes(it->second); Size bestSize; for (const auto &size : sizes) { - /* Skip sensor sizes that are smaller than the max YUV size. */ - if (maxYuvSize.width > size.width || - maxYuvSize.height > size.height) + if (minSensorSize.width > size.width || + minSensorSize.height > size.height) continue; - uint16_t dist = std::abs(static_cast(maxYuvSize.width) - + uint16_t dist = std::abs(static_cast(minSensorSize.width) - static_cast(size.width)) + - std::abs(static_cast(maxYuvSize.height) - + std::abs(static_cast(minSensorSize.height) - static_cast(size.height)); if (dist < distance) { dist = distance; From patchwork Tue Jul 9 14:39:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20620 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B0D13BDB1C for ; Tue, 9 Jul 2024 14:39:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D075763374; Tue, 9 Jul 2024 16:39:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="drLbg3x+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7B5AC6336D for ; Tue, 9 Jul 2024 16:39:20 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0BCAD13D7; Tue, 9 Jul 2024 16:38:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535928; bh=MIe+6qUxwUjckLd+0Baz5RY09JdELA6KYgrUDWYzquA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=drLbg3x+8/8368azsW/1WVA0NmopZZ5zLfrKPd0dUpLDP52xBZm9ruy2h+ZHVMwjx z4wZNP+JLL3CE4lKzDySqE8tJ1fmagVX/eNYbLalopoPZgpkE99e4qS18heU+B1CkK QmX44IHXkGuqT1QJr1McbhYwR8+MLj4fUDW7SRVI= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Umang Jain , Kieran Bingham , Jacopo Mondi Subject: [PATCH v4 03/13] libcamera: mali-c55: Init camera properties Date: Tue, 9 Jul 2024 15:39:03 +0100 Message-Id: <20240709143913.3276983-4-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Initialise the camera properties using the CameraSensor properties. Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index d1ae1a61..bffd842c 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -965,7 +965,7 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) if (data->init()) return false; - /* \todo: Init properties and controls. */ + data->properties_ = data->sensor_->properties(); registerMaliCamera(std::move(data), sensor->name()); } From patchwork Tue Jul 9 14:39:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20621 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BF89DBDB1C for ; Tue, 9 Jul 2024 14:39:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D07946337F; Tue, 9 Jul 2024 16:39:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="A4gkK3YG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C30176336E for ; Tue, 9 Jul 2024 16:39:20 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5A1592393; Tue, 9 Jul 2024 16:38:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535928; bh=l/3TbggOxDUmjAdNufPjtBf5vLp92L1CHmIOBM+RRr4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A4gkK3YGKASSwDKTBQYsZ0nCEXITanPI9Pf7CimqbxFCmBXPZSqcFJ6emNQxcmcH8 qvIMicf+wjs1tMZND0jZ30PiDQKUuuIcmu/DtwPCQc5TUWojcdoAcxpRFpaTOATgQ9 q+/nryONL8CG4HYW64B/Ox/S4dhJ9XbC/jmAKXc0= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham , Laurent Pinchart Subject: [PATCH v4 04/13] libcamera: mali-c55: Simplify bufferReady() Date: Tue, 9 Jul 2024 15:39:04 +0100 Message-Id: <20240709143913.3276983-5-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The PipelineHandler::completeBuffer() base class function returns a boolean to indicate if there still are pending buffers in the Request. Simplify the bufferReady() function in the Mali-C55 pipeline handler using the completeBuffer() return value. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index bffd842c..12e32f23 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -894,12 +894,8 @@ void PipelineHandlerMaliC55::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); - completeBuffer(request, buffer); - - if (request->hasPendingBuffers()) - return; - - completeRequest(request); + if (completeBuffer(request, buffer)) + completeRequest(request); } void PipelineHandlerMaliC55::registerMaliCamera(std::unique_ptr data, From patchwork Tue Jul 9 14:39:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20622 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2C05ABDB1C for ; Tue, 9 Jul 2024 14:39:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9B05563384; Tue, 9 Jul 2024 16:39:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Hi4i5DkI"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 22D036336F for ; Tue, 9 Jul 2024 16:39:21 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A66A62394; Tue, 9 Jul 2024 16:38:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535928; bh=ccOYQBOvCSDWpdECsCA4CfPxCWj6FpW7ZnXdda6qAqU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hi4i5DkIEldJdZEH/wp2nTR4Ozwvbm6HZpCGELxecc/0bODWT3I2JRzfQNqDWtx2h GqglQ0F4AzpFww5KUzLaHXJyMdL4/eJd1dec6bo6BBe0d1LpmlWBXiE+fkp0SqUZeh IjyrndKnhyeURPRAy5p3un71MvFDySWLdktjVjQ8= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham , Laurent Pinchart Subject: [PATCH v4 05/13] libcamera: mali-c55: Remove MaliC55CameraData::mbusCodes() Date: Tue, 9 Jul 2024 15:39:05 +0100 Message-Id: <20240709143913.3276983-6-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The MaliC55CameraData::mbusCodes() function is unused. Remove it. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 9 --------- 1 file changed, 9 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 12e32f23..877a156c 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -95,7 +95,6 @@ public: int init(); /* Deflect these functionalities to either TPG or CameraSensor. */ - const std::vector mbusCodes() const; const std::vector sizes(unsigned int mbusCode) const; const Size resolution() const; @@ -178,14 +177,6 @@ void MaliC55CameraData::initTPGData() tpgResolution_ = tpgSizes_.back(); } -const std::vector MaliC55CameraData::mbusCodes() const -{ - if (sensor_) - return sensor_->mbusCodes(); - - return tpgCodes_; -} - const std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const { if (sensor_) From patchwork Tue Jul 9 14:39:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20623 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5E734BDB1C for ; Tue, 9 Jul 2024 14:39:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6980263389; Tue, 9 Jul 2024 16:39:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="o4KXq1Ym"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6FCCF619B9 for ; Tue, 9 Jul 2024 16:39:21 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 021AB1894; Tue, 9 Jul 2024 16:38:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535929; bh=k2mtghCpGfjrgjDQNn46L2jMOl9Vfuq4RSuNdaV6Yw0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=o4KXq1Ym3TFXQssmNVlEEuLijAO/MnzcyqQHIr70dFcPqycmvpqerBrmzfPx/Uixq o8okIyVEyeKYcWwd8K8HZXrWa58+Vii7H2W+bzXLy/CjGOkTSPa7ISlJljMe0Lb+kq Jbu4AXb26BvSi4GUvKD32WWmNxShxfPILK8oScR0= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally , Jacopo Mondi , Kieran Bingham , Umang Jain Subject: [PATCH v4 06/13] libcamera: mali-c55: Add stride and size to rawConfig Date: Tue, 9 Jul 2024 15:39:06 +0100 Message-Id: <20240709143913.3276983-7-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Complete the RAW StreamConfiguration by populating the frame stride and the frame size. Set the minimum required alignment to 4 bytes as the Mali C55 ISP output expands RAW output to 16 bits and a RAW Bayer macro-pixel requires two samples to be complete. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 877a156c..28ad0172 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -367,6 +367,10 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() maxSize = rawSize; + const PixelFormatInfo &info = PixelFormatInfo::info(rawConfig->pixelFormat); + rawConfig->stride = info.stride(rawConfig->size.width, 0, 4); + rawConfig->frameSize = info.frameSize(rawConfig->size, 4); + rawConfig->setStream(const_cast(&data_->frStream_)); frPipeAvailable = false; } From patchwork Tue Jul 9 14:39:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20624 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CF329BDB1C for ; Tue, 9 Jul 2024 14:39:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 715AF63379; Tue, 9 Jul 2024 16:39:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gnOicEzu"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BFD5863372 for ; Tue, 9 Jul 2024 16:39:21 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 51E6E2395; Tue, 9 Jul 2024 16:38:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535929; bh=AYrQtemEzWv6W+Y7RBVRUOqpwN0nhxUSD2F+BaU9Ae8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gnOicEzueQK3hPKknG0qtslBoSEpGnVqZDKEqIUZPv3vrxGkYy7LWVbK3sOLNQPwR QZuZfqmxhg89uI1Ny0d4EsA12Z+z6Tk+nf1GY4tTvbJ5DS6yKtn/8Sgyrsgq7LAR1V Lg49JRMjCfU64h6egNgQY9P+yJX9cEXhneVayZT4= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v4 07/13] include: media-bus-formats: Add Mali-C55 mbus codes Date: Tue, 9 Jul 2024 15:39:07 +0100 Message-Id: <20240709143913.3276983-8-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add media bus codes introduced by the Mali C55 ISP support to describe the 20-bit input formats supported by the ISP. Add the following formats - MEDIA_BUS_FMT_RGB202020_1X60 for processed input formats https://lore.kernel.org/linux-media/20240529152858.183799-2-dan.scally@ideasonboard.com/ - MEDIA_BUS_FMT_SBGGR20_1X20 MEDIA_BUS_FMT_SGBRG20_1X20 MEDIA_BUS_FMT_SGRBG20_1X20 MEDIA_BUS_FMT_SRGGB20_1X20 for the RAW bayer input format https://lore.kernel.org/linux-media/20240529152858.183799-3-dan.scally@ideasonboard.com/ Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- include/linux/media-bus-format.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/media-bus-format.h b/include/linux/media-bus-format.h index d4c1d991..b6acf8c8 100644 --- a/include/linux/media-bus-format.h +++ b/include/linux/media-bus-format.h @@ -34,7 +34,7 @@ #define MEDIA_BUS_FMT_FIXED 0x0001 -/* RGB - next is 0x1026 */ +/* RGB - next is 0x1027 */ #define MEDIA_BUS_FMT_RGB444_1X12 0x1016 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_BE 0x1001 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_LE 0x1002 @@ -72,6 +72,7 @@ #define MEDIA_BUS_FMT_RGB888_1X36_CPADLO 0x1021 #define MEDIA_BUS_FMT_RGB121212_1X36 0x1019 #define MEDIA_BUS_FMT_RGB161616_1X48 0x101a +#define MEDIA_BUS_FMT_RGB202020_1X60 0x1026 /* YUV (including grey) - next is 0x202f */ #define MEDIA_BUS_FMT_Y8_1X8 0x2001 @@ -121,7 +122,7 @@ #define MEDIA_BUS_FMT_YUV16_1X48 0x202a #define MEDIA_BUS_FMT_UYYVYY16_0_5X48 0x202b -/* Bayer - next is 0x3021 */ +/* Bayer - next is 0x3025 */ #define MEDIA_BUS_FMT_SBGGR8_1X8 0x3001 #define MEDIA_BUS_FMT_SGBRG8_1X8 0x3013 #define MEDIA_BUS_FMT_SGRBG8_1X8 0x3002 @@ -154,6 +155,10 @@ #define MEDIA_BUS_FMT_SGBRG16_1X16 0x301e #define MEDIA_BUS_FMT_SGRBG16_1X16 0x301f #define MEDIA_BUS_FMT_SRGGB16_1X16 0x3020 +#define MEDIA_BUS_FMT_SBGGR20_1X20 0x3021 +#define MEDIA_BUS_FMT_SGBRG20_1X20 0x3022 +#define MEDIA_BUS_FMT_SGRBG20_1X20 0x3023 +#define MEDIA_BUS_FMT_SRGGB20_1X20 0x3024 /* JPEG compressed formats - next is 0x4002 */ #define MEDIA_BUS_FMT_JPEG_1X8 0x4001 From patchwork Tue Jul 9 14:39:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20625 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E41DEBDB1C for ; Tue, 9 Jul 2024 14:39:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 94A876338B; Tue, 9 Jul 2024 16:39:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OpsOCB5e"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1D43163377 for ; Tue, 9 Jul 2024 16:39:22 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9CBC116C3; Tue, 9 Jul 2024 16:38:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535929; bh=M4ZBq0GeQnRpQ/4MRxsuA/nnG+GKhfxcRJzivycYOvs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OpsOCB5eCda3SJNQgXxwcGf3nZrx24mdz6Z3XW156t5BFEeGASCvwnY+5XUtNYFdJ w19KU75Dym3YrCnhvUQrzSchC6x9FqOZ6mcEJ3kQOgPDKtRmKLycnMcgl6/n1MWoNE TuTU6Ao6UxOGoBRaE7Hy+CJ/kUQfjagN/SagMIdY= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v4 08/13] libcamera: v4l2-subdevice: Add Mali C55 media bus formats Date: Tue, 9 Jul 2024 15:39:08 +0100 Message-Id: <20240709143913.3276983-9-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add support in the mediaBusFormatInfo map in the v4l2-subdevice.c file for the media bus formats used by the Mali C55 ISP. Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- src/libcamera/v4l2_subdevice.cpp | 42 ++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 82824433..6f154caf 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -189,6 +189,20 @@ const std::map mediaBusFormatInfo{ .bitsPerPixel = 24, .colourEncoding = PixelFormatInfo::ColourEncodingRGB, } }, + { MEDIA_BUS_FMT_RGB121212_1X36, { + .name = "RGB121212_1X36", + .code = MEDIA_BUS_FMT_RGB121212_1X36, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 36, + .colourEncoding = PixelFormatInfo::ColourEncodingRGB, + } }, + { MEDIA_BUS_FMT_RGB202020_1X60, { + .name = "RGB202020_1X60", + .code = MEDIA_BUS_FMT_RGB202020_1X60, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 60, + .colourEncoding = PixelFormatInfo::ColourEncodingRGB, + } }, { MEDIA_BUS_FMT_ARGB8888_1X32, { .name = "ARGB8888_1X32", .code = MEDIA_BUS_FMT_ARGB8888_1X32, @@ -679,6 +693,34 @@ const std::map mediaBusFormatInfo{ .bitsPerPixel = 16, .colourEncoding = PixelFormatInfo::ColourEncodingRAW } }, + { MEDIA_BUS_FMT_SBGGR20_1X20, { + .name = "SBGGR20_1X20", + .code = MEDIA_BUS_FMT_SBGGR20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, + { MEDIA_BUS_FMT_SGBRG20_1X20, { + .name = "SGBRG20_1X20", + .code = MEDIA_BUS_FMT_SGBRG20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, + { MEDIA_BUS_FMT_SGRBG20_1X20, { + .name = "SGRBG20_1X20", + .code = MEDIA_BUS_FMT_SGRBG20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, + { MEDIA_BUS_FMT_SRGGB20_1X20, { + .name = "SRGGB20_1X20", + .code = MEDIA_BUS_FMT_SRGGB20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, /* \todo Clarify colour encoding for HSV formats */ { MEDIA_BUS_FMT_AHSV8888_1X32, { .name = "AHSV8888_1X32", From patchwork Tue Jul 9 14:39:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20627 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 649CCBDB1C for ; Tue, 9 Jul 2024 14:39:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C74A6338A; Tue, 9 Jul 2024 16:39:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tyVKEFdE"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5DAD363379 for ; Tue, 9 Jul 2024 16:39:22 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E773D13D7; Tue, 9 Jul 2024 16:38:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535930; bh=/OxeJWjN+CSvgbaccenY2VnzeNmDSr0IY9wjvD2Csss=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tyVKEFdETa1RxC1Rdz8GtpsFsfQdDvtry5CWcc6vHDg6bV6uT2Mn0xGcZa5dqcYku s8FHl+qd/EdBNPW0F4xuPGkfEYqUV5KsM0aycJ93Aqec6pVxYNFwMXiUtgi06pvrOd rLH/lkdilZ+c/7NLC067Au5ss4iAFOtFb+VrpNHY= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v4 09/13] libcamera: mali-c55: Propagate CSI-2 format to ISP Date: Tue, 9 Jul 2024 15:39:09 +0100 Message-Id: <20240709143913.3276983-10-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The latest version of the Mali C55 driver has changed the format accepted by the ISP sink pad to be the 20-bit wide. The CSI-2 receiver handles the format expansion internally by propagating the sensor produced format from its sink to the 20-bit expanded version on its source pad. Instead of re-applying the sensor format to the CSI-2 receiver source pad (which is now an invalid operation) read from there the format propagated by the driver internally and further propagate it to the ISP subdevice. Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Latest version of the Mali C55 support https://lore.kernel.org/linux-media/20240529152858.183799-1-dan.scally@ideasonboard.com/ --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 28ad0172..156560c1 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -768,7 +768,7 @@ int PipelineHandlerMaliC55::configure(Camera *camera, if (ret) return ret; - ret = data->csi_->setFormat(1, &subdevFormat); + ret = data->csi_->getFormat(1, &subdevFormat); if (ret) return ret; } From patchwork Tue Jul 9 14:39:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20626 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0F80ABDB1C for ; Tue, 9 Jul 2024 14:39:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7965C63379; Tue, 9 Jul 2024 16:39:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bY93LNdB"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9BA086337B for ; Tue, 9 Jul 2024 16:39:22 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3DF6816C3; Tue, 9 Jul 2024 16:38:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535930; bh=nVK7tK9zsAMVY3xmZMWkvx3KQmv7zU4kI128EiCnYLo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bY93LNdBI929HNMBbqo8OzGR1KzxY7XOEl7sNo4yAF4RKIIdtCwIhG+gUwnkJ/lgL 2VPT9iL/dW/yJ3qhAqTYhRvMb60wgMK+owFsNx8Dyidg0VQIG3O2R6gD2mwmudgyZ6 vwS/WiIYFxHzq9yROB7qEN8wzsDItC9BNnhGVFTg= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally Subject: [PATCH v4 10/13] libcamera: mali-c55: Correct input/output format representation Date: Tue, 9 Jul 2024 15:39:10 +0100 Message-Id: <20240709143913.3276983-11-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" At present we configure raw streams by looping through the pixel formats we support and finding one with an associated media bus format code that the sensor can produce. In the new representation of raw data from the kernel driver this will not work - the sensor could produce 8, 10, 12, 14 or 16 bit data and the ISP will force it to RAW16, which is the only actually supported output. To fix the issue move to simply finding a pixel format with a bayer order that matches that of the media bus format produced by the sensor. If the sensor can produce multiple formats with the same bayer order use the one with the largest bitdepth. Finally, remove the claim to support RAW formats of less than 16 bits. Signed-off-by: Daniel Scally Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 121 +++++++++++-------- 1 file changed, 73 insertions(+), 48 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 156560c1..c456a798 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -57,22 +57,6 @@ const std::map maliC55FmtToCode = { { formats::NV21, MEDIA_BUS_FMT_YUV10_1X30 }, /* RAW formats, FR pipe only. */ - { formats::SGBRG8, MEDIA_BUS_FMT_SGBRG8_1X8 }, - { formats::SRGGB8, MEDIA_BUS_FMT_SRGGB8_1X8 }, - { formats::SBGGR8, MEDIA_BUS_FMT_SBGGR8_1X8 }, - { formats::SGRBG8, MEDIA_BUS_FMT_SGRBG8_1X8 }, - { formats::SGBRG10, MEDIA_BUS_FMT_SGBRG10_1X10 }, - { formats::SRGGB10, MEDIA_BUS_FMT_SRGGB10_1X10 }, - { formats::SBGGR10, MEDIA_BUS_FMT_SBGGR10_1X10 }, - { formats::SGRBG10, MEDIA_BUS_FMT_SGRBG10_1X10 }, - { formats::SGBRG12, MEDIA_BUS_FMT_SGBRG12_1X12 }, - { formats::SRGGB12, MEDIA_BUS_FMT_SRGGB12_1X12 }, - { formats::SBGGR12, MEDIA_BUS_FMT_SBGGR12_1X12 }, - { formats::SGRBG12, MEDIA_BUS_FMT_SGRBG12_1X12 }, - { formats::SGBRG14, MEDIA_BUS_FMT_SGBRG14_1X14 }, - { formats::SRGGB14, MEDIA_BUS_FMT_SRGGB14_1X14 }, - { formats::SBGGR14, MEDIA_BUS_FMT_SBGGR14_1X14 }, - { formats::SGRBG14, MEDIA_BUS_FMT_SGRBG14_1X14 }, { formats::SGBRG16, MEDIA_BUS_FMT_SGBRG16_1X16 }, { formats::SRGGB16, MEDIA_BUS_FMT_SRGGB16_1X16 }, { formats::SBGGR16, MEDIA_BUS_FMT_SBGGR16_1X16 }, @@ -98,7 +82,8 @@ public: const std::vector sizes(unsigned int mbusCode) const; const Size resolution() const; - PixelFormat bestRawFormat() const; + int pixfmtToMbusCode(const PixelFormat &pixFmt) const; + const PixelFormat &bestRawFormat() const; PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; @@ -208,33 +193,75 @@ const Size MaliC55CameraData::resolution() const return tpgResolution_; } -PixelFormat MaliC55CameraData::bestRawFormat() const +/* + * The Mali C55 ISP can only produce 16-bit RAW output in bypass modes, but the + * sensors connected to it might produce 8/10/12/16 bits. We simply search the + * sensor's supported formats for the one with a matching bayer order and the + * greatest bitdepth. + */ +int MaliC55CameraData::pixfmtToMbusCode(const PixelFormat &pixFmt) const { + auto it = maliC55FmtToCode.find(pixFmt); + if (it == maliC55FmtToCode.end()) + return -EINVAL; + + BayerFormat bayerFormat = BayerFormat::fromMbusCode(it->second); + if (!bayerFormat.isValid()) + return -EINVAL; + + V4L2Subdevice::Formats formats = sd_->formats(0); + unsigned int sensorMbusCode = 0; unsigned int bitDepth = 0; - PixelFormat rawFormat; - /* - * Iterate over all the supported PixelFormat and find the one - * supported by the camera with the largest bitdepth. - */ - for (const auto &maliFormat : maliC55FmtToCode) { - PixelFormat pixFmt = maliFormat.first; - if (!isFormatRaw(pixFmt)) + for (const auto &[code, sizes] : formats) { + BayerFormat sdBayerFormat = BayerFormat::fromMbusCode(code); + if (!sdBayerFormat.isValid()) continue; - unsigned int rawCode = maliFormat.second; - const auto rawSizes = sizes(rawCode); - if (rawSizes.empty()) + if (sdBayerFormat.order != bayerFormat.order) continue; - BayerFormat bayer = BayerFormat::fromMbusCode(rawCode); - if (bayer.bitDepth > bitDepth) { - bitDepth = bayer.bitDepth; - rawFormat = pixFmt; + if (sdBayerFormat.bitDepth > bitDepth) { + bitDepth = sdBayerFormat.bitDepth; + sensorMbusCode = code; } } - return rawFormat; + if (!sensorMbusCode) + return -EINVAL; + + return sensorMbusCode; +} + +/* + * Find a RAW PixelFormat supported by both the ISP and the sensor. + * + * The situation is mildly complicated by the fact that we expect the sensor to + * output something like RAW8/10/12/16, but the ISP can only accept as input + * RAW20 and can only produce as output RAW16. The one constant in that is the + * bayer order of the data, so we'll simply check that the sensor produces a + * format with a bayer order that matches that of one of the formats we support, + * and select that. + */ +const PixelFormat &MaliC55CameraData::bestRawFormat() const +{ + static const PixelFormat invalidPixFmt = {}; + + for (const auto &fmt : sd_->formats(0)) { + BayerFormat sensorBayer = BayerFormat::fromMbusCode(fmt.first); + + for (const auto &[pixFmt, rawCode] : maliC55FmtToCode) { + if (!isFormatRaw(pixFmt)) + continue; + + BayerFormat bayer = BayerFormat::fromMbusCode(rawCode); + if (bayer.order == sensorBayer.order) + return pixFmt; + } + } + + LOG(MaliC55, Fatal) << "Sensor doesn't provide a compatible format"; + return invalidPixFmt; } /* @@ -243,13 +270,11 @@ PixelFormat MaliC55CameraData::bestRawFormat() const */ PixelFormat MaliC55CameraData::adjustRawFormat(const PixelFormat &rawFmt) const { - /* Make sure the provided raw format is supported by the pipeline. */ - auto it = maliC55FmtToCode.find(rawFmt); - if (it == maliC55FmtToCode.end()) + /* Make sure the RAW mbus code is supported by the image source. */ + int rawCode = pixfmtToMbusCode(rawFmt); + if (rawCode < 0) return bestRawFormat(); - /* Now make sure the RAW mbus code is supported by the image source. */ - unsigned int rawCode = it->second; const auto rawSizes = sizes(rawCode); if (rawSizes.empty()) return bestRawFormat(); @@ -259,16 +284,14 @@ PixelFormat MaliC55CameraData::adjustRawFormat(const PixelFormat &rawFmt) const Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &size) const { - /* Just make sure the format is supported. */ - auto it = maliC55FmtToCode.find(rawFmt); - if (it == maliC55FmtToCode.end()) - return {}; - /* Expand the RAW size to the minimum ISP input size. */ Size rawSize = size.expandedTo(kMaliC55MinInputSize); /* Check if the size is natively supported. */ - unsigned int rawCode = it->second; + int rawCode = pixfmtToMbusCode(rawFmt); + if (rawCode < 0) + return {}; + const auto rawSizes = sizes(rawCode); auto sizeIt = std::find(rawSizes.begin(), rawSizes.end(), rawSize); if (sizeIt != rawSizes.end()) @@ -418,8 +441,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() /* If there's a RAW config, sensor configuration follows it. */ if (rawConfig) { - const auto it = maliC55FmtToCode.find(rawConfig->pixelFormat); - sensorFormat_.code = it->second; + sensorFormat_.code = data_->pixfmtToMbusCode(rawConfig->pixelFormat); sensorFormat_.size = rawConfig->size.expandedTo(minSensorSize); return status; @@ -614,7 +636,10 @@ PipelineHandlerMaliC55::generateConfiguration(Camera *camera, if (isRaw) { /* Make sure the mbus code is supported. */ - unsigned int rawCode = maliFormat.second; + int rawCode = data->pixfmtToMbusCode(pixFmt); + if (rawCode < 0) + continue; + const auto sizes = data->sizes(rawCode); if (sizes.empty()) continue; From patchwork Tue Jul 9 14:39:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20628 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C4763BDB1C for ; Tue, 9 Jul 2024 14:39:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0203B6338C; Tue, 9 Jul 2024 16:39:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="LcOiVyyO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D35A86337C for ; Tue, 9 Jul 2024 16:39:22 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 763692393; Tue, 9 Jul 2024 16:38:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535930; bh=7HjZW+B3x78J3wRUkUtBFxMgqtv/VD1c96N6Ydcnhek=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LcOiVyyOWHPL69QN5Qdn0m6vvhRwUh1ujYaT9HH8l4Ib/T/X2duVDn4L505ajjrA1 olsXyEDdmHMu2FXXB1SfREPA3u/7Uwk1Jh3uXwmpCPy26kg6vfiioUlUWhckRZiuh+ 4iuBDY9LZgCuNeFAmvcK7MkxdXDUUSd/qibAUuNs= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Daniel Scally Subject: [PATCH v4 11/13] libcamera: mali-c55: Enable links between resizer and video node Date: Tue, 9 Jul 2024 15:39:11 +0100 Message-Id: <20240709143913.3276983-12-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The mali-c55 driver now expects links to video devices to be enabled in order for those devices to be streamed from / to. Enable the media link between the resizers and their associated video device to fulfil the requirement. Signed-off-by: Daniel Scally Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index c456a798..512a7489 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -827,6 +827,17 @@ int PipelineHandlerMaliC55::configure(Camera *camera, Stream *stream = streamConfig.stream(); MaliC55Pipe *pipe = pipeFromStream(data, stream); + /* + * Enable the media link between the pipe's resizer and the + * capture video device + */ + const MediaEntity *rszEntity = pipe->resizer->entity(); + ret = rszEntity->getPadByIndex(1)->links()[0]->setEnabled(true); + if (ret) { + LOG(MaliC55, Error) << "Couldn't enable resizer's link"; + return ret; + } + if (isFormatRaw(streamConfig.pixelFormat)) ret = configureRawStream(data, streamConfig, subdevFormat); else From patchwork Tue Jul 9 14:39:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20629 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0E272BDB1C for ; Tue, 9 Jul 2024 14:39:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5BA4163393; Tue, 9 Jul 2024 16:39:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oqC4IbQP"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1EABB63375 for ; Tue, 9 Jul 2024 16:39:23 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AFDF913D7; Tue, 9 Jul 2024 16:38:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535930; bh=Z+fg35l1hrsPQMFJ/K/TidYVBhhjsTJQBMhmue70dFQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oqC4IbQP02IzZnQbY68+fQN1Vxar4670+TmUxJ/VALZ/VQdS9dHeLxFW0QABcWc/m 7rAybTgovlx8BO5jKfnkD5up/L8IJ6jcguvvCFNRwl8Ev2sKX2DM42aqhtUrVz7h72 beod4+/aFZS2q1NJXgm0RLPZU94vOwSzPeaEZ1oA= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally Subject: [PATCH v4 12/13] libcamera: mali-c55: Enable usage of scaler Date: Tue, 9 Jul 2024 15:39:12 +0100 Message-Id: <20240709143913.3276983-13-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The Mali C55 ISP has a resizing pipeline that allows to crop and scale images. So far the mali-c55 pipeline has only supported cropping without using the scaling functionalities. Now that the kernel has gained support for the scaling operations, make the libcamera pipeline use it by combining it with a first cropping step to align the input and output images FOV ratio, and then scale to the desired output size. Reviewed-by: Daniel Scally Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 512a7489..f5ca2ca4 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -747,16 +747,28 @@ int PipelineHandlerMaliC55::configureProcessedStream(MaliC55CameraData *data, if (ret) return ret; - /* \todo Configure the resizer crop/compose rectangles. */ - Rectangle ispCrop = { 0, 0, config.size }; + /* + * Compute the scaler-in to scaler-out ratio: first center-crop to align + * the FOV to the desired resolution, then scale to the desired size. + */ + Size scalerIn = subdevFormat.size.boundedToAspectRatio(config.size); + int xCrop = (subdevFormat.size.width - scalerIn.width) / 2; + int yCrop = (subdevFormat.size.height - scalerIn.height) / 2; + Rectangle ispCrop = { xCrop, yCrop, scalerIn }; ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_CROP, &ispCrop); if (ret) return ret; - ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_COMPOSE, &ispCrop); + Rectangle ispCompose = { 0, 0, config.size }; + ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_COMPOSE, &ispCompose); if (ret) return ret; + /* + * The source pad format size comes directly from the sink + * compose rectangle. + */ + subdevFormat.size = ispCompose.size(); subdevFormat.code = maliC55FmtToCode.find(config.pixelFormat)->second; return pipe->resizer->setFormat(1, &subdevFormat); } From patchwork Tue Jul 9 14:39:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 20630 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 10CAAC328C for ; Tue, 9 Jul 2024 14:39:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 37F7D63394; Tue, 9 Jul 2024 16:39:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gWvjBNy6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 67D00619C1 for ; Tue, 9 Jul 2024 16:39:23 +0200 (CEST) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EEBC616C3; Tue, 9 Jul 2024 16:38:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720535931; bh=PoTohEHTQjw6sCpFMMHR7W2xyEIvLtoKtNezhRH0VEY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gWvjBNy6EXFb5wehAF0KteYtzCoVJhq+eML4dPECvLDf3xka9srCqBQy3EdMf01yT gwOUnFPr0kKS03sk4T2M6h6vNqQ/g1jpZiF9Yx1uOZWFsO/y08igspcAqWb8qDpTcS kIreohkEl9Xx4hzIoKalDakGvtfKQS9zV+UqONmo= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Daniel Scally Subject: [PATCH v4 13/13] mali-c55: implement support for ScalerCrop Date: Tue, 9 Jul 2024 15:39:13 +0100 Message-Id: <20240709143913.3276983-14-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240709143913.3276983-1-dan.scally@ideasonboard.com> References: <20240709143913.3276983-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Implement support for the ScalerCrop control that allows to apply a digital zoom to the captured streams. Initialize the camera controls at camera registration time and update them at configure time as the sensor's analogue crop size might change depending on the desired Camera configuration. Reviewed-by: Daniel Scally Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 133 +++++++++++++++++++ 1 file changed, 133 insertions(+) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index f5ca2ca4..916b1d30 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -85,6 +85,8 @@ public: int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; + void updateControls(); + PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; @@ -264,6 +266,27 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const return invalidPixFmt; } +void MaliC55CameraData::updateControls() +{ + if (!sensor_) + return; + + IPACameraSensorInfo sensorInfo; + int ret = sensor_->sensorInfo(&sensorInfo); + if (ret) { + LOG(MaliC55, Error) << "Failed to retrieve sensor info"; + return; + } + + ControlInfoMap::Map controls; + Rectangle ispMinCrop{ 0, 0, 640, 480 }; + controls[&controls::ScalerCrop] = + ControlInfo(ispMinCrop, sensorInfo.analogCrop, + sensorInfo.analogCrop); + + controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); +} + /* * Make sure the provided raw pixel format is supported and adjust it to * one of the supported ones if it's not. @@ -544,6 +567,8 @@ private: const StreamConfiguration &config, V4L2SubdeviceFormat &subdevFormat); + void applyScalerCrop(Camera *camera, const ControlList &controls); + void registerMaliCamera(std::unique_ptr data, const std::string &name); bool registerTPGCamera(MediaLink *link); @@ -871,6 +896,8 @@ int PipelineHandlerMaliC55::configure(Camera *camera, pipe->stream = stream; } + data->updateControls(); + return 0; } @@ -918,6 +945,102 @@ void PipelineHandlerMaliC55::stopDevice([[maybe_unused]] Camera *camera) } } +void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, + const ControlList &controls) +{ + MaliC55CameraData *data = cameraData(camera); + + const auto &scalerCrop = controls.get(controls::ScalerCrop); + if (!scalerCrop) + return; + + if (!data->sensor_) { + LOG(MaliC55, Error) << "ScalerCrop not supported for TPG"; + return; + } + + Rectangle nativeCrop = *scalerCrop; + + IPACameraSensorInfo sensorInfo; + int ret = data->sensor_->sensorInfo(&sensorInfo); + if (ret) { + LOG(MaliC55, Error) << "Failed to retrieve sensor info"; + return; + } + + /* + * The ScalerCrop rectangle re-scaling in the ISP crop rectangle + * comes straight from the RPi pipeline handler. + * + * Create a version of the crop rectangle aligned to the analogue crop + * rectangle top-left coordinates and scaled in the [analogue crop to + * output frame] ratio to take into account binning/skipping on the + * sensor. + */ + Rectangle ispCrop = nativeCrop.translatedBy(-sensorInfo.analogCrop + .topLeft()); + ispCrop.scaleBy(sensorInfo.outputSize, sensorInfo.analogCrop.size()); + + /* + * The crop rectangle should be: + * 1. At least as big as ispMinCropSize_, once that's been + * enlarged to the same aspect ratio. + * 2. With the same mid-point, if possible. + * 3. But it can't go outside the sensor area. + */ + Rectangle ispMinCrop{ 0, 0, 640, 480 }; + Size minSize = ispMinCrop.size().expandedToAspectRatio(nativeCrop.size()); + Size size = ispCrop.size().expandedTo(minSize); + ispCrop = size.centeredTo(ispCrop.center()) + .enclosedIn(Rectangle(sensorInfo.outputSize)); + + /* + * As the resizer can't upscale, the crop rectangle has to be larger + * than the larger stream output size. + */ + Size maxYuvSize; + for (MaliC55Pipe &pipe : pipes_) { + if (!pipe.stream) + continue; + + const StreamConfiguration &config = pipe.stream->configuration(); + if (isFormatRaw(config.pixelFormat)) { + LOG(MaliC55, Debug) << "Cannot crop with a RAW stream"; + return; + } + + Size streamSize = config.size; + if (streamSize.width > maxYuvSize.width) + maxYuvSize.width = streamSize.width; + if (streamSize.height > maxYuvSize.height) + maxYuvSize.height = streamSize.height; + } + + ispCrop.size().expandTo(maxYuvSize); + + /* + * Now apply the scaler crop to each enabled output. This overrides the + * crop configuration performed at configure() time and can cause + * square pixels if the crop rectangle and scaler output FOV ratio are + * different. + */ + for (MaliC55Pipe &pipe : pipes_) { + if (!pipe.stream) + continue; + + /* Create a copy to avoid setSelection() to modify ispCrop. */ + Rectangle pipeCrop = ispCrop; + ret = pipe.resizer->setSelection(0, V4L2_SEL_TGT_CROP, &pipeCrop); + if (ret) { + LOG(MaliC55, Error) + << "Failed to apply crop to " + << (pipe.stream == &data->frStream_ ? + "FR" : "DS") << " pipe"; + return; + } + } +} + int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { int ret; @@ -930,6 +1053,15 @@ int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) return ret; } + /* + * Some controls need to be applied immediately, as in example, + * the ScalerCrop one. + * + * \todo Move it buffer queue time (likely after the IPA has filled in + * the parameters buffer) once we have plumbed the IPA loop in. + */ + applyScalerCrop(camera, request->controls()); + return 0; } @@ -1005,6 +1137,7 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return false; data->properties_ = data->sensor_->properties(); + data->updateControls(); registerMaliCamera(std::move(data), sensor->name()); }