From patchwork Fri Nov 15 10:13:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21904 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0A356C0F1B for ; Fri, 15 Nov 2024 10:13:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4E6A565877; Fri, 15 Nov 2024 11:13:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VlsvYGck"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 24CF365860 for ; Fri, 15 Nov 2024 11:13:48 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0E52CE0D; Fri, 15 Nov 2024 11:13:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665613; bh=cinsoAfR3z1VnpqG5w1re9WpRuIu3VLCiSwau/9of+g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VlsvYGckIpJbLzdfSesW96L1NLUFhTY5zhyPygh8jFsYXVgqkoVsrEI9UFtl7/Xnq 1sfUUKGCY2tbcA9pFgqEQjymFXkT1399sZGRMqt/kfZjz5sg7XTI+g4oVBmH9M3L3n HVXaHBMKyAPXD8lxpBzpRjrBWWH7x2x4CeRbCvRM= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Jacopo Mondi , Daniel Scally , Umang Jain , Kieran Bingham Subject: [PATCH v6 01/14] libcamera: mali-c55: Limit max size to sensor resolution Date: Fri, 15 Nov 2024 10:13:21 +0000 Message-Id: <20241115101334.453104-2-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The Mali C55 ISP cannot upscale. The maximum allowed output size is the sensor's resolution. For RAW streams this is already handled in adjustRawSizes(), while for processed streams the maximum allowed resolution was wrongly set to the ISP maximum output size (8192x8192). Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v6: - None Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index e40025b4..662b34ef 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -341,7 +341,11 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() rawConfig = &config; } - Size maxSize = kMaliC55MaxSize; + /* + * The C55 can not upscale. Limit the configuration to the ISP + * capabilities and the sensor resolution. + */ + Size maxSize = kMaliC55MaxSize.boundedTo(data_->resolution()); if (rawConfig) { /* * \todo Take into account the Bayer components ordering once From patchwork Fri Nov 15 10:13:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21905 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6FCBCC0F1B for ; Fri, 15 Nov 2024 10:13:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 05FC865880; Fri, 15 Nov 2024 11:13:55 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="C3vxoET3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 58F3765865 for ; Fri, 15 Nov 2024 11:13:48 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7559CFC5; Fri, 15 Nov 2024 11:13:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665613; bh=K1sbtq5yxj+kwESVaQF000/mTHHODuQSKMpDcF6YARw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C3vxoET3At3EXmHP+GdaZinOi8gWR1m6e2nKuyjfWv9tqcP88HeQMGJNaKBwyZGzw 5zJcL1kFNM1sY9icZLXO6ThmVMTqRYOghQC810eioQnJhjOkr/pOv0M0X01uJzXP3B sLsvz61WumZotEp2gjLTfsiWQGQ/tz45ttv1xaCg= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v6 02/14] libcamera: mali-c55: Limit ISP input size Date: Fri, 15 Nov 2024 10:13:22 +0000 Message-Id: <20241115101334.453104-3-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The Mali-C55 ISP has a minimum input size limit of 640x480. Filter out resolutions smaller than this when selecting the sensor format. While at it, rename 'maxYuvSize' to a more appropriate 'minSensorSize'. Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v6: - None Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 38 ++++++++++++-------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 662b34ef..44f56ab7 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -79,6 +79,7 @@ const std::map maliC55FmtToCode = { { formats::SGRBG16, MEDIA_BUS_FMT_SGRBG16_1X16 }, }; +constexpr Size kMaliC55MinInputSize = { 640, 480 }; constexpr Size kMaliC55MinSize = { 128, 128 }; constexpr Size kMaliC55MaxSize = { 8192, 8192 }; constexpr unsigned int kMaliC55ISPInternalFormat = MEDIA_BUS_FMT_RGB121212_1X36; @@ -264,13 +265,16 @@ PixelFormat MaliC55CameraData::adjustRawFormat(const PixelFormat &rawFmt) const return rawFmt; } -Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &rawSize) const +Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &size) const { /* Just make sure the format is supported. */ auto it = maliC55FmtToCode.find(rawFmt); if (it == maliC55FmtToCode.end()) return {}; + /* Expand the RAW size to the minimum ISP input size. */ + Size rawSize = size.expandedTo(kMaliC55MinInputSize); + /* Check if the size is natively supported. */ unsigned int rawCode = it->second; const auto rawSizes = sizes(rawCode); @@ -281,14 +285,14 @@ Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &ra /* Or adjust it to the closest supported size. */ uint16_t distance = std::numeric_limits::max(); Size bestSize; - for (const Size &size : rawSizes) { + for (const Size &sz : rawSizes) { uint16_t dist = std::abs(static_cast(rawSize.width) - - static_cast(size.width)) + + static_cast(sz.width)) + std::abs(static_cast(rawSize.height) - - static_cast(size.height)); + static_cast(sz.height)); if (dist < distance) { dist = distance; - bestSize = size; + bestSize = sz; } } @@ -375,8 +379,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() frPipeAvailable = false; } - /* Adjust processed streams. */ - Size maxYuvSize; + /* + * Adjust processed streams. + * + * Compute the minimum sensor size to be later used to select the + * sensor configuration. + */ + Size minSensorSize = kMaliC55MinInputSize; for (StreamConfiguration &config : config_) { if (isFormatRaw(config.pixelFormat)) continue; @@ -398,8 +407,8 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() status = Adjusted; } - if (maxYuvSize < size) - maxYuvSize = size; + if (minSensorSize < size) + minSensorSize = size; if (frPipeAvailable) { config.setStream(const_cast(&data_->frStream_)); @@ -415,7 +424,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() if (rawConfig) { const auto it = maliC55FmtToCode.find(rawConfig->pixelFormat); sensorFormat_.code = it->second; - sensorFormat_.size = rawConfig->size; + sensorFormat_.size = rawConfig->size.expandedTo(minSensorSize); return status; } @@ -429,14 +438,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() const auto sizes = data_->sizes(it->second); Size bestSize; for (const auto &size : sizes) { - /* Skip sensor sizes that are smaller than the max YUV size. */ - if (maxYuvSize.width > size.width || - maxYuvSize.height > size.height) + if (minSensorSize.width > size.width || + minSensorSize.height > size.height) continue; - uint16_t dist = std::abs(static_cast(maxYuvSize.width) - + uint16_t dist = std::abs(static_cast(minSensorSize.width) - static_cast(size.width)) + - std::abs(static_cast(maxYuvSize.height) - + std::abs(static_cast(minSensorSize.height) - static_cast(size.height)); if (dist < distance) { dist = distance; From patchwork Fri Nov 15 10:13:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21907 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 48A79C3261 for ; Fri, 15 Nov 2024 10:14:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 73A046587D; Fri, 15 Nov 2024 11:13:58 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bauTVhlG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8B9D86586A for ; Fri, 15 Nov 2024 11:13:48 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D2A0D9CE; Fri, 15 Nov 2024 11:13:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665614; bh=VpZC8PTz+1Nj554gevqPvU57r63g+PiZwc9DzX0/NkM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bauTVhlG7kCu5Re1FRELo38R6oYsEDk0emK96NX3qB7XcOFK8Vio4ywa/OwKMIy87 pTnHkPk7TXi7ffOR46Yc9pQKn1EQAmNmSmhwqGTBPo7NzO9jkEavO4/qhQfTXfaSDy XKastdLXwHCYFREHULZem7WJsza2nZLSD6cuts+4= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Daniel Scally , Umang Jain , Kieran Bingham , Jacopo Mondi Subject: [PATCH v6 03/14] libcamera: mali-c55: Init camera properties Date: Fri, 15 Nov 2024 10:13:23 +0000 Message-Id: <20241115101334.453104-4-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Initialise the camera properties using the CameraSensor properties. Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v6: - None Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 44f56ab7..80cd98b3 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -964,7 +964,7 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) if (data->init()) return false; - /* \todo: Init properties and controls. */ + data->properties_ = data->sensor_->properties(); registerMaliCamera(std::move(data), sensor->name()); } From patchwork Fri Nov 15 10:13:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21908 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CB0C9C32CE for ; Fri, 15 Nov 2024 10:14:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9B47E65886; Fri, 15 Nov 2024 11:13:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PFQRkBPT"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DFF036586C for ; Fri, 15 Nov 2024 11:13:48 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 43EE21114; Fri, 15 Nov 2024 11:13:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665614; bh=hquK9PGF8VK4hnbDPiF0tdxZq2zhcxSdUi1q/tVBgd8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PFQRkBPTXVPjd4poh8nxpT2ji7Y0qitZGIt9Fsm3H9uZcvVA/Lwv3fqM+0RwV4zHN QX6dtX1A8JzxzO4ckq3qg3xVKiJuaJV9+TI9CHDMQisYB+y/Tfsq0wBKAEaQKDzVoz K7nSHbmRUQubf3AJ1yK7tqcyLJueUDfpt70kGcWs= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Daniel Scally , Jacopo Mondi , Kieran Bingham , Laurent Pinchart Subject: [PATCH v6 04/14] libcamera: mali-c55: Simplify bufferReady() Date: Fri, 15 Nov 2024 10:13:24 +0000 Message-Id: <20241115101334.453104-5-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The PipelineHandler::completeBuffer() base class function returns a boolean to indicate if there still are pending buffers in the Request. Simplify the bufferReady() function in the Mali-C55 pipeline handler using the completeBuffer() return value. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- Changes in v6: - None Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 80cd98b3..7d0d0293 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -893,12 +893,8 @@ void PipelineHandlerMaliC55::imageBufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); - completeBuffer(request, buffer); - - if (request->hasPendingBuffers()) - return; - - completeRequest(request); + if (completeBuffer(request, buffer)) + completeRequest(request); } void PipelineHandlerMaliC55::registerMaliCamera(std::unique_ptr data, From patchwork Fri Nov 15 10:13:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21906 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9A249C0F1B for ; Fri, 15 Nov 2024 10:13:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 178B26587C; Fri, 15 Nov 2024 11:13:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oxDD6Qrk"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 502A86586F for ; Fri, 15 Nov 2024 11:13:49 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A7FE811D6; Fri, 15 Nov 2024 11:13:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665614; bh=VoE9Jga2m8ZHw6g7g/gcQavDRo1oqk7hMUEZBgf7Q5A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oxDD6QrkN4mwFgAEwwj1sTdtK4P5O8UzfWbtErIRpQ/CaNaUqtIKZjkyuM38arNpI TY26y4ps47Mo+usQ/CXtToe3FWFfBo2efTlgZvLbcpqkBae8nBvpfupvYyTWGdQo5x rB33AHSkhDWnWAwpkXhh1VW9VFNZ1jB+5BugpwP4= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Daniel Scally , Jacopo Mondi , Kieran Bingham , Laurent Pinchart Subject: [PATCH v6 05/14] libcamera: mali-c55: Remove MaliC55CameraData::mbusCodes() Date: Fri, 15 Nov 2024 10:13:25 +0000 Message-Id: <20241115101334.453104-6-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The MaliC55CameraData::mbusCodes() function is unused. Remove it. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- Changes in v6: - None Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 9 --------- 1 file changed, 9 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 7d0d0293..330006db 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -95,7 +95,6 @@ public: int init(); /* Deflect these functionalities to either TPG or CameraSensor. */ - const std::vector mbusCodes() const; const std::vector sizes(unsigned int mbusCode) const; const Size resolution() const; @@ -177,14 +176,6 @@ void MaliC55CameraData::initTPGData() tpgResolution_ = tpgSizes_.back(); } -const std::vector MaliC55CameraData::mbusCodes() const -{ - if (sensor_) - return sensor_->mbusCodes(); - - return tpgCodes_; -} - const std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const { if (sensor_) From patchwork Fri Nov 15 10:13:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21910 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 79723C3261 for ; Fri, 15 Nov 2024 10:14:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AF10A65888; Fri, 15 Nov 2024 11:14:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HfbLU0nr"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B4196600F2 for ; Fri, 15 Nov 2024 11:13:49 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 181C71340; Fri, 15 Nov 2024 11:13:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665615; bh=B9Q3EyGYlyZ0IRqms02XCEZhyCH1rICvMkeSp7dJQvQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HfbLU0nr/daNIDOLwyYNwtTYLZfBrXyyytgnhESwFmMsbVBb26/7MveDTwVt33UiP VNdcDhN3QqR45PawWmIU+UHLS01a82gdBQICvbhtaz52FUEqgDlG0NIQE4mvnGw/hL oBax9o3Qk96qXFTQOR3nrOXRv2nuJ+er106Msb/4= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Daniel Scally , Jacopo Mondi , Kieran Bingham , Umang Jain Subject: [PATCH v6 06/14] libcamera: mali-c55: Add stride and size to rawConfig Date: Fri, 15 Nov 2024 10:13:26 +0000 Message-Id: <20241115101334.453104-7-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Complete the RAW StreamConfiguration by populating the frame stride and the frame size. Set the minimum required alignment to 4 bytes as the Mali C55 ISP output expands RAW output to 16 bits and a RAW Bayer macro-pixel requires two samples to be complete. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Umang Jain --- Changes in v6: - None Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 330006db..a35aa92a 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -366,6 +366,10 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() maxSize = rawSize; + const PixelFormatInfo &info = PixelFormatInfo::info(rawConfig->pixelFormat); + rawConfig->stride = info.stride(rawConfig->size.width, 0, 4); + rawConfig->frameSize = info.frameSize(rawConfig->size, 4); + rawConfig->setStream(const_cast(&data_->frStream_)); frPipeAvailable = false; } From patchwork Fri Nov 15 10:13:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21909 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EDEECC0F1B for ; Fri, 15 Nov 2024 10:14:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4B4EF6588B; Fri, 15 Nov 2024 11:14:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GZBeXh1s"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 20C7265870 for ; Fri, 15 Nov 2024 11:13:50 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7EC0713AB; Fri, 15 Nov 2024 11:13:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665615; bh=yauI/z9OGcIQs8LvdT1Iyt3kQ6vGlDLkp0BD4Nu6sLI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GZBeXh1sr4Q5ipniUO67clOQjn0jzSEd7c6+EH3w2ZStJNp4xDQia4krEJz1UPdph pjxXu4p7IFuDSE7oPyjAjp4hypwAcYysVr1m6wX1ND8lo2jkUFEXrecLnav9oyzDlU GhSiWOhI+9RIY6kXJJLqNRhWJODwAKybVkXZa6tc= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Jacopo Mondi , Laurent Pinchart , Daniel Scally , Umang Jain Subject: [PATCH v6 07/14] include: media-bus-formats: Add Mali-C55 mbus codes Date: Fri, 15 Nov 2024 10:13:27 +0000 Message-Id: <20241115101334.453104-8-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add media bus codes introduced by the Mali C55 ISP support to describe the 20-bit input formats supported by the ISP. This manual addition is a temporary measure until the kernel driver changes are merged, from which point this will be managed through the usual merge of the upstream kernel headers. Add the following formats - MEDIA_BUS_FMT_RGB202020_1X60 for processed input formats https://lore.kernel.org/linux-media/20241106100534.768400-2-dan.scally@ideasonboard.com/ - MEDIA_BUS_FMT_SBGGR20_1X20 MEDIA_BUS_FMT_SGBRG20_1X20 MEDIA_BUS_FMT_SGRBG20_1X20 MEDIA_BUS_FMT_SRGGB20_1X20 for the RAW bayer input format https://lore.kernel.org/linux-media/20241106100534.768400-3-dan.scally@ideasonboard.com/ Reviewed-by: Laurent Pinchart Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v6: - Updated commit message Changes in v5: - None include/linux/media-bus-format.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/media-bus-format.h b/include/linux/media-bus-format.h index d4c1d991..b6acf8c8 100644 --- a/include/linux/media-bus-format.h +++ b/include/linux/media-bus-format.h @@ -34,7 +34,7 @@ #define MEDIA_BUS_FMT_FIXED 0x0001 -/* RGB - next is 0x1026 */ +/* RGB - next is 0x1027 */ #define MEDIA_BUS_FMT_RGB444_1X12 0x1016 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_BE 0x1001 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_LE 0x1002 @@ -72,6 +72,7 @@ #define MEDIA_BUS_FMT_RGB888_1X36_CPADLO 0x1021 #define MEDIA_BUS_FMT_RGB121212_1X36 0x1019 #define MEDIA_BUS_FMT_RGB161616_1X48 0x101a +#define MEDIA_BUS_FMT_RGB202020_1X60 0x1026 /* YUV (including grey) - next is 0x202f */ #define MEDIA_BUS_FMT_Y8_1X8 0x2001 @@ -121,7 +122,7 @@ #define MEDIA_BUS_FMT_YUV16_1X48 0x202a #define MEDIA_BUS_FMT_UYYVYY16_0_5X48 0x202b -/* Bayer - next is 0x3021 */ +/* Bayer - next is 0x3025 */ #define MEDIA_BUS_FMT_SBGGR8_1X8 0x3001 #define MEDIA_BUS_FMT_SGBRG8_1X8 0x3013 #define MEDIA_BUS_FMT_SGRBG8_1X8 0x3002 @@ -154,6 +155,10 @@ #define MEDIA_BUS_FMT_SGBRG16_1X16 0x301e #define MEDIA_BUS_FMT_SGRBG16_1X16 0x301f #define MEDIA_BUS_FMT_SRGGB16_1X16 0x3020 +#define MEDIA_BUS_FMT_SBGGR20_1X20 0x3021 +#define MEDIA_BUS_FMT_SGBRG20_1X20 0x3022 +#define MEDIA_BUS_FMT_SGRBG20_1X20 0x3023 +#define MEDIA_BUS_FMT_SRGGB20_1X20 0x3024 /* JPEG compressed formats - next is 0x4002 */ #define MEDIA_BUS_FMT_JPEG_1X8 0x4001 From patchwork Fri Nov 15 10:13:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21912 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4DDEFC330B for ; Fri, 15 Nov 2024 10:14:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3831E65885; Fri, 15 Nov 2024 11:14:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="boSJ+icY"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7CB1B65862 for ; Fri, 15 Nov 2024 11:13:50 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E3ABF13CC; Fri, 15 Nov 2024 11:13:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665616; bh=6dtQYcGpGaLyTQreNBero10OkdcLP5DleWd0dJFuEd4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=boSJ+icYp5DfBW2lQKMFdkOOHZdZhpkAwTwBpIJK35qNErKaelBXAPmXUyWSWyE8V p256ZCBCoJmV7aKp4p1TWKY9gWnTeJxr1J2OWDo7ZguCmWJJNRI7jPfDeqiKXbtaAr ojQrkI22J7UzX84JAxV1iykyzwhV4m47HjQbBJXM= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v6 08/14] libcamera: v4l2-subdevice: Add Mali C55 media bus formats Date: Fri, 15 Nov 2024 10:13:28 +0000 Message-Id: <20241115101334.453104-9-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Add support in the mediaBusFormatInfo map in the v4l2-subdevice.c file for the media bus formats used by the Mali C55 ISP. Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v6: - None Changes in v5: - None src/libcamera/v4l2_subdevice.cpp | 42 ++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 9f2ec479..3a0d075f 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -188,6 +188,20 @@ const std::map mediaBusFormatInfo{ .bitsPerPixel = 24, .colourEncoding = PixelFormatInfo::ColourEncodingRGB, } }, + { MEDIA_BUS_FMT_RGB121212_1X36, { + .name = "RGB121212_1X36", + .code = MEDIA_BUS_FMT_RGB121212_1X36, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 36, + .colourEncoding = PixelFormatInfo::ColourEncodingRGB, + } }, + { MEDIA_BUS_FMT_RGB202020_1X60, { + .name = "RGB202020_1X60", + .code = MEDIA_BUS_FMT_RGB202020_1X60, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 60, + .colourEncoding = PixelFormatInfo::ColourEncodingRGB, + } }, { MEDIA_BUS_FMT_ARGB8888_1X32, { .name = "ARGB8888_1X32", .code = MEDIA_BUS_FMT_ARGB8888_1X32, @@ -678,6 +692,34 @@ const std::map mediaBusFormatInfo{ .bitsPerPixel = 16, .colourEncoding = PixelFormatInfo::ColourEncodingRAW } }, + { MEDIA_BUS_FMT_SBGGR20_1X20, { + .name = "SBGGR20_1X20", + .code = MEDIA_BUS_FMT_SBGGR20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, + { MEDIA_BUS_FMT_SGBRG20_1X20, { + .name = "SGBRG20_1X20", + .code = MEDIA_BUS_FMT_SGBRG20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, + { MEDIA_BUS_FMT_SGRBG20_1X20, { + .name = "SGRBG20_1X20", + .code = MEDIA_BUS_FMT_SGRBG20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, + { MEDIA_BUS_FMT_SRGGB20_1X20, { + .name = "SRGGB20_1X20", + .code = MEDIA_BUS_FMT_SRGGB20_1X20, + .type = MediaBusFormatInfo::Type::Image, + .bitsPerPixel = 20, + .colourEncoding = PixelFormatInfo::ColourEncodingRAW + } }, /* \todo Clarify colour encoding for HSV formats */ { MEDIA_BUS_FMT_AHSV8888_1X32, { .name = "AHSV8888_1X32", From patchwork Fri Nov 15 10:13:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21914 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F3312C0F1B for ; Fri, 15 Nov 2024 10:14:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1696A65889; Fri, 15 Nov 2024 11:14:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="IZUW12dG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CF42665872 for ; Fri, 15 Nov 2024 11:13:50 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 49E40FC5; Fri, 15 Nov 2024 11:13:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665616; bh=B2xSIXh8hz8P+0jQr3ZWarvea1lsu8RU8Nlx87yk+Pk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IZUW12dG/jNI3Q832x5twElwiKBEdj0ASG3OoHutHt1JYUdoisEp1XwXYYU8voQ29 YyN6CuWRLJRfeCO/g8TRg/0zdK4uy+eZEbiui4vCBcP0UfCzUzmm+IV2i+kchulOi9 ZIXEeMyzDNIEeouM8MmfjftaHszT8I4qR7cWdZ0M= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Daniel Scally Subject: [PATCH v6 09/14] libcamera: bayer_format: Add 20-bit bayer formats Date: Fri, 15 Nov 2024 10:13:29 +0000 Message-Id: <20241115101334.453104-10-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for the 20-bit bayer formats to libcamera's BayerFormat class. Signed-off-by: Daniel Scally Reviewed-by: Kieran Bingham --- Changes in v6: - New patch src/libcamera/bayer_format.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/bayer_format.cpp b/src/libcamera/bayer_format.cpp index c2120d1c..3dab91fc 100644 --- a/src/libcamera/bayer_format.cpp +++ b/src/libcamera/bayer_format.cpp @@ -225,6 +225,10 @@ const std::unordered_map mbusCodeToBayer{ { MEDIA_BUS_FMT_SGBRG16_1X16, { BayerFormat::GBRG, 16, BayerFormat::Packing::None } }, { MEDIA_BUS_FMT_SGRBG16_1X16, { BayerFormat::GRBG, 16, BayerFormat::Packing::None } }, { MEDIA_BUS_FMT_SRGGB16_1X16, { BayerFormat::RGGB, 16, BayerFormat::Packing::None } }, + { MEDIA_BUS_FMT_SBGGR20_1X20, { BayerFormat::BGGR, 20, BayerFormat::Packing::None } }, + { MEDIA_BUS_FMT_SGBRG20_1X20, { BayerFormat::GBRG, 20, BayerFormat::Packing::None } }, + { MEDIA_BUS_FMT_SGRBG20_1X20, { BayerFormat::GRBG, 20, BayerFormat::Packing::None } }, + { MEDIA_BUS_FMT_SRGGB20_1X20, { BayerFormat::RGGB, 20, BayerFormat::Packing::None } }, { MEDIA_BUS_FMT_Y8_1X8, { BayerFormat::MONO, 8, BayerFormat::Packing::None } }, { MEDIA_BUS_FMT_Y10_1X10, { BayerFormat::MONO, 10, BayerFormat::Packing::None } }, { MEDIA_BUS_FMT_Y12_1X12, { BayerFormat::MONO, 12, BayerFormat::Packing::None } }, From patchwork Fri Nov 15 10:13:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21911 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C0AA9C32CE for ; Fri, 15 Nov 2024 10:14:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 228DE65895; Fri, 15 Nov 2024 11:14:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qlFwRiHR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 390C465873 for ; Fri, 15 Nov 2024 11:13:51 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9913913E2; Fri, 15 Nov 2024 11:13:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665616; bh=68tj4ZJrZx5nClInhJT5zmSE1bjOyQJ+GobodhlqCko=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qlFwRiHRihH2vsreGuIqkKj6FWxvT+2j3S6ykmNCC1YmSobzZJXl1Iq2fu/dK0HJc /6llx6Fl++Lj5sH9jBZ+Hpv7T96QhwSHxFpSrUFmbp9nZAy+OyJsvtts03XtFj5ecA XHbYruI3WOCx0DK8KTWZMYeOuCn80oNqbFq4Lkaw= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Jacopo Mondi , Daniel Scally , Umang Jain Subject: [PATCH v6 10/14] libcamera: mali-c55: Propagate CSI-2 format to ISP Date: Fri, 15 Nov 2024 10:13:30 +0000 Message-Id: <20241115101334.453104-11-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The latest version of the Mali C55 driver has changed the format accepted by the ISP sink pad to be the 20-bit wide. The CSI-2 receiver handles the format expansion internally by propagating the sensor produced format from its sink to the 20-bit expanded version on its source pad. Instead of re-applying the sensor format to the CSI-2 receiver source pad (which is now an invalid operation) read from there the format propagated by the driver internally and further propagate it to the ISP subdevice. Reviewed-by: Daniel Scally Reviewed-by: Umang Jain Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v6: - None Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index a35aa92a..f5f98c59 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -767,7 +767,7 @@ int PipelineHandlerMaliC55::configure(Camera *camera, if (ret) return ret; - ret = data->csi_->setFormat(1, &subdevFormat); + ret = data->csi_->getFormat(1, &subdevFormat); if (ret) return ret; } From patchwork Fri Nov 15 10:13:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21915 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A7AECC3261 for ; Fri, 15 Nov 2024 10:14:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DBB1165890; Fri, 15 Nov 2024 11:14:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="G1LNzA+N"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8BC5765874 for ; Fri, 15 Nov 2024 11:13:51 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 07EC0149B; Fri, 15 Nov 2024 11:13:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665617; bh=t+ZNSusZ/ezFxb5QednvAcwq78SoKupgLZkOZ3SVw6I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G1LNzA+NQ521Fkka83r2hnIeeTPmc3mkXsglLD5nXBPuboX0WeGbtcJZDny0U204r /PdFgSVs3MuGWk7qiO/ZedTn237YcslBfDVMMQZINwDPJra4FOLdcpZpu4TaudrUy5 FLl8x7Eyz2d9YTD2FpDlnlNqCVra/4RME1TQ1GGA= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Daniel Scally , Kieran Bingham Subject: [PATCH v6 11/14] libcamera: mali-c55: Correct input/output format representation Date: Fri, 15 Nov 2024 10:13:31 +0000 Message-Id: <20241115101334.453104-12-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" At present we configure raw streams by looping through the pixel formats we support and finding one with an associated media bus format code that the sensor can produce. In the new representation of raw data from the kernel driver this will not work - the sensor could produce 8, 10, 12, 14 or 16 bit data and the ISP will force it to RAW16, which is the only actually supported output. To fix the issue move to simply finding a pixel format with a bayer order that matches that of the media bus format produced by the sensor. If the sensor can produce multiple formats with the same bayer order use the one with the largest bitdepth. Finally, remove the claim to support RAW formats of less than 16 bits. Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v6: - In ::validate(), ran the returned pixel format from bestRawFormat() through pixfmtToMbusCode() to guarantee we get a sensor-supported code. Changes in v5: - Dropped the Fatal log. src/libcamera/pipeline/mali-c55/mali-c55.cpp | 136 ++++++++++++------- 1 file changed, 85 insertions(+), 51 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index f5f98c59..e973abc2 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -57,22 +57,6 @@ const std::map maliC55FmtToCode = { { formats::NV21, MEDIA_BUS_FMT_YUV10_1X30 }, /* RAW formats, FR pipe only. */ - { formats::SGBRG8, MEDIA_BUS_FMT_SGBRG8_1X8 }, - { formats::SRGGB8, MEDIA_BUS_FMT_SRGGB8_1X8 }, - { formats::SBGGR8, MEDIA_BUS_FMT_SBGGR8_1X8 }, - { formats::SGRBG8, MEDIA_BUS_FMT_SGRBG8_1X8 }, - { formats::SGBRG10, MEDIA_BUS_FMT_SGBRG10_1X10 }, - { formats::SRGGB10, MEDIA_BUS_FMT_SRGGB10_1X10 }, - { formats::SBGGR10, MEDIA_BUS_FMT_SBGGR10_1X10 }, - { formats::SGRBG10, MEDIA_BUS_FMT_SGRBG10_1X10 }, - { formats::SGBRG12, MEDIA_BUS_FMT_SGBRG12_1X12 }, - { formats::SRGGB12, MEDIA_BUS_FMT_SRGGB12_1X12 }, - { formats::SBGGR12, MEDIA_BUS_FMT_SBGGR12_1X12 }, - { formats::SGRBG12, MEDIA_BUS_FMT_SGRBG12_1X12 }, - { formats::SGBRG14, MEDIA_BUS_FMT_SGBRG14_1X14 }, - { formats::SRGGB14, MEDIA_BUS_FMT_SRGGB14_1X14 }, - { formats::SBGGR14, MEDIA_BUS_FMT_SBGGR14_1X14 }, - { formats::SGRBG14, MEDIA_BUS_FMT_SGRBG14_1X14 }, { formats::SGBRG16, MEDIA_BUS_FMT_SGBRG16_1X16 }, { formats::SRGGB16, MEDIA_BUS_FMT_SRGGB16_1X16 }, { formats::SBGGR16, MEDIA_BUS_FMT_SBGGR16_1X16 }, @@ -98,7 +82,8 @@ public: const std::vector sizes(unsigned int mbusCode) const; const Size resolution() const; - PixelFormat bestRawFormat() const; + int pixfmtToMbusCode(const PixelFormat &pixFmt) const; + const PixelFormat &bestRawFormat() const; PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; @@ -207,33 +192,78 @@ const Size MaliC55CameraData::resolution() const return tpgResolution_; } -PixelFormat MaliC55CameraData::bestRawFormat() const +/* + * The Mali C55 ISP can only produce 16-bit RAW output in bypass modes, but the + * sensors connected to it might produce 8/10/12/16 bits. We simply search the + * sensor's supported formats for the one with a matching bayer order and the + * greatest bitdepth. + */ +int MaliC55CameraData::pixfmtToMbusCode(const PixelFormat &pixFmt) const { + auto it = maliC55FmtToCode.find(pixFmt); + if (it == maliC55FmtToCode.end()) + return -EINVAL; + + BayerFormat bayerFormat = BayerFormat::fromMbusCode(it->second); + if (!bayerFormat.isValid()) + return -EINVAL; + + V4L2Subdevice::Formats formats = sd_->formats(0); + unsigned int sensorMbusCode = 0; unsigned int bitDepth = 0; - PixelFormat rawFormat; - /* - * Iterate over all the supported PixelFormat and find the one - * supported by the camera with the largest bitdepth. - */ - for (const auto &maliFormat : maliC55FmtToCode) { - PixelFormat pixFmt = maliFormat.first; - if (!isFormatRaw(pixFmt)) + for (const auto &[code, sizes] : formats) { + BayerFormat sdBayerFormat = BayerFormat::fromMbusCode(code); + if (!sdBayerFormat.isValid()) continue; - unsigned int rawCode = maliFormat.second; - const auto rawSizes = sizes(rawCode); - if (rawSizes.empty()) + if (sdBayerFormat.order != bayerFormat.order) + continue; + + if (sdBayerFormat.bitDepth > bitDepth) { + bitDepth = sdBayerFormat.bitDepth; + sensorMbusCode = code; + } + } + + if (!sensorMbusCode) + return -EINVAL; + + return sensorMbusCode; +} + +/* + * Find a RAW PixelFormat supported by both the ISP and the sensor. + * + * The situation is mildly complicated by the fact that we expect the sensor to + * output something like RAW8/10/12/16, but the ISP can only accept as input + * RAW20 and can only produce as output RAW16. The one constant in that is the + * bayer order of the data, so we'll simply check that the sensor produces a + * format with a bayer order that matches that of one of the formats we support, + * and select that. + */ +const PixelFormat &MaliC55CameraData::bestRawFormat() const +{ + static const PixelFormat invalidPixFmt = {}; + + for (const auto &fmt : sd_->formats(0)) { + BayerFormat sensorBayer = BayerFormat::fromMbusCode(fmt.first); + + if (!sensorBayer.isValid()) continue; - BayerFormat bayer = BayerFormat::fromMbusCode(rawCode); - if (bayer.bitDepth > bitDepth) { - bitDepth = bayer.bitDepth; - rawFormat = pixFmt; + for (const auto &[pixFmt, rawCode] : maliC55FmtToCode) { + if (!isFormatRaw(pixFmt)) + continue; + + BayerFormat bayer = BayerFormat::fromMbusCode(rawCode); + if (bayer.order == sensorBayer.order) + return pixFmt; } } - return rawFormat; + LOG(MaliC55, Error) << "Sensor doesn't provide a compatible format"; + return invalidPixFmt; } /* @@ -242,13 +272,11 @@ PixelFormat MaliC55CameraData::bestRawFormat() const */ PixelFormat MaliC55CameraData::adjustRawFormat(const PixelFormat &rawFmt) const { - /* Make sure the provided raw format is supported by the pipeline. */ - auto it = maliC55FmtToCode.find(rawFmt); - if (it == maliC55FmtToCode.end()) + /* Make sure the RAW mbus code is supported by the image source. */ + int rawCode = pixfmtToMbusCode(rawFmt); + if (rawCode < 0) return bestRawFormat(); - /* Now make sure the RAW mbus code is supported by the image source. */ - unsigned int rawCode = it->second; const auto rawSizes = sizes(rawCode); if (rawSizes.empty()) return bestRawFormat(); @@ -258,16 +286,14 @@ PixelFormat MaliC55CameraData::adjustRawFormat(const PixelFormat &rawFmt) const Size MaliC55CameraData::adjustRawSizes(const PixelFormat &rawFmt, const Size &size) const { - /* Just make sure the format is supported. */ - auto it = maliC55FmtToCode.find(rawFmt); - if (it == maliC55FmtToCode.end()) - return {}; - /* Expand the RAW size to the minimum ISP input size. */ Size rawSize = size.expandedTo(kMaliC55MinInputSize); /* Check if the size is natively supported. */ - unsigned int rawCode = it->second; + int rawCode = pixfmtToMbusCode(rawFmt); + if (rawCode < 0) + return {}; + const auto rawSizes = sizes(rawCode); auto sizeIt = std::find(rawSizes.begin(), rawSizes.end(), rawSize); if (sizeIt != rawSizes.end()) @@ -348,6 +374,10 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() */ PixelFormat rawFormat = data_->adjustRawFormat(rawConfig->pixelFormat); + + if (!rawFormat.isValid()) + return Invalid; + if (rawFormat != rawConfig->pixelFormat) { LOG(MaliC55, Debug) << "RAW format adjusted to " << rawFormat; @@ -417,8 +447,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() /* If there's a RAW config, sensor configuration follows it. */ if (rawConfig) { - const auto it = maliC55FmtToCode.find(rawConfig->pixelFormat); - sensorFormat_.code = it->second; + sensorFormat_.code = data_->pixfmtToMbusCode(rawConfig->pixelFormat); sensorFormat_.size = rawConfig->size.expandedTo(minSensorSize); return status; @@ -426,11 +455,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() /* If there's no RAW config, compute the sensor configuration here. */ PixelFormat rawFormat = data_->bestRawFormat(); - const auto it = maliC55FmtToCode.find(rawFormat); - sensorFormat_.code = it->second; + if (!rawFormat.isValid()) + return Invalid; + + sensorFormat_.code = data_->pixfmtToMbusCode(rawFormat); uint16_t distance = std::numeric_limits::max(); - const auto sizes = data_->sizes(it->second); + const auto sizes = data_->sizes(sensorFormat_.code); Size bestSize; for (const auto &size : sizes) { if (minSensorSize.width > size.width || @@ -613,7 +644,10 @@ PipelineHandlerMaliC55::generateConfiguration(Camera *camera, if (isRaw) { /* Make sure the mbus code is supported. */ - unsigned int rawCode = maliFormat.second; + int rawCode = data->pixfmtToMbusCode(pixFmt); + if (rawCode < 0) + continue; + const auto sizes = data->sizes(rawCode); if (sizes.empty()) continue; From patchwork Fri Nov 15 10:13:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21913 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A6961C330C for ; Fri, 15 Nov 2024 10:14:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B36C765884; Fri, 15 Nov 2024 11:14:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="u5f7HGHm"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2F02265875 for ; Fri, 15 Nov 2024 11:13:52 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8EFA8157E; Fri, 15 Nov 2024 11:13:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665617; bh=6V0GJ1y1Ty69KjFXvve9kGzrRfWqUvoCB5D1UZb8kbo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u5f7HGHmLrm/1wMBeBuCp31HiwhS8SmYbUswjNvF6MoNuK/BX6NwrOjg5R8GB+4OF HCLDIB5fNddYurD0Tj35iHh6mjRrtKmjuafsLE1I1x1S3oHba/4BZnWQIAdCo8iaQh MfzVd3UNSYVWt6NoqS2Bgi5RUsV/jibiq1G+U9p4= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Daniel Scally , Jacopo Mondi , Kieran Bingham Subject: [PATCH v6 12/14] libcamera: mali-c55: Enable links between resizer and video node Date: Fri, 15 Nov 2024 10:13:32 +0000 Message-Id: <20241115101334.453104-13-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The mali-c55 driver now expects links to video devices to be enabled in order for those devices to be streamed from / to. Enable the media link between the resizers and their associated video device to fulfil the requirement. Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham Signed-off-by: Daniel Scally --- Changes in v6: - Used MediaDevice::link() to find and cache the links in MaliC55Pipe, which simplies their enablement in ::configure() Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 25 ++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index e973abc2..c0206ea2 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -509,6 +509,7 @@ private: struct MaliC55Pipe { std::unique_ptr resizer; std::unique_ptr cap; + MediaLink *link; Stream *stream; }; @@ -835,6 +836,17 @@ int PipelineHandlerMaliC55::configure(Camera *camera, Stream *stream = streamConfig.stream(); MaliC55Pipe *pipe = pipeFromStream(data, stream); + /* + * Enable the media link between the pipe's resizer and the + * capture video device + */ + + ret = pipe->link->setEnabled(true); + if (ret) { + LOG(MaliC55, Error) << "Couldn't enable resizer's link"; + return ret; + } + if (isFormatRaw(streamConfig.pixelFormat)) ret = configureRawStream(data, streamConfig, subdevFormat); else @@ -1028,6 +1040,12 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) if (frPipe->cap->open() < 0) return false; + frPipe->link = media_->link("mali-c55 resizer fr", 1, "mali-c55 fr", 0); + if (!frPipe->link) { + LOG(MaliC55, Error) << "No link between fr resizer and video node"; + return false; + } + frPipe->cap->bufferReady.connect(this, &PipelineHandlerMaliC55::imageBufferReady); dsFitted_ = !!media_->getEntityByName("mali-c55 ds"); @@ -1044,6 +1062,13 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) if (dsPipe->cap->open() < 0) return false; + dsPipe->link = media_->link("mali-c55 resizer ds", 1, + "mali-c55 ds", 0); + if (!dsPipe->link) { + LOG(MaliC55, Error) << "No link between ds resizer and video node"; + return false; + } + dsPipe->cap->bufferReady.connect(this, &PipelineHandlerMaliC55::imageBufferReady); } From patchwork Fri Nov 15 10:13:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21917 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D1ED6C32CE for ; Fri, 15 Nov 2024 10:14:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E08DB65890; Fri, 15 Nov 2024 11:14:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="P/KtW7Iz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8A73A6587A for ; Fri, 15 Nov 2024 11:13:52 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F1DCB1590; Fri, 15 Nov 2024 11:13:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665618; bh=4j1rsI/VC751cgoYwUXU/Ry6Htj9ynzYyDZUJLDbliM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P/KtW7IzTL0wGg2EiKqCES/ikzyig9LYKxdKyDlbndITGhj9ulVHxFoYdJjcuElnU Q+jaDUuKO8pZo3hN6Smf0Ywv7rbMGkEWp2kEDSpTKTB9gUmDpbQSvf9DNW15q2m+d5 OSuSjsOUOFfLIPvX/gDQTomP+e5fZy+krmNrqIkQ= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Jacopo Mondi , Kieran Bingham , Daniel Scally Subject: [PATCH v6 13/14] libcamera: mali-c55: Enable usage of scaler Date: Fri, 15 Nov 2024 10:13:33 +0000 Message-Id: <20241115101334.453104-14-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The Mali C55 ISP has a resizing pipeline that allows to crop and scale images. So far the mali-c55 pipeline has only supported cropping without using the scaling functionalities. Now that the kernel has gained support for the scaling operations, make the libcamera pipeline use it by combining it with a first cropping step to align the input and output images FOV ratio, and then scale to the desired output size. Reviewed-by: Kieran Bingham Reviewed-by: Daniel Scally Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v6: - None Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index c0206ea2..d0218349 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -756,16 +756,28 @@ int PipelineHandlerMaliC55::configureProcessedStream(MaliC55CameraData *data, if (ret) return ret; - /* \todo Configure the resizer crop/compose rectangles. */ - Rectangle ispCrop = { 0, 0, config.size }; + /* + * Compute the scaler-in to scaler-out ratio: first center-crop to align + * the FOV to the desired resolution, then scale to the desired size. + */ + Size scalerIn = subdevFormat.size.boundedToAspectRatio(config.size); + int xCrop = (subdevFormat.size.width - scalerIn.width) / 2; + int yCrop = (subdevFormat.size.height - scalerIn.height) / 2; + Rectangle ispCrop = { xCrop, yCrop, scalerIn }; ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_CROP, &ispCrop); if (ret) return ret; - ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_COMPOSE, &ispCrop); + Rectangle ispCompose = { 0, 0, config.size }; + ret = pipe->resizer->setSelection(0, V4L2_SEL_TGT_COMPOSE, &ispCompose); if (ret) return ret; + /* + * The source pad format size comes directly from the sink + * compose rectangle. + */ + subdevFormat.size = ispCompose.size(); subdevFormat.code = maliC55FmtToCode.find(config.pixelFormat)->second; return pipe->resizer->setFormat(1, &subdevFormat); } From patchwork Fri Nov 15 10:13:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Scally X-Patchwork-Id: 21916 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 49BD7C330D for ; Fri, 15 Nov 2024 10:14:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4E6E16589B; Fri, 15 Nov 2024 11:14:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="liVdqd3G"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ECADB65876 for ; Fri, 15 Nov 2024 11:13:52 +0100 (CET) Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 59BDE15D2; Fri, 15 Nov 2024 11:13:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1731665618; bh=aeGAhW1AW3dO2WH6PyXxomXS0mwwEiltIZLIZ0J/5kk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=liVdqd3GFzOBXqbmbu6Jhl1U42LlmaCtAqF7rSJPE7SUtcCdaS9+p6OIZXBaSBaMR 9Jhn6DtJ8WICoEeMby5/Ubx0bUOKdy9lMNkMYQb3InBexA8YfRrODRQQXOd7LaYA8U 6tTv458T3496bQm7M1dlo9POBZYmzFn2GauCLWEw= From: Daniel Scally To: libcamera-devel@lists.libcamera.org Cc: Anthony.McGivern@arm.com, Jacopo Mondi , Kieran Bingham , Daniel Scally Subject: [PATCH v6 14/14] libcamera: mali-c55: implement support for ScalerCrop Date: Fri, 15 Nov 2024 10:13:34 +0000 Message-Id: <20241115101334.453104-15-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241115101334.453104-1-dan.scally@ideasonboard.com> References: <20241115101334.453104-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi Implement support for the ScalerCrop control that allows to apply a digital zoom to the captured streams. Initialize the camera controls at camera registration time and update them at configure time as the sensor's analogue crop size might change depending on the desired Camera configuration. Reviewed-by: Kieran Bingham Reviewed-by: Daniel Scally Signed-off-by: Jacopo Mondi Signed-off-by: Daniel Scally --- Changes in v6: - None Changes in v5: - None src/libcamera/pipeline/mali-c55/mali-c55.cpp | 135 +++++++++++++++++++ 1 file changed, 135 insertions(+) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index d0218349..3fc07a5d 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -22,6 +22,8 @@ #include #include +#include + #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" @@ -85,6 +87,8 @@ public: int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; + void updateControls(); + PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; @@ -266,6 +270,27 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const return invalidPixFmt; } +void MaliC55CameraData::updateControls() +{ + if (!sensor_) + return; + + IPACameraSensorInfo sensorInfo; + int ret = sensor_->sensorInfo(&sensorInfo); + if (ret) { + LOG(MaliC55, Error) << "Failed to retrieve sensor info"; + return; + } + + ControlInfoMap::Map controls; + Rectangle ispMinCrop{ 0, 0, 640, 480 }; + controls[&controls::ScalerCrop] = + ControlInfo(ispMinCrop, sensorInfo.analogCrop, + sensorInfo.analogCrop); + + controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); +} + /* * Make sure the provided raw pixel format is supported and adjust it to * one of the supported ones if it's not. @@ -553,6 +578,8 @@ private: const StreamConfiguration &config, V4L2SubdeviceFormat &subdevFormat); + void applyScalerCrop(Camera *camera, const ControlList &controls); + void registerMaliCamera(std::unique_ptr data, const std::string &name); bool registerTPGCamera(MediaLink *link); @@ -880,6 +907,8 @@ int PipelineHandlerMaliC55::configure(Camera *camera, pipe->stream = stream; } + data->updateControls(); + return 0; } @@ -927,6 +956,102 @@ void PipelineHandlerMaliC55::stopDevice([[maybe_unused]] Camera *camera) } } +void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, + const ControlList &controls) +{ + MaliC55CameraData *data = cameraData(camera); + + const auto &scalerCrop = controls.get(controls::ScalerCrop); + if (!scalerCrop) + return; + + if (!data->sensor_) { + LOG(MaliC55, Error) << "ScalerCrop not supported for TPG"; + return; + } + + Rectangle nativeCrop = *scalerCrop; + + IPACameraSensorInfo sensorInfo; + int ret = data->sensor_->sensorInfo(&sensorInfo); + if (ret) { + LOG(MaliC55, Error) << "Failed to retrieve sensor info"; + return; + } + + /* + * The ScalerCrop rectangle re-scaling in the ISP crop rectangle + * comes straight from the RPi pipeline handler. + * + * Create a version of the crop rectangle aligned to the analogue crop + * rectangle top-left coordinates and scaled in the [analogue crop to + * output frame] ratio to take into account binning/skipping on the + * sensor. + */ + Rectangle ispCrop = nativeCrop.translatedBy(-sensorInfo.analogCrop + .topLeft()); + ispCrop.scaleBy(sensorInfo.outputSize, sensorInfo.analogCrop.size()); + + /* + * The crop rectangle should be: + * 1. At least as big as ispMinCropSize_, once that's been + * enlarged to the same aspect ratio. + * 2. With the same mid-point, if possible. + * 3. But it can't go outside the sensor area. + */ + Rectangle ispMinCrop{ 0, 0, 640, 480 }; + Size minSize = ispMinCrop.size().expandedToAspectRatio(nativeCrop.size()); + Size size = ispCrop.size().expandedTo(minSize); + ispCrop = size.centeredTo(ispCrop.center()) + .enclosedIn(Rectangle(sensorInfo.outputSize)); + + /* + * As the resizer can't upscale, the crop rectangle has to be larger + * than the larger stream output size. + */ + Size maxYuvSize; + for (MaliC55Pipe &pipe : pipes_) { + if (!pipe.stream) + continue; + + const StreamConfiguration &config = pipe.stream->configuration(); + if (isFormatRaw(config.pixelFormat)) { + LOG(MaliC55, Debug) << "Cannot crop with a RAW stream"; + return; + } + + Size streamSize = config.size; + if (streamSize.width > maxYuvSize.width) + maxYuvSize.width = streamSize.width; + if (streamSize.height > maxYuvSize.height) + maxYuvSize.height = streamSize.height; + } + + ispCrop.size().expandTo(maxYuvSize); + + /* + * Now apply the scaler crop to each enabled output. This overrides the + * crop configuration performed at configure() time and can cause + * square pixels if the crop rectangle and scaler output FOV ratio are + * different. + */ + for (MaliC55Pipe &pipe : pipes_) { + if (!pipe.stream) + continue; + + /* Create a copy to avoid setSelection() to modify ispCrop. */ + Rectangle pipeCrop = ispCrop; + ret = pipe.resizer->setSelection(0, V4L2_SEL_TGT_CROP, &pipeCrop); + if (ret) { + LOG(MaliC55, Error) + << "Failed to apply crop to " + << (pipe.stream == &data->frStream_ ? + "FR" : "DS") << " pipe"; + return; + } + } +} + int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { int ret; @@ -939,6 +1064,15 @@ int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) return ret; } + /* + * Some controls need to be applied immediately, as in example, + * the ScalerCrop one. + * + * \todo Move it buffer queue time (likely after the IPA has filled in + * the parameters buffer) once we have plumbed the IPA loop in. + */ + applyScalerCrop(camera, request->controls()); + return 0; } @@ -1014,6 +1148,7 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return false; data->properties_ = data->sensor_->properties(); + data->updateControls(); registerMaliCamera(std::move(data), sensor->name()); }