From patchwork Thu Nov 24 12:12:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 17887 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 754C3BDE6B for ; Thu, 24 Nov 2022 12:12:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 45FCC63321; Thu, 24 Nov 2022 13:12:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669291950; bh=cMyh8jAUSIrQxf42ySkdTyCgVYV2cIDPeSWBXmpbK9I=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=av5Tn1KkMTdW7ere3Nb7GgcTsL2K+34eSj7ZgMrgbrCFJa1HQ9UsWLMuM3w5H88Zk +tJ7NdBGQnL3Q3tqzUGTlOQOJfkRnpepKOPyJ34/8TQowhnWpMzhxB7iLVKLJCw90o V8Fzn6xg9aLGH7t4B6ZqZCdDtDT/LalCMufVqjnnsAGNblNVaLXXTuWjKSY7/5r94X Dbmp80ijNjMnl3du3t3FC0ah5CHbu8oQjSs41AzFlXsTI90CKd+raYRaHIBaV9xPMB g0lasnw1Z08iPpgJd2jMhjz+q6vii1QAdQmCDCY7VBNHf4C5RX4TlqGmwhEM/zXtPi YZYgYQ5lbJSfg== Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::222]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A5A36331A for ; Thu, 24 Nov 2022 13:12:27 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id 6B1D24000E; Thu, 24 Nov 2022 12:12:26 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 13:12:12 +0100 Message-Id: <20221124121220.47000-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221124121220.47000-1-jacopo@jmondi.org> References: <20221124121220.47000-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/9] libcamera: bayer_format: Add toMbusCode method X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman This makes it easier to perform transformations on Bayer type mbus codes by converting them to a BayerFormat, doing the transform, and then converting them back again. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi --- include/libcamera/internal/bayer_format.h | 1 + src/libcamera/bayer_format.cpp | 11 +++++++++++ 2 files changed, 12 insertions(+) diff --git a/include/libcamera/internal/bayer_format.h b/include/libcamera/internal/bayer_format.h index 78ba3969913d..3601dccb7228 100644 --- a/include/libcamera/internal/bayer_format.h +++ b/include/libcamera/internal/bayer_format.h @@ -47,6 +47,7 @@ public: } static const BayerFormat &fromMbusCode(unsigned int mbusCode); + unsigned int toMbusCode() const; bool isValid() const { return bitDepth != 0; } std::string toString() const; diff --git a/src/libcamera/bayer_format.cpp b/src/libcamera/bayer_format.cpp index f27cc1662d25..fdbc4af1dcc0 100644 --- a/src/libcamera/bayer_format.cpp +++ b/src/libcamera/bayer_format.cpp @@ -226,6 +226,17 @@ const BayerFormat &BayerFormat::fromMbusCode(unsigned int mbusCode) return it->second; } +/** + * \brief Retrieve the media bus code corresponding this this BayerFormat + * \return The corresponding media bus code, or zero if none was found + */ +unsigned int BayerFormat::toMbusCode() const +{ + auto it = std::find_if(mbusCodeToBayer.begin(), mbusCodeToBayer.end(), + [this](const auto &i) { return i.second == *this; }); + return it != mbusCodeToBayer.end() ? it->first : 0; +} + /** * \fn BayerFormat::isValid() * \brief Return whether a BayerFormat is valid From patchwork Thu Nov 24 12:12:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 17888 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D9A76BDE6B for ; Thu, 24 Nov 2022 12:12:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D5EF963323; Thu, 24 Nov 2022 13:12:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669291950; bh=rkqY+rdxPsZzbM+wm8W6vy87mJYXqpr2dW4Y5OAN7AM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=wSV4ZrawThIRzxSy5UEyglwKGhHnhyb972nuiUtD2jCozPbcqqcN/CWqsOB97vJMy Nr4gZ6iy/JYHI+zyCxIen1OSCO3oqLzaQlJw4K8dM18W3MwYjdkSnNWbgp6a6ntzj1 D0n8Zn+ZOXLlFRqDOaP0uQKPdIeBaAEBzjoVwl8xmqIvY7xD1/Uny2N88yvfrP0/VR QXaDcLiC2Z2a2XpJJ7vavwuTpJvPxRaJqrR4nIaC2dCTjWBLG+qQ76yCKZFoFq24++ etQ/oufCqygJd+FgpUSLf213VsB6HziBOU3KEd5uQt+uS9Zy1bWn7Yaq1V+Zs7xHoz 0lw7pP/BZaTbA== Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 92F6063319 for ; Thu, 24 Nov 2022 13:12:28 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id A49DD40002; Thu, 24 Nov 2022 12:12:27 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 13:12:13 +0100 Message-Id: <20221124121220.47000-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221124121220.47000-1-jacopo@jmondi.org> References: <20221124121220.47000-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/9] libcamera: camera_sensor: Do not clear camera flips when listing formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman Previously the code used to clear the camnera's h and v flip bits when enumerating the supported formats so as to obtain any Bayer formats in the sensor's native (untransformed) orientation. However this fails when the camera is already in use elsewhere. Instead, we query the current state of the flip bits and transform the formats - which we obtain in their flipped orientation - back into their native orientation to be stored. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi --- src/libcamera/camera_sensor.cpp | 49 ++++++++++++++++++++++++++------- 1 file changed, 39 insertions(+), 10 deletions(-) diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 572a313a8f99..cbac9e7801ae 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -19,6 +19,8 @@ #include +#include + #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera_lens.h" #include "libcamera/internal/camera_sensor_properties.h" @@ -108,18 +110,45 @@ int CameraSensor::init() return ret; /* - * Clear any flips to be sure we get the "native" Bayer order. This is - * harmless for sensors where the flips don't affect the Bayer order. + * We want to get the native mbus codes for the sensor, without any flips. + * We can't clear any flips here, so we have to read the current values + * (if the flip controls exist), decide whether they actually modify any + * output Bayer pattern, and finally undo their effect on the formats. + * + * First, check if the flip controls exist and if so read them. */ - ControlList ctrls(subdev_->controls()); - if (subdev_->controls().find(V4L2_CID_HFLIP) != subdev_->controls().end()) - ctrls.set(V4L2_CID_HFLIP, 0); - if (subdev_->controls().find(V4L2_CID_VFLIP) != subdev_->controls().end()) - ctrls.set(V4L2_CID_VFLIP, 0); - subdev_->setControls(&ctrls); + std::vector flipCtrlIds; + const struct v4l2_query_ext_ctrl *hflipInfo = subdev_->controlInfo(V4L2_CID_HFLIP); + const struct v4l2_query_ext_ctrl *vflipInfo = subdev_->controlInfo(V4L2_CID_VFLIP); + if (hflipInfo) + flipCtrlIds.push_back(V4L2_CID_HFLIP); + if (vflipInfo) + flipCtrlIds.push_back(V4L2_CID_VFLIP); + ControlList flipCtrls = subdev_->getControls(flipCtrlIds); + + /* Now construct a transform that would undo any flips. */ + Transform transform = Transform::Identity; + if (hflipInfo && flipCtrls.get(V4L2_CID_HFLIP).get() && + (hflipInfo->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT)) + transform |= Transform::HFlip; + if (vflipInfo && flipCtrls.get(V4L2_CID_VFLIP).get() && + (vflipInfo->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT)) + transform |= Transform::VFlip; + + /* Finally get the formats, and apply the transform to the mbus codes. */ + auto formats = subdev_->formats(pad_); + for (const auto &format : formats) { + unsigned int mbusCode = format.first; + BayerFormat bayerFormat = BayerFormat::fromMbusCode(mbusCode); + + if (bayerFormat.isValid()) + mbusCode = bayerFormat.transform(transform).toMbusCode(); + + if (mbusCode) + formats_[mbusCode] = std::move(format.second); + } /* Enumerate, sort and cache media bus codes and sizes. */ - formats_ = subdev_->formats(pad_); if (formats_.empty()) { LOG(CameraSensor, Error) << "No image format found"; return -EINVAL; @@ -189,7 +218,7 @@ int CameraSensor::init() * \todo The control API ought to have a flag to specify if a control * is read-only which could be used below. */ - const ControlInfo hblank = ctrls.infoMap()->at(V4L2_CID_HBLANK); + const ControlInfo hblank = subdev_->controls().at(V4L2_CID_HBLANK); const int32_t hblankMin = hblank.min().get(); const int32_t hblankMax = hblank.max().get(); From patchwork Thu Nov 24 12:12:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 17889 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9177FBDE6B for ; Thu, 24 Nov 2022 12:12:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 633CC63325; Thu, 24 Nov 2022 13:12:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669291953; bh=cbKXWWxWYaYtfLqqRFZdvvo2THZ5IDkTdGBLyyy/Zuo=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=MIvrqVl4+NDUq2oCPpE94Wb6E4tmHykgxv3ADTtxilwRS0OAW9YGMoQwVCRSMR783 SNZblvR1WV1RxOMxEsabADGO2vwV2xsx5u25FjOoX95asXGWBryG0R5ltJe8rHoYdt QnK8N+yQ5I9TAaVoUwPSxeEtMqPlKe+t0/iFJfgiZBFtTeHMTG5b+OkBinnT/hSaVM AiCvPxvoF05An8Wal4Y7wxRyLY/vOmlwYBB7/Qr23SHdkuohvSELd9GhXmWm0uqg57 VoLkDUSZofO0NndugqSKSVJtQXj+zVgUu9Pght57p6Cl1LuCFT7IAcIB92Zdr65Nk7 IjbLs5y3AhM5w== Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::222]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 870E863319 for ; Thu, 24 Nov 2022 13:12:29 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id 9AEAA40004; Thu, 24 Nov 2022 12:12:28 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 13:12:14 +0100 Message-Id: <20221124121220.47000-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221124121220.47000-1-jacopo@jmondi.org> References: <20221124121220.47000-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/9] libcamera: bayer_format: Expand documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The current documentation of the BayerFormat::transform() function reports examples on the Bayer components ordering transformation for horizontal flip (mirroring) but not for vertical flip or for the combination of the two. It might be useful to complete the documentation to eases understanding of the transform() function on a sensor's Bayer pattern. Signed-off-by: Jacopo Mondi Reviewed-by: David Plowman --- src/libcamera/bayer_format.cpp | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/src/libcamera/bayer_format.cpp b/src/libcamera/bayer_format.cpp index fdbc4af1dcc0..dd8109ba85c2 100644 --- a/src/libcamera/bayer_format.cpp +++ b/src/libcamera/bayer_format.cpp @@ -367,11 +367,14 @@ BayerFormat BayerFormat::fromPixelFormat(PixelFormat format) * \brief Apply a transform to this BayerFormat * \param[in] t The transform to apply * - * Appplying a transform to an image stored in a Bayer format affects the Bayer - * order. For example, performing a horizontal flip on the Bayer pattern - * RGGB causes the RG rows of pixels to become GR, and the GB rows to become BG. - * The transformed image would have a GRBG order. The bit depth and modifiers - * are not affected. + * Applying a transform to an image stored in a Bayer format affects the Bayer + * order. For example, performing a horizontal flip on the Bayer pattern RGGB + * causes the RG rows of pixels to become GR, and the GB rows to become BG. The + * transformed image would have a GRBG order. Performing a vertical flip on + * the Bayer pattern RGGB causes the GB row to be read before the RG one and the + * transformed image would have GBRG order. Applying both vertical and + * horizontal flips on the Bayer patter RGGB results in transformed images with + * BGGR order. The bit depth and modifiers are not affected. * * Horizontal and vertical flips are applied before transpose. * @@ -386,8 +389,11 @@ BayerFormat BayerFormat::transform(Transform t) const /* * Observe that flipping bit 0 of the Order enum performs a horizontal - * mirror on the Bayer pattern (e.g. RGGB goes to GRBG). Similarly, - * flipping bit 1 performs a vertical mirror operation on it. Hence: + * mirror on the Bayer pattern (e.g. RG/GB goes to GR/BG). Similarly, + * flipping bit 1 performs a vertical mirror operation on it (e.g RG/GB + * goes to GB/RG). Applying both vertical and horizontal flips + * combines vertical and horizontal mirroring on the Bayer pattern + * (e.g. RG/GB goes to BG/GR). Hence: */ if (!!(t & Transform::HFlip)) result.order = static_cast(result.order ^ 1); From patchwork Thu Nov 24 12:12:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 17890 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5FB2BBDE6B for ; Thu, 24 Nov 2022 12:12:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0388463313; Thu, 24 Nov 2022 13:12:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669291954; bh=r+f33DwLy79JD2IOnj7rEgDwXTCozTBwrDQB5q1rtjU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=iKOnmtrMTzz0CKb8dI8vm8NSYatWKA4frzPHsxqzhOu+CZ3gn3v7pM/fXEEnUd0TY sdYdK04FaW/Xx02aJApPrGgq+xGTRFFmGR2ci/Vhu0tASE4bJTj0PH4mVCE7pMUqw9 BH/c+O+/fCTstRpEqcxgtX43IZqIB/HTb95b33BWELX6WPZkr3ZAbRrPdbehOMMOL+ YUvBZuimJQvig7moo7oOMNAQKAargqo3vr3Olu2r/tWUcMz8jWW/s9d4go5pttJEq2 d4szTC18T9+a2m+W1qWCThh/t/FNqoQ19Kn9smZGa1mqOfBKptJ9iGKvOmHLriwDMk gUm20YVWEwWsQ== Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BB58863319 for ; Thu, 24 Nov 2022 13:12:30 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id C3C2940002; Thu, 24 Nov 2022 12:12:29 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 13:12:15 +0100 Message-Id: <20221124121220.47000-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221124121220.47000-1-jacopo@jmondi.org> References: <20221124121220.47000-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/9] libcamera: camera_sensor: Verify flips support X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" During the camera sensor driver validation, verify if the sensor supports horizontal and vertical flips and store a flag as CameraSensor::supportFlips_ class member. The flag will be later inspected when applying flips. Signed-off-by: Jacopo Mondi Reviewed-by: David Plowman --- include/libcamera/internal/camera_sensor.h | 1 + src/libcamera/camera_sensor.cpp | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index b9f4d7867854..878f3c28a3c9 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -101,6 +101,7 @@ private: Size pixelArraySize_; Rectangle activeArea_; const BayerFormat *bayerFormat_; + bool supportFlips_; ControlList properties_; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index cbac9e7801ae..3afcbc482095 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -57,7 +57,8 @@ LOG_DEFINE_CATEGORY(CameraSensor) */ CameraSensor::CameraSensor(const MediaEntity *entity) : entity_(entity), pad_(UINT_MAX), staticProps_(nullptr), - bayerFormat_(nullptr), properties_(properties::properties) + bayerFormat_(nullptr), supportFlips_(false), + properties_(properties::properties) { } @@ -271,6 +272,18 @@ int CameraSensor::validateSensorDriver() } } + /* Verify if sensor supports horizontal/vertical flips. */ + const struct v4l2_query_ext_ctrl *hflipInfo = subdev_->controlInfo(V4L2_CID_HFLIP); + const struct v4l2_query_ext_ctrl *vflipInfo = subdev_->controlInfo(V4L2_CID_VFLIP); + if (hflipInfo && vflipInfo && + !(hflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY) && + !(vflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY)) + supportFlips_ = true; + + if (!supportFlips_) + LOG(CameraSensor, Warning) + << "Camera sensor does not support horizontal/vertical flip"; + /* * Make sure the required selection targets are supported. * From patchwork Thu Nov 24 12:12:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 17891 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 04545C3286 for ; Thu, 24 Nov 2022 12:12:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9C6BA63323; Thu, 24 Nov 2022 13:12:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669291954; bh=ISbHCbvzW3SqlCAN6p1qOlLOTMF5PhOrgqUNBfFReCY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=V+VNGJWKIicpROxAG3bkUBZ356WR0h8hRdHjnmS9b9VLiXzd6lePLCu8l6wcb+5pn PRVpjEIt9M9PhhAwURVM+Gd57YeazBnf3OMLsX4A4mHVyPRyjrW94JT/hBO7l/rEHM 1XrM46Pvp7LRY6IAcrNOeZ+M20/JKhEeTVyzY+HTH/83ttxvqv3fOpdXowNxJQidzR xHEbxtmDdPDjV7gEgtxxvo9OVG1j8+942wYoDjfMHWA923jY41wBvrrrayx3aWljaR UhXS09Uf9pt0IzAwJxZV/hClQ9BaLPPFX4HX+C75+/iTh7Crx1FScVnC0gzQlEX2xd jLVQFHCxNVuKQ== Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::222]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AB9846331E for ; Thu, 24 Nov 2022 13:12:31 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id CFF5F40004; Thu, 24 Nov 2022 12:12:30 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 13:12:16 +0100 Message-Id: <20221124121220.47000-6-jacopo@jmondi.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221124121220.47000-1-jacopo@jmondi.org> References: <20221124121220.47000-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/9] libcamera: camera_sensor: Validate Transform X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The two pipeline handlers that currently support Transform do so by operating H/V flips on the image sensor, instead of rotating on the ISP. As the image sensor performs the actual rotation, centralize the code that validates a requested Transform against the camera sensor rotation and capabilities in the CameraSensor class. The implementation in the IPU3 pipeline handler was copied from the RaspberryPi implementation, and is now centralized in CameraSensor to make it easier for other platforms that do not rotate on the ISP to implement support for Transform using the CameraSensor class. Signed-off-by: Jacopo Mondi Reviewed-by: David Plowman --- include/libcamera/internal/camera_sensor.h | 4 ++ src/libcamera/camera_sensor.cpp | 59 +++++++++++++++++++ src/libcamera/pipeline/ipu3/ipu3.cpp | 45 ++------------ .../pipeline/raspberrypi/raspberrypi.cpp | 59 +++---------------- 4 files changed, 76 insertions(+), 91 deletions(-) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index 878f3c28a3c9..bea52badaff7 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -29,6 +29,8 @@ class BayerFormat; class CameraLens; class MediaEntity; +enum class Transform; + struct CameraSensorProperties; class CameraSensor : protected Loggable @@ -68,6 +70,8 @@ public: CameraLens *focusLens() { return focusLens_.get(); } + Transform validateTransform(Transform *transform) const; + protected: std::string logPrefix() const override; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 3afcbc482095..a4ad0ba9a099 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -982,6 +982,65 @@ void CameraSensor::updateControlInfo() * connected to the sensor */ +/** + * \brief Validate a transform request against the sensor capabilities + * \param[inout] transform The requested transformation, updated to match + * the sensor capabilities + * + * The requested \a transform is adjusted against the sensor rotation and its + * capabilities. + * + * In example, if the requested \a transform is Transform::Identity and the + * sensor rotation is 180 degrees, the resulting transform returned by the + * function is Transform::Rot180 to automatically correct the image, but only if + * the sensor can actually apply horizontal and vertical flips. + * + * \return A Transform instance that represents how \a transform is applied to + * the camera sensor + */ +Transform CameraSensor::validateTransform(Transform *transform) const +{ + /* Adjust the requested transform to the sensor rotation. */ + int32_t rotation = properties().get(properties::Rotation).value_or(0); + bool success; + + Transform rotationTransform = transformFromRotation(rotation, &success); + if (!success) + LOG(CameraSensor, Warning) << "Invalid rotation of " << rotation + << " degrees - ignoring"; + + Transform combined = *transform * rotationTransform; + + /* + * The camera sensor cannot do Transpose. Adjust any combined result + * that includes a transpose by flipping the transpose bit to notify + * applications they either get the transform they requested, or have + * to do a simple transpose themselves (they don't have to worry about + * the other possible cases). + */ + if (!!(combined & Transform::Transpose)) { + /* + * Flipping the transpose bit in "transform" flips it in the + * combined result too (as it's the last thing that happens), + * which is of course clearing it. + */ + *transform ^= Transform::Transpose; + combined &= ~Transform::Transpose; + } + + /* + * If the sensor can do no transforms, then combined must be changed to + * the identity and the sensor rotation must be cleared from the + * requested "transform". + */ + if (!supportFlips_ && !!combined) { + *transform = -rotationTransform; + combined = Transform::Identity; + } + + return combined; +} + std::string CameraSensor::logPrefix() const { return "'" + entity_->name() + "'"; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e4d79ea44aed..a424ac914859 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -184,48 +184,15 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() if (config_.empty()) return Invalid; - Transform combined = transform * data_->rotationTransform_; - - /* - * We combine the platform and user transform, but must "adjust away" - * any combined result that includes a transposition, as we can't do - * those. In this case, flipping only the transpose bit is helpful to - * applications - they either get the transform they requested, or have - * to do a simple transpose themselves (they don't have to worry about - * the other possible cases). - */ - if (!!(combined & Transform::Transpose)) { - /* - * Flipping the transpose bit in "transform" flips it in the - * combined result too (as it's the last thing that happens), - * which is of course clearing it. - */ - transform ^= Transform::Transpose; - combined &= ~Transform::Transpose; - status = Adjusted; - } - /* - * We also check if the sensor doesn't do h/vflips at all, in which - * case we clear them, and the application will have to do everything. + * Validate the requested transform against the sensor capabilities and + * rotation and store the final combined transform that configure() will + * need to apply to the sensor to save us working it out again. */ - if (!data_->supportsFlips_ && !!combined) { - /* - * If the sensor can do no transforms, then combined must be - * changed to the identity. The only user transform that gives - * rise to this is the inverse of the rotation. (Recall that - * combined = transform * rotationTransform.) - */ - transform = -data_->rotationTransform_; - combined = Transform::Identity; + Transform requestedTransform = transform; + combinedTransform_ = data_->cio2_.sensor()->validateTransform(&transform); + if (transform != requestedTransform) status = Adjusted; - } - - /* - * Store the final combined transform that configure() will need to - * apply to the sensor to save us working it out again. - */ - combinedTransform_ = combined; /* Cap the number of entries to the available streams. */ if (config_.size() > kMaxStreams) { diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 087c71b65700..e83984985bce 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -366,59 +366,14 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); /* - * What if the platform has a non-90 degree rotation? We can't even - * "adjust" the configuration and carry on. Alternatively, raising an - * error means the platform can never run. Let's just print a warning - * and continue regardless; the rotation is effectively set to zero. + * Validate the requested transform against the sensor capabilities and + * rotation and store the final combined transform that configure() will + * need to apply to the sensor to save us working it out again. */ - int32_t rotation = data_->sensor_->properties().get(properties::Rotation).value_or(0); - bool success; - Transform rotationTransform = transformFromRotation(rotation, &success); - if (!success) - LOG(RPI, Warning) << "Invalid rotation of " << rotation - << " degrees - ignoring"; - Transform combined = transform * rotationTransform; - - /* - * We combine the platform and user transform, but must "adjust away" - * any combined result that includes a transform, as we can't do those. - * In this case, flipping only the transpose bit is helpful to - * applications - they either get the transform they requested, or have - * to do a simple transpose themselves (they don't have to worry about - * the other possible cases). - */ - if (!!(combined & Transform::Transpose)) { - /* - * Flipping the transpose bit in "transform" flips it in the - * combined result too (as it's the last thing that happens), - * which is of course clearing it. - */ - transform ^= Transform::Transpose; - combined &= ~Transform::Transpose; - status = Adjusted; - } - - /* - * We also check if the sensor doesn't do h/vflips at all, in which - * case we clear them, and the application will have to do everything. - */ - if (!data_->supportsFlips_ && !!combined) { - /* - * If the sensor can do no transforms, then combined must be - * changed to the identity. The only user transform that gives - * rise to this the inverse of the rotation. (Recall that - * combined = transform * rotationTransform.) - */ - transform = -rotationTransform; - combined = Transform::Identity; + Transform requestedTransform = transform; + combinedTransform_ = data_->sensor_->validateTransform(&transform); + if (transform != requestedTransform) status = Adjusted; - } - - /* - * Store the final combined transform that configure() will need to - * apply to the sensor to save us working it out again. - */ - combinedTransform_ = combined; unsigned int rawCount = 0, outCount = 0, count = 0, maxIndex = 0; std::pair outSize[2]; @@ -453,7 +408,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (data_->flipsAlterBayerOrder_) { BayerFormat bayer = BayerFormat::fromV4L2PixelFormat(fourcc); bayer.order = data_->nativeBayerOrder_; - bayer = bayer.transform(combined); + bayer = bayer.transform(combinedTransform_); fourcc = bayer.toV4L2PixelFormat(); } From patchwork Thu Nov 24 12:12:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 17892 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4AE0EBDE6B for ; Thu, 24 Nov 2022 12:12:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 08F7A63319; Thu, 24 Nov 2022 13:12:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669291956; bh=EIxGp0Ho9OrceTK8NhYeG8gPBtr/iyRig4gkxDUWv9I=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=2xojUo2dyFtj0/gk0br00ndZq5uxTUTXnwKqf5e1T4ifch/ytmVWI2ca8pu0e+wi6 D/s6XHUI+xYa80ZMv6fxqibXwzTlRkMJlDY5oVcOCrvusHHLoHF1BptGo8svoM93Ao sFoYkysA+H5W8MuXRPw24+uPpZUKFqn2+H03y4jxEg3sG1ZlAR/yWgOgrr11+f5Ndl 2zAVh9lSahG5AfTBxSeV3/PuaotGTrdoTEuhNc3gm0TVKenaVMUe1AOCCrVBxARidc oQd9T4tP00iz1Aytt7kNAm1QsZNIoLMjJqAYIE5gVTqNdWfqmBsQ34Iue0WW0VcEvs 0B1YOKSGj+PDA== Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B2F2F63325 for ; Thu, 24 Nov 2022 13:12:32 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id D7E144000C; Thu, 24 Nov 2022 12:12:31 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 13:12:17 +0100 Message-Id: <20221124121220.47000-7-jacopo@jmondi.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221124121220.47000-1-jacopo@jmondi.org> References: <20221124121220.47000-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 6/9] libcamera: camera_sensor: Apply flips at setFormat() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Augment the CameraSensor::setFormat() function to configure horizontal and vertical flips before appllying the image format on the sensor. Applying flips before format is crucial as they might change the Bayer pattern ordering. To allow users of the CameraSensor class to easily pass the requested Transform add to the V4L2SubdeviceFormat class a 'transform' member, by default initialized to Transform::Identity. Moving the handling of H/V flips to the CameraSensor class allows to remove quite some boilerplate code from the IPU3 and RaspberryPi pipeline handlers. Signed-off-by: Jacopo Mondi --- include/libcamera/internal/v4l2_subdevice.h | 2 ++ src/libcamera/camera_sensor.cpp | 26 +++++++++++++++++ src/libcamera/pipeline/ipu3/cio2.cpp | 6 +++- src/libcamera/pipeline/ipu3/cio2.h | 4 ++- src/libcamera/pipeline/ipu3/ipu3.cpp | 28 ++----------------- .../pipeline/raspberrypi/raspberrypi.cpp | 27 +++++------------- src/libcamera/v4l2_subdevice.cpp | 10 +++++++ 7 files changed, 55 insertions(+), 48 deletions(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index 69862de0585a..576faf971a05 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -20,6 +20,7 @@ #include #include +#include #include "libcamera/internal/formats.h" #include "libcamera/internal/media_object.h" @@ -44,6 +45,7 @@ struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; std::optional colorSpace; + Transform transform = Transform::Identity; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index a4ad0ba9a099..e6d99e4881d1 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -770,6 +770,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu .mbus_code = bestCode, .size = *bestSize, .colorSpace = ColorSpace::Raw, + .transform = Transform::Identity, }; return format; @@ -779,12 +780,37 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu * \brief Set the sensor output format * \param[in] format The desired sensor output format * + * If flips are writable they are configured according to the desired Transform. + * Transform::Identity always corresponds to H/V flip being disabled if the + * controls are writable. Flips are set before the new format is applied as + * they can effectively change the Bayer pattern ordering. + * * The ranges of any controls associated with the sensor are also updated. * * \return 0 on success or a negative error code otherwise */ int CameraSensor::setFormat(V4L2SubdeviceFormat *format) { + /* + * Configure flips. If the sensor has configurable flips, we assume that + * Transform::Identity means both vertical and horizontal flips are + * disabled. + */ + ControlList flipCtrls(subdev_->controls()); + if (supportFlips_) { + flipCtrls.set(V4L2_CID_HFLIP, + static_cast(!!(format->transform & + Transform::HFlip))); + flipCtrls.set(V4L2_CID_VFLIP, + static_cast(!!(format->transform & + Transform::VFlip))); + + int ret = subdev_->setControls(&flipCtrls); + if (ret) + return ret; + } + + /* Apply format on the subdev. */ int ret = subdev_->setFormat(pad_, format); if (ret) return ret; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index d4e523af24b4..a819884f762d 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -15,6 +15,7 @@ #include #include #include +#include #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/framebuffer.h" @@ -177,10 +178,12 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index) /** * \brief Configure the CIO2 unit * \param[in] size The requested CIO2 output frame size + * \param[in] transform The transformation to be applied on the image sensor * \param[out] outputFormat The CIO2 unit output image format * \return 0 on success or a negative error code otherwise */ -int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) +int CIO2Device::configure(const Size &size, const Transform &transform, + V4L2DeviceFormat *outputFormat) { V4L2SubdeviceFormat sensorFormat; int ret; @@ -191,6 +194,7 @@ int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) */ std::vector mbusCodes = utils::map_keys(mbusCodesToPixelFormat); sensorFormat = getSensorFormat(mbusCodes, size); + sensorFormat.transform = transform; ret = sensor_->setFormat(&sensorFormat); if (ret) return ret; diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index 68504a2da89d..bbd87eb8ceb6 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -26,6 +26,7 @@ class Request; class Size; class SizeRange; struct StreamConfiguration; +enum class Transform; class CIO2Device { @@ -38,7 +39,8 @@ public: std::vector sizes(const PixelFormat &format) const; int init(const MediaDevice *media, unsigned int index); - int configure(const Size &size, V4L2DeviceFormat *outputFormat); + int configure(const Size &size, const Transform &transform, + V4L2DeviceFormat *outputFormat); StreamConfiguration generateConfiguration(Size size) const; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index a424ac914859..3a569c7e0031 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -51,7 +51,7 @@ class IPU3CameraData : public Camera::Private { public: IPU3CameraData(PipelineHandler *pipe) - : Camera::Private(pipe), supportsFlips_(false) + : Camera::Private(pipe) { } @@ -73,7 +73,6 @@ public: Stream rawStream_; Rectangle cropRegion_; - bool supportsFlips_; Transform rotationTransform_; std::unique_ptr delayedCtrls_; @@ -539,7 +538,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) */ const Size &sensorSize = config->cio2Format().size; V4L2DeviceFormat cio2Format; - ret = cio2->configure(sensorSize, &cio2Format); + ret = cio2->configure(sensorSize, config->combinedTransform_, &cio2Format); if (ret) return ret; @@ -547,24 +546,6 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) cio2->sensor()->sensorInfo(&sensorInfo); data->cropRegion_ = sensorInfo.analogCrop; - /* - * Configure the H/V flip controls based on the combination of - * the sensor and user transform. - */ - if (data->supportsFlips_) { - ControlList sensorCtrls(cio2->sensor()->controls()); - sensorCtrls.set(V4L2_CID_HFLIP, - static_cast(!!(config->combinedTransform_ - & Transform::HFlip))); - sensorCtrls.set(V4L2_CID_VFLIP, - static_cast(!!(config->combinedTransform_ - & Transform::VFlip))); - - ret = cio2->sensor()->setControls(&sensorCtrls); - if (ret) - return ret; - } - /* * If the ImgU gets configured, its driver seems to expect that * buffers will be queued to its outputs, as otherwise the next @@ -1127,11 +1108,6 @@ int PipelineHandlerIPU3::registerCameras() LOG(IPU3, Warning) << "Invalid rotation of " << rotationValue << " degrees: ignoring"; - ControlList ctrls = cio2->sensor()->getControls({ V4L2_CID_HFLIP }); - if (!ctrls.empty()) - /* We assume the sensor supports VFLIP too. */ - data->supportsFlips_ = true; - /** * \todo Dynamically assign ImgU and output devices to each * stream and camera; as of now, limit support to two cameras diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index e83984985bce..a74d2fc65ffe 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -690,24 +690,12 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) } } - /* - * Configure the H/V flip controls based on the combination of - * the sensor and user transform. - */ - if (data->supportsFlips_) { - const RPiCameraConfiguration *rpiConfig = - static_cast(config); - ControlList controls; - - controls.set(V4L2_CID_HFLIP, - static_cast(!!(rpiConfig->combinedTransform_ & Transform::HFlip))); - controls.set(V4L2_CID_VFLIP, - static_cast(!!(rpiConfig->combinedTransform_ & Transform::VFlip))); - data->setSensorControls(controls); - } - /* First calculate the best sensor mode we can use based on the user request. */ V4L2SubdeviceFormat sensorFormat = findBestFormat(data->sensorFormats_, rawStream ? sensorSize : maxSize, bitDepth); + /* Apply any cached transform. */ + const RPiCameraConfiguration *rpiConfig = static_cast(config); + sensorFormat.transform = rpiConfig->combinedTransform_; + /* Finally apply the format on the sensor. */ ret = data->sensor_->setFormat(&sensorFormat); if (ret) return ret; @@ -1292,10 +1280,9 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me * We cache three things about the sensor in relation to transforms * (meaning horizontal and vertical flips). * - * Firstly, does it support them? - * Secondly, if you use them does it affect the Bayer ordering? - * Thirdly, what is the "native" Bayer order, when no transforms are - * applied? + * If flips are supported verify if they affect the Bayer ordering + * and what the "native" Bayer order is, when no transforms are + * applied. * * We note that the sensor's cached list of supported formats is * already in the "native" order, with any flips having been undone. diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 15e8206a915c..78e05a90f711 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -216,6 +216,16 @@ const std::map formatInfoMap = { * resulting color space is acceptable. */ +/** + * \var V4L2SubdeviceFormat::transform + * \brief The transformations (vertical/horizontal flips) to be applied on the image + * + * Transformations applied on RAW image sensor subdevices might alter the Bayer + * pattern ordering. + * + * Default initialized to Identity (no transform). + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat From patchwork Thu Nov 24 12:12:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 17893 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D059FC3286 for ; Thu, 24 Nov 2022 12:12:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7DA196331C; Thu, 24 Nov 2022 13:12:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669291956; bh=vSMmQ4yA17arU9532ACoTt7taXLmykxR2xU0L3KTQWs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=pbusvtMvFXb3j4yhNRxXJb0OP3yufktU0s6KRDVtthoxHcmoxM6K17zSbomMicCiK hBLjxuy0pLkImaJX1DaY43zmAgyU/7DXI+NUyS5u9lCWkYrLv+AF2l2AX5rYe+B2ol smTe5raZOTmF5KqlPNQOpKKHoyVgyLIkdGgHMPl8jfdfv9kphyA40MsTQxiIVVnznx 6Jckiw4NpOvYp80JUmVotqHBHeiBVLfviH27B5/36kkJMPARe6v1Um756vHY2wZHgz UTvchFlZUEDkxzy9ijJTWGlMGJhmZJDrqBX0ej7ZSBX7P/A1O1GjIUNV1wQ5bQy5uc W8VeOiJipY/Mw== Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::222]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B03396332A for ; Thu, 24 Nov 2022 13:12:33 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id D645640002; Thu, 24 Nov 2022 12:12:32 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 13:12:18 +0100 Message-Id: <20221124121220.47000-8-jacopo@jmondi.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221124121220.47000-1-jacopo@jmondi.org> References: <20221124121220.47000-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 7/9] libcamera: rkisp1: Re-sort includes X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Comply with a checkstyle suggestion and separate inclusion directives. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 3d3a7086c03a..14741e73e050 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -23,11 +23,13 @@ #include #include #include +#include +#include +#include + #include #include #include -#include -#include #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" From patchwork Thu Nov 24 12:12:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 17894 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 885BABDE6B for ; Thu, 24 Nov 2022 12:12:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 45F1863325; Thu, 24 Nov 2022 13:12:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669291957; bh=UGOOsJM9Ac1SccGfvBu28Xir5AaMsbXHKUYraf86zNY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ChfrGvrQzxYCiaSstYQJksiFXBn52m3VC6isYfhLrztB6V1RNoZWAagzflTavUVi0 kV5KLj08kAD+JaWUs6FAcDWvH47Xe2s1WEq6EU8k1ANx+/piiJWDTg+CZqoNuGtb8A u1RHKlicTAsJL12K+k76YaJOWk4i6hDVPYh8gTDFjDfMkgEkIBqwRUCzCF91xYK6GE 2O2KCZ5j7csGrVNTGOH21nm8qZgXx6URWn12c7I6zWyJd1K7WDFXLYtuy2f7E2I6hH xWW1CozK55ZMlQssg0H23IJMHJ74JoYiGTrz7EhSiladQ8HcLWAmp5A28jteRToeR4 4mwa8awa4nrZw== Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AD8AF63327 for ; Thu, 24 Nov 2022 13:12:34 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id D701E4000C; Thu, 24 Nov 2022 12:12:33 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 13:12:19 +0100 Message-Id: <20221124121220.47000-9-jacopo@jmondi.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221124121220.47000-1-jacopo@jmondi.org> References: <20221124121220.47000-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 8/9] libcamera: rkisp1: Add support for Transform X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for Transform to the RkISP1 pipeline handler. The pipeline rotates using the sensor's V/H flips, hence use the CameraSensor helpers to handle transformation requests from applications. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 14741e73e050..41afc98ae2b2 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -26,6 +26,7 @@ #include #include #include +#include #include #include @@ -430,17 +431,17 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); - if (transform != Transform::Identity) { - transform = Transform::Identity; - status = Adjusted; - } - /* Cap the number of entries to the available streams. */ if (config_.size() > pathCount) { config_.resize(pathCount); status = Adjusted; } + Transform requestedTransform = transform; + Transform combined = sensor->validateTransform(&transform); + if (transform != requestedTransform) + status = Adjusted; + /* * If there are more than one stream in the configuration figure out the * order to evaluate the streams. The first stream has the highest @@ -529,6 +530,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() if (sensorFormat_.size.isNull()) sensorFormat_.size = sensor->resolution(); + sensorFormat_.transform = combined; + return status; } From patchwork Thu Nov 24 12:12:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 17895 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 49E03BDE6B for ; Thu, 24 Nov 2022 12:12:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 08CED6331E; Thu, 24 Nov 2022 13:12:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669291959; bh=1Y9XURxrGeqSini5H73FcuTfYF7+giYbLY/41RfhXVs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=0WDXC+s+md/+5rzgDMmvuy2n0UvpHsg0kNCr3ubYEwoAziAE9FQnQJKmD/LjTa8Hg LOUPb/hQ3iQKFLqxlnhnHldoFnFBJoBRjBlEkBYDRDARo8R5QaLD1I1rRouUl/KCR4 JitfGgIxwxpi11mMIVaNbmG7e72OY313HzzNFUpJXJ6hlNaOrHFUjP0NGeLzLk0WSQ DYHww7plVxTUYxDsL8xsG4OoFgNiNbQSNZRC9InUrVfMcpp0L9h0c86e97SckLxl7q 87GsnDHK0GU9aOKycACBRJKgUx251E+P78bKnkU5WtXMEFPbBXDBnlmXh9nnEcTOxt Lm3/NbV9IT9fQ== Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B306D6331A for ; Thu, 24 Nov 2022 13:12:35 +0100 (CET) Received: (Authenticated sender: jacopo@jmondi.org) by mail.gandi.net (Postfix) with ESMTPSA id DECDC40002; Thu, 24 Nov 2022 12:12:34 +0000 (UTC) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 13:12:20 +0100 Message-Id: <20221124121220.47000-10-jacopo@jmondi.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221124121220.47000-1-jacopo@jmondi.org> References: <20221124121220.47000-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 9/9] documentation: sensor_drivers: Document H/V flip X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Briefly document the optional requirement support for H/V flip controls. Signed-off-by: Jacopo Mondi --- Documentation/sensor_driver_requirements.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/Documentation/sensor_driver_requirements.rst b/Documentation/sensor_driver_requirements.rst index b0854be3328a..e9f998ca8d31 100644 --- a/Documentation/sensor_driver_requirements.rst +++ b/Documentation/sensor_driver_requirements.rst @@ -55,6 +55,19 @@ The sensor driver should support the following V4L2 controls: The controls are used to register the camera location and rotation. +In order to support rotating the image according to the above registered +properties the sensor driver should support + +* `V4L2_CID_HFLIP`_ +* `V4L2_CID_VFLIP`_ + +.. _V4L2_CID_HFLIP: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html +.. _V4L2_CID_VFLIP: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html + +and have them writable from userspace. In case of a RAW Bayer sensors, drivers +should correctly report if vertical/horizontal flips modify the Bayer pattern +ordering by reporting the `V4L2_CTRL_FLAG_MODIFY_LAYOUT` control flag. + The sensor driver should implement support for the V4L2 Selection API, specifically it should implement support for the `VIDIOC_SUBDEV_G_SELECTION`_ ioctl with support for the following selection