From patchwork Sat Jan 14 19:47:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 18112 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AEA1AC3240 for ; Sat, 14 Jan 2023 19:47:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A5F4C625E7; Sat, 14 Jan 2023 20:47:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1673725647; bh=fcPrR1vBAffWHMMstDC1JL0aPnhAGq2eJMsR3VGqptU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=0+6gIbtBYdTRmzM+2KsZ5pis7HEfzprh0jFkv3khKHufPR5F3QlODb8RnDtd+LBpq lYyT1zTb8WAv7mz6llKi2o057qf7ra1vqNgED1Dr1DjiOH9JY5WSWm+BLJ5lSoFIcx iULOlJhhKrKFbWfu5jIeRebMSVkNfc1wcn4UQfWZN6KWSTAcT7DkX8wvTj6X5c+jWF ctfh1VzRfn+AtulKA1IhbcmhR/Z+iWx0OFmEDSVMpu4t4ojjjrvkgDCaRpc/qTSv2u Vmgg//+JzQoEz9RLLBy9WN0vB79HqdzjSefx4p4NebcPC4KRjrMkdlL49krpQ3p16m M/k1qFCHrd7aw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 03368625D0 for ; Sat, 14 Jan 2023 20:47:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cRmJZqbg"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 289BF4DD; Sat, 14 Jan 2023 20:47:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1673725644; bh=fcPrR1vBAffWHMMstDC1JL0aPnhAGq2eJMsR3VGqptU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cRmJZqbgQgCqoik5uGqxtT5dgkdadR75d8lOS/dbyIM58nl12K2g+1jvTsRc4WEzj 9WhnqLkE/bgxeKYZeH74EvVzJ2AKSn/2Km+zLQHVthmm2WOU12fdciOf2u3CsBniPX wIWWvIMWSKdiOg5cOHRBktedhpDDb2acDWwee03E= To: libcamera-devel@lists.libcamera.org Date: Sat, 14 Jan 2023 20:47:08 +0100 Message-Id: <20230114194712.23272-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> References: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/5] libcamera: camera_sensor: Verify flips support X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" During the camera sensor driver validation, verify if the sensor supports horizontal and vertical flips and store a flag as CameraSensor::supportFlips_ class member. The flag will be later inspected when applying flips. Signed-off-by: Jacopo Mondi Reviewed-by: David Plowman --- include/libcamera/internal/camera_sensor.h | 1 + src/libcamera/camera_sensor.cpp | 18 +++++++++++++++++- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index b9f4d7867854..878f3c28a3c9 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -101,6 +101,7 @@ private: Size pixelArraySize_; Rectangle activeArea_; const BayerFormat *bayerFormat_; + bool supportFlips_; ControlList properties_; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index a210aa4fa370..83ac075a4265 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -55,7 +55,8 @@ LOG_DEFINE_CATEGORY(CameraSensor) */ CameraSensor::CameraSensor(const MediaEntity *entity) : entity_(entity), pad_(UINT_MAX), staticProps_(nullptr), - bayerFormat_(nullptr), properties_(properties::properties) + bayerFormat_(nullptr), supportFlips_(false), + properties_(properties::properties) { } @@ -247,6 +248,21 @@ int CameraSensor::validateSensorDriver() } } + /* + * Verify if sensor supports horizontal/vertical flips + * + * \todo Handle horizontal and vertical flips independently. + */ + const struct v4l2_query_ext_ctrl *hflipInfo = subdev_->controlInfo(V4L2_CID_HFLIP); + const struct v4l2_query_ext_ctrl *vflipInfo = subdev_->controlInfo(V4L2_CID_VFLIP); + if (hflipInfo && !(hflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY) && + vflipInfo && !(vflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY)) + supportFlips_ = true; + + if (!supportFlips_) + LOG(CameraSensor, Warning) + << "Camera sensor does not support horizontal/vertical flip"; + /* * Make sure the required selection targets are supported. * From patchwork Sat Jan 14 19:47:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 18113 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1FCD5C3240 for ; Sat, 14 Jan 2023 19:47:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BB986625EC; Sat, 14 Jan 2023 20:47:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1673725648; bh=CtRTJpV4d+DsfdLbYm0nj4IZEqeFkpUvjLsEMsL4KgY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=iMbaIN0fkKSXlkG+v7pdrcqms6fkut9pJucy+eJ/3wnVYuVnrVN+Lbu20G/0XmirI KnBMVvEg7st44nOJpZnv/iHOwh9Ida2mHtf5hOnzSkwwPDlH+jF6FB4vLgHB45TC+s ry1jXCVGBMR1CaoC2E4rogaTLwaf8KJdzhQTS8I0AutZCvd187Fko7ZyL1XnGUUlu0 4kK+hdmT2bEHGcJXWiS+PCrYz0wJEnAQPWYpQ2AuT7RN6y3xjM88R1Ven3Unr0/uwz oDyeH9NS3QkzuyWjDNivLvuwI8V3fuSBihj3UaJt9jAe17w7ONmvBcQip/SyWPim6e 5XhKTT6ZaZfag== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 81F05625D0 for ; Sat, 14 Jan 2023 20:47:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AZrHpNhi"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EAAA84D4; Sat, 14 Jan 2023 20:47:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1673725645; bh=CtRTJpV4d+DsfdLbYm0nj4IZEqeFkpUvjLsEMsL4KgY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AZrHpNhi/Dv2tGiAl4F5CxzbmYXDqu1CaJkEZ9XB7S/hjosa6QFMmbOWQ4xn0Slo6 erATmIngOulW+8XKKzr0edtDc0pyUJQFh0W0sn9lEJO0p5pv0TwZz/UfO6OY898QMP yFXSBFt+GzDofWgikx3uPSgjR5tWb3WL9xjbMs6o= To: libcamera-devel@lists.libcamera.org Date: Sat, 14 Jan 2023 20:47:09 +0100 Message-Id: <20230114194712.23272-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> References: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/5] libcamera: camera_sensor: Validate Transform X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The two pipeline handlers that currently support Transform (IPU3 and RkISP1) implement it by operating H/V flips on the image sensor. Centralize the code that validates a Transform request against the sensor rotation capabilities in the CameraSensor class. The implementation in the IPU3 pipeline handler was copied from the RaspberryPi implementation, and is now centralized in CameraSensor to make it easier for other platforms. The CameraSensor::validateTransform() implementation comes directly from the RaspberryPi pipeline handler, no functional changes intended. Signed-off-by: Jacopo Mondi Reviewed-by: David Plowman --- include/libcamera/internal/camera_sensor.h | 4 ++ src/libcamera/camera_sensor.cpp | 65 +++++++++++++++++++ src/libcamera/pipeline/ipu3/ipu3.cpp | 45 ++----------- .../pipeline/raspberrypi/raspberrypi.cpp | 59 ++--------------- 4 files changed, 82 insertions(+), 91 deletions(-) diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index 878f3c28a3c9..bea52badaff7 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -29,6 +29,8 @@ class BayerFormat; class CameraLens; class MediaEntity; +enum class Transform; + struct CameraSensorProperties; class CameraSensor : protected Loggable @@ -68,6 +70,8 @@ public: CameraLens *focusLens() { return focusLens_.get(); } + Transform validateTransform(Transform *transform) const; + protected: std::string logPrefix() const override; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 83ac075a4265..3518d3e3b02a 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -962,6 +962,71 @@ void CameraSensor::updateControlInfo() * connected to the sensor */ +/** + * \brief Validate a transform request against the sensor capabilities + * \param[inout] transform The requested transformation, updated to match + * the sensor capabilities + * + * The requested \a transform is adjusted to compensate for the sensor's + * mounting rotation and validated agains the sensor capabilities. + * + * For example, if the requested \a transform is Transform::Identity and the + * sensor rotation is 180 degrees, the resulting transform returned by the + * function is Transform::Rot180 to automatically correct the image, but only if + * the sensor can actually apply horizontal and vertical flips. + * + * \return A Transform instance that represents which transformation has been + * applied to the camera sensor + */ +Transform CameraSensor::validateTransform(Transform *transform) const +{ + /* Adjust the requested transform to compensate the sensor rotation. */ + int32_t rotation = properties().get(properties::Rotation).value_or(0); + bool success; + + Transform rotationTransform = transformFromRotation(rotation, &success); + if (!success) + LOG(CameraSensor, Warning) << "Invalid rotation of " << rotation + << " degrees - ignoring"; + + Transform combined = *transform * rotationTransform; + + /* + * We combine the platform and user transform, but must "adjust away" + * any combined result that includes a transform, as we can't do those. + * In this case, flipping only the transpose bit is helpful to + * applications - they either get the transform they requested, or have + * to do a simple transpose themselves (they don't have to worry about + * the other possible cases). + */ + if (!!(combined & Transform::Transpose)) { + /* + * Flipping the transpose bit in "transform" flips it in the + * combined result too (as it's the last thing that happens), + * which is of course clearing it. + */ + *transform ^= Transform::Transpose; + combined &= ~Transform::Transpose; + } + + /* + * We also check if the sensor doesn't do h/vflips at all, in which + * case we clear them, and the application will have to do everything. + */ + if (!supportFlips_ && !!combined) { + /* + * If the sensor can do no transforms, then combined must be + * changed to the identity. The only user transform that gives + * rise to this the inverse of the rotation. (Recall that + * combined = transform * rotationTransform.) + */ + *transform = -rotationTransform; + combined = Transform::Identity; + } + + return combined; +} + std::string CameraSensor::logPrefix() const { return "'" + entity_->name() + "'"; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e4d79ea44aed..a424ac914859 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -184,48 +184,15 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() if (config_.empty()) return Invalid; - Transform combined = transform * data_->rotationTransform_; - - /* - * We combine the platform and user transform, but must "adjust away" - * any combined result that includes a transposition, as we can't do - * those. In this case, flipping only the transpose bit is helpful to - * applications - they either get the transform they requested, or have - * to do a simple transpose themselves (they don't have to worry about - * the other possible cases). - */ - if (!!(combined & Transform::Transpose)) { - /* - * Flipping the transpose bit in "transform" flips it in the - * combined result too (as it's the last thing that happens), - * which is of course clearing it. - */ - transform ^= Transform::Transpose; - combined &= ~Transform::Transpose; - status = Adjusted; - } - /* - * We also check if the sensor doesn't do h/vflips at all, in which - * case we clear them, and the application will have to do everything. + * Validate the requested transform against the sensor capabilities and + * rotation and store the final combined transform that configure() will + * need to apply to the sensor to save us working it out again. */ - if (!data_->supportsFlips_ && !!combined) { - /* - * If the sensor can do no transforms, then combined must be - * changed to the identity. The only user transform that gives - * rise to this is the inverse of the rotation. (Recall that - * combined = transform * rotationTransform.) - */ - transform = -data_->rotationTransform_; - combined = Transform::Identity; + Transform requestedTransform = transform; + combinedTransform_ = data_->cio2_.sensor()->validateTransform(&transform); + if (transform != requestedTransform) status = Adjusted; - } - - /* - * Store the final combined transform that configure() will need to - * apply to the sensor to save us working it out again. - */ - combinedTransform_ = combined; /* Cap the number of entries to the available streams. */ if (config_.size() > kMaxStreams) { diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 8569df17976a..c086a69a913d 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -367,59 +367,14 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); /* - * What if the platform has a non-90 degree rotation? We can't even - * "adjust" the configuration and carry on. Alternatively, raising an - * error means the platform can never run. Let's just print a warning - * and continue regardless; the rotation is effectively set to zero. + * Validate the requested transform against the sensor capabilities and + * rotation and store the final combined transform that configure() will + * need to apply to the sensor to save us working it out again. */ - int32_t rotation = data_->sensor_->properties().get(properties::Rotation).value_or(0); - bool success; - Transform rotationTransform = transformFromRotation(rotation, &success); - if (!success) - LOG(RPI, Warning) << "Invalid rotation of " << rotation - << " degrees - ignoring"; - Transform combined = transform * rotationTransform; - - /* - * We combine the platform and user transform, but must "adjust away" - * any combined result that includes a transform, as we can't do those. - * In this case, flipping only the transpose bit is helpful to - * applications - they either get the transform they requested, or have - * to do a simple transpose themselves (they don't have to worry about - * the other possible cases). - */ - if (!!(combined & Transform::Transpose)) { - /* - * Flipping the transpose bit in "transform" flips it in the - * combined result too (as it's the last thing that happens), - * which is of course clearing it. - */ - transform ^= Transform::Transpose; - combined &= ~Transform::Transpose; - status = Adjusted; - } - - /* - * We also check if the sensor doesn't do h/vflips at all, in which - * case we clear them, and the application will have to do everything. - */ - if (!data_->supportsFlips_ && !!combined) { - /* - * If the sensor can do no transforms, then combined must be - * changed to the identity. The only user transform that gives - * rise to this the inverse of the rotation. (Recall that - * combined = transform * rotationTransform.) - */ - transform = -rotationTransform; - combined = Transform::Identity; + Transform requestedTransform = transform; + combinedTransform_ = data_->sensor_->validateTransform(&transform); + if (transform != requestedTransform) status = Adjusted; - } - - /* - * Store the final combined transform that configure() will need to - * apply to the sensor to save us working it out again. - */ - combinedTransform_ = combined; unsigned int rawCount = 0, outCount = 0, count = 0, maxIndex = 0; std::pair outSize[2]; @@ -454,7 +409,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (data_->flipsAlterBayerOrder_) { BayerFormat bayer = BayerFormat::fromV4L2PixelFormat(fourcc); bayer.order = data_->nativeBayerOrder_; - bayer = bayer.transform(combined); + bayer = bayer.transform(combinedTransform_); fourcc = bayer.toV4L2PixelFormat(); } From patchwork Sat Jan 14 19:47:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 18114 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9758C3293 for ; Sat, 14 Jan 2023 19:47:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 54567625EB; Sat, 14 Jan 2023 20:47:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1673725649; bh=8sBhE2PAmURrrlB3B3CqN6E5XAEjx6Y/yqzQIH5iXgQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=aJoj7wq2kQq6mPCSBeRchzeBR6tFmzg51ruFjT0Yg4HsGXK3JN3SE+zoE9SgR72jf RylMFoitfajcl12sjwB8Q/4mFcXEB60kLxsPI9AD9f8ydGDBSY+LqUhDkpgrs10EFR v65ETf/Fy65qBdiqCsmCqJvsBkRgO8w9HYiC7XOvy6fVRiXO1zbeLyuCFYmFbRKhWg ZrLe2i6QH6pg3CQ43SxFAzAKsb7UGIfei0vqCyZ/Fl1QrqgX9269PtNxxFhe35m0uI gj0biYuQrlV6D+TvxARGmTpxRsVyQgm/wWmVtELKSPElBnf8M+7s7ZDdUN3/b1ydO6 zhGlybl3PWJxg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0C6B4625D0 for ; Sat, 14 Jan 2023 20:47:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="g6lS0Tm+"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 755096CF; Sat, 14 Jan 2023 20:47:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1673725645; bh=8sBhE2PAmURrrlB3B3CqN6E5XAEjx6Y/yqzQIH5iXgQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g6lS0Tm+MeIpHinv4HwiYMKIYr/Txjd4yrgXVTgTbjQ5FqQAjYeAbXwcNVJ2rZgPa shGiijCc6tx2wLYe2VcvQXuvdJPsxet67hJxkKU7xznEzEniO5D4lEAPorib74Ajrd vMVO6lOqSTmJKYWUyEtgJ6TfgwTzgHQoLBZYbncc= To: libcamera-devel@lists.libcamera.org Date: Sat, 14 Jan 2023 20:47:10 +0100 Message-Id: <20230114194712.23272-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> References: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/5] libcamera: camera_sensor: Apply flips at setFormat() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Augment the CameraSensor::setFormat() function to configure horizontal and vertical flips before applying the image format on the sensor. Applying flips before format is crucial as they might change the Bayer pattern ordering. To allow users of the CameraSensor class to specify a Transform, add to the V4L2SubdeviceFormat class a 'transform' member, by default initialized to Transform::Identity. Moving the handling of H/V flips to the CameraSensor class allows to remove quite some boilerplate code from the IPU3 and RaspberryPi pipeline handlers. No functional changes intended. Signed-off-by: Jacopo Mondi Reviewed-by: David Plowman --- include/libcamera/internal/v4l2_subdevice.h | 2 ++ src/libcamera/camera_sensor.cpp | 23 +++++++++++++++ src/libcamera/pipeline/ipu3/cio2.cpp | 6 +++- src/libcamera/pipeline/ipu3/cio2.h | 4 ++- src/libcamera/pipeline/ipu3/ipu3.cpp | 28 ++----------------- .../pipeline/raspberrypi/raspberrypi.cpp | 27 +++++------------- src/libcamera/v4l2_subdevice.cpp | 7 +++++ 7 files changed, 49 insertions(+), 48 deletions(-) diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h index 69862de0585a..576faf971a05 100644 --- a/include/libcamera/internal/v4l2_subdevice.h +++ b/include/libcamera/internal/v4l2_subdevice.h @@ -20,6 +20,7 @@ #include #include +#include #include "libcamera/internal/formats.h" #include "libcamera/internal/media_object.h" @@ -44,6 +45,7 @@ struct V4L2SubdeviceFormat { uint32_t mbus_code; Size size; std::optional colorSpace; + Transform transform = Transform::Identity; const std::string toString() const; uint8_t bitsPerPixel() const; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 3518d3e3b02a..6d5c2317e0fe 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -750,6 +750,7 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu .mbus_code = bestCode, .size = *bestSize, .colorSpace = ColorSpace::Raw, + .transform = Transform::Identity, }; return format; @@ -759,12 +760,34 @@ V4L2SubdeviceFormat CameraSensor::getFormat(const std::vector &mbu * \brief Set the sensor output format * \param[in] format The desired sensor output format * + * If flips are writable they are configured according to the desired Transform. + * Transform::Identity always corresponds to H/V flip being disabled if the + * controls are writable. Flips are set before the new format is applied as + * they can effectively change the Bayer pattern ordering. + * * The ranges of any controls associated with the sensor are also updated. * * \return 0 on success or a negative error code otherwise */ int CameraSensor::setFormat(V4L2SubdeviceFormat *format) { + /* Configure flips if the sensor supports that. */ + if (supportFlips_) { + ControlList flipCtrls(subdev_->controls()); + + flipCtrls.set(V4L2_CID_HFLIP, + static_cast(!!(format->transform & + Transform::HFlip))); + flipCtrls.set(V4L2_CID_VFLIP, + static_cast(!!(format->transform & + Transform::VFlip))); + + int ret = subdev_->setControls(&flipCtrls); + if (ret) + return ret; + } + + /* Apply format on the subdev. */ int ret = subdev_->setFormat(pad_, format); if (ret) return ret; diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index d4e523af24b4..a819884f762d 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -15,6 +15,7 @@ #include #include #include +#include #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/framebuffer.h" @@ -177,10 +178,12 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index) /** * \brief Configure the CIO2 unit * \param[in] size The requested CIO2 output frame size + * \param[in] transform The transformation to be applied on the image sensor * \param[out] outputFormat The CIO2 unit output image format * \return 0 on success or a negative error code otherwise */ -int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) +int CIO2Device::configure(const Size &size, const Transform &transform, + V4L2DeviceFormat *outputFormat) { V4L2SubdeviceFormat sensorFormat; int ret; @@ -191,6 +194,7 @@ int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) */ std::vector mbusCodes = utils::map_keys(mbusCodesToPixelFormat); sensorFormat = getSensorFormat(mbusCodes, size); + sensorFormat.transform = transform; ret = sensor_->setFormat(&sensorFormat); if (ret) return ret; diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index 68504a2da89d..bbd87eb8ceb6 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -26,6 +26,7 @@ class Request; class Size; class SizeRange; struct StreamConfiguration; +enum class Transform; class CIO2Device { @@ -38,7 +39,8 @@ public: std::vector sizes(const PixelFormat &format) const; int init(const MediaDevice *media, unsigned int index); - int configure(const Size &size, V4L2DeviceFormat *outputFormat); + int configure(const Size &size, const Transform &transform, + V4L2DeviceFormat *outputFormat); StreamConfiguration generateConfiguration(Size size) const; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index a424ac914859..3a569c7e0031 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -51,7 +51,7 @@ class IPU3CameraData : public Camera::Private { public: IPU3CameraData(PipelineHandler *pipe) - : Camera::Private(pipe), supportsFlips_(false) + : Camera::Private(pipe) { } @@ -73,7 +73,6 @@ public: Stream rawStream_; Rectangle cropRegion_; - bool supportsFlips_; Transform rotationTransform_; std::unique_ptr delayedCtrls_; @@ -539,7 +538,7 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) */ const Size &sensorSize = config->cio2Format().size; V4L2DeviceFormat cio2Format; - ret = cio2->configure(sensorSize, &cio2Format); + ret = cio2->configure(sensorSize, config->combinedTransform_, &cio2Format); if (ret) return ret; @@ -547,24 +546,6 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) cio2->sensor()->sensorInfo(&sensorInfo); data->cropRegion_ = sensorInfo.analogCrop; - /* - * Configure the H/V flip controls based on the combination of - * the sensor and user transform. - */ - if (data->supportsFlips_) { - ControlList sensorCtrls(cio2->sensor()->controls()); - sensorCtrls.set(V4L2_CID_HFLIP, - static_cast(!!(config->combinedTransform_ - & Transform::HFlip))); - sensorCtrls.set(V4L2_CID_VFLIP, - static_cast(!!(config->combinedTransform_ - & Transform::VFlip))); - - ret = cio2->sensor()->setControls(&sensorCtrls); - if (ret) - return ret; - } - /* * If the ImgU gets configured, its driver seems to expect that * buffers will be queued to its outputs, as otherwise the next @@ -1127,11 +1108,6 @@ int PipelineHandlerIPU3::registerCameras() LOG(IPU3, Warning) << "Invalid rotation of " << rotationValue << " degrees: ignoring"; - ControlList ctrls = cio2->sensor()->getControls({ V4L2_CID_HFLIP }); - if (!ctrls.empty()) - /* We assume the sensor supports VFLIP too. */ - data->supportsFlips_ = true; - /** * \todo Dynamically assign ImgU and output devices to each * stream and camera; as of now, limit support to two cameras diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index c086a69a913d..d8232ff8e065 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -691,24 +691,12 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) } } - /* - * Configure the H/V flip controls based on the combination of - * the sensor and user transform. - */ - if (data->supportsFlips_) { - const RPiCameraConfiguration *rpiConfig = - static_cast(config); - ControlList controls; - - controls.set(V4L2_CID_HFLIP, - static_cast(!!(rpiConfig->combinedTransform_ & Transform::HFlip))); - controls.set(V4L2_CID_VFLIP, - static_cast(!!(rpiConfig->combinedTransform_ & Transform::VFlip))); - data->setSensorControls(controls); - } - /* First calculate the best sensor mode we can use based on the user request. */ V4L2SubdeviceFormat sensorFormat = findBestFormat(data->sensorFormats_, rawStream ? sensorSize : maxSize, bitDepth); + /* Apply any cached transform. */ + const RPiCameraConfiguration *rpiConfig = static_cast(config); + sensorFormat.transform = rpiConfig->combinedTransform_; + /* Finally apply the format on the sensor. */ ret = data->sensor_->setFormat(&sensorFormat); if (ret) return ret; @@ -1293,10 +1281,9 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me * We cache three things about the sensor in relation to transforms * (meaning horizontal and vertical flips). * - * Firstly, does it support them? - * Secondly, if you use them does it affect the Bayer ordering? - * Thirdly, what is the "native" Bayer order, when no transforms are - * applied? + * If flips are supported verify if they affect the Bayer ordering + * and what the "native" Bayer order is, when no transforms are + * applied. * * We note that the sensor's cached list of supported formats is * already in the "native" order, with any flips having been undone. diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp index 15e8206a915c..38ff8b0c605b 100644 --- a/src/libcamera/v4l2_subdevice.cpp +++ b/src/libcamera/v4l2_subdevice.cpp @@ -216,6 +216,13 @@ const std::map formatInfoMap = { * resulting color space is acceptable. */ +/** + * \var V4L2SubdeviceFormat::transform + * \brief The transform (vertical/horizontal flips) to be applied on the subdev + * + * Default initialized to Identity (no transform). + */ + /** * \brief Assemble and return a string describing the format * \return A string describing the V4L2SubdeviceFormat From patchwork Sat Jan 14 19:47:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 18115 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0602CC3294 for ; Sat, 14 Jan 2023 19:47:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3A15A625DD; Sat, 14 Jan 2023 20:47:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1673725650; bh=BKsSbLfnR3Agv3+/w4J99xXG8aa2GLCFMdmlQ3kkPOo=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=T/PfXCuUYuAjuSh4+rhKvQH8xUqT+toaz01776RZqVW0PoOJd37HefmCy4YksvjTE LTZXS79LOyMXXkNHvSSIU5bPymEW6wil2axSJoeppepORNpkSG6Om1mnnmBULxkZRm ItID4zaijj3LciJat6ZQT28GPoK2do/gjN9ifQTVJvOxYWglYmoPf3oN4iPDMeN9FI pvMph9gxrMJZLGAFl4a3FUDezu3C4odxFdihRu2nJuHEtrgfxYp9hu2oIEUvDcRnoZ VR1Cd3qIjfVNdPhzbqdZZzhMOXiRxdewh+2R2WHi6aoSfbSAhJEL2cDOYEQDbCuL/3 xI4NrXXew3l8g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 86C9D625DD for ; Sat, 14 Jan 2023 20:47:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cjt5Lnq7"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EE35C4D4; Sat, 14 Jan 2023 20:47:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1673725646; bh=BKsSbLfnR3Agv3+/w4J99xXG8aa2GLCFMdmlQ3kkPOo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cjt5Lnq7+ltt1q+Io1Nj5GIYkeQ493K7qry/DZkBM2kVgxgkd9mXsqvuA90u/DoR1 Of8PrQnKVUmN18VBdL5+enIi1rqeLT7Jr3o2n4LFTDq8vEFcZrg7x9hW0O91wuxUDa yFG/C+/B+7eCYumRI1CrDT3qKexnZbNkRb6D37vo= To: libcamera-devel@lists.libcamera.org Date: Sat, 14 Jan 2023 20:47:11 +0100 Message-Id: <20230114194712.23272-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> References: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/5] libcamera: rkisp1: Add support for Transform X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for Transform to the RkISP1 pipeline handler. The pipeline rotates using the sensor's V/H flips, hence use the CameraSensor helpers to handle transformation requests from applications. Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c27b3ef9bd8e..b81ad647be79 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -23,6 +23,10 @@ #include #include #include +#include +#include +#include + #include #include #include @@ -469,17 +473,17 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); - if (transform != Transform::Identity) { - transform = Transform::Identity; - status = Adjusted; - } - /* Cap the number of entries to the available streams. */ if (config_.size() > pathCount) { config_.resize(pathCount); status = Adjusted; } + Transform requestedTransform = transform; + Transform combined = sensor->validateTransform(&transform); + if (transform != requestedTransform) + status = Adjusted; + /* * Simultaneous capture of raw and processed streams isn't possible. If * there is any raw stream, cap the number of streams to one. @@ -589,6 +593,8 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() if (sensorFormat_.size.isNull()) sensorFormat_.size = sensor->resolution(); + sensorFormat_.transform = combined; + return status; } From patchwork Sat Jan 14 19:47:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 18116 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 63D5CC3295 for ; Sat, 14 Jan 2023 19:47:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C3E2E625F6; Sat, 14 Jan 2023 20:47:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1673725651; bh=7Y5lZedw3sQCQu5BnIQLNkw7XghiBC6Se9sQDo5K4PU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=28L9Fv7MnYdvrwuBoD0zue0O61sZCZ0op9rtiSfX9hQjaDWtK/yS5KCnjnbTQxTqo nDOCXWyitTZVea4MuzY243x1V+G87GCeXhoJtfDIRfKLZ/tSQ8gMtArqFkF8Ukpg95 SxWHEqXdW0kfI8/NV2XZF+vbr9Gl7K4Ke7SuFU9QtTiXufCzlyiu6nnCJghRd/33hG In78U7wY5lUG5gWMzK7rlSYBxFKDcaawP1eb9CxuEWQWRDi5NRIm0kg74Szfo/+sYP fhR4FZZ7O5svWSNuo4hSOCwkcRRCgdHxDuqUZPJyOQDDrOip5NgzhoJGScwcb9OiK0 EMjSrwJxN3F9g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 63115625E6 for ; Sat, 14 Jan 2023 20:47:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="u4/1hqeo"; dkim-atps=neutral Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 77DE54DD; Sat, 14 Jan 2023 20:47:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1673725646; bh=7Y5lZedw3sQCQu5BnIQLNkw7XghiBC6Se9sQDo5K4PU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u4/1hqeot/o7PGNmPBXs0X1n99YLNZHETzqnQJNt3gsCu3/dao7zF3ElXnPHyspVr FFxeGwLousnG2o9bYL536kJ7/jNCz4crvLRB3jXDw79Wbl9sf9KiZutojqqLhindom kLoyTIMDKhSuC5euy+XBCkNHcTbclNBd52K3Uk2k= To: libcamera-devel@lists.libcamera.org Date: Sat, 14 Jan 2023 20:47:12 +0100 Message-Id: <20230114194712.23272-6-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> References: <20230114194712.23272-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/5] documentation: sensor_drivers: Document H/V flip X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Briefly document the optional requirement support for H/V flip controls. Signed-off-by: Jacopo Mondi Reviewed-by: David Plowman --- Documentation/sensor_driver_requirements.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/Documentation/sensor_driver_requirements.rst b/Documentation/sensor_driver_requirements.rst index 3abc8f35924a..0e516b34a215 100644 --- a/Documentation/sensor_driver_requirements.rst +++ b/Documentation/sensor_driver_requirements.rst @@ -62,6 +62,18 @@ The sensor driver should support the following V4L2 controls: The controls are used to register the camera location and rotation. +In order to support rotating the image the sensor driver should support + +* `V4L2_CID_HFLIP`_ +* `V4L2_CID_VFLIP`_ + +.. _V4L2_CID_HFLIP: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html +.. _V4L2_CID_VFLIP: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/control.html + +The controls must be writable from userspace. In case of a RAW Bayer sensors, +drivers should correctly report if vertical/horizontal flips modify the Bayer +pattern ordering by reporting the `V4L2_CTRL_FLAG_MODIFY_LAYOUT` control flag. + The sensor driver should implement support for the V4L2 Selection API, specifically it should implement support for the `VIDIOC_SUBDEV_G_SELECTION`_ ioctl with support for the following selection