From patchwork Tue Jun 20 14:29:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 18751 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7EEAFBD78E for ; Tue, 20 Jun 2023 14:29:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 971D9628C4; Tue, 20 Jun 2023 16:29:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1687271359; bh=532uo/0a1TVcjNDAKzFvpD9O1ULqaKihMWI3Kw1Ptc8=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=4VVe91w6FmJ8Jej30CynBSH4E5qXgwph/rdNI+AyB9syia+m6iAnSYW0YqRyXnHhl +6t5P3Yz7rn2Y/T8Ft/m9FlIYGfI6J9Dz8zVjlnYOgqaGADbK/7oS/aOwBCYP0UZML 56W9sWnLzrVisN85D8LpoETjSUy/KeHjSxvnwXZfHQt37W7ldeDJ91Rcs2n8vD8aH6 SssSRJ5wf262+3w4+8wwd6cq5qKCHeWubAr9Ta0MD7GtJBDa1SdfRN1pPj2jdbeBpD JI+I59tfs5jn6E3aeQewuZ/XB6q6zGmAt7glc3Wh2vBfnEmUszOD8NVo0eBXJrJfwo IdymAXeZRdaxg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E6132617B2 for ; Tue, 20 Jun 2023 16:29:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="niK6bumC"; dkim-atps=neutral Received: from uno.lan (unknown [IPv6:2001:b07:5d2e:52c9:72c3:346:a663:c82d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9C9792B3; Tue, 20 Jun 2023 16:28:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1687271323; bh=532uo/0a1TVcjNDAKzFvpD9O1ULqaKihMWI3Kw1Ptc8=; h=From:To:Cc:Subject:Date:From; b=niK6bumCirJ+UdFlPe+SEZZPqMsdk3KeQXZSksnJBLPaLIvEIEqrGJF70MZCSyV35 Rk6Ijbf9dALYDMOcm+I/Oi9DkV4R8ebGBrl6gV5y0oLoQg0N7Hlwfgs//Oj7hl/Rbf BzSoJHqDxZMQ9xUk+6Ar6Yk430+YKjFKugNJxo/0= To: libcamera-devel@lists.libcamera.org Date: Tue, 20 Jun 2023 16:29:00 +0200 Message-Id: <20230620142904.72600-1-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Subject: [libcamera-devel] [RFC 0/4] libcamera: Replace CameraConfiguration::transform X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jacopo Mondi via libcamera-devel From: Jacopo Mondi Reply-To: Jacopo Mondi Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello everyone We have been discussing rotation/transformations for a long time, and the currently implemented CameraConfiguration::transform behaviour has proven to be confusing for several reasons: it was poorly documented, translating it to something consumable for upper layer frameworks was difficult and it behaved differently than any other field part of the CameraConfiguration, as it was meant to tell libcamera "what to do" instead of expressing what applications want. For these reasons this series proposes to replace the usage to Transform in the public API in favour of a new CameraConfiguration::Orientation type, defined based on the EXIF specification, tag 274, 'orientation'. For reference this is the same as implemented by gstreamer: https://gstreamer.freedesktop.org/documentation/gstreamer/gsttaglist.html?gi-language=c#GST_TAG_IMAGE_ORIENTATION The newly introduced CameraConfiguration::orientation replaces the existing CameraConfiguration::tranform, and it is meant for application to express how they would like the images to be oriented, not to tell libcamera what to do. As an example, passing in 'rotation0' means that the application expects the images to be rotated upright, and doesn't tell libcamera not to apply any rotation like passing in "Transform::Identity" did. The value CameraConfiguration::orientation is set to after a validate() also differs in meaning, as instead of reporting "what applications have to do to obtain what they originally asked for" it simply reports the actual orientation of the stream: this means that if libcamera cannot fully satisfy the user request it will set ::orientation to report the native images rotation and the CameraConfiguration::status will be set to Adjusted. Handling of 90 and 270 degrees rotation has also changed: as the camera sensor cannot correct rotations that include a transposition, requests for a 90/270 degrees are ignored, and cameras with a mounting rotation of 90/270 degrees are never re-oriented. This makes it clear and less confusing for applications that they have to deal with correction fully by themselves. As an example, with the current implementation if the application requires a Rot270 (HFlip + Transpose) libcamera will do the HFlip and leave transposition to the upper layers. There is no clear advantage in doing so, and display compositors are better suited for handling transpositions and flipping in a single pass instead of having the library try to handle part of that. This series clearly breaks the application API as it removes a member from CameraConfiguration, so it should be introduced probably only when a new release is cut. Tested on RPi imx219 with the newly introduced cam option, flip, mirror and rotation work as expected. Tested on Pinephone Pro, where sensors are 90/270 degrees rotated (libcamera performs no correction) Comments welcome of course. Thanks j Jacopo Mondi (4): libcamera: camera_sensor: Cache rotationTransform_ libcamera: camera: Introduce CameraConfiguration::orientation libcamera: Move to use CameraConfiguration::orientation apsp: cam: Add option to set stream orientation Documentation/Doxyfile.in | 2 + Documentation/rotation/flip-rotate-0.eps | 170 +++++++++++++++ Documentation/rotation/flip-rotate-0.png | Bin 0 -> 16488 bytes Documentation/rotation/flip-rotate-180.eps | 191 +++++++++++++++++ Documentation/rotation/flip-rotate-180.png | Bin 0 -> 22198 bytes Documentation/rotation/flip-rotate-270.eps | 197 ++++++++++++++++++ Documentation/rotation/flip-rotate-270.png | Bin 0 -> 21367 bytes Documentation/rotation/flip-rotate-90.eps | 195 +++++++++++++++++ Documentation/rotation/flip-rotate-90.png | Bin 0 -> 21020 bytes Documentation/rotation/rotate-0.eps | 169 +++++++++++++++ Documentation/rotation/rotate-0.png | Bin 0 -> 9086 bytes Documentation/rotation/rotate-180.eps | 189 +++++++++++++++++ Documentation/rotation/rotate-180.png | Bin 0 -> 22182 bytes Documentation/rotation/rotate-270.eps | 196 +++++++++++++++++ Documentation/rotation/rotate-270.png | Bin 0 -> 21016 bytes Documentation/rotation/rotate-90.eps | 196 +++++++++++++++++ Documentation/rotation/rotate-90.png | Bin 0 -> 21528 bytes include/libcamera/camera.h | 14 +- include/libcamera/internal/camera_sensor.h | 4 +- include/libcamera/transform.h | 4 + src/apps/cam/camera_session.cpp | 17 ++ src/apps/cam/main.cpp | 5 + src/apps/cam/main.h | 1 + src/libcamera/camera.cpp | 54 ++++- src/libcamera/camera_sensor.cpp | 137 ++++++------ src/libcamera/pipeline/ipu3/ipu3.cpp | 6 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 8 +- .../pipeline/rpi/common/pipeline_base.cpp | 9 +- src/libcamera/pipeline/simple/simple.cpp | 6 +- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 +- src/libcamera/pipeline/vimc/vimc.cpp | 4 +- src/libcamera/transform.cpp | 58 ++++++ 32 files changed, 1739 insertions(+), 97 deletions(-) create mode 100644 Documentation/rotation/flip-rotate-0.eps create mode 100644 Documentation/rotation/flip-rotate-0.png create mode 100644 Documentation/rotation/flip-rotate-180.eps create mode 100644 Documentation/rotation/flip-rotate-180.png create mode 100644 Documentation/rotation/flip-rotate-270.eps create mode 100644 Documentation/rotation/flip-rotate-270.png create mode 100644 Documentation/rotation/flip-rotate-90.eps create mode 100644 Documentation/rotation/flip-rotate-90.png create mode 100644 Documentation/rotation/rotate-0.eps create mode 100644 Documentation/rotation/rotate-0.png create mode 100644 Documentation/rotation/rotate-180.eps create mode 100644 Documentation/rotation/rotate-180.png create mode 100644 Documentation/rotation/rotate-270.eps create mode 100644 Documentation/rotation/rotate-270.png create mode 100644 Documentation/rotation/rotate-90.eps create mode 100644 Documentation/rotation/rotate-90.png --- 2.40.1