From patchwork Thu Feb 2 11:12:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthias Fend X-Patchwork-Id: 18229 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 08BB5C323E for ; Thu, 2 Feb 2023 11:13:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1DD01625E7; Thu, 2 Feb 2023 12:13:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1675336385; bh=lehFmaCtGSjQqupTtJdt35QhfASUdACeq7GbZsOXiYs=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=k8dQEV/ZTNAV6AhGRph15YIssfbsbLlm4q6Z82mPd4/dhpn2k/gWcGw6gLvoMc+Jp 8NCmMgXEhBMh4dvJUprNicvZflQFvPEJnBTiHmNYLNRT5B1KzlysPtw3+D2wBUpv43 4qIQWJU0FsDu5LyzVa4+vIVUp33/aWlF85Re+X0BcctqNamcEbFJqUyLWfll31VqTm B48bLK7xxCHgF/EHbADGrM8QUSkdukn/FKDNroZnpVB9b0BFn+geJraSYWO+OtOVaB vOBtlQqXXaNH+1EPLLFJA1r6fPEF6PN+VqLZxDeAfzNie3OqJytj4hNWNIm13dOU5G RWJlQb152W94A== Received: from lx20.hoststar.hosting (lx20.hoststar.hosting [168.119.41.54]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9F727603BF for ; Thu, 2 Feb 2023 12:13:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=emfend.at header.i=@emfend.at header.b="fDsGFWgQ"; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=emfend.at; s=mail; h=Content-Transfer-Encoding:MIME-Version:Message-Id:Date:Subject:Cc: To:From:Sender:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=tUFFdqmjZ/RYMhgydynCEWAokJxPaGLe9XEpQodQKr4=; b=fDsGFWgQ5en66C4fQUdb2IXipc 9vot7SzaidUnONbrmhCJEUnfzai1l2mA5QilRuR22xiVr2XueTkNwt/ydpz29h+VUQosb5Rq7HkHc dpVxZ1JKKYi8muJcyODiDXFmqc7t1E9Wq+UidC1dPcqk5n/DK6aPGJrFRCw/v0FXdNew=; Received: from 194-208-208-245.tele.net ([194.208.208.245]:51050 helo=localhost.localdomain) by lx20.hoststar.hosting with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.93) (envelope-from ) id 1pNXWX-008Uhs-DO; Thu, 02 Feb 2023 12:13:01 +0100 To: libcamera-devel@lists.libcamera.org Date: Thu, 2 Feb 2023 12:12:45 +0100 Message-Id: <20230202111252.3366729-1-matthias.fend@emfend.at> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Spam-Score: X-Spam-Bar: X-Spam-Report: Subject: [libcamera-devel] [RFC PATCH 0/7] add support for more complex optics X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Matthias Fend via libcamera-devel From: Matthias Fend Reply-To: Matthias Fend Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hello, as mentioned on the mailing list some time ago, I was wondering how to use more complex optics together with libcamera IPAs. Currently, only optics with a VCM focus lens are considered in libcamera. With these, an absolute position is set and the lens is almost immediately at the new desired position. Pretty simple. For a slightly more complex optics there are the following differences: - There may be multiple lens groups with different functions (zoom, focus, or auxiliary lens). - The lens groups are often controlled by stepper motors and thus need some time to reach the new position. - It may be necessary to move some lens group(s) synchronously (in realtime) depending on another lens group(s). - To implement these requirements, dedicated intelligent controller hardware is often necessary. - Depending on the application, calibration of the optics or lens groups may be necessary. - With such an optics controller, the current status or position of the lens group(s) can change at any time, and the application must read this information from it. In my current case, the control of the optics controller, in analogy to the VCM drivers, was also implemented as a v4l2 device. Since this controller requires controls that go beyond the currently available v4l2 controls, this driver must use some custom or at least adapted existing v4l2 controls. The intention is to bring the missing v4l2 controls upstream as well, but if that doesn't fully succeed, there should still be a way to use those controls in an IPA. Currently, v4l2 controls are used by libcamera to transfer control information for the lens from the IPA to the pipeline handler. This transport form is probably just a utility, but I used the circumstance to transparently bring v4l2 controls from the IPA to the v4l-subdev via the pipeline handler. The IPA can thus use all possible controls at will and assume that they arrive in the driver exactly the same way. With this, I was able to meet my (control) requirements with minor changes. As mentioned at the beginning, it may be that only the dedicated optic controller knows where the lens groups actually are. To report back the current state or positions of the lens groups, the v4l2 subdev can provide read-only volatile controls. Something like this was not available in libcamera until now. To have the same interface for this direction, I decided to pass these controls transparently to the IPA as well. So that not all controls must be transferred, I filtered only controls, which can change themselves (volatile). To give you a little better idea, I have created this series of patches. The question now is whether this approach can in principle be integrated into libcamera? I look forward to feedback of all kinds. Thanks ~Matthias Matthias Fend (7): ipa: rkisp1: add basic camera lens support libcamera: camera_lens: add method to set controls on subdev pipeline: rkisp1: transparently pass lens controls from the IPA to the lens libcamera: camera_lens: add method to get controls from subdev libcamera: v4l2_device: add function to get a list of volatile control ids libcamera: camera_lens: add method to get a list of volatile controls ipa: rkisp1: pass volatile lens controls to IPA include/libcamera/internal/camera_lens.h | 3 +++ include/libcamera/internal/v4l2_device.h | 1 + include/libcamera/ipa/rkisp1.mojom | 7 ++++-- src/libcamera/camera_lens.cpp | 24 +++++++++++++++++++++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 27 ++++++++++++++++++++---- src/libcamera/v4l2_device.cpp | 15 +++++++++++++ 6 files changed, 71 insertions(+), 6 deletions(-)