From patchwork Thu Nov 26 09:51:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 10498 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 353D6BE08A for ; Thu, 26 Nov 2020 09:51:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 04D056343E; Thu, 26 Nov 2020 10:51:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PMbwYXZ+"; dkim-atps=neutral Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0EA8C60331 for ; Thu, 26 Nov 2020 10:51:31 +0100 (CET) Received: by mail-wr1-x432.google.com with SMTP id g14so1406722wrm.13 for ; Thu, 26 Nov 2020 01:51:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=rQ0WGBNuG8Fj3/AkxZ/Oo9TYfuJoTSqCXy8VdHSOzYw=; b=PMbwYXZ+8ZqyTiX7WgFknJB8bc9DL1PDGjqnlN1HGGAfA5hv8h264wWrHYlFCTOZtI bO9I1yHI5OYoa4gkPgXeKVPro5DkchF0NmoENSxk5tn3gz4NVPf6+6ORLx+GIqkcwFfC 4yAJ5yq9y7asDm7ZdeFveCLx6wXjZ3Vt8N/xTwsQjoEHue5uMfvgYtktfXZLj2V13NsG +pJaMO90V9rwISTSqYGOneGVaTTRYhSGDsthTdF9Geh2nsj7L9Av9b8CCXumN0AM6Nhq T4cKIgUmxUGEK6CicorGHzjj/osGJ/VPbWLXFpkKWbMAO0GvMEgc+CfaNoiMiNiwRNGF efkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=rQ0WGBNuG8Fj3/AkxZ/Oo9TYfuJoTSqCXy8VdHSOzYw=; b=BDRAxa8oUtj5zW9N4jAR7/JTk0hfbQgkVBYChZxUoYl5tWMhgyZc8BzfrANqjK9pTd vDa+R1FV7xQJN7cDFv+klLL0voPiWANgWAM2D6nt/SEccB8dwt0+5hqcB/esMhH4YteZ 3GVdq3zjWjGaQYB9i/ii6FkBb1DH7Fq83RYEyy25rQMdmyQfw1Dq5AVA4jZbInmQhCJm B/Vtsm+nTS4NFy/Itw6DDM/VOm3OYkoEZRvWQR9DUcOPUt2BFo+Oa3IVBU8WFuBHXw9a Erh0avqKMGTnz1wVicXuviSXP8v8gAtKYqBCUrjLLxWZyMx15crx5G2W6OUnnUZ+KfzL GR6w== X-Gm-Message-State: AOAM530p89OYcJiTkeuI6sRhFsBTDx9OJplELvAAZqkpeAciTpR4rGs5 hgsv4wZ82b3trny/M4Sgc5XIcsrWXdAvsg== X-Google-Smtp-Source: ABdhPJx28D5Mar3dqMWJDK70tAjuJK3H9zM63C7+K5SA9xZMkDK5jGO+PavXGTO0og/ADIOEiF8Isw== X-Received: by 2002:a5d:46c6:: with SMTP id g6mr2873907wrs.170.1606384290202; Thu, 26 Nov 2020 01:51:30 -0800 (PST) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id i11sm8176827wrm.1.2020.11.26.01.51.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Nov 2020 01:51:29 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Thu, 26 Nov 2020 09:51:24 +0000 Message-Id: <20201126095126.997055-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/3] libcamera: pipeline: Pass libcamera controls into pipeline_handler::start() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Applications now have the ability to pass in controls that need to be applied on startup, rather than doing it through Request where there might be some frames of delay in getting the controls applied. This commit adds the ability to pass in a set of libcamera controls into the pipeline handlers through the pipeline_handler::start() method. These controls are provided by the application through the camera::start() public API. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Jacopo Mondi --- Documentation/guides/pipeline-handler.rst | 4 ++-- include/libcamera/camera.h | 2 +- include/libcamera/internal/pipeline_handler.h | 2 +- src/libcamera/camera.cpp | 11 ++++++----- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 ++-- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 4 ++-- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 ++-- src/libcamera/pipeline/simple/simple.cpp | 4 ++-- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 ++-- src/libcamera/pipeline/vimc/vimc.cpp | 4 ++-- src/libcamera/pipeline_handler.cpp | 1 + 11 files changed, 23 insertions(+), 21 deletions(-) diff --git a/Documentation/guides/pipeline-handler.rst b/Documentation/guides/pipeline-handler.rst index 57aee455..63275a12 100644 --- a/Documentation/guides/pipeline-handler.rst +++ b/Documentation/guides/pipeline-handler.rst @@ -209,7 +209,7 @@ methods for the overridden class members. int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) override; - int start(Camera *camera) override; + int start(Camera *camera, ControlList *controls) override; void stop(Camera *camera) override; int queueRequestDevice(Camera *camera, Request *request) override; @@ -239,7 +239,7 @@ methods for the overridden class members. return -1; } - int PipelineHandlerVivid::start(Camera *camera) + int PipelineHandlerVivid::start(Camera *camera, ControlList *controls) { return -1; } diff --git a/include/libcamera/camera.h b/include/libcamera/camera.h index 5c5f1a05..f94f8599 100644 --- a/include/libcamera/camera.h +++ b/include/libcamera/camera.h @@ -103,7 +103,7 @@ public: std::unique_ptr createRequest(uint64_t cookie = 0); int queueRequest(Request *request); - int start(); + int start(ControlList *controls = nullptr); int stop(); private: diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index c12c8904..bd3c4a81 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -78,7 +78,7 @@ public: virtual int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) = 0; - virtual int start(Camera *camera) = 0; + virtual int start(Camera *camera, ControlList *controls) = 0; virtual void stop(Camera *camera) = 0; int queueRequest(Camera *camera, Request *request); diff --git a/src/libcamera/camera.cpp b/src/libcamera/camera.cpp index dffbd6bd..de9c6c86 100644 --- a/src/libcamera/camera.cpp +++ b/src/libcamera/camera.cpp @@ -943,9 +943,10 @@ int Camera::queueRequest(Request *request) /** * \brief Start capture from camera * - * Start the camera capture session. Once the camera is started the application - * can queue requests to the camera to process and return to the application - * until the capture session is terminated with \a stop(). + * Start the camera capture session, optionally providing a list of controls to + * action before starting. Once the camera is started the application can queue + * requests to the camera to process and return to the application until the + * capture session is terminated with \a stop(). * * \context This function may only be called when the camera is in the * Configured state as defined in \ref camera_operation, and shall be @@ -956,7 +957,7 @@ int Camera::queueRequest(Request *request) * \retval -ENODEV The camera has been disconnected from the system * \retval -EACCES The camera is not in a state where it can be started */ -int Camera::start() +int Camera::start(ControlList *controls) { Private *const d = LIBCAMERA_D_PTR(); @@ -967,7 +968,7 @@ int Camera::start() LOG(Camera, Debug) << "Starting capture"; ret = d->pipe_->invokeMethod(&PipelineHandler::start, - ConnectionTypeBlocking, this); + ConnectionTypeBlocking, this, controls); if (ret) return ret; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 4cedb32b..8a1918d5 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -105,7 +105,7 @@ public: int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) override; - int start(Camera *camera) override; + int start(Camera *camera, ControlList *controls) override; void stop(Camera *camera) override; int queueRequestDevice(Camera *camera, Request *request) override; @@ -596,7 +596,7 @@ int PipelineHandlerIPU3::freeBuffers(Camera *camera) return 0; } -int PipelineHandlerIPU3::start(Camera *camera) +int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] ControlList *controls) { IPU3CameraData *data = cameraData(camera); CIO2Device *cio2 = &data->cio2_; diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 7ad66f21..ddb30e49 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -237,7 +237,7 @@ public: int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) override; - int start(Camera *camera) override; + int start(Camera *camera, ControlList *controls) override; void stop(Camera *camera) override; int queueRequestDevice(Camera *camera, Request *request) override; @@ -726,7 +726,7 @@ int PipelineHandlerRPi::exportFrameBuffers([[maybe_unused]] Camera *camera, Stre return ret; } -int PipelineHandlerRPi::start(Camera *camera) +int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *controls) { RPiCameraData *data = cameraData(camera); int ret; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 1b1922a9..2e8d2930 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -187,7 +187,7 @@ public: int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) override; - int start(Camera *camera) override; + int start(Camera *camera, ControlList *controls) override; void stop(Camera *camera) override; int queueRequestDevice(Camera *camera, Request *request) override; @@ -822,7 +822,7 @@ int PipelineHandlerRkISP1::freeBuffers(Camera *camera) return 0; } -int PipelineHandlerRkISP1::start(Camera *camera) +int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] ControlList *controls) { RkISP1CameraData *data = cameraData(camera); int ret; diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 0d3078f7..b047aeb9 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -126,7 +126,7 @@ public: int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) override; - int start(Camera *camera) override; + int start(Camera *camera, ControlList *controls) override; void stop(Camera *camera) override; bool match(DeviceEnumerator *enumerator) override; @@ -646,7 +646,7 @@ int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, return data->video_->exportBuffers(count, buffers); } -int SimplePipelineHandler::start(Camera *camera) +int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] ControlList *controls) { SimpleCameraData *data = cameraData(camera); V4L2VideoDevice *video = data->video_; diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 0f3241cc..87b0f03d 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -76,7 +76,7 @@ public: int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) override; - int start(Camera *camera) override; + int start(Camera *camera, ControlList *controls) override; void stop(Camera *camera) override; int queueRequestDevice(Camera *camera, Request *request) override; @@ -236,7 +236,7 @@ int PipelineHandlerUVC::exportFrameBuffers(Camera *camera, Stream *stream, return data->video_->exportBuffers(count, buffers); } -int PipelineHandlerUVC::start(Camera *camera) +int PipelineHandlerUVC::start(Camera *camera, [[maybe_unused]] ControlList *controls) { UVCCameraData *data = cameraData(camera); unsigned int count = data->stream_.configuration().bufferCount; diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 914b6b54..d81b8598 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -92,7 +92,7 @@ public: int exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) override; - int start(Camera *camera) override; + int start(Camera *camera, ControlList *controls) override; void stop(Camera *camera) override; int queueRequestDevice(Camera *camera, Request *request) override; @@ -313,7 +313,7 @@ int PipelineHandlerVimc::exportFrameBuffers(Camera *camera, Stream *stream, return data->video_->exportBuffers(count, buffers); } -int PipelineHandlerVimc::start(Camera *camera) +int PipelineHandlerVimc::start(Camera *camera, [[maybe_unused]] ControlList *controls) { VimcCameraData *data = cameraData(camera); unsigned int count = data->stream_.configuration().bufferCount; diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index 894200ee..bafcf21b 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -351,6 +351,7 @@ const ControlList &PipelineHandler::properties(const Camera *camera) const * \fn PipelineHandler::start() * \brief Start capturing from a group of streams * \param[in] camera The camera to start + * \param[in] controls Controls for the IPA to action before starting the camera * * Start the group of streams that have been configured for capture by * \a configure(). The intended caller of this method is the Camera class which From patchwork Thu Nov 26 09:51:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 10499 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9BF61BE08A for ; Thu, 26 Nov 2020 09:51:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4E45A63446; Thu, 26 Nov 2020 10:51:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Z6u+p9+I"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B0EA560331 for ; Thu, 26 Nov 2020 10:51:31 +0100 (CET) Received: by mail-wr1-x435.google.com with SMTP id p8so1434196wrx.5 for ; Thu, 26 Nov 2020 01:51:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dbthfx5bvmZxckK65DBFn+CwnKpKLWJDelW51rLTDIE=; b=Z6u+p9+I+nSVXdb3u6SfD7Of0xpDRVbV56891zBbb0N5BBa2G7CedbsEiEgN/REzOY apX4pMmvE6XxRw/l9gHCuNxhpMpffKxgYsXWVWwUq6oUJKqxSCr0Qk9yiCb9J6cD5/WW RzrO1jCDsgIzHyAuAOBZ41RIq1qpPkVYuudqpjbLeX+77M2KMbkwIhg0Ejz82oTdc43u 2mKzZZSAnyOwJ0QdKZdedI2qhZzV5oagmUs260UwnRFAZCV7N1g0bvHKE+PYgwRwoP5H tkuYNEbsyslD+8mAylmq/2fNE4W8frwHEDvfAe3J/Prf5zhXeGw0NcFsph2xQiKrxhSa JszQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dbthfx5bvmZxckK65DBFn+CwnKpKLWJDelW51rLTDIE=; b=ka9JpblJSM1juJoxH/d8RbrKr4srALqtCGHQrNxllueW8asqhi2s/jZU/6u16cg4bQ WQoS0qSDqk9qdLd/p398/Lth/KJkw8tnEnvBaGkKn/3YYlpRYRHReDkA8mEbcMRX5Srn po2n8Rw4FWMyWvO7TD8aS4qm71tQbvhAOaklAfFDnVJHND7QGOIz4td8mFNWqn8CD0g4 wHRdEoxcJ5TBKTlkgZBJ9TjMVZXNAiBDGQ9o8o1fvsxhFVdO0/16HxQ8arHTnmTo3R8J GdEDQt+nm+ByYRQ307vbMkW9a+Ryxg47m42qKmm38SOceIM38yt0VMqO/cfPyPL/lfpf iYiQ== X-Gm-Message-State: AOAM53055HuZDIZTFHcFqMeC3OGnadEMNOyXCjlYirH5P6Zbeepndki9 FdGDSyjb9C0IxwA3dRo61eu/LErGgu2brg== X-Google-Smtp-Source: ABdhPJy4CAZ6GvbFVcTqGpOBH5zGI7X42O+Dbo3cEIka/CXE7H5RYkIiufVkM+B1EegKYakMoVTM5A== X-Received: by 2002:adf:e502:: with SMTP id j2mr2765398wrm.73.1606384291012; Thu, 26 Nov 2020 01:51:31 -0800 (PST) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id i11sm8176827wrm.1.2020.11.26.01.51.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Nov 2020 01:51:30 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Thu, 26 Nov 2020 09:51:25 +0000 Message-Id: <20201126095126.997055-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201126095126.997055-1-naush@raspberrypi.com> References: <20201126095126.997055-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/3] libcamera: ipa: Pass a set of controls and return results from ipa::start() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This change allows controls passed into PipelineHandler::start to be forwarded onto IPAInterface::start(). We also add a return channel if the pipeline handler must action any of these controls, e.g. setting the analogue gain or shutter speed in the sensor device. todo: The IPA interface wrapper needs a translation for passing IPAOperationData into start() and configure() Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman --- include/libcamera/internal/ipa_context_wrapper.h | 3 ++- include/libcamera/ipa/ipa_interface.h | 3 ++- src/ipa/libipa/ipa_interface_wrapper.cpp | 4 +++- src/ipa/raspberrypi/raspberrypi.cpp | 3 ++- src/ipa/rkisp1/rkisp1.cpp | 3 ++- src/ipa/vimc/vimc.cpp | 6 ++++-- src/libcamera/ipa_context_wrapper.cpp | 6 ++++-- src/libcamera/ipa_interface.cpp | 7 +++++++ src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 6 ++++-- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 +++-- src/libcamera/pipeline/vimc/vimc.cpp | 3 ++- src/libcamera/proxy/ipa_proxy_linux.cpp | 3 ++- src/libcamera/proxy/ipa_proxy_thread.cpp | 13 ++++++++----- test/ipa/ipa_interface_test.cpp | 3 ++- test/ipa/ipa_wrappers_test.cpp | 5 +++-- 15 files changed, 50 insertions(+), 23 deletions(-) diff --git a/include/libcamera/internal/ipa_context_wrapper.h b/include/libcamera/internal/ipa_context_wrapper.h index 8f767e84..1878a615 100644 --- a/include/libcamera/internal/ipa_context_wrapper.h +++ b/include/libcamera/internal/ipa_context_wrapper.h @@ -20,7 +20,8 @@ public: ~IPAContextWrapper(); int init(const IPASettings &settings) override; - int start() override; + int start(const IPAOperationData &ipaConfig, + IPAOperationData *result) override; void stop() override; void configure(const CameraSensorInfo &sensorInfo, const std::map &streamConfig, diff --git a/include/libcamera/ipa/ipa_interface.h b/include/libcamera/ipa/ipa_interface.h index 322b7079..b44e2538 100644 --- a/include/libcamera/ipa/ipa_interface.h +++ b/include/libcamera/ipa/ipa_interface.h @@ -153,7 +153,8 @@ public: virtual ~IPAInterface() = default; virtual int init(const IPASettings &settings) = 0; - virtual int start() = 0; + virtual int start(const IPAOperationData &ipaConfig, + IPAOperationData *result) = 0; virtual void stop() = 0; virtual void configure(const CameraSensorInfo &sensorInfo, diff --git a/src/ipa/libipa/ipa_interface_wrapper.cpp b/src/ipa/libipa/ipa_interface_wrapper.cpp index cee532e3..2b305b56 100644 --- a/src/ipa/libipa/ipa_interface_wrapper.cpp +++ b/src/ipa/libipa/ipa_interface_wrapper.cpp @@ -95,7 +95,9 @@ int IPAInterfaceWrapper::start(struct ipa_context *_ctx) { IPAInterfaceWrapper *ctx = static_cast(_ctx); - return ctx->ipa_->start(); + /* \todo Translate the ipaConfig and result. */ + IPAOperationData ipaConfig = {}; + return ctx->ipa_->start(ipaConfig, nullptr); } void IPAInterfaceWrapper::stop(struct ipa_context *_ctx) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index f338ff8b..7a07b477 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -77,7 +77,8 @@ public: } int init(const IPASettings &settings) override; - int start() override { return 0; } + int start([[maybe_unused]] const IPAOperationData &ipaConfig, + [[maybe_unused]] IPAOperationData *result) override { return 0; } void stop() override {} void configure(const CameraSensorInfo &sensorInfo, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 07d7f1b2..0b8a9096 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -37,7 +37,8 @@ public: { return 0; } - int start() override { return 0; } + int start([[maybe_unused]] const IPAOperationData &ipaConfig, + [[maybe_unused]] IPAOperationData *result) override { return 0; } void stop() override {} void configure(const CameraSensorInfo &info, diff --git a/src/ipa/vimc/vimc.cpp b/src/ipa/vimc/vimc.cpp index cf841135..ed26331d 100644 --- a/src/ipa/vimc/vimc.cpp +++ b/src/ipa/vimc/vimc.cpp @@ -34,7 +34,8 @@ public: int init(const IPASettings &settings) override; - int start() override; + int start(const IPAOperationData &ipaConfig, + IPAOperationData *result) override; void stop() override; void configure([[maybe_unused]] const CameraSensorInfo &sensorInfo, @@ -82,7 +83,8 @@ int IPAVimc::init(const IPASettings &settings) return 0; } -int IPAVimc::start() +int IPAVimc::start([[maybe_unused]] const IPAOperationData &ipaConfig, + [[maybe_unused]] IPAOperationData *result) { trace(IPAOperationStart); diff --git a/src/libcamera/ipa_context_wrapper.cpp b/src/libcamera/ipa_context_wrapper.cpp index 231300ce..8c4ec40a 100644 --- a/src/libcamera/ipa_context_wrapper.cpp +++ b/src/libcamera/ipa_context_wrapper.cpp @@ -86,14 +86,16 @@ int IPAContextWrapper::init(const IPASettings &settings) return 0; } -int IPAContextWrapper::start() +int IPAContextWrapper::start(const IPAOperationData &ipaConfig, + IPAOperationData *result) { if (intf_) - return intf_->start(); + return intf_->start(ipaConfig, result); if (!ctx_) return 0; + /* \todo Translate the ipaConfig and response */ return ctx_->ops->start(ctx_); } diff --git a/src/libcamera/ipa_interface.cpp b/src/libcamera/ipa_interface.cpp index 23fc56d7..282c3c0f 100644 --- a/src/libcamera/ipa_interface.cpp +++ b/src/libcamera/ipa_interface.cpp @@ -536,10 +536,17 @@ namespace libcamera { /** * \fn IPAInterface::start() * \brief Start the IPA + * \param[in] ipaConfig Pipeline-handler-specific configuration data + * \param[out] result Pipeline-handler-specific configuration result * * This method informs the IPA module that the camera is about to be started. * The IPA module shall prepare any resources it needs to operate. * + * The \a ipaConfig and \a result parameters carry custom data passed by the + * pipeline handler to the IPA and back. The pipeline handler may set the \a + * result parameter to null if the IPA protocol doesn't need to pass a result + * back through the configure() function. + * * \return 0 on success or a negative error code otherwise */ diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index ddb30e49..29bcef07 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -747,7 +747,9 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont } /* Start the IPA. */ - ret = data->ipa_->start(); + IPAOperationData ipaConfig = {}, result = {}; + ipaConfig.controls.emplace_back(*controls); + ret = data->ipa_->start(ipaConfig, &result); if (ret) { LOG(RPI, Error) << "Failed to start IPA for " << camera->id(); @@ -1176,7 +1178,7 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) } /* Ready the IPA - it must know about the sensor resolution. */ - IPAOperationData result; + IPAOperationData result = {}; ipa_->configure(sensorInfo_, streamConfig, entityControls, ipaConfig, &result); diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2e8d2930..a6c82a48 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -832,7 +832,8 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] ControlList *c if (ret) return ret; - ret = data->ipa_->start(); + IPAOperationData ipaConfig = {}; + ret = data->ipa_->start(ipaConfig, nullptr); if (ret) { freeBuffers(camera); LOG(RkISP1, Error) @@ -911,7 +912,7 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] ControlList *c std::map entityControls; entityControls.emplace(0, data->sensor_->controls()); - IPAOperationData ipaConfig; + ipaConfig = {}; data->ipa_->configure(sensorInfo, streamConfig, entityControls, ipaConfig, nullptr); diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index d81b8598..b4773cf5 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -322,7 +322,8 @@ int PipelineHandlerVimc::start(Camera *camera, [[maybe_unused]] ControlList *con if (ret < 0) return ret; - ret = data->ipa_->start(); + IPAOperationData ipaConfig = {}; + ret = data->ipa_->start(ipaConfig, nullptr); if (ret) { data->video_->releaseBuffers(); return ret; diff --git a/src/libcamera/proxy/ipa_proxy_linux.cpp b/src/libcamera/proxy/ipa_proxy_linux.cpp index b78a0e45..619976e5 100644 --- a/src/libcamera/proxy/ipa_proxy_linux.cpp +++ b/src/libcamera/proxy/ipa_proxy_linux.cpp @@ -30,7 +30,8 @@ public: { return 0; } - int start() override { return 0; } + int start([[maybe_unused]] const IPAOperationData &ipaConfig, + [[maybe_unused]] IPAOperationData *result) override { return 0; } void stop() override {} void configure([[maybe_unused]] const CameraSensorInfo &sensorInfo, [[maybe_unused]] const std::map &streamConfig, diff --git a/src/libcamera/proxy/ipa_proxy_thread.cpp b/src/libcamera/proxy/ipa_proxy_thread.cpp index eead2883..9a81b6e7 100644 --- a/src/libcamera/proxy/ipa_proxy_thread.cpp +++ b/src/libcamera/proxy/ipa_proxy_thread.cpp @@ -26,7 +26,8 @@ public: IPAProxyThread(IPAModule *ipam); int init(const IPASettings &settings) override; - int start() override; + int start(const IPAOperationData &ipaConfig, + IPAOperationData *result) override; void stop() override; void configure(const CameraSensorInfo &sensorInfo, @@ -50,9 +51,9 @@ private: ipa_ = ipa; } - int start() + int start(const IPAOperationData &ipaConfig, IPAOperationData *result) { - return ipa_->start(); + return ipa_->start(ipaConfig, result); } void stop() @@ -111,12 +112,14 @@ int IPAProxyThread::init(const IPASettings &settings) return 0; } -int IPAProxyThread::start() +int IPAProxyThread::start(const IPAOperationData &ipaConfig, + IPAOperationData *result) { running_ = true; thread_.start(); - return proxy_.invokeMethod(&ThreadProxy::start, ConnectionTypeBlocking); + return proxy_.invokeMethod(&ThreadProxy::start, ConnectionTypeBlocking, + ipaConfig, result); } void IPAProxyThread::stop() diff --git a/test/ipa/ipa_interface_test.cpp b/test/ipa/ipa_interface_test.cpp index 67488409..03b7f0ad 100644 --- a/test/ipa/ipa_interface_test.cpp +++ b/test/ipa/ipa_interface_test.cpp @@ -120,7 +120,8 @@ protected: } /* Test start of IPA module. */ - ipa_->start(); + IPAOperationData ipaConfig = {}; + ipa_->start(ipaConfig, nullptr); timer.start(1000); while (timer.isRunning() && trace_ != IPAOperationStart) dispatcher->processEvents(); diff --git a/test/ipa/ipa_wrappers_test.cpp b/test/ipa/ipa_wrappers_test.cpp index 59d991cb..7b8ae77b 100644 --- a/test/ipa/ipa_wrappers_test.cpp +++ b/test/ipa/ipa_wrappers_test.cpp @@ -56,7 +56,8 @@ public: return 0; } - int start() override + int start([[maybe_unused]] const IPAOperationData &ipaConfig, + [[maybe_unused]] IPAOperationData *result) override { report(Op_start, TestPass); return 0; @@ -376,7 +377,7 @@ protected: return TestFail; } - ret = INVOKE(start); + ret = INVOKE(start, ipaConfig, nullptr); if (ret == TestFail) { cerr << "Failed to run start()"; return TestFail; From patchwork Thu Nov 26 09:51:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 10500 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 02067BE08A for ; Thu, 26 Nov 2020 09:51:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C0D2C63449; Thu, 26 Nov 2020 10:51:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="mN8sFW+Z"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4C0A863445 for ; Thu, 26 Nov 2020 10:51:33 +0100 (CET) Received: by mail-wr1-x429.google.com with SMTP id s8so1417014wrw.10 for ; Thu, 26 Nov 2020 01:51:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LgcW+QiPUg9btLTFemG0qL47tIsFP5c/CIoGpSHdOog=; b=mN8sFW+ZNhEWMiDzWYaygIPRsrSfEesFvPe06SChU30SLCM8o1nbT8oIXoBoSLZzpm Aft6n/9GXET3Eq3/q9X8Ny0yfCWojyTOJQNVNi99hsqS6VCRi3CQG9lrJJ4ln0LXNDuA cp5FW/wRkWw/wf6uLQpG/VBZ/PG/U1YXhtSGZ4B48VYCjm0/9qF+f79clkxkSq8CTstR 1pIoFeJciYDLVEDsW8x3/sw7n1xdpsOvdhgnmmA3f7fcG9yhWpYlwySZV4SIwFNoz0c3 NWogZVpzFM3F7IU43UFBXsApcRCeJiXdnC7p9XsvgdyhP+E7z73L76NDC7cH0Y6bXsNW iuuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LgcW+QiPUg9btLTFemG0qL47tIsFP5c/CIoGpSHdOog=; b=cryAo9URzgueAb9oSdx5yUOUmc9kygN5zQlQeDAefLOaRODI0qbjevGmFbwM0cUjzK DkNIVY/almB98n9uN8EgQdVEGdJIkLvR7lT3VwlCTPLfWKSa8oVc2N9ksRtjhdobFVb1 zSqfkZROYb0ZV/76eLuwkSISeo+fQwZUETiJntts/IR0tWhFox8X6asKu6XmAPwIT1CC BE7E/OsmaBuJWpIIrd4OPlXVm9I6jjMXwwrNdlhNMpFORe0xt4TQSliSDCxcQ5mjIBUq UoUEVROJkG23cDJgw3pkX6m6JBdYZkHgK3cIrQpAxMFJ0xccGgumB3n18PjeX/dtFOwQ o7wg== X-Gm-Message-State: AOAM530FIgtwe0/5xNvislupogm77EbDx96iwDihlpd1Y5u5ZOduBHm2 suIUyddDUM4f9gLcGeuqS1GB/V2JzJEvSg== X-Google-Smtp-Source: ABdhPJzoNJ1tS1OhwQqpwCbrj/L4LWTjccMw8gtLA4H/UxTXRKIpyPTAUOnz/0Qo7rWgrrPgGgSDQA== X-Received: by 2002:a5d:60cb:: with SMTP id x11mr2861625wrt.0.1606384292579; Thu, 26 Nov 2020 01:51:32 -0800 (PST) Received: from naushir-VirtualBox.pitowers.org ([2a00:1098:3142:14:a00:27ff:fe4d:f6a2]) by smtp.gmail.com with ESMTPSA id i11sm8176827wrm.1.2020.11.26.01.51.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Nov 2020 01:51:32 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Thu, 26 Nov 2020 09:51:26 +0000 Message-Id: <20201126095126.997055-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201126095126.997055-1-naush@raspberrypi.com> References: <20201126095126.997055-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/3] pipeline: ipa: raspberrypi: Pass controls to IPA on start X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Forward any controls passed into the pipeline handler to the IPA. The IPA then sets up the Raspberry Pi controller with these settings appropriately, and passes back any V4L2 sensor controls that need to be applied. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman --- include/libcamera/ipa/raspberrypi.h | 1 + src/ipa/raspberrypi/raspberrypi.cpp | 53 ++++++++++++------- .../pipeline/raspberrypi/raspberrypi.cpp | 14 ++++- 3 files changed, 47 insertions(+), 21 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index 2b55dbfc..c91a14bd 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -21,6 +21,7 @@ enum ConfigParameters { IPA_CONFIG_STAGGERED_WRITE = (1 << 1), IPA_CONFIG_SENSOR = (1 << 2), IPA_CONFIG_DROP_FRAMES = (1 << 3), + IPA_CONFIG_STARTUP = (1 << 4) }; enum Operations { diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 7a07b477..c09b3d07 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -77,8 +77,7 @@ public: } int init(const IPASettings &settings) override; - int start([[maybe_unused]] const IPAOperationData &ipaConfig, - [[maybe_unused]] IPAOperationData *result) override { return 0; } + int start(const IPAOperationData &ipaConfig, IPAOperationData *result) override; void stop() override {} void configure(const CameraSensorInfo &sensorInfo, @@ -154,6 +153,35 @@ int IPARPi::init(const IPASettings &settings) return 0; } +int IPARPi::start(const IPAOperationData &ipaConfig, IPAOperationData *result) +{ + RPiController::Metadata metadata; + + result->operation = 0; + if (ipaConfig.operation & RPi::IPA_CONFIG_STARTUP) { + /* We have been given some controls to action before start. */ + queueRequest(ipaConfig.controls[0]); + } + + controller_.SwitchMode(mode_, &metadata); + + /* SwitchMode may supply updated exposure/gain values to use. */ + AgcStatus agcStatus; + agcStatus.shutter_time = 0.0; + agcStatus.analogue_gain = 0.0; + + /* SwitchMode may supply updated exposure/gain values to use. */ + metadata.Get("agc.status", agcStatus); + if (agcStatus.shutter_time != 0.0 && agcStatus.analogue_gain != 0.0) { + ControlList ctrls(unicamCtrls_); + applyAGC(&agcStatus, ctrls); + result->controls.emplace_back(ctrls); + result->operation |= RPi::IPA_CONFIG_SENSOR; + } + + return 0; +} + void IPARPi::setMode(const CameraSensorInfo &sensorInfo) { mode_.bitdepth = sensorInfo.bitsPerPixel; @@ -229,7 +257,6 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, result->data.push_back(gainDelay); result->data.push_back(exposureDelay); result->data.push_back(sensorMetadata); - result->operation |= RPi::IPA_CONFIG_STAGGERED_WRITE; } @@ -285,11 +312,6 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, result->data.push_back(dropFrame); result->operation |= RPi::IPA_CONFIG_DROP_FRAMES; - /* These zero values mean not program anything (unless overwritten). */ - struct AgcStatus agcStatus; - agcStatus.shutter_time = 0.0; - agcStatus.analogue_gain = 0.0; - if (!controllerInit_) { /* Load the tuning file for this sensor. */ controller_.Read(tuningFile_.c_str()); @@ -297,20 +319,13 @@ void IPARPi::configure(const CameraSensorInfo &sensorInfo, controllerInit_ = true; /* Supply initial values for gain and exposure. */ + ControlList ctrls(unicamCtrls_); + AgcStatus agcStatus; agcStatus.shutter_time = DefaultExposureTime; agcStatus.analogue_gain = DefaultAnalogueGain; - } - - RPiController::Metadata metadata; - controller_.SwitchMode(mode_, &metadata); - - /* SwitchMode may supply updated exposure/gain values to use. */ - metadata.Get("agc.status", agcStatus); - if (agcStatus.shutter_time != 0.0 && agcStatus.analogue_gain != 0.0) { - ControlList ctrls(unicamCtrls_); applyAGC(&agcStatus, ctrls); - result->controls.push_back(ctrls); + result->controls.emplace_back(ctrls); result->operation |= RPi::IPA_CONFIG_SENSOR; } @@ -843,7 +858,7 @@ void IPARPi::processStats(unsigned int bufferId) IPAOperationData op; op.operation = RPi::IPA_ACTION_V4L2_SET_STAGGERED; - op.controls.push_back(ctrls); + op.controls.emplace_back(ctrls); queueFrameAction.emit(0, op); } } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 29bcef07..3eb8c190 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -748,7 +748,10 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont /* Start the IPA. */ IPAOperationData ipaConfig = {}, result = {}; - ipaConfig.controls.emplace_back(*controls); + if (controls) { + ipaConfig.operation = RPi::IPA_CONFIG_STARTUP; + ipaConfig.controls.emplace_back(*controls); + } ret = data->ipa_->start(ipaConfig, &result); if (ret) { LOG(RPI, Error) @@ -757,6 +760,14 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont return ret; } + /* Apply any gain/exposure settings that the IPA may have passed back. */ + ASSERT(data->staggeredCtrl_); + if (result.operation & RPi::IPA_CONFIG_SENSOR) { + const ControlList &ctrls = result.controls[0]; + if (!data->staggeredCtrl_.set(ctrls)) + LOG(RPI, Error) << "V4L2 staggered set failed"; + } + /* * IPA configure may have changed the sensor flips - hence the bayer * order. Get the sensor format and set the ISP input now. @@ -777,7 +788,6 @@ int PipelineHandlerRPi::start(Camera *camera, [[maybe_unused]] ControlList *cont * starting. First check that the staggered ctrl has been initialised * by configure(). */ - ASSERT(data->staggeredCtrl_); data->staggeredCtrl_.reset(); data->staggeredCtrl_.write(); data->expectedSequence_ = 0;