From patchwork Fri Sep 25 08:51:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9826 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09CDFC3B5C for ; Fri, 25 Sep 2020 08:51:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CB55F63043; Fri, 25 Sep 2020 10:51:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="m9CofgmN"; dkim-atps=neutral Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C434B63030 for ; Fri, 25 Sep 2020 10:51:36 +0200 (CEST) Received: by mail-wr1-x430.google.com with SMTP id k15so2638637wrn.10 for ; Fri, 25 Sep 2020 01:51:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gP5B4J/bWhecyfZY4eiJ4gLYV2WzmrTl87X3aXvikXI=; b=m9CofgmNEEjSRvI9gGpfQxSv4DESKoC1q600JMtafc3RQ5Zi4D79ovA0WiHqpeLJwF ExvZkHbfYGfXRkNXnXviaMC0RXOTag7x6mS9/uHTASeNaSkiUVsJIAlBKkZ/2PW+smc5 wLRfLuvc6AiixGcrIIBq5Ab8l+VD3g+o1frvSimFaRk5ij5PJzHpKUymkbPhs4PAPfjX aQ+kVpBAvi/9sPeWGipXcpLBByqgOAlR//lYwHliYY9NPZpm0WDwB1p5O21L+mCB8+K1 oPFIjnpZPoXa19RIlHbAW9Uyav/jkIWOO6C4dkhzlcdvECSvxcoiDOYAQ97XP2SfAaoc CeMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gP5B4J/bWhecyfZY4eiJ4gLYV2WzmrTl87X3aXvikXI=; b=miINWtOO1T+Ma/vL8Dr6wUQNzX6IkmvgC3XBnZbLiayS1a7ITR8fupypj05cU2cYnL yZRtIkH0cD0gRt7e/vU6TqYDMjO5GluhMunHsAk7Tdwr2NgOTIxA/QBGS0bJwKaZNc3M /PNg+9+BtWJSIIcLeD5h6K/ZMJjTZtoesggO0eZFTOYh7eGHCmoCkgU0NnKC+WVbQFYU azfmN4UE5DCyFcxlvM/ILj2OHGVSFKB1Ab4ff01Nu2i6AsskRlPafso2cXZZ0dWeQgpX 60jLKAOrlL+gHlTByKDUv+SPCTKeVhHztd5ZeG9huRhF+1Q1PivCXv/12eh8yOrOJWZC fJvA== X-Gm-Message-State: AOAM530FJ9qRHFDwLtsZcBSIZtp9ZG59R9s9fdC6uoYDdoN89hNXRr4Q Pqistbcf4HkwMFTTgI6zIELrV4mKU+BlzA== X-Google-Smtp-Source: ABdhPJxs37QXobXyEd7ehlHDBKnwKrDTKjvO3odnqcMqND+UOEQIlZjrlfwqwr5SYnGm+wizG6RLCg== X-Received: by 2002:a5d:4709:: with SMTP id y9mr3311460wrq.59.1601023896145; Fri, 25 Sep 2020 01:51:36 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id n4sm2001383wrp.61.2020.09.25.01.51.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Sep 2020 01:51:35 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Fri, 25 Sep 2020 09:51:26 +0100 Message-Id: <20200925085127.17214-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925085127.17214-1-david.plowman@raspberrypi.com> References: <20200925085127.17214-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/6] libcamera: pipeline: raspberrypi: Implementation of digital zoom X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" During configure() we update the SensorOutputSize to the correct value for this camera mode (which we also note internally) and work out the minimum crop size allowed by the ISP. Whenever a new IspCrop request is received we check it's valid and apply it to the ISP V4L2 device. We also forward it to the IPA so that the IPA can return the values used in the image metadata. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi --- include/libcamera/ipa/raspberrypi.h | 1 + src/ipa/raspberrypi/raspberrypi.cpp | 7 +++ .../pipeline/raspberrypi/raspberrypi.cpp | 47 +++++++++++++++++++ 3 files changed, 55 insertions(+) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index dd6ebea..9dd6323 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -58,6 +58,7 @@ static const ControlInfoMap RPiControls = { { &controls::Saturation, ControlInfo(0.0f, 32.0f) }, { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, + { &controls::IspCrop, ControlInfo({}, Rectangle(65535, 65535, 65535, 65535), {}) }, }; } /* namespace libcamera */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 0555cc4..64f0e35 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -687,6 +687,13 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::ISP_CROP: { + /* Just copy the information back. */ + Rectangle crop = ctrl.second.get(); + libcameraMetadata_.set(controls::IspCrop, crop); + break; + } + default: LOG(IPARPI, Warning) << "Ctrl " << controls::controls.at(ctrl.first)->name() diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 50f0718..dd5feac 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -182,6 +182,11 @@ public: std::queue embeddedQueue_; std::deque requestQueue_; + /* For handling digital zoom. */ + Size ispMinSize_; + Size sensorOutputSize_; + Rectangle lastIspCrop_; + unsigned int dropFrameCount_; private: @@ -502,6 +507,14 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) */ ret = data->isp_[Isp::Input].dev()->setFormat(&sensorFormat); + /* + * Update the SensorOutputSize to the correct value for this camera mode. + * + * \todo Move this property to CameraConfiguration when that + * feature will be made available and set it at validate() time + */ + data->properties_.set(properties::SensorOutputSize, sensorFormat.size); + /* * See which streams are requested, and route the user * StreamConfiguration appropriately. @@ -592,6 +605,16 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return ret; } + /* + * Figure out the smallest selection the ISP will allow. We also store + * the output image size, in pixels, from the sensor. These will be + * used for digital zoom. + */ + Rectangle testCrop(0, 0, 1, 1); + data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &testCrop); + data->ispMinSize_ = testCrop.size(); + data->sensorOutputSize_ = sensorFormat.size; + /* Adjust aspect ratio by providing crops on the input image. */ Rectangle crop{ 0, 0, sensorFormat.size }; @@ -608,6 +631,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) crop.y = (sensorFormat.size.height - crop.height) >> 1; data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); + data->lastIspCrop_ = crop; + ret = data->configureIPA(); if (ret) LOG(RPI, Error) << "Failed to configure the IPA: " << ret; @@ -1449,6 +1474,28 @@ void RPiCameraData::tryRunPipeline() /* Take the first request from the queue and action the IPA. */ Request *request = requestQueue_.front(); + if (request->controls().contains(controls::IspCrop)) { + Rectangle crop = request->controls().get(controls::IspCrop); + if (crop.width && crop.height) { + /* + * The crop that we set must be: + * 1. At least as big as ispMinSize_, once that's been + * enlarged to the same aspect ratio. + * 2. With the same mid-point, if possible. + * 3. But it can't go outside the sensor area. + */ + Size minSize = ispMinSize_.alignedUpToAspectRatio(crop.size()); + Size size = crop.size().expandedTo(minSize); + crop = size.centredTo(crop).boundedTo(Rectangle(sensorOutputSize_)); + + if (crop != lastIspCrop_) + isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); + lastIspCrop_ = crop; + } + + request->controls().set(controls::IspCrop, lastIspCrop_); + } + /* * Process all the user controls by the IPA. Once this is complete, we * queue the ISP output buffer listed in the request to start the HW