From patchwork Tue Sep 29 16:39:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9868 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5510CC3B5B for ; Tue, 29 Sep 2020 16:40:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0AF086221D; Tue, 29 Sep 2020 18:40:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Uck40433"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 02143621EA for ; Tue, 29 Sep 2020 18:40:10 +0200 (CEST) Received: by mail-wr1-x436.google.com with SMTP id k10so6162206wru.6 for ; Tue, 29 Sep 2020 09:40:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U07BFgIs43AOzvJHddqiQga4NTJKjxfhWxvHzgV1B3M=; b=Uck40433/9a5V1F/Es2ZnjVSef7VdeseZ2ALqCgFm76A7klfLwlN2v3cXh2dW8dIrI QMALrQePc/1vlZyN3Jsq+Y73BsNvVmO5ms3RLns+7gZB2j0IPvX+Sh/7yq1JCtzfWU9+ haTQW3ww5pSm6qqE1zv1l7CGv6ENumQGILtko8u9HhJMXOnoJgguyxlVhUUhZKPTOZtp 250MuFaQ1/gX/bxqr+Sxi3XiVns4SnlVkB7vS5JqfWK6ER8DEipYkihrx02mc2DXc0P0 07Yt/0PwOuL+AnBL6GqVSV6kUOJQmz7Zs9eqUuVfUYr+9On8UzXAx2ensUALz+7aVsbK tnzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U07BFgIs43AOzvJHddqiQga4NTJKjxfhWxvHzgV1B3M=; b=q2wHZvB/EcMMIz8gDq4PGDCyuDIPdtTVedQdCG6OYGxteevlCkmmbKcBjYO40GXqM0 qwnn8Z+yD8EspxSUjJ3ACkqFkqfydgKAAbVFKXLgvQrQ3nLqZ6jJc59A1RnJzBY4kQlp Hiz/eg8X4hxygTvIaIDMyX4v6zOquvfRTCmCn0/I0AuZ4RGjciWfkdb7egnRSRTKFO4J ZiaLntfhsp0QZYmH+GAdEIx8XmMS2tx1HovFbXwC+2n+XD3vKdPHkjFjrh9i9NwzGPh0 Zxmb8n9K+ER50A5AG4PQyIBlXxDRHTyPzPp5559qPKLF+XHS3CeWNhQ6hKDxeADYLCNg IRBw== X-Gm-Message-State: AOAM530kpsmU3vZ0abkQbjtFtvzWtr3hlbhS6fT6UpvDou/2CDDdKblV e373sM0QIera9PjiGk6guTusfZt0gDPyfA== X-Google-Smtp-Source: ABdhPJz4DfhL36AfbjDrbvYaeaV1n6+rtyx9ZJ95NldnIHx8qXLNqBW4O04JRQGLL8BJh0kQWoMX/w== X-Received: by 2002:adf:f34f:: with SMTP id e15mr5092303wrp.387.1601397609303; Tue, 29 Sep 2020 09:40:09 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id q12sm6977393wrs.48.2020.09.29.09.40.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Sep 2020 09:40:08 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 29 Sep 2020 17:39:59 +0100 Message-Id: <20200929164000.15429-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200929164000.15429-1-david.plowman@raspberrypi.com> References: <20200929164000.15429-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/6] libcamera: pipeline: raspberrypi: Implementation of digital zoom X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" During configure() we update the SensorOutputSize to the correct value for this camera mode (which we also note internally) and work out the minimum crop size allowed by the ISP. Whenever a new IspCrop request is received we check it's valid and apply it to the ISP V4L2 device. We also forward it to the IPA so that the IPA can return the values used in the image metadata. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- include/libcamera/ipa/raspberrypi.h | 1 + src/ipa/raspberrypi/raspberrypi.cpp | 7 +++ .../pipeline/raspberrypi/raspberrypi.cpp | 47 +++++++++++++++++++ 3 files changed, 55 insertions(+) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index b3041591..9ef5f6fc 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -60,6 +60,7 @@ static const ControlInfoMap Controls = { { &controls::Saturation, ControlInfo(0.0f, 32.0f) }, { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, + { &controls::IspCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, }; } /* namespace RPi */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b0c7d1c1..dc73c20b 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -699,6 +699,13 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::ISP_CROP: { + /* Just copy the information back. */ + Rectangle crop = ctrl.second.get(); + libcameraMetadata_.set(controls::IspCrop, crop); + break; + } + default: LOG(IPARPI, Warning) << "Ctrl " << controls::controls.at(ctrl.first)->name() diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index d2bee150..4a6d0bf4 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -193,6 +193,11 @@ public: bool flipsAlterBayerOrder_; BayerFormat::Order nativeBayerOrder_; + /* For handling digital zoom. */ + Size ispMinSize_; + Size sensorOutputSize_; + Rectangle lastIspCrop_; + unsigned int dropFrameCount_; private: @@ -587,6 +592,14 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) */ ret = data->isp_[Isp::Input].dev()->setFormat(&sensorFormat); + /* + * Update the SensorOutputSize to the correct value for this camera mode. + * + * \todo Move this property to CameraConfiguration when that + * feature will be made available and set it at validate() time + */ + data->properties_.set(properties::SensorOutputSize, sensorFormat.size); + /* * See which streams are requested, and route the user * StreamConfiguration appropriately. @@ -677,6 +690,16 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return ret; } + /* + * Figure out the smallest selection the ISP will allow. We also store + * the output image size, in pixels, from the sensor. These will be + * used for digital zoom. + */ + Rectangle testCrop(0, 0, 1, 1); + data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &testCrop); + data->ispMinSize_ = testCrop.size(); + data->sensorOutputSize_ = sensorFormat.size; + /* Adjust aspect ratio by providing crops on the input image. */ Rectangle crop{ 0, 0, sensorFormat.size }; @@ -693,6 +716,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) crop.y = (sensorFormat.size.height - crop.height) >> 1; data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); + data->lastIspCrop_ = crop; + ret = data->configureIPA(config); if (ret) LOG(RPI, Error) << "Failed to configure the IPA: " << ret; @@ -1588,6 +1613,28 @@ void RPiCameraData::tryRunPipeline() /* Take the first request from the queue and action the IPA. */ Request *request = requestQueue_.front(); + if (request->controls().contains(controls::IspCrop)) { + Rectangle crop = request->controls().get(controls::IspCrop); + if (crop.width && crop.height) { + /* + * The crop that we set must be: + * 1. At least as big as ispMinSize_, once that's been + * enlarged to the same aspect ratio. + * 2. With the same mid-point, if possible. + * 3. But it can't go outside the sensor area. + */ + Size minSize = ispMinSize_.alignedUpToAspectRatio(crop.size()); + Size size = crop.size().expandedTo(minSize); + crop = size.centredTo(crop).boundedTo(Rectangle(sensorOutputSize_)); + + if (crop != lastIspCrop_) + isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); + lastIspCrop_ = crop; + } + + request->controls().set(controls::IspCrop, lastIspCrop_); + } + /* * Process all the user controls by the IPA. Once this is complete, we * queue the ISP output buffer listed in the request to start the HW