From patchwork Tue Sep 22 10:03:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9703 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 475DCBF01C for ; Tue, 22 Sep 2020 10:04:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1009A62FDD; Tue, 22 Sep 2020 12:04:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="UH66klOF"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C05F462FDE for ; Tue, 22 Sep 2020 12:04:11 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id v12so870331wmh.3 for ; Tue, 22 Sep 2020 03:04:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z8SrpSgEre5Ub0KUWcgKZOT5DdvwoJNRhtRvy7mkJDc=; b=UH66klOFOyHdgLxev0DIrrKMRajL7Q7tvgqW3/9zYyPdYYPl4E04+XRl2Ir3obNmsr foRCo3nxRc4msL4zTtwvAqsCXwed/F7r9cj6dXFnN6YF0c3NiAQUzWRHvR6gxlBeY6QX 23enUqK7caOwjMd5uVJWraB8BEVsJquRBKpXlRdv8FTaPx2lE652hE1dI98NQ0biz6wk sO0pb/E/PbzE5Rm6Oj7v5bzOoRdG5HeljsUvMw5wdeDDQ/r7t+XXf6jDzDJ9GrpTluRv USIioCb71Xr0Du0+CHl03vVjoF2jwPs1zZOeaxx9UAdzV7NXzfx4vLDqxbh4sPEqQUXS jEEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z8SrpSgEre5Ub0KUWcgKZOT5DdvwoJNRhtRvy7mkJDc=; b=pGRROBGk2tEk8QVOsVsYtCo7PiH2QFhEp2OsWSrkKwgOKHQf2QQy+qVO7Q1jQXB4Co iDIHHQ/x0PG1H1Wi7WY7HZvcyDN5E105mIV5OYS0liOLVYmBU9MXO4MuJ5AbhiSVEyEU nvFwJVxxx6Wh0WyVi01oh2mvT/DH1K5sckQs0vUXzHtuCQ1GueDvWInDeHPln9m7/NHn BhK43sZv9xadBirWdSZj35KQBi0FYG58hnCh3+yNnESDGqnDdD+s8FqTYTElb5Rh8NMF /w36ovSdEWInR+vJFXSajE73U/Qn4veou7psMvNpr5U8OVL4Gb2AYSLmZ+M3uruZFHGP YaDw== X-Gm-Message-State: AOAM533fqh1SMcfhonJ8OcSDImxBpjpfPA0GEIzLZxrYZ9jAh65G2gL2 4aKxs4tPIozFEKcbRKP44uZlwEIir+kgnA== X-Google-Smtp-Source: ABdhPJyVyHBAWEg2ZqcSjKxIEC07SODzp2KGWCGZ+WHUex3ndoVFIr9qWSpoujHuN0JoL5VgO51Rxg== X-Received: by 2002:a05:600c:245:: with SMTP id 5mr133657wmj.33.1600769051128; Tue, 22 Sep 2020 03:04:11 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id g14sm26369595wrv.25.2020.09.22.03.04.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2020 03:04:10 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Tue, 22 Sep 2020 11:03:59 +0100 Message-Id: <20200922100400.30766-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200922100400.30766-1-david.plowman@raspberrypi.com> References: <20200922100400.30766-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/6] libcamera: pipeline: raspberrypi: Implementation of digital zoom X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" During configure() we update the SensorOutputSize to the correct value for this camera mode (which we also note internally) and work out the minimum crop size allowed by the ISP. Whenever a new IspCrop request is received we check it's valid and apply it to the ISP V4L2 device. We also forward it to the IPA so that the IPA can return the values used in the image metadata. Signed-off-by: David Plowman Reviewed-by: Jacopo Mondi --- include/libcamera/ipa/raspberrypi.h | 1 + src/ipa/raspberrypi/raspberrypi.cpp | 7 +++ .../pipeline/raspberrypi/raspberrypi.cpp | 47 +++++++++++++++++++ 3 files changed, 55 insertions(+) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index dd6ebea..91a1892 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -58,6 +58,7 @@ static const ControlInfoMap RPiControls = { { &controls::Saturation, ControlInfo(0.0f, 32.0f) }, { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, + { &controls::IspCrop, ControlInfo(Rectangle(0, 0, 0, 0), Rectangle(65535, 65535, 65535, 65535), Rectangle(0, 0, 0, 0)) }, }; } /* namespace libcamera */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 0555cc4..64f0e35 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -687,6 +687,13 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::ISP_CROP: { + /* Just copy the information back. */ + Rectangle crop = ctrl.second.get(); + libcameraMetadata_.set(controls::IspCrop, crop); + break; + } + default: LOG(IPARPI, Warning) << "Ctrl " << controls::controls.at(ctrl.first)->name() diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 50f0718..1674f8f 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -182,6 +182,11 @@ public: std::queue embeddedQueue_; std::deque requestQueue_; + /* For handling digital zoom. */ + Size ispMinSize_; + Size sensorOutputSize_; + Rectangle lastIspCrop_; + unsigned int dropFrameCount_; private: @@ -496,6 +501,14 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) LOG(RPI, Info) << "Sensor: " << camera->id() << " - Selected mode: " << sensorFormat.toString(); + /* + * Update the SensorOutputSize to the correct value for this camera mode. + * + * \todo Move this property to CameraConfiguration when that + * feature will be made available and set it at validate() time + */ + data->properties_.set(properties::SensorOutputSize, sensorFormat.size); + /* * This format may be reset on start() if the bayer order has changed * because of flips in the sensor. @@ -592,6 +605,16 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return ret; } + /* + * Figure out the smallest selection the ISP will allow. We also store + * the output image size, in pixels, from the sensor. These will be + * used for digital zoom. + */ + Rectangle testCrop(0, 0, 1, 1); + data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &testCrop); + data->ispMinSize_ = testCrop.size(); + data->sensorOutputSize_ = sensorFormat.size; + /* Adjust aspect ratio by providing crops on the input image. */ Rectangle crop{ 0, 0, sensorFormat.size }; @@ -608,6 +631,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) crop.y = (sensorFormat.size.height - crop.height) >> 1; data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); + data->lastIspCrop_ = crop; + ret = data->configureIPA(); if (ret) LOG(RPI, Error) << "Failed to configure the IPA: " << ret; @@ -1449,6 +1474,28 @@ void RPiCameraData::tryRunPipeline() /* Take the first request from the queue and action the IPA. */ Request *request = requestQueue_.front(); + if (request->controls().contains(controls::IspCrop)) { + Rectangle crop = request->controls().get(controls::IspCrop); + if (crop.width && crop.height) { + /* + * The crop that we set must be: + * 1. At least as big as ispMinSize_, once that's been + * enlarged to the same aspect ratio. + * 2. With the same mid-point, if possible. + * 3. But it can't go outside the sensor area. + */ + Size minSize = ispMinSize_.alignedUpToAspectRatio(crop.size()); + Size size = crop.size().expandedTo(minSize); + crop = size.centredTo(crop).boundedTo(Rectangle(sensorOutputSize_)); + + if (crop != lastIspCrop_) + isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); + lastIspCrop_ = crop; + } + + request->controls().set(controls::IspCrop, lastIspCrop_); + } + /* * Process all the user controls by the IPA. Once this is complete, we * queue the ISP output buffer listed in the request to start the HW