From patchwork Mon Sep 7 16:44:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9515 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 425DEBDB1E for ; Mon, 7 Sep 2020 16:45:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 026BE62C15; Mon, 7 Sep 2020 18:45:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="mkTU+JS4"; dkim-atps=neutral Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D43960372 for ; Mon, 7 Sep 2020 18:44:59 +0200 (CEST) Received: by mail-wr1-x42b.google.com with SMTP id m6so16472951wrn.0 for ; Mon, 07 Sep 2020 09:44:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cLq8fMZ6wax4QpFQ10/ZmQl82O46CeZqaT5ztQBHqlQ=; b=mkTU+JS4xNFJTI838MsBxzKXWdCYGOuAxz9oidmS03AyoIkScaUejjivABci/8AdgC DfDfw0WZexTOAF0ENYghjyA+bdpwSN6ucJxa7R/M1F5dmKufIdzLXzkzvL40s1+fT789 SjHYkHhHRr2kvpc1dHEn1spmY/NpwcRNmFUY40DwZVBmE3uI1L6COCizjBZqrADtSs9b vV2cPomwz6l08mBZ+OXRmJEvhcjOt0RnOFu8BHdpMgnY+1obihvKU0iFzg9yELJeyRxX ccB5/sIOnQlMoTHWPiNsV21A2Jf1ZOeVyQlvILeaWDn7uVnr5B+DDTHAH4QSpkE/nV7Z yRKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cLq8fMZ6wax4QpFQ10/ZmQl82O46CeZqaT5ztQBHqlQ=; b=G2xsR1wYWP0P6XtqB6HxSwWbEo58EcOim5IleXucu8nRZBeDEWVAtN5wA8OBYanF8l j9G/YE1E05tmIp+Lell1AcEOdH7SVJHCQWhLZaLERlXFo/r81ZJcB4fIZTxc1ibTpSOH 5A4oR5yvNemCIeklOoRVdIjCIJG9ctJ5U8VSaHQ1GEjquo6AqI6/iIRvuKPEzkfjmrLO vLDGo31mIbnpsNNb4VZl5zV2URH1brFjN+MA/oRQMIlDmqIytRVGuJh74Fdq9o62V4Zc +FqD5CcTWTyV+j+QTdzLJ09aW7fqMxZnn8mbHTls81MM1nScemjeLtfeez2ZTirseA+p 3Kcg== X-Gm-Message-State: AOAM532VFJzUUeHS47RsJly1NCf2IchoQukuX6LFW+mMz6eMk+f8kzdL HR9GdCH93aZmSnP7Qs3wmpuoWLWQYxRtyQ== X-Google-Smtp-Source: ABdhPJyj8EjZOEn3VqX4p1Ac7S/iEVtp+Uy2bBwKP+mb+iNWYP4OkiS+ibxmRNFaE7xyajp0IuVPKw== X-Received: by 2002:a5d:6806:: with SMTP id w6mr22489405wru.395.1599497098538; Mon, 07 Sep 2020 09:44:58 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id v3sm27707033wmh.6.2020.09.07.09.44.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 09:44:57 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Mon, 7 Sep 2020 17:44:50 +0100 Message-Id: <20200907164450.13082-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200907164450.13082-1-david.plowman@raspberrypi.com> References: <20200907164450.13082-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 4/4] libcamera: pipeline: raspberrypi: Implementation of digital zoom X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" During configure() we make a note of the minimum crop size allowed by the ISP, and the image output size of the chosen sensor mode. Whenever a new SensorCrop request is received we check it's valid and apply it to the ISP V4L2 device. We also forward it to the IPA so that the IPA can return the values used in the image metadata. --- include/libcamera/ipa/raspberrypi.h | 1 + src/ipa/raspberrypi/raspberrypi.cpp | 7 ++++ .../pipeline/raspberrypi/raspberrypi.cpp | 39 +++++++++++++++++++ 3 files changed, 47 insertions(+) diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h index ca62990..361b47d 100644 --- a/include/libcamera/ipa/raspberrypi.h +++ b/include/libcamera/ipa/raspberrypi.h @@ -57,6 +57,7 @@ static const ControlInfoMap RPiControls = { { &controls::Saturation, ControlInfo(0.0f, 32.0f) }, { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, + { &controls::SensorCrop, ControlInfo(Rectangle(0, 0, 0, 0), Rectangle(65535, 65535, 65535, 65535), Rectangle(0, 0, 0, 0)) }, }; } /* namespace libcamera */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 4557016..4a364b7 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -687,6 +687,13 @@ void IPARPi::queueRequest(const ControlList &controls) break; } + case controls::SENSOR_CROP: { + /* Just copy the information back. */ + Rectangle crop = ctrl.second.get(); + libcameraMetadata_.set(controls::SensorCrop, crop); + break; + } + default: LOG(IPARPI, Warning) << "Ctrl " << controls::controls.at(ctrl.first)->name() diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 7f151cb..3196a61 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -335,6 +335,11 @@ public: std::queue embeddedQueue_; std::deque requestQueue_; + /* For handling digital zoom. */ + Size ispMinSize_; + Size sensorOutputSize_; + Rectangle lastSensorCrop_; + private: void checkRequestCompleted(); void tryRunPipeline(); @@ -737,6 +742,16 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return ret; } + /* + * Figure out the smallest selection the ISP will allow. We also store + * the output image size, in pixels, from the sensor. These will be + * used for digital zoom. + */ + Rectangle testCrop(0, 0, 1, 1); + data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &testCrop); + data->ispMinSize_ = testCrop.size(); + data->sensorOutputSize_ = sensorFormat.size; + /* Adjust aspect ratio by providing crops on the input image. */ Rectangle crop{ 0, 0, sensorFormat.size }; @@ -753,6 +768,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) crop.y = (sensorFormat.size.height - crop.height) >> 1; data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); + data->lastSensorCrop_ = crop; + ret = data->configureIPA(); if (ret) LOG(RPI, Error) << "Failed to configure the IPA: " << ret; @@ -1545,6 +1562,28 @@ void RPiCameraData::tryRunPipeline() */ Request *request = requestQueue_.front(); + if (request->controls().contains(controls::SensorCrop)) { + Rectangle crop = request->controls().get(controls::SensorCrop); + if (crop.width && crop.height) { + /* + * The crop that we set must be: + * 1. At least as big as ispMinSize_, once that's been + * enlarged to the same aspect ratio. + * 2. With the same mid-point, if possible. + * 3. But it can't go outside the sensor area. + */ + Size minSize = ispMinSize_.aspectRatioUp(crop.size()); + Size size = crop.size().expandedTo(minSize); + crop = size.centre(crop).clamp(Rectangle(sensorOutputSize_)); + + if (crop != lastSensorCrop_) + isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); + lastSensorCrop_ = crop; + } + + request->controls().set(controls::SensorCrop, lastSensorCrop_); + } + /* * Process all the user controls by the IPA. Once this is complete, we * queue the ISP output buffer listed in the request to start the HW