{"id":8714,"url":"https://patchwork.libcamera.org/api/patches/8714/?format=json","web_url":"https://patchwork.libcamera.org/patch/8714/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200709091555.1617-3-david.plowman@raspberrypi.com>","date":"2020-07-09T09:15:55","name":"[libcamera-devel,v2,2/2] libcamera: raspberrypi: Implement digital zoom","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"559258eb9345f325d998d17e95beb0da09213ee3","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/?format=json","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/8714/mbox/","series":[{"id":1092,"url":"https://patchwork.libcamera.org/api/series/1092/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1092","date":"2020-07-09T09:15:53","name":"Digital zoom implementation","version":2,"mbox":"https://patchwork.libcamera.org/series/1092/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/8714/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/8714/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 85ADEBD790\n\tfor ;\n\tThu, 9 Jul 2020 09:16:11 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 508F961190;\n\tThu, 9 Jul 2020 11:16:11 +0200 (CEST)","from mail-wm1-x330.google.com (mail-wm1-x330.google.com\n\t[IPv6:2a00:1450:4864:20::330])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C382561190\n\tfor ;\n\tThu, 9 Jul 2020 11:16:08 +0200 (CEST)","by mail-wm1-x330.google.com with SMTP id l2so1066905wmf.0\n\tfor ;\n\tThu, 09 Jul 2020 02:16:08 -0700 (PDT)","from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72])\n\tby smtp.gmail.com with ESMTPSA id\n\th84sm4135765wme.22.2020.07.09.02.16.07\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 09 Jul 2020 02:16:07 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"PsfAul/6\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=UC8cy68txEeNI8Pq3ScFqRcZuj6BWeZjqq2AYqY2pqE=;\n\tb=PsfAul/6dHW2zpwis5a4R9cxo3VBizbKwmjaCOwQWbA/HKetUg/4tOrVvWTOW86FNd\n\tLyGqNLcYSTLOPhXKfUFtRWxuq42OAp02OXrzEOhMY3oBGWNkXMcONajuVNaqKamjusiv\n\tvAbflUv2jEAh0X8RAd5qPyQjej2G20ByizhzVEdOVZYOGMKsOmvBC5gCL0jzW7hH4+Pp\n\t4RN7R5phW8/c0O2qwvi06BC7Nl8hHLqRrPvHCm2lgYwO+FrRsahETG5emRycd7E36eEE\n\tIkA4CCfaD3h8UdWna7y9joNXqMdIdH+VdJCw2+fiqJgUuW+yS9xdNtQRqmq7MMk1B9TH\n\tpG2Q==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=UC8cy68txEeNI8Pq3ScFqRcZuj6BWeZjqq2AYqY2pqE=;\n\tb=YSTclfHb6jYHHZXejh7c5xXBvmFkCE1UWDbBzVN5RxVnxk1u01vob7ZsSrWsycTsMb\n\t1VMqTCPWKjWvsN4JLxkUNTymIvYaq+MQhqw9gdAMqC4gGhRLMzHTH6wvAL0sqRUer3mB\n\tShn3MOSI9PXNpu1ex0xYTxfoQJ18mWd4ZWz/QHicm3VrxDlxBTVXzEHZCZbdE0MkAwjb\n\tLC78KmpAqgKkQTtAZqV0R0P2E8bKYceV8HfyvucCYw92EtVZT7Q01DmjJ5+ju/5hXH61\n\tief4nUtyW5PVAXwoXv9ZKzJb28H0DLGwU8d1PE7QdIhKA3fYhkMsj0Mgqx90uK2XTOCC\n\ttnaQ==","X-Gm-Message-State":"AOAM5306ZWVwaa4TWY3jZ38W+ASGAItl9x5DyLQIeRm0Cr5bSm89FUUy\n\tRRuRFlu2tqJv65WrjbkvxRxcmDY2tVM=","X-Google-Smtp-Source":"ABdhPJwWb5u9GU+keBeN0Mc8kEZ0XhfgSoEVfSDIokifudtWsKIZ2NYdDYwmRa9InB5E3TsObh0aXg==","X-Received":"by 2002:a1c:238d:: with SMTP id\n\tj135mr12857809wmj.71.1594286168036; \n\tThu, 09 Jul 2020 02:16:08 -0700 (PDT)","From":"David Plowman ","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 9 Jul 2020 10:15:55 +0100","Message-Id":"<20200709091555.1617-3-david.plowman@raspberrypi.com>","X-Mailer":"git-send-email 2.20.1","In-Reply-To":"<20200709091555.1617-1-david.plowman@raspberrypi.com>","References":"<20200709091555.1617-1-david.plowman@raspberrypi.com>","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v2 2/2] libcamera: raspberrypi: Implement\n\tdigital zoom","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"These changes implement digital zoom for the Raspberry Pi. We detect\nthe digital zoom control and update the V4L2 \"selection\" before\nstarting the ISP. We also update the value in the control that we send\nto the IPA, so that it has the correct value.\n---\n include/libcamera/ipa/raspberrypi.h | 1 +\n src/ipa/raspberrypi/raspberrypi.cpp | 10 ++++\n .../pipeline/raspberrypi/raspberrypi.cpp | 56 ++++++++++++++++++-\n 3 files changed, 66 insertions(+), 1 deletion(-)","diff":"diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h\nindex a18ce9a..e66402e 100644\n--- a/include/libcamera/ipa/raspberrypi.h\n+++ b/include/libcamera/ipa/raspberrypi.h\n@@ -52,6 +52,7 @@ static const ControlInfoMap RPiControls = {\n \t{ &controls::Contrast, ControlInfo(0.0f, 32.0f) },\n \t{ &controls::Saturation, ControlInfo(0.0f, 32.0f) },\n \t{ &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) },\n+\t{ &controls::DigitalZoom, ControlInfo(Rectangle{ 0, 0, 0, 0 }, Rectangle{ 65535, 65535, 65535, 65535 }, Rectangle{ 0, 0, 0, 0 }) },\n };\n \n } /* namespace libcamera */\ndiff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp\nindex b1f2786..3c68078 100644\n--- a/src/ipa/raspberrypi/raspberrypi.cpp\n+++ b/src/ipa/raspberrypi/raspberrypi.cpp\n@@ -658,6 +658,16 @@ void IPARPi::queueRequest(const ControlList &controls)\n \t\t\tbreak;\n \t\t}\n \n+\t\tcase controls::DIGITAL_ZOOM: {\n+\t\t\t/*\n+\t\t\t * We send the zoom info back in the metadata where the pipeline\n+\t\t\t * handler will update it to the values actually used.\n+\t\t\t */\n+\t\t\tRectangle crop = ctrl.second.get();\n+\t\t\tlibcameraMetadata_.set(controls::DigitalZoom, crop);\n+\t\t\tbreak;\n+\t\t}\n+\n \t\tdefault:\n \t\t\tLOG(IPARPI, Warning)\n \t\t\t\t<< \"Ctrl \" << controls::controls.at(ctrl.first)->name()\ndiff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\nindex f4966f8..55db11d 100644\n--- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n+++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n@@ -287,7 +287,8 @@ class RPiCameraData : public CameraData\n public:\n \tRPiCameraData(PipelineHandler *pipe)\n \t\t: CameraData(pipe), sensor_(nullptr), lsTable_(nullptr),\n-\t\t state_(State::Stopped), dropFrame_(false), ispOutputCount_(0)\n+\t\t state_(State::Stopped), dropFrame_(false), ispOutputCount_(0),\n+\t\t ispMinSize_(0)\n \t{\n \t}\n \n@@ -322,6 +323,14 @@ public:\n \tvoid handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream);\n \tvoid handleState();\n \n+\tvoid initSensorCrop(Rectangle const &crop, unsigned int ispMinSize)\n+\t{\n+\t\t/* The initial zoom region is the whole of the sensorCrop_. */\n+\t\tsensorCrop_ = crop;\n+\t\tzoomRect_ = Rectangle{ 0, 0, crop.width, crop.height };\n+\t\tispMinSize_ = ispMinSize;\n+\t}\n+\n \tCameraSensor *sensor_;\n \t/* Array of Unicam and ISP device streams and associated buffers/streams. */\n \tRPiDevice unicam_;\n@@ -358,6 +367,15 @@ private:\n \n \tbool dropFrame_;\n \tint ispOutputCount_;\n+\t/*\n+\t * sensorCrop_ is the largest region of the full sensor output that matches\n+\t * the desired aspect ratio, and therefore represents the area within\n+\t * which we can pan and zoom. zoomRect_ is the portion from within the\n+\t * sensorCrop_ that pan/zoom is currently using.\n+\t */\n+\tRectangle sensorCrop_;\n+\tRectangle zoomRect_;\n+\tunsigned int ispMinSize_;\n };\n \n class RPiCameraConfiguration : public CameraConfiguration\n@@ -744,6 +762,10 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)\n \t\treturn ret;\n \t}\n \n+\tRectangle testCrop = { 0, 0, 1, 1 };\n+\tdata->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &testCrop);\n+\tconst unsigned int ispMinSize = testCrop.width;\n+\n \t/* Adjust aspect ratio by providing crops on the input image. */\n \tRectangle crop = {\n \t\t.x = 0,\n@@ -763,8 +785,12 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)\n \n \tcrop.x = (sensorFormat.size.width - crop.width) >> 1;\n \tcrop.y = (sensorFormat.size.height - crop.height) >> 1;\n+\n \tdata->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop);\n \n+\tsensorCrop_ = Size(crop.width, crop.height);\n+\tdata->initSensorCrop(crop, ispMinSize);\n+\n \tret = configureIPA(camera);\n \tif (ret)\n \t\tLOG(RPI, Error) << \"Failed to configure the IPA: \" << ret;\n@@ -1553,6 +1579,34 @@ void RPiCameraData::tryRunPipeline()\n \t */\n \tRequest *request = requestQueue_.front();\n \n+\tif (request->controls().contains(controls::DigitalZoom)) {\n+\t\tRectangle rect = request->controls().get(controls::DigitalZoom);\n+\t\t/*\n+\t\t * If a new digital zoom value was given, check that it lies within the\n+\t\t * available sensorCrop_, coercing it if necessary.\n+\t\t */\n+\t\tif (rect.width && rect.height) {\n+\t\t\tzoomRect_ = rect;\n+\t\t\tif (zoomRect_.width < ispMinSize_)\n+\t\t\t\tzoomRect_.width = ispMinSize_;\n+\t\t\telse if (zoomRect_.width > sensorCrop_.width)\n+\t\t\t\tzoomRect_.width = sensorCrop_.width;\n+\t\t\tif (zoomRect_.height < ispMinSize_)\n+\t\t\t\tzoomRect_.height = ispMinSize_;\n+\t\t\telse if (zoomRect_.height > sensorCrop_.height)\n+\t\t\t\tzoomRect_.height = sensorCrop_.height;\n+\t\t\tif (zoomRect_.x + zoomRect_.width > sensorCrop_.width)\n+\t\t\t\tzoomRect_.x = sensorCrop_.width - zoomRect_.width;\n+\t\t\tif (zoomRect_.y + zoomRect_.height > sensorCrop_.height)\n+\t\t\t\tzoomRect_.y = sensorCrop_.height - zoomRect_.height;\n+\t\t}\n+\t\tRectangle sensorRect = zoomRect_;\n+\t\tsensorRect.x += sensorCrop_.x;\n+\t\tsensorRect.y += sensorCrop_.y;\n+\t\tisp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &sensorRect);\n+\t\trequest->controls().set(controls::DigitalZoom, zoomRect_);\n+\t}\n+\n \t/*\n \t * Process all the user controls by the IPA. Once this is complete, we\n \t * queue the ISP output buffer listed in the request to start the HW\n","prefixes":["libcamera-devel","v2","2/2"]}