{"id":8937,"url":"https://patchwork.libcamera.org/api/1.1/patches/8937/?format=json","web_url":"https://patchwork.libcamera.org/patch/8937/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20200723084338.13711-3-david.plowman@raspberrypi.com>","date":"2020-07-23T08:43:38","name":"[libcamera-devel,v3,2/2] libcamera: raspberrypi: Implement digital zoom","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"b9b4251d72fd31a3616558d4b9171334e3d1cfff","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/1.1/people/42/?format=json","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/8937/mbox/","series":[{"id":1129,"url":"https://patchwork.libcamera.org/api/1.1/series/1129/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=1129","date":"2020-07-23T08:43:36","name":"Digital zoom implementatino","version":3,"mbox":"https://patchwork.libcamera.org/series/1129/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/8937/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/8937/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id EE1C2BD86F\n\tfor ;\n\tThu, 23 Jul 2020 08:43:47 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1AC2A61146;\n\tThu, 23 Jul 2020 10:43:47 +0200 (CEST)","from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com\n\t[IPv6:2a00:1450:4864:20::32d])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9664B60536\n\tfor ;\n\tThu, 23 Jul 2020 10:43:45 +0200 (CEST)","by mail-wm1-x32d.google.com with SMTP id y24so231931wma.1\n\tfor ;\n\tThu, 23 Jul 2020 01:43:45 -0700 (PDT)","from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72])\n\tby smtp.gmail.com with ESMTPSA id\n\tt2sm2809450wmb.25.2020.07.23.01.43.44\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 23 Jul 2020 01:43:44 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"edfQQA/7\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references\n\t:mime-version:content-transfer-encoding;\n\tbh=ueNwn2zsSD1A5AY+qG+4p8wVmR8jVQtrsuCUmF/L9V8=;\n\tb=edfQQA/73RiTwe5PWCuWIMCQkGKaW9UT63yBjbM+3/D00LUGfOvG2YbOvgAkiC2TYF\n\t7AN9FDaC6mu9D/lmkDqzwDJLUEBRNmw/DtHNHN5bJmafd1LDIPpXz+YAkDkRWG4Cpw3h\n\tvkRcJb1IVoOnAiDbd4CHftyFoLLs45fXK17Dh8hvwoYNDNWz++mHXU4+PG7EFM1FsU51\n\tifAF3/4qAhUCbysYml5+b38d8HhxokOgfYT5C7krTDDXEyvJqNzizpRMKWraVRsaiZdn\n\tT2njPxoTs1KrlWweehxIX2TrFRR9XIBI548ThCcgW/NBqGNNd42TWmtydceU+6sPIvKZ\n\tsR6A==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-transfer-encoding;\n\tbh=ueNwn2zsSD1A5AY+qG+4p8wVmR8jVQtrsuCUmF/L9V8=;\n\tb=KcQYtyCsO1pJpC1veKvc4kzd9sEEqEVG6gGCwAus0njO7PWYHRhsJcQX5GVzobQU4D\n\tSzYfpjaKTVpqEyWKrS6g3OnrsiBKKbqHI40HKuw456/ClXxxBEEqPh++hC0zreSv9Hyk\n\trkrpyDY65Fv8GF+iG4rokYmP/ITNAfiP75qkuwV6jOnpqxQSze5JG/WK4+RICA5LxJ0p\n\tOWjML8Cli8nhuRhdp6+RPAYdySU276H9EN6iOf1zNsm3WLfczcOkG8JWW/Zrpyc52f+x\n\tug2+WGcgZOI7/qng5RPdn8TOeljrksx9+0z6MuFmZxVU6L8d4sMw5e3QSHc6ult+UmBo\n\tpd6w==","X-Gm-Message-State":"AOAM5331YuHxVFmaFJ/OPEQu8NM4TPgjq+rVnCahrhfJ+zzZT2cxL6E3\n\twAs78DOdhMiuoO4UzsOMDydLvzuHo0CMSA==","X-Google-Smtp-Source":"ABdhPJziqnoyNRdpwQWIPoaV/29q1EI9Cag3VDbFs5q+gsz8oh1cXjqN8buxWgHt2nmtmu3h1QWwRg==","X-Received":"by 2002:a1c:18e:: with SMTP id 136mr3093026wmb.93.1595493824921; \n\tThu, 23 Jul 2020 01:43:44 -0700 (PDT)","From":"David Plowman ","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 23 Jul 2020 09:43:38 +0100","Message-Id":"<20200723084338.13711-3-david.plowman@raspberrypi.com>","X-Mailer":"git-send-email 2.20.1","In-Reply-To":"<20200723084338.13711-1-david.plowman@raspberrypi.com>","References":"<20200723084338.13711-1-david.plowman@raspberrypi.com>","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v3 2/2] libcamera: raspberrypi: Implement\n\tdigital zoom","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"These changes implement digital zoom for the Raspberry Pi. We detect\nthe digital zoom control and update the V4L2 \"selection\" before\nstarting the ISP. We also update the value in the control that we send\nto the IPA, so that it has the correct value.\n\nSigned-off-by: David Plowman \n---\n include/libcamera/ipa/raspberrypi.h | 1 +\n src/ipa/raspberrypi/raspberrypi.cpp | 10 +++\n .../pipeline/raspberrypi/raspberrypi.cpp | 72 +++++++++++++++++++\n 3 files changed, 83 insertions(+)","diff":"diff --git a/include/libcamera/ipa/raspberrypi.h b/include/libcamera/ipa/raspberrypi.h\nindex a493776..f42e13e 100644\n--- a/include/libcamera/ipa/raspberrypi.h\n+++ b/include/libcamera/ipa/raspberrypi.h\n@@ -56,6 +56,7 @@ static const ControlInfoMap RPiControls = {\n \t{ &controls::Contrast, ControlInfo(0.0f, 32.0f) },\n \t{ &controls::Saturation, ControlInfo(0.0f, 32.0f) },\n \t{ &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) },\n+\t{ &controls::DigitalZoom, ControlInfo(Rectangle(0, 0, 0, 0), Rectangle(65535, 65535, 65535, 65535), Rectangle(0, 0, 0, 0)) },\n };\n \n } /* namespace libcamera */\ndiff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp\nindex 7bd0488..f759d4d 100644\n--- a/src/ipa/raspberrypi/raspberrypi.cpp\n+++ b/src/ipa/raspberrypi/raspberrypi.cpp\n@@ -679,6 +679,16 @@ void IPARPi::queueRequest(const ControlList &controls)\n \t\t\tbreak;\n \t\t}\n \n+\t\tcase controls::DIGITAL_ZOOM: {\n+\t\t\t/*\n+\t\t\t * We send the zoom info back in the metadata where the pipeline\n+\t\t\t * handler will update it to the values actually used.\n+\t\t\t */\n+\t\t\tRectangle crop = ctrl.second.get();\n+\t\t\tlibcameraMetadata_.set(controls::DigitalZoom, crop);\n+\t\t\tbreak;\n+\t\t}\n+\n \t\tdefault:\n \t\t\tLOG(IPARPI, Warning)\n \t\t\t\t<< \"Ctrl \" << controls::controls.at(ctrl.first)->name()\ndiff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\nindex bf1c771..6bc595f 100644\n--- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n+++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n@@ -308,6 +308,14 @@ public:\n \tvoid handleStreamBuffer(FrameBuffer *buffer, const RPiStream *stream);\n \tvoid handleState();\n \n+\tvoid initPipelineCrop(Rectangle const &crop, Size const &ispMinSize)\n+\t{\n+\t\t/* The initial zoom region is the whole of the pipelineCrop_. */\n+\t\tpipelineCrop_ = crop;\n+\t\tzoomRect_ = Rectangle(0, 0, crop.width, crop.height);\n+\t\tispMinSize_ = ispMinSize;\n+\t}\n+\n \tCameraSensor *sensor_;\n \t/* Array of Unicam and ISP device streams and associated buffers/streams. */\n \tRPiDevice unicam_;\n@@ -344,6 +352,15 @@ private:\n \n \tbool dropFrame_;\n \tint ispOutputCount_;\n+\t/*\n+\t * pipelineCrop_ is the largest region of the full sensor output that matches\n+\t * the desired aspect ratio, and therefore represents the area within\n+\t * which we can pan and zoom. zoomRect_ is the portion from within the\n+\t * sensorCrop_ that pan/zoom is currently using.\n+\t */\n+\tRectangle pipelineCrop_;\n+\tRectangle zoomRect_;\n+\tSize ispMinSize_;\n };\n \n class RPiCameraConfiguration : public CameraConfiguration\n@@ -736,6 +753,10 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)\n \t\treturn ret;\n \t}\n \n+\tRectangle testCrop(0, 0, 1, 1);\n+\tdata->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &testCrop);\n+\tunsigned int minSize = std::max(testCrop.width, testCrop.height);\n+\n \t/* Adjust aspect ratio by providing crops on the input image. */\n \tRectangle crop{ 0, 0, sensorFormat.size };\n \n@@ -750,8 +771,17 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)\n \n \tcrop.x = (sensorFormat.size.width - crop.width) >> 1;\n \tcrop.y = (sensorFormat.size.height - crop.height) >> 1;\n+\n \tdata->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop);\n \n+\tpipelineCrop_ = Size(crop.width, crop.height);\n+\tSize ispMinSize(minSize, minSize);\n+\tif (crop.width > crop.height)\n+\t\tispMinSize.width = minSize * crop.width / crop.height;\n+\telse if (crop.width < crop.height)\n+\t\tispMinSize.height = minSize * crop.height / crop.width;\n+\tdata->initPipelineCrop(crop, ispMinSize);\n+\n \tret = data->configureIPA();\n \tif (ret)\n \t\tLOG(RPI, Error) << \"Failed to configure the IPA: \" << ret;\n@@ -1488,6 +1518,30 @@ void RPiCameraData::checkRequestCompleted()\n \t}\n }\n \n+static void fixup_dimension(int &start, unsigned int &size,\n+\t\t\t unsigned int minSize, unsigned int maxSize)\n+{\n+\t/*\n+\t * If the size being requested it too small we must increase it, but\n+\t * we try to keep the mid point the same. Though if this would mean\n+\t * extending beyond the available bounds, then we have to line ourselves\n+\t * up flush.\n+\t */\n+\tint mid = start + size / 2;\n+\tsize = std::min(std::max(size, minSize), maxSize);\n+\tstart = mid - size / 2;\n+\tif (start < 0)\n+\t\tstart = 0;\n+\telse if (start + size > maxSize)\n+\t\tstart = maxSize - size;\n+}\n+\n+static void fixup_rectangle(Rectangle ®ion, Size const &minSize, Size const &maxSize)\n+{\n+\tfixup_dimension(region.x, region.width, minSize.width, maxSize.width);\n+\tfixup_dimension(region.y, region.height, minSize.height, maxSize.height);\n+}\n+\n void RPiCameraData::tryRunPipeline()\n {\n \tFrameBuffer *bayerBuffer, *embeddedBuffer;\n@@ -1541,6 +1595,24 @@ void RPiCameraData::tryRunPipeline()\n \t */\n \tRequest *request = requestQueue_.front();\n \n+\tif (request->controls().contains(controls::DigitalZoom)) {\n+\t\tRectangle rect = request->controls().get(controls::DigitalZoom);\n+\t\t/*\n+\t\t * If a new digital zoom value was given, check that it lies within the\n+\t\t * available pipelineCrop_, and that it's not smaller than permitted.\n+\t\t */\n+\t\tif (rect.width && rect.height) {\n+\t\t\tzoomRect_ = rect;\n+\t\t\tfixup_rectangle(zoomRect_, ispMinSize_,\n+\t\t\t\t\tSize(pipelineCrop_.width, pipelineCrop_.height));\n+\t\t}\n+\t\tRectangle sensorRect = zoomRect_;\n+\t\tsensorRect.x += pipelineCrop_.x;\n+\t\tsensorRect.y += pipelineCrop_.y;\n+\t\tisp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &sensorRect);\n+\t\trequest->controls().set(controls::DigitalZoom, zoomRect_);\n+\t}\n+\n \t/*\n \t * Process all the user controls by the IPA. Once this is complete, we\n \t * queue the ISP output buffer listed in the request to start the HW\n","prefixes":["libcamera-devel","v3","2/2"]}