[{"id":26945,"web_url":"https://patchwork.libcamera.org/comment/26945/","msgid":"","date":"2023-04-27T00:51:10","subject":"Re: [libcamera-devel] Suboptimal resolution for digital zoom\n\t(controls::ScalerCrop)","submitter":{"id":159,"url":"https://patchwork.libcamera.org/api/people/159/","name":"Elias Naur","email":"mail@eliasnaur.com"},"content":"On Tue, 25 Apr 2023 at 05:24, David Plowman\n wrote:\n>\n> Hi\n>\n> It should also be possible to request a raw stream at the full sensor\n> resolution, but set the output size to the crop that you really want.\n> Initially you'll obviously find that the full image is being\n> downscaled to your output resolution. But then set the ScalerCrop to\n> match the output size that you asked for, and you should be able to\n> get a 1:1 crop from the full resolution image. Does that help?\n>\n> David\n\nThis sounds similar to Laurent's idea below. It worked, thank you.\nFor future reference, I spent some time figuring out that my dummy raw\nstream needed two properties to function as a sensor resolution setting,\nonly:\n\n rawConfig.colorSpace = ColorSpace::Raw;\n rawConfig.bufferCount = 0;\n\nWithout the colorSpace setting, Camera::configure fails even with a prior\nCameraConfiguration::validate. Without the bufferCount setting, you\nwon't get any request callbacks.\n\n>\n> On Tue, 25 Apr 2023 at 07:34, Laurent Pinchart via libcamera-devel\n> wrote:\n> >\n> > Hi Elias,\n> >\n> > On Mon, Apr 24, 2023 at 08:04:22PM -0600, Elias Naur via libcamera-devel wrote:\n> > > Hi,\n> > >\n> > > I'd like to squeeze out as much resolution as possible from my Raspberry Pi camera v1\n> > > for scanning QR codes. The camera has a fixed focus at ~50cm from the lens, so the\n> > > resolution gain from moving the QR codes closer is lost to blur from the loss of focus.\n> > >\n> > > My current strategy is to scan codes at a distance, but use the considerable resolution\n> > > of the sensor (~2500x1900) to gain enough resolution for successful scans.\n> > >\n> > > However, I fail to find any way to achieve that with libcamera. I tried three options, all\n> > > suboptimal:\n> > >\n> > > 1. Configure the camera for full resolution and crop to a smaller section in my program.\n> > > That works surprisingly well, except that the white-balance etc. algorithms run on the full\n> > > image and not just the cropped part.\n> > > 2. Configure the camera for full resolution and set controls::ScalerCrop to the area\n> > > of interest. This seems to fix the white-balance algorithms, but results in the cropped\n> > > area being *upscaled* to full resolution when I receive them. Scanning for QR codes\n> > > at full resolution is way too slow on a RPi Zero.\n> > > 3. Configure the camera for the same resolution of the area of interest, and set\n> > > controls::ScalerCrop. This is efficient, but the sensor is configured for the low\n> > > resolution and thus ruins the resolution gain.\n> > >\n> > > Questions:\n> > > 1. Is there another way?\n> >\n> > How about capturing full resolution frames, without cropping in the\n> > capture pipeline, and scanning for the QR code in the area of interest\n> > only ?\n> >\n\nI believe this is my option 1. above. It works, except that auto-white balance\netc. runs on the full field of view and not just the cropped subimage.\n\n> > > 2. The following hack works for me,\n> > >\n> > > diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n> > > b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n> > > index 00600441..6ed81dc1 100644\n> > > --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n> > > +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n> > > @@ -842,7 +842,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)\n> > > * request, and apply it to the sensor with the cached transform, if\n> > > * any.\n> > > */\n> > > - V4L2SubdeviceFormat sensorFormat = findBestFormat(data->sensorFormats_, rawStream ? sensorSize : maxSize, bitDepth);\n> > > + V4L2SubdeviceFormat sensorFormat = findBestFormat(data->sensorFormats_, Size{2592,1944}, bitDepth);\n> > > const RPiCameraConfiguration *rpiConfig = static_cast(config);\n> > > ret = data->sensor_->setFormat(&sensorFormat, rpiConfig->combinedTransform_);\n> > > if (ret)\n> > >\n> > > Would a similar change that *always* uses the full sensor resolution be acceptable?\n> > > I can't discern any performance issues between full resolution and lower resolution.\n> >\n> > You can't hardcode a specific resolution, as this wouldn't work properly\n> > with other sensors. Always selecting the sensor full resolution is also\n> > not acceptable, as that will limit the frame rate on high resolution\n> > sensors (e.g. the Raspberry Pi HQ camera will be limited to 15fps).\n> >\n\nI thought so.\n\n> > > 3. If the above fix is unacceptable, what is the right way to fix this\n> > > issue in libcamera?\n> >\n> > The current way to select the sensor resolution is a bit of a hack. You\n> > need to add a raw stream to your camera configuration, set to the full\n> > sensor resolution. You don't need to submit buffers for that stream when\n> > queuing requests, so you can just capture from the processed YUV stream\n> > as you do now.\n> >\n> > We will likely implement in the future an easier API to select the\n> > sensor resolution.\n> >\n\nThis idea worked, thank you! As you may have expected it took me a\nwhile to figure out\nthe right incantations, so a more intuitive way of setting the sensor resolution\nwould be greatly appreciated.\n\nElias","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8DADEC0DA4\n\tfor ;\n\tThu, 27 Apr 2023 00:51:26 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EA705627DF;\n\tThu, 27 Apr 2023 02:51:25 +0200 (CEST)","from mail-qk1-x734.google.com (mail-qk1-x734.google.com\n\t[IPv6:2607:f8b0:4864:20::734])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0194461EB1\n\tfor ;\n\tThu, 27 Apr 2023 02:51:22 +0200 (CEST)","by mail-qk1-x734.google.com with SMTP id\n\taf79cd13be357-74de9ce136cso375860685a.1\n\tfor ;\n\tWed, 26 Apr 2023 17:51:22 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1682556686;\n\tbh=NK+8uuwoY/yB07iVbCc4mk+suE0EhngJH7/bAUeohCI=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=nIevcZh6iAmEPvFfoB5wK7DtktCv6VIvAdOF1esAtAUyLfbM7T3j9GMC2hLE27wOV\n\tXhsGv3QKBe3PKUzpbOP08Dm7mwqOLohHqpHWk9M+A/nn1Kx5Oz0TleoNSW6xuc1Shy\n\tZWPJAtUUJZOpJbnq7lgqU8wkTR39yEC9utdv0lwA7WjSWNswMmWLxG6PJbwLvRN0d6\n\t2wXDbj7WK4BcuFMgp6/r84UmVskVDzeu7v74x2Xv5GB8ytpVrRQhy2i/qhhG1aAr9a\n\tj+Wz2R80vyVu1IxwVt0nrtTI7Qksu4wenyTUV6Ohd1UUWNMQLRYJW5erDkWwI4+TQh\n\taog4/HAD91tqw==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=eliasnaur.com; s=google; t=1682556681; x=1685148681;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to;\n\tbh=jy/3U7ORgFIQf3Yda9UhdIo9ra0iRlmyI2CnrIjVSBQ=;\n\tb=jvdUqZ/rL4SSE7jLefcm/cpSsRh8ppxcODOa6d0LenHfD7Hozd6eIwGYqeg+5JBlTx\n\tyDhM2eAH15S41FqQgXRwYam/oHiXyv9oCND+asx4OItvkcTxlckLjW30BjbFhyyT7BJi\n\ttVDB3i8Lzj1rMChQcpwkoyOO1LrFpiXDVgMEyRcJKYYHNLd/ZhF1NYwmb2AhAPGGl6oE\n\tWvD0WfwryXUrtKwOkvIPDkTcJgKtITLQNr4rl6byp2dg5nnRNjg3ZHLYYmmXRDxpu0jh\n\tUnB3PQ36mgqTSpuB/PHHi/7i0F8ngZD8Z1bqwjk5l+ah38+H6sC1cKAUiTzY5tKbe7n4\n\timcA=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=eliasnaur.com\n\theader.i=@eliasnaur.com\n\theader.b=\"jvdUqZ/r\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20221208; t=1682556681; x=1685148681;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=jy/3U7ORgFIQf3Yda9UhdIo9ra0iRlmyI2CnrIjVSBQ=;\n\tb=duuf/CdE1uP+U2H4QKeh7ZQFEepRk4lwoB4WFiGOowbulXJiqd98xWXMSgIdzArfDu\n\txiVE55Kj38vy2chdrryHreBhHDx7G+/tN8KDKDe5f4RZHm/tNEXNrX794l5aQkT+OC70\n\t/YNBy5F6UCRL8cJ/lg6TJDJN4IWt9VGhjWphsopy8WAkha7hwwbcAyDG5G55xtUOro/O\n\tLJQf9Q8Se9E/Q0Muz+DNt5oYh0OoMmN7MV+0Ds/vAkVN32s9v5iaN/EZZ62jyRH+yBgu\n\t55Z95zjFo8u/f2kJXnh9NH8LsFj9ldcajwUsyfVj5+Lx1sQPjNKyLxeXWdVtWPsyhcSJ\n\tkL1Q==","X-Gm-Message-State":"AAQBX9cB0gGAh710LP4sCr7/XHNe+/yYsp+BnM8K+qLpYtMekmgUhlZG\n\tPRk0sH41tXBd8++nVdZRlHRa7StmY1s4REVWxbLwRQ==","X-Google-Smtp-Source":"AKy350bAnlcLSzITt9Cj3B72grGP4UqkRPkyKnSqJfD0TleRC3/aWqSBL6zxkuIaSihBkxrkp9yWNig/5I1Xg+fSo1g=","X-Received":"by 2002:a05:6214:da7:b0:56e:9986:4fa9 with SMTP id\n\th7-20020a0562140da700b0056e99864fa9mr42050125qvh.7.1682556681649;\n\tWed, 26 Apr 2023 17:51:21 -0700 (PDT)","MIME-Version":"1.0","References":"\n\t<20230425063426.GD11371@pendragon.ideasonboard.com>\n\t","In-Reply-To":"","Date":"Wed, 26 Apr 2023 18:51:10 -0600","Message-ID":"","To":"David Plowman ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] Suboptimal resolution for digital zoom\n\t(controls::ScalerCrop)","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Elias Naur via libcamera-devel ","Reply-To":"Elias Naur ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]