[{"id":12027,"web_url":"https://patchwork.libcamera.org/comment/12027/","msgid":"","date":"2020-08-17T17:02:52","subject":"Re: [libcamera-devel] [PATCH v2 0/5] Transform implementation","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Hi everyone\n\nI was wondering if I could give this a gentle nudge? I also thought it\nmight help if I explain a bit more what we're up to - it's a bit less\nrandom than it might seem!\n\nSome of you will know our existing \"raspicam\" apps, raspistill,\nraspivid etc. The plan is to introduce libcamera equivalents hoping to\nencourage more of our users to move to libcamera. (PiCamera - Python\nbindings for our existing stack - is also popular of course, but we're\ntackling the \"raspicam\" apps first.)\n\nThe aim is therefore to plug the most obvious functionality gaps,\nwhich we've identified as:\n\n1. Raw stream alongside video. This is addressed by some of Naush's\nrecent patches on zero-copy raw buffers.\n\n2. Ability to set camera configuration (esp. exposure/gain) before the\ncamera starts. This is very important to many users.\n\n3. User-level transforms of the image (e.g. 180 degree rotation).\n\n4. Lack of digital zoom.\n\nHaving addressed those we plan to start opening the apps to users to try,\nand being libcamera-based they could be helpful to non-Pi users too.\n\nI hope that extra information is useful!\n\nThanks and best regards\nDavid\n\nOn Thu, 6 Aug 2020 at 17:36, David Plowman\n wrote:\n>\n> Hi everyone\n>\n> Here's a second version of the 2d transform implementation (still for\n> discussion only). It follows the same principles as the previous\n> version, but the ALSC issues that I ran into are no longer part of\n> this set (indeed they have already been merged).\n>\n> There are now 5 patches. I'll add the relevant documentation once we\n> can feel confident there won't be too much more churn. That leaves:\n>\n> 1. The first merely adds the Transform enum. I to-ed and fro-ed a bit\n> here, and decided, after doing it both ways, that wrapping a class\n> round the enum didn't add a whole lot, and just left me with more\n> questions and more boilerplate. Anyway, see what you think...\n>\n> 2. The second patch adds the Transform enum to the\n> CameraConfiguration. I've also amended all the pipeline handlers\n> (including the Raspberry Pi one) simply to coerce it to the Identity\n> if it wasn't, marking the configuration as \"adjusted\".\n>\n> 3. This patch updates the Raspberry Pi pipeline handler to allow all\n> non-transposing transforms, and to set the flip bits in the sensor.\n>\n> 4. Here I just plumb the Transform through to the Raspberry Pi IPA\n> where it gets put into the CameraMode structure. This bothered me\n> first time round, but on reflection I think it's pragmatic to treat it\n> like that from the point of view of the IPAs.\n>\n> 5. This updates the ALSC algorithm to transform the calibrated tables\n> correctly. Note that we regenerate the tables from scratch if the\n> transform changes.\n>\n> Best regards\n> David\n>\n> David Plowman (5):\n> libcamera: Add Transform enum to represet 2d plane transforms.\n> libcamera: Add user Transform to CameraConfiguration\n> libcamera: raspberrypi: Set camera flips correctly from user transform\n> libcamera: raspberrypi: Plumb user transform through to IPA\n> libcamera: ipa: raspberrypi: ALSC: Handle user transform\n>\n> include/libcamera/camera.h | 3 +\n> include/libcamera/meson.build | 1 +\n> include/libcamera/transform.h | 58 +++++++++++++\n> src/ipa/raspberrypi/controller/camera_mode.h | 4 +\n> src/ipa/raspberrypi/controller/rpi/alsc.cpp | 13 ++-\n> src/ipa/raspberrypi/raspberrypi.cpp | 48 ++++++-----\n> src/libcamera/camera.cpp | 2 +-\n> src/libcamera/meson.build | 1 +\n> src/libcamera/pipeline/ipu3/ipu3.cpp | 5 ++\n> .../pipeline/raspberrypi/raspberrypi.cpp | 26 +++++-\n> src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 ++\n> src/libcamera/pipeline/simple/simple.cpp | 5 ++\n> src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 5 ++\n> src/libcamera/pipeline/vimc/vimc.cpp | 5 ++\n> src/libcamera/transform.cpp | 83 +++++++++++++++++++\n> 15 files changed, 238 insertions(+), 26 deletions(-)\n> create mode 100644 include/libcamera/transform.h\n> create mode 100644 src/libcamera/transform.cpp\n>\n> --\n> 2.20.1\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E1758BD879\n\tfor ;\n\tMon, 17 Aug 2020 17:03:06 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4BA3361930;\n\tMon, 17 Aug 2020 19:03:06 +0200 (CEST)","from mail-ot1-x334.google.com (mail-ot1-x334.google.com\n\t[IPv6:2607:f8b0:4864:20::334])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 675AB60384\n\tfor ;\n\tMon, 17 Aug 2020 19:03:04 +0200 (CEST)","by mail-ot1-x334.google.com with SMTP id q9so13967097oth.5\n\tfor ;\n\tMon, 17 Aug 2020 10:03:04 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"SETFDPvg\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to;\n\tbh=m1GzyZoOSzBEa9atlm581UHPRmRaSG7XVEReuhRCFmw=;\n\tb=SETFDPvgqlSuRjwNbyKWu4mcZuIdXHUJVpMJJmdshrWZKyIAJkYt6QWvIdQk413bxg\n\tOpNZ2n90O+HvuNMRSuyhd53yCrkw9+qTP+vGYgjumCMo4+vzdM2mtyIX4HVulOnOOIbb\n\t7o4RjTCkxJeZ/pREcBm7eH3itq3u7YH/M6p/sOhipjzEjlRCSbsbuEQTvpkAucu59KlA\n\tgN5iXZwExx7bgJVHfNK98Yfdq5GrBrxHf7AwfMNsqpRqDkApEMDrxZ9l4R3Qv9TF++as\n\tUKXRPpWJ3sb7s6z6YK0wAEbLevbC+A0frj7KmxBRz9CfWF0cwMr6A6tQThnKBoq8NlMa\n\thnCw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to;\n\tbh=m1GzyZoOSzBEa9atlm581UHPRmRaSG7XVEReuhRCFmw=;\n\tb=FoyXOx6fn6Y+U/bsiv0sANS/brjAHuCO9bYAM4635b2m3f1CcN9qoDtwnLgSXh+e3u\n\tUxOVu2ZKDeB3+kcX5p7oScRab3pszazULmx8JP0I1FbTz+C1fXqvu+H020XRu/94qopD\n\tDsqOidkKO3JDaElKQA63OjYcojbAaEVSmB5cxvdelMe1C9P38w1u1AH+zy4W6/dvoaGs\n\tCcUAJIX35aKq2X+2Zv0yLncdyAQxE+vxosOCXOSZSE0BFZDINN7UsKMLxm9TX+oosYmt\n\t+khbd8dEMTnNIGKEkO5PBp4dYNO2J8g3bOXWKk6dznxx4JQHJqiA8vurYC+ucPdaKQui\n\tZctw==","X-Gm-Message-State":"AOAM533vEZX3yDQvI+Ar11MeBQd4gmYlL2M6uQaLwPMCA96fIqJc1Uss\n\tPQOb6mk/NxeYUiAm0X1CSMaM3niWkD3LZS6a2LwyU9o9AM0g8A==","X-Google-Smtp-Source":"ABdhPJwQqpuCRNBLpkKuy9HOjFiVR4Mfu4Db3FhIQpv1mNWqlRWzq3V5MpB52u7Ew1A/VrFMaJwRoK3z5/W53djnlks=","X-Received":"by 2002:a05:6830:19d8:: with SMTP id\n\tp24mr11906663otp.166.1597683782518; \n\tMon, 17 Aug 2020 10:03:02 -0700 (PDT)","MIME-Version":"1.0","References":"<20200806163639.12971-1-david.plowman@raspberrypi.com>","In-Reply-To":"<20200806163639.12971-1-david.plowman@raspberrypi.com>","From":"David Plowman ","Date":"Mon, 17 Aug 2020 18:02:52 +0100","Message-ID":"","To":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-devel] [PATCH v2 0/5] Transform implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":12039,"web_url":"https://patchwork.libcamera.org/comment/12039/","msgid":"<20200819021627.GP2360@pendragon.ideasonboard.com>","date":"2020-08-19T02:16:27","subject":"Re: [libcamera-devel] [PATCH v2 0/5] Transform implementation","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi David,\n\nOn Mon, Aug 17, 2020 at 06:02:52PM +0100, David Plowman wrote:\n> Hi everyone\n> \n> I was wondering if I could give this a gentle nudge? I also thought it\n> might help if I explain a bit more what we're up to - it's a bit less\n> random than it might seem!\n\nI've now reviewed the patches. Sorry again about the delay.\n\n> Some of you will know our existing \"raspicam\" apps, raspistill,\n> raspivid etc. The plan is to introduce libcamera equivalents hoping to\n> encourage more of our users to move to libcamera. (PiCamera - Python\n> bindings for our existing stack - is also popular of course, but we're\n> tackling the \"raspicam\" apps first.)\n> \n> The aim is therefore to plug the most obvious functionality gaps,\n> which we've identified as:\n> \n> 1. Raw stream alongside video. This is addressed by some of Naush's\n> recent patches on zero-copy raw buffers.\n> \n> 2. Ability to set camera configuration (esp. exposure/gain) before the\n> camera starts. This is very important to many users.\n> \n> 3. User-level transforms of the image (e.g. 180 degree rotation).\n> \n> 4. Lack of digital zoom.\n> \n> Having addressed those we plan to start opening the apps to users to try,\n> and being libcamera-based they could be helpful to non-Pi users too.\n\nI'll continue with the review of the other pending series :-)\n\n> I hope that extra information is useful!\n> \n> On Thu, 6 Aug 2020 at 17:36, David Plowman wrote:\n> >\n> > Hi everyone\n> >\n> > Here's a second version of the 2d transform implementation (still for\n> > discussion only). It follows the same principles as the previous\n> > version, but the ALSC issues that I ran into are no longer part of\n> > this set (indeed they have already been merged).\n> >\n> > There are now 5 patches. I'll add the relevant documentation once we\n> > can feel confident there won't be too much more churn. That leaves:\n> >\n> > 1. The first merely adds the Transform enum. I to-ed and fro-ed a bit\n> > here, and decided, after doing it both ways, that wrapping a class\n> > round the enum didn't add a whole lot, and just left me with more\n> > questions and more boilerplate. Anyway, see what you think...\n> >\n> > 2. The second patch adds the Transform enum to the\n> > CameraConfiguration. I've also amended all the pipeline handlers\n> > (including the Raspberry Pi one) simply to coerce it to the Identity\n> > if it wasn't, marking the configuration as \"adjusted\".\n> >\n> > 3. This patch updates the Raspberry Pi pipeline handler to allow all\n> > non-transposing transforms, and to set the flip bits in the sensor.\n> >\n> > 4. Here I just plumb the Transform through to the Raspberry Pi IPA\n> > where it gets put into the CameraMode structure. This bothered me\n> > first time round, but on reflection I think it's pragmatic to treat it\n> > like that from the point of view of the IPAs.\n> >\n> > 5. This updates the ALSC algorithm to transform the calibrated tables\n> > correctly. Note that we regenerate the tables from scratch if the\n> > transform changes.\n> >\n> > David Plowman (5):\n> > libcamera: Add Transform enum to represet 2d plane transforms.\n> > libcamera: Add user Transform to CameraConfiguration\n> > libcamera: raspberrypi: Set camera flips correctly from user transform\n> > libcamera: raspberrypi: Plumb user transform through to IPA\n> > libcamera: ipa: raspberrypi: ALSC: Handle user transform\n> >\n> > include/libcamera/camera.h | 3 +\n> > include/libcamera/meson.build | 1 +\n> > include/libcamera/transform.h | 58 +++++++++++++\n> > src/ipa/raspberrypi/controller/camera_mode.h | 4 +\n> > src/ipa/raspberrypi/controller/rpi/alsc.cpp | 13 ++-\n> > src/ipa/raspberrypi/raspberrypi.cpp | 48 ++++++-----\n> > src/libcamera/camera.cpp | 2 +-\n> > src/libcamera/meson.build | 1 +\n> > src/libcamera/pipeline/ipu3/ipu3.cpp | 5 ++\n> > .../pipeline/raspberrypi/raspberrypi.cpp | 26 +++++-\n> > src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 ++\n> > src/libcamera/pipeline/simple/simple.cpp | 5 ++\n> > src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 5 ++\n> > src/libcamera/pipeline/vimc/vimc.cpp | 5 ++\n> > src/libcamera/transform.cpp | 83 +++++++++++++++++++\n> > 15 files changed, 238 insertions(+), 26 deletions(-)\n> > create mode 100644 include/libcamera/transform.h\n> > create mode 100644 src/libcamera/transform.cpp","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 87CAABD87C\n\tfor ;\n\tWed, 19 Aug 2020 02:16:47 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id F26B561E66;\n\tWed, 19 Aug 2020 04:16:46 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2635E60384\n\tfor ;\n\tWed, 19 Aug 2020 04:16:45 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A2B8129E;\n\tWed, 19 Aug 2020 04:16:44 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"JZhQXkpe\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1597803404;\n\tbh=A5+UgNiVsWzYQOS+UQ836HWUDnOAyLpi2aHPlg5l4TY=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=JZhQXkpeJ5R6VVoJBXubkxqAhYE8YBwnPGKWNwukMe9t8P+TXDUVdDWsvR2sp0VnW\n\t9qa0SN5u5TxE6ruH3k9RsfjwmCZJCtZnAcUaeau+Z66zALcfQtUCM3FljI+4Wc4hOn\n\t+7apidW219F5tMijUmQEVGgKUECMmuM1ftf69NYM=","Date":"Wed, 19 Aug 2020 05:16:27 +0300","From":"Laurent Pinchart ","To":"David Plowman ","Message-ID":"<20200819021627.GP2360@pendragon.ideasonboard.com>","References":"<20200806163639.12971-1-david.plowman@raspberrypi.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v2 0/5] Transform implementation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]