[{"id":27928,"web_url":"https://patchwork.libcamera.org/comment/27928/","msgid":"<169701964499.2915094.17960336807330474858@ping.linuxembedded.co.uk>","date":"2023-10-11T10:20:44","subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi David,\n\nQuoting David Plowman via libcamera-devel (2023-10-05 11:41:33)\n> We add an HdrMode control (to enable and disable HDR processing)\n> and an HdrChannel, which indicates what kind of HDR frame (short, long\n> or otherwise) has just arrived.\n> \n> Currently the HdrMode supports the following values:\n> \n> * Off - no HDR processing at all.\n> * MultiExposureUnmerged - frames at multiple different exposures are\n>   produced, but not merged together. They are returned \"as is\".\n> * MultiExposure - frames at multiple different exposures are merged\n>   to create HDR images.\n> * SingleExposure - multiple frames all at the same exposure are\n>   merged to create HDR images.\n> * Night - multiple frames will be combined to create \"night mode\"\n>   images.\n> \n> Signed-off-by: David Plowman <david.plowman@raspberrypi.com>\n> Reviewed-by: Naushir Patuck <naush@raspberrypi.com>\n> ---\n>  src/libcamera/control_ids.yaml | 63 ++++++++++++++++++++++++++++++++++\n>  1 file changed, 63 insertions(+)\n> \n> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> index f2e542f4..e3699e74 100644\n> --- a/src/libcamera/control_ids.yaml\n> +++ b/src/libcamera/control_ids.yaml\n> @@ -774,6 +774,69 @@ controls:\n>              Continuous AF is paused. No further state changes or lens movements\n>              will occur until the AfPauseResume control is sent.\n>  \n> +  - HdrMode:\n> +      type: int32_t\n> +      description: |\n> +        Control to set the mode to be used for High Dynamic Range (HDR)\n> +        imaging.\n> +\n> +      enum:\n> +        - name: HdrModeOff\n> +          value: 0\n> +          description: |\n> +            HDR is not enabled.\n> +        - name: HdrModeMultiExposureUnmerged\n> +          value: 1\n> +          description: |\n> +            Multiple exposures will be generated in an alternating fashion.\n> +            However, they will not be merged together and will be returned to\n> +            the application as they are. The expectation is that, if necessary,\n> +            the application can merge them to create HDR images for itself.\n> +        - name: HdrModeMultiExposure\n> +          value: 2\n> +          description: |\n> +            Multiple exposures will be generated and merged to create HDR\n> +            images.\n> +        - name: HdrModeSingleExposure\n> +          value: 3\n> +          description: |\n> +            Multiple frames all at a single exposure will be used to create HDR\n> +            images.\n> +        - name: HdrModeNight\n> +          value: 4\n> +          description: |\n> +            Multiple frames will be combined to produce \"night mode\" images.\n\nMdrModeMultiExposureUnmerged has been documented to describe what that\nis doing. HdrModeMultiExposure, and HdrModeSingleExposure are more self\nexplanatory - but HdrModeNight might need more explaining.\n\nDoes this just mean sequential images are stacked? Are there different\nexposures? Are there any other expectations that a user may need or\nassumptions they could get wrong?\n\nOr is it just 'Hey do anything you like to make pictures look better at\nnight time or in low light conditions'?.\n\n\n\n> +\n> +  - HdrChannel:\n> +      type: int32_t\n> +      description: |\n> +        This value is reported back to the application so that it can discover\n> +        whether this capture corresponds to the short or long exposure image (or\n> +        any other image used by the HDR procedure).\n> +\n> +      enum:\n> +        - name: HdrChannelNone\n> +          value: 0\n> +          description: |\n> +            This image does not correspond to any of the captures used to create\n> +            an HDR image.\n> +        - name: HdrChannelShort\n> +          value: 1\n> +          description: |\n> +            This is a short exposure image.\n> +        - name: HdrChannelMedium\n> +          value: 2\n> +          description: |\n> +            This is a medium exposure image.\n> +        - name: HdrChannelLong\n> +          value: 3\n> +          description: |\n> +            This is a long exposure image.\n> +        - name: HdrChannelNight\n> +          value: 4\n> +          description: |\n> +            This frame has been used to produce a \"night mode\" image.\n\nWhat does it mean if a 'single' frame has been used to produce a night\nmode image. Does that mean it's blended with previous frames? Or is this\nframe only 'one' of a set?\n\n\"Frame has been used to produce\" makes it sound like the result of using\nthis frame will be output later and maybe this frame should be discarded\nbecause it's not the \"final\" night mode image ?\n\n\n\n> +\n>    # ----------------------------------------------------------------------------\n>    # Draft controls section\n>  \n> -- \n> 2.30.2\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5FD0EC3272\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed, 11 Oct 2023 10:20:50 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id ACA8F62970;\n\tWed, 11 Oct 2023 12:20:49 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C4F5B61DDA\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 11 Oct 2023 12:20:47 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(aztw-30-b2-v4wan-166917-cust845.vm26.cable.virginm.net\n\t[82.37.23.78])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 3BE9847C;\n\tWed, 11 Oct 2023 12:20:45 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1697019649;\n\tbh=6dPpDbG25G7BZQOjlUGGGELeNgSjZpX8y1oSEsd17i4=;\n\th=In-Reply-To:References:To:Date:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=uxvI0RXPRmBAyhp8WjX3+zF+HNwWArnmHJqOX0c9IkmnUh+Fnsgd0P05tFn14Vg8+\n\tC55Wjs2qsijChRWCC1nxpq10XC0gsI/R4DtvZkkwHs1R117CDT6nbQTcX1n7rFQIsM\n\tqmBOlnrUSayKeW1lVKFBWb9NNL2WzplVHOSk1cf4RGoZT6M6fqWTwbXNzkkCzTgKAZ\n\tNs9Q7LyIXzHrtF66ArkK0M+psa/f7kt0oQMteJfJFvSMPbdC9lnHR4CDrLBcqAZrpk\n\tGehSwMCn4TR+stbdLEgkzabi1Jo1FBuMmeqeMEgj973PExO0DSs085/12h2IV6wR7Q\n\tTxBOZ9TSx8X/w==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1697019645;\n\tbh=6dPpDbG25G7BZQOjlUGGGELeNgSjZpX8y1oSEsd17i4=;\n\th=In-Reply-To:References:Subject:From:To:Date:From;\n\tb=i2dNDcmFXkeo3cPoJvNwdvEu1JSE+OCeIJ+2QoACVtBOXz4oSD3+Zk1ALCNA0QTAL\n\t7NSIG7e7aqfGagjCv97oZUH6xox0m2IvjhaKGD2EuSqoUjud/Up5niX4B102lWHfNe\n\tpu9f4AyORAO2LWLWCNrLzUmXYKIaorucFmEMgRjQ="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"i2dNDcmF\"; dkim-atps=neutral","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20231005104133.51901-1-david.plowman@raspberrypi.com>","References":"<20231005104133.51901-1-david.plowman@raspberrypi.com>","To":"David Plowman <david.plowman@raspberrypi.com>,\n\tlibcamera-devel@lists.libcamera.org","Date":"Wed, 11 Oct 2023 11:20:44 +0100","Message-ID":"<169701964499.2915094.17960336807330474858@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","Subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","From":"Kieran Bingham via libcamera-devel\n\t<libcamera-devel@lists.libcamera.org>","Reply-To":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":27930,"web_url":"https://patchwork.libcamera.org/comment/27930/","msgid":"<CAHW6GY+P6yVG-HSGaZ=qTGY8vwi6oiAqsEQqq3yYjfjAh_cmGQ@mail.gmail.com>","date":"2023-10-11T10:39:26","subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Hi Kieran\n\nThanks for the questions!\n\nOn Wed, 11 Oct 2023 at 11:20, Kieran Bingham\n<kieran.bingham@ideasonboard.com> wrote:\n>\n> Hi David,\n>\n> Quoting David Plowman via libcamera-devel (2023-10-05 11:41:33)\n> > We add an HdrMode control (to enable and disable HDR processing)\n> > and an HdrChannel, which indicates what kind of HDR frame (short, long\n> > or otherwise) has just arrived.\n> >\n> > Currently the HdrMode supports the following values:\n> >\n> > * Off - no HDR processing at all.\n> > * MultiExposureUnmerged - frames at multiple different exposures are\n> >   produced, but not merged together. They are returned \"as is\".\n> > * MultiExposure - frames at multiple different exposures are merged\n> >   to create HDR images.\n> > * SingleExposure - multiple frames all at the same exposure are\n> >   merged to create HDR images.\n> > * Night - multiple frames will be combined to create \"night mode\"\n> >   images.\n> >\n> > Signed-off-by: David Plowman <david.plowman@raspberrypi.com>\n> > Reviewed-by: Naushir Patuck <naush@raspberrypi.com>\n> > ---\n> >  src/libcamera/control_ids.yaml | 63 ++++++++++++++++++++++++++++++++++\n> >  1 file changed, 63 insertions(+)\n> >\n> > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > index f2e542f4..e3699e74 100644\n> > --- a/src/libcamera/control_ids.yaml\n> > +++ b/src/libcamera/control_ids.yaml\n> > @@ -774,6 +774,69 @@ controls:\n> >              Continuous AF is paused. No further state changes or lens movements\n> >              will occur until the AfPauseResume control is sent.\n> >\n> > +  - HdrMode:\n> > +      type: int32_t\n> > +      description: |\n> > +        Control to set the mode to be used for High Dynamic Range (HDR)\n> > +        imaging.\n> > +\n> > +      enum:\n> > +        - name: HdrModeOff\n> > +          value: 0\n> > +          description: |\n> > +            HDR is not enabled.\n> > +        - name: HdrModeMultiExposureUnmerged\n> > +          value: 1\n> > +          description: |\n> > +            Multiple exposures will be generated in an alternating fashion.\n> > +            However, they will not be merged together and will be returned to\n> > +            the application as they are. The expectation is that, if necessary,\n> > +            the application can merge them to create HDR images for itself.\n> > +        - name: HdrModeMultiExposure\n> > +          value: 2\n> > +          description: |\n> > +            Multiple exposures will be generated and merged to create HDR\n> > +            images.\n> > +        - name: HdrModeSingleExposure\n> > +          value: 3\n> > +          description: |\n> > +            Multiple frames all at a single exposure will be used to create HDR\n> > +            images.\n> > +        - name: HdrModeNight\n> > +          value: 4\n> > +          description: |\n> > +            Multiple frames will be combined to produce \"night mode\" images.\n>\n> MdrModeMultiExposureUnmerged has been documented to describe what that\n> is doing. HdrModeMultiExposure, and HdrModeSingleExposure are more self\n> explanatory - but HdrModeNight might need more explaining.\n>\n> Does this just mean sequential images are stacked? Are there different\n> exposures? Are there any other expectations that a user may need or\n> assumptions they could get wrong?\n>\n> Or is it just 'Hey do anything you like to make pictures look better at\n> night time or in low light conditions'?.\n\nIt means this last thing. I don't feel I can judge what any other\nvendor will do HDR-wise, so these modes are the most generic \"hooks\" I\ncan think of that stand a fair chance of vendors at large being able\nto hang their HDR implementations off them.\n\nFor night mode in particular - it's \"do whatever it is that you do\".\nThough there's an implicit \"probably do it with some kind of frame\nmerging or HDR-like procedure\", because I think a number of vendors do\nindeed do this. But there's no knowing for sure. \"Night modes\" might\nbe completely different things too, like we have this magic HDR mode\nin the imx708 sensor which just doesn't fit in here.\n\n>\n>\n>\n> > +\n> > +  - HdrChannel:\n> > +      type: int32_t\n> > +      description: |\n> > +        This value is reported back to the application so that it can discover\n> > +        whether this capture corresponds to the short or long exposure image (or\n> > +        any other image used by the HDR procedure).\n> > +\n> > +      enum:\n> > +        - name: HdrChannelNone\n> > +          value: 0\n> > +          description: |\n> > +            This image does not correspond to any of the captures used to create\n> > +            an HDR image.\n> > +        - name: HdrChannelShort\n> > +          value: 1\n> > +          description: |\n> > +            This is a short exposure image.\n> > +        - name: HdrChannelMedium\n> > +          value: 2\n> > +          description: |\n> > +            This is a medium exposure image.\n> > +        - name: HdrChannelLong\n> > +          value: 3\n> > +          description: |\n> > +            This is a long exposure image.\n> > +        - name: HdrChannelNight\n> > +          value: 4\n> > +          description: |\n> > +            This frame has been used to produce a \"night mode\" image.\n>\n> What does it mean if a 'single' frame has been used to produce a night\n> mode image. Does that mean it's blended with previous frames? Or is this\n> frame only 'one' of a set?\n\nIt's hard to say what this really means, as I think different vendors\nwill do all kinds of different things. Unfortunately I suspect the HDR\nor \"night mode\" is likely to be very vendor specific, and we may find\nthey work quite differently.\n\nIn our case, all it really means is that the exposure for the frame\nwas calculated by AEC/AGC running with its \"night mode\" settings. It\nwill have been combined with previous \"night mode\" frames (if any).\n\n>\n> \"Frame has been used to produce\" makes it sound like the result of using\n> this frame will be output later and maybe this frame should be discarded\n> because it's not the \"final\" night mode image ?\n\nTo some extent this is me using vague language because I can't say for\nsure what anyone else will do. Actually it does so happen that in our\n\"night mode\" you will need to discard a few frames to give the image\nmerging time to work (you could take the very first image - but it\nwill be noisy). But other vendors may not do it like this. Or they\nmay. But probably not. Who knows!\n\nBut I'm definitely open to rewording any of this if we can agree on\nthe level of vaguess/specificness that's required...\n\nDavid\n\n>\n>\n>\n> > +\n> >    # ----------------------------------------------------------------------------\n> >    # Draft controls section\n> >\n> > --\n> > 2.30.2\n> >","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C670EC3272\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed, 11 Oct 2023 10:39:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2A31562970;\n\tWed, 11 Oct 2023 12:39:40 +0200 (CEST)","from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com\n\t[IPv6:2607:f8b0:4864:20::f2a])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A71F961DDA\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 11 Oct 2023 12:39:38 +0200 (CEST)","by mail-qv1-xf2a.google.com with SMTP id\n\t6a1803df08f44-66b024d26e2so7094656d6.0\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 11 Oct 2023 03:39:38 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1697020780;\n\tbh=/JN9XrxJCC63+HYxeNThZxZklIG8qYMLxzt1tyY3dSU=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=wi2Ot2QeLuNGGeqn4s8zVh79VqcCxMZhBbB+Act3g9p+J1nHw6bQUC34abAyJ+Hi+\n\tUXqn4sPHXHcq1JlIfxitVGhahiDnWfQnxWWhMhksias9VNvOPSARfXUo3JhWXrIcbA\n\tmayxDB1CtLIDkPLp6tN31C1yX6HuKLjgAnYW2J1xF3s8RQKwOx+WxsvS3U1Yv0OPFG\n\teLb5w64RUEKeO8ww4HxflfkT9iSatKIdjYhQ/VcWZ4SwOqBUE0bDYsuSIzdbkc2qmq\n\tONr2Q1vzlFMJB46EI1YDA48cHkvV6IsbBGSvxtcfeujHr5jhiPmXwPLP5TfvTynTti\n\tkhBRokBKz8WNQ==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1697020777; x=1697625577;\n\tdarn=lists.libcamera.org; \n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to;\n\tbh=WeCY9kCel0dvQqiuDfjY0HbsZoryYsdggHW0z9WU7ag=;\n\tb=YwVcbH5p2uW2mBK7mHNHcU4UOLk9sqlwPY+voBnGUEiwNqm+52p5qAKIzlfBo+1S52\n\tXu2m6jbPvMVp5VSj8nvNnow+2qQqGaZwNrpbGct88qWVp8i8Z22xLAblT0xKMMs+yEQO\n\tPPq3eCwPLCookBbN6jO4FJQc8sZk4+Mn0YqUbLAGwaR5kr6lQM+81ld22ov+MzJCb2Gg\n\tiABusItQcywGU7HK5B9acqxgRWu7fH2g/3DhYuw6d78scKczEuiUnoRDc17VQrviHhI+\n\t9X6CCTQPsawVGSgy4rwCwaEwBGWLOWjujWe8TfeXZ79zWkOEmRVNyIUjaXJ6yv71Ivfz\n\t5rdQ=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"YwVcbH5p\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1697020777; x=1697625577;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=WeCY9kCel0dvQqiuDfjY0HbsZoryYsdggHW0z9WU7ag=;\n\tb=hkZj/47ZOiAAvYIgT1u0AxCywBMR2XYDBXs/nAoUPz4RMaaejtWW2gj3QjWtosWI/f\n\tXDzpV3X1y0FbAzUMhIxIIkAAjdH7fwuuZmEOYrrEcBH66RQLI9Hk6qi07bwSJJGYvRUr\n\tyxGPEGd7IRXQkhc8W4OZKh3A1UL/xwzOJz+t7K8Qx2OuDwzslLMTW9bUkpLgtSM9px/m\n\t1xgzeKwLoe66eWw9Ko7qrCZHE/QLTBZFmrpaLqbambX6eEehqMHw9gltO091rO9Moecw\n\t/QH2s7GBo56oe3aXnwbKIe6cBnXl5KFay2dbXjtqEdrqvQpuTW3oSnSiHzqHmiKIQqtt\n\tsMZg==","X-Gm-Message-State":"AOJu0YymFdwIZ2FhAZNKczGJ2kxtKh2oq4GPrDE6j0FX7aeaQdsOlBaw\n\tbdRLENA8FEMHa/XlZ5PQ0oL/b1UUqZnA3foHi20n8BMhmqqJiN2O","X-Google-Smtp-Source":"AGHT+IEL2rCBu+T5Eu37/IvO6eXCGGNLXHcptP8pUzxcNI4tky4jxZYDIkIzJ5kST0wWlVUaCl4M3SkE7YO7dTdPs/s=","X-Received":"by 2002:ad4:5be5:0:b0:65b:c89:f44c with SMTP id\n\tk5-20020ad45be5000000b0065b0c89f44cmr21229604qvc.7.1697020777391;\n\tWed, 11 Oct 2023 03:39:37 -0700 (PDT)","MIME-Version":"1.0","References":"<20231005104133.51901-1-david.plowman@raspberrypi.com>\n\t<169701964499.2915094.17960336807330474858@ping.linuxembedded.co.uk>","In-Reply-To":"<169701964499.2915094.17960336807330474858@ping.linuxembedded.co.uk>","Date":"Wed, 11 Oct 2023 11:39:26 +0100","Message-ID":"<CAHW6GY+P6yVG-HSGaZ=qTGY8vwi6oiAqsEQqq3yYjfjAh_cmGQ@mail.gmail.com>","To":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","From":"David Plowman via libcamera-devel <libcamera-devel@lists.libcamera.org>","Reply-To":"David Plowman <david.plowman@raspberrypi.com>","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":27991,"web_url":"https://patchwork.libcamera.org/comment/27991/","msgid":"<20231018235427.GV1512@pendragon.ideasonboard.com>","date":"2023-10-18T23:54:27","subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hello David,\n\nChiming in at last, sorry for the delay.\n\nOn Wed, Oct 11, 2023 at 11:39:26AM +0100, David Plowman via libcamera-devel wrote:\n> On Wed, 11 Oct 2023 at 11:20, Kieran Bingham wrote:\n> > Quoting David Plowman via libcamera-devel (2023-10-05 11:41:33)\n> > > We add an HdrMode control (to enable and disable HDR processing)\n> > > and an HdrChannel, which indicates what kind of HDR frame (short, long\n> > > or otherwise) has just arrived.\n> > >\n> > > Currently the HdrMode supports the following values:\n> > >\n> > > * Off - no HDR processing at all.\n> > > * MultiExposureUnmerged - frames at multiple different exposures are\n> > >   produced, but not merged together. They are returned \"as is\".\n> > > * MultiExposure - frames at multiple different exposures are merged\n> > >   to create HDR images.\n> > > * SingleExposure - multiple frames all at the same exposure are\n> > >   merged to create HDR images.\n> > > * Night - multiple frames will be combined to create \"night mode\"\n> > >   images.\n> > >\n> > > Signed-off-by: David Plowman <david.plowman@raspberrypi.com>\n> > > Reviewed-by: Naushir Patuck <naush@raspberrypi.com>\n> > > ---\n> > >  src/libcamera/control_ids.yaml | 63 ++++++++++++++++++++++++++++++++++\n> > >  1 file changed, 63 insertions(+)\n> > >\n> > > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > > index f2e542f4..e3699e74 100644\n> > > --- a/src/libcamera/control_ids.yaml\n> > > +++ b/src/libcamera/control_ids.yaml\n> > > @@ -774,6 +774,69 @@ controls:\n> > >              Continuous AF is paused. No further state changes or lens movements\n> > >              will occur until the AfPauseResume control is sent.\n> > >\n> > > +  - HdrMode:\n> > > +      type: int32_t\n> > > +      description: |\n> > > +        Control to set the mode to be used for High Dynamic Range (HDR)\n> > > +        imaging.\n> > > +\n> > > +      enum:\n> > > +        - name: HdrModeOff\n> > > +          value: 0\n> > > +          description: |\n> > > +            HDR is not enabled.\n\nSounds quite non controversial :-) Maybe s/not enabled/disabled/\n\n> > > +        - name: HdrModeMultiExposureUnmerged\n> > > +          value: 1\n> > > +          description: |\n> > > +            Multiple exposures will be generated in an alternating fashion.\n> > > +            However, they will not be merged together and will be returned to\n> > > +            the application as they are. The expectation is that, if necessary,\n> > > +            the application can merge them to create HDR images for itself.\n> > > +        - name: HdrModeMultiExposure\n> > > +          value: 2\n> > > +          description: |\n> > > +            Multiple exposures will be generated and merged to create HDR\n> > > +            images.\n> > > +        - name: HdrModeSingleExposure\n> > > +          value: 3\n> > > +          description: |\n> > > +            Multiple frames all at a single exposure will be used to create HDR\n> > > +            images.\n\nIf a platform supports both the multi-exposure and single-exposure\nmodes, how will an application decide which one to use, what are the\npros and cons ?\n\n> > > +        - name: HdrModeNight\n> > > +          value: 4\n> > > +          description: |\n> > > +            Multiple frames will be combined to produce \"night mode\" images.\n> >\n> > MdrModeMultiExposureUnmerged has been documented to describe what that\n> > is doing. HdrModeMultiExposure, and HdrModeSingleExposure are more self\n> > explanatory\n\nIt feels a bit like we're mixing different concepts in the same control,\nbut I'm OK with it for now.\n\nWhat does bother me though is that there's no definition of HDR\nanywhere. I think that needs to be fixed, in order to clearly set the\nboundaries of what falls in the category of HDR-related controls, and\nwhat is entirely separate.\n\n> > - but HdrModeNight might need more explaining.\n> >\n> > Does this just mean sequential images are stacked? Are there different\n> > exposures? Are there any other expectations that a user may need or\n> > assumptions they could get wrong?\n> >\n> > Or is it just 'Hey do anything you like to make pictures look better at\n> > night time or in low light conditions'?.\n> \n> It means this last thing. I don't feel I can judge what any other\n> vendor will do HDR-wise, so these modes are the most generic \"hooks\" I\n> can think of that stand a fair chance of vendors at large being able\n> to hang their HDR implementations off them.\n> \n> For night mode in particular - it's \"do whatever it is that you do\".\n> Though there's an implicit \"probably do it with some kind of frame\n> merging or HDR-like procedure\", because I think a number of vendors do\n> indeed do this. But there's no knowing for sure. \"Night modes\" might\n> be completely different things too,\n\nIf it's not HDR, then does it really belong to this control ?\n\n> like we have this magic HDR mode\n> in the imx708 sensor which just doesn't fit in here.\n\nWhat is that magic HDR mode ?\n\n> > > +\n> > > +  - HdrChannel:\n> > > +      type: int32_t\n> > > +      description: |\n> > > +        This value is reported back to the application so that it can discover\n> > > +        whether this capture corresponds to the short or long exposure image (or\n> > > +        any other image used by the HDR procedure).\n\nYou mention short and long here, but there's also a medium value below.\nIf the implementation uses two exposures only, which ones will it be ?\nThis should be documented. Also, in the single exposure case, will the\nchannel be set to HdrChannelNone ? This should also be documented.\nSimilarly, in the HdrModeMultiExposure mode, what will be reported here,\ngiven that all images will be the result of merging multiple exposures ?\n\n> > > +\n> > > +      enum:\n> > > +        - name: HdrChannelNone\n> > > +          value: 0\n> > > +          description: |\n> > > +            This image does not correspond to any of the captures used to create\n> > > +            an HDR image.\n> > > +        - name: HdrChannelShort\n> > > +          value: 1\n> > > +          description: |\n> > > +            This is a short exposure image.\n> > > +        - name: HdrChannelMedium\n> > > +          value: 2\n> > > +          description: |\n> > > +            This is a medium exposure image.\n> > > +        - name: HdrChannelLong\n> > > +          value: 3\n> > > +          description: |\n> > > +            This is a long exposure image.\n> > > +        - name: HdrChannelNight\n> > > +          value: 4\n> > > +          description: |\n> > > +            This frame has been used to produce a \"night mode\" image.\n\nThis reminds me of the three-states boolean:\nhttps://thedailywtf.com/articles/What_Is_Truth_0x3f_.\n\n> > What does it mean if a 'single' frame has been used to produce a night\n> > mode image. Does that mean it's blended with previous frames? Or is this\n> > frame only 'one' of a set?\n> \n> It's hard to say what this really means, as I think different vendors\n> will do all kinds of different things. Unfortunately I suspect the HDR\n> or \"night mode\" is likely to be very vendor specific, and we may find\n> they work quite differently.\n> \n> In our case, all it really means is that the exposure for the frame\n> was calculated by AEC/AGC running with its \"night mode\" settings. It\n> will have been combined with previous \"night mode\" frames (if any).\n\nIf I understand you correctly, this is single exposure HDR, but with AGC\nrunning in night mode. Do you then need this value ? It seems to me that\nyou could report HdrChannelNone here, and applications would still know\nthat night mode is being used from the value of HdrMode.\n\n> > \"Frame has been used to produce\" makes it sound like the result of using\n> > this frame will be output later and maybe this frame should be discarded\n> > because it's not the \"final\" night mode image ?\n> \n> To some extent this is me using vague language because I can't say for\n> sure what anyone else will do. Actually it does so happen that in our\n> \"night mode\" you will need to discard a few frames to give the image\n> merging time to work (you could take the very first image - but it\n> will be noisy). But other vendors may not do it like this. Or they\n> may. But probably not. Who knows!\n> \n> But I'm definitely open to rewording any of this if we can agree on\n> the level of vaguess/specificness that's required...\n\nI want controls to be defined in an unambiguous way, for the benefit of\napplication developers who will then know what they get, and IPA\ndevelopers who will know what to implement. There's some leeway in the\nsense that implementations can be given some latitude, but that latitude\nneeds to be documented with clear boundaries.\n\nAs guessing what other platforms would do, I would rather be more\nprecise here and document the use case(s) you want to target. When we\nwill implement support for other platforms, we'll revisit it.\n\n> > > +\n> > >    # ----------------------------------------------------------------------------\n> > >    # Draft controls section\n> > >","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 552C2C3272\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed, 18 Oct 2023 23:54:24 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8A2AA6297F;\n\tThu, 19 Oct 2023 01:54:23 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4A11A6295F\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 19 Oct 2023 01:54:21 +0200 (CEST)","from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi\n\t[213.243.189.158])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 650EFB2A;\n\tThu, 19 Oct 2023 01:54:13 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1697673263;\n\tbh=5tHvDFt4dudUp+vZbBTHmXQVHEuOXtTO7BFKXj0gQJg=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=pBrHT6GWukIwDtcuaRiXSkAV84A9EcWr7u/cL3h+eTfh1XPME7Qz0EGH0xhDmrQ/p\n\ttwIblR1zaI+xViGqKb7vtse/fLnH4VllTgNeD8Xs9cyYOgSIHFAm9Nlg300uW584hI\n\tI5xK74relGTYn3b3s55iHtFO6sIJIEIt6+WJPTIC7Ta1sarkljDQd3P0kWvvF079g4\n\t4+v0htIGev19zjefVLBYdlHoKAOasr1h+NblfN0YLPwR55R8O0tTbgcT0AWRWaip05\n\tWQKnWF7Q6vOiiy1gbm3vwYtCcwz13XzIBBZZesI0104o0nPDjT3ZxoD4v24xNoigvA\n\t8B8q3UGgEOM4Q==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1697673253;\n\tbh=5tHvDFt4dudUp+vZbBTHmXQVHEuOXtTO7BFKXj0gQJg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=dRmOdx8zInPdFYO2t5qcSwQY+0XVmwoS3wF0DdGdo3amhvoY1eEg7YtQDjq7h/ACm\n\t8ACoIkpy0CTDjw2LUoeYNCe7bLnDbJM+ASuiMGk4pNRRDbmzQCvJP8asFX52p19+Ij\n\tZwBTzD7ca03HDNa6/EKgbs/j5/8mngDcrK4e96tM="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"dRmOdx8z\"; dkim-atps=neutral","Date":"Thu, 19 Oct 2023 02:54:27 +0300","To":"David Plowman <david.plowman@raspberrypi.com>","Message-ID":"<20231018235427.GV1512@pendragon.ideasonboard.com>","References":"<20231005104133.51901-1-david.plowman@raspberrypi.com>\n\t<169701964499.2915094.17960336807330474858@ping.linuxembedded.co.uk>\n\t<CAHW6GY+P6yVG-HSGaZ=qTGY8vwi6oiAqsEQqq3yYjfjAh_cmGQ@mail.gmail.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<CAHW6GY+P6yVG-HSGaZ=qTGY8vwi6oiAqsEQqq3yYjfjAh_cmGQ@mail.gmail.com>","Subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","From":"Laurent Pinchart via libcamera-devel\n\t<libcamera-devel@lists.libcamera.org>","Reply-To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":27995,"web_url":"https://patchwork.libcamera.org/comment/27995/","msgid":"<CAHW6GY+u6y-7_PfNp3jhbRZiNoMd-t0KB7Q0yjGS86yKDAMWbw@mail.gmail.com>","date":"2023-10-19T08:34:48","subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Hi Laurent\n\nThanks for the comments!\n\nOn Thu, 19 Oct 2023 at 00:54, Laurent Pinchart\n<laurent.pinchart@ideasonboard.com> wrote:\n>\n> Hello David,\n>\n> Chiming in at last, sorry for the delay.\n>\n> On Wed, Oct 11, 2023 at 11:39:26AM +0100, David Plowman via libcamera-devel wrote:\n> > On Wed, 11 Oct 2023 at 11:20, Kieran Bingham wrote:\n> > > Quoting David Plowman via libcamera-devel (2023-10-05 11:41:33)\n> > > > We add an HdrMode control (to enable and disable HDR processing)\n> > > > and an HdrChannel, which indicates what kind of HDR frame (short, long\n> > > > or otherwise) has just arrived.\n> > > >\n> > > > Currently the HdrMode supports the following values:\n> > > >\n> > > > * Off - no HDR processing at all.\n> > > > * MultiExposureUnmerged - frames at multiple different exposures are\n> > > >   produced, but not merged together. They are returned \"as is\".\n> > > > * MultiExposure - frames at multiple different exposures are merged\n> > > >   to create HDR images.\n> > > > * SingleExposure - multiple frames all at the same exposure are\n> > > >   merged to create HDR images.\n> > > > * Night - multiple frames will be combined to create \"night mode\"\n> > > >   images.\n> > > >\n> > > > Signed-off-by: David Plowman <david.plowman@raspberrypi.com>\n> > > > Reviewed-by: Naushir Patuck <naush@raspberrypi.com>\n> > > > ---\n> > > >  src/libcamera/control_ids.yaml | 63 ++++++++++++++++++++++++++++++++++\n> > > >  1 file changed, 63 insertions(+)\n> > > >\n> > > > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > > > index f2e542f4..e3699e74 100644\n> > > > --- a/src/libcamera/control_ids.yaml\n> > > > +++ b/src/libcamera/control_ids.yaml\n> > > > @@ -774,6 +774,69 @@ controls:\n> > > >              Continuous AF is paused. No further state changes or lens movements\n> > > >              will occur until the AfPauseResume control is sent.\n> > > >\n> > > > +  - HdrMode:\n> > > > +      type: int32_t\n> > > > +      description: |\n> > > > +        Control to set the mode to be used for High Dynamic Range (HDR)\n> > > > +        imaging.\n> > > > +\n> > > > +      enum:\n> > > > +        - name: HdrModeOff\n> > > > +          value: 0\n> > > > +          description: |\n> > > > +            HDR is not enabled.\n>\n> Sounds quite non controversial :-) Maybe s/not enabled/disabled/\n\nSure, can change that.\n\n>\n> > > > +        - name: HdrModeMultiExposureUnmerged\n> > > > +          value: 1\n> > > > +          description: |\n> > > > +            Multiple exposures will be generated in an alternating fashion.\n> > > > +            However, they will not be merged together and will be returned to\n> > > > +            the application as they are. The expectation is that, if necessary,\n> > > > +            the application can merge them to create HDR images for itself.\n> > > > +        - name: HdrModeMultiExposure\n> > > > +          value: 2\n> > > > +          description: |\n> > > > +            Multiple exposures will be generated and merged to create HDR\n> > > > +            images.\n> > > > +        - name: HdrModeSingleExposure\n> > > > +          value: 3\n> > > > +          description: |\n> > > > +            Multiple frames all at a single exposure will be used to create HDR\n> > > > +            images.\n>\n> If a platform supports both the multi-exposure and single-exposure\n> modes, how will an application decide which one to use, what are the\n> pros and cons ?\n\nI think it's a case of \"read the platform documentation\". A Pi 5 will\ndo both, but we very strongly recommend use of single-exposure mode.\n\n>\n> > > > +        - name: HdrModeNight\n> > > > +          value: 4\n> > > > +          description: |\n> > > > +            Multiple frames will be combined to produce \"night mode\" images.\n> > >\n> > > MdrModeMultiExposureUnmerged has been documented to describe what that\n> > > is doing. HdrModeMultiExposure, and HdrModeSingleExposure are more self\n> > > explanatory\n>\n> It feels a bit like we're mixing different concepts in the same control,\n> but I'm OK with it for now.\n>\n> What does bother me though is that there's no definition of HDR\n> anywhere. I think that needs to be fixed, in order to clearly set the\n> boundaries of what falls in the category of HDR-related controls, and\n> what is entirely separate.\n\nI think that's a fair point, though it's quite hard to describe.\n\nFor us, it's a process of producing multiple frames at either a single\nor multiple different exposures, and then optionally processing them\nusing methods that may include merging/fusing frames, and various\ntypes of tonemapping.\n\nBut basically yeah, it's a real mixed bag of imaging techniques. We\nwant to bring to bear any functionality or tricks that we have in our\npipeline to accomplish certain outcomes, and we can probably expect\nother vendors to take a similar view.\n\n>\n> > > - but HdrModeNight might need more explaining.\n> > >\n> > > Does this just mean sequential images are stacked? Are there different\n> > > exposures? Are there any other expectations that a user may need or\n> > > assumptions they could get wrong?\n> > >\n> > > Or is it just 'Hey do anything you like to make pictures look better at\n> > > night time or in low light conditions'?.\n> >\n> > It means this last thing. I don't feel I can judge what any other\n> > vendor will do HDR-wise, so these modes are the most generic \"hooks\" I\n> > can think of that stand a fair chance of vendors at large being able\n> > to hang their HDR implementations off them.\n> >\n> > For night mode in particular - it's \"do whatever it is that you do\".\n> > Though there's an implicit \"probably do it with some kind of frame\n> > merging or HDR-like procedure\", because I think a number of vendors do\n> > indeed do this. But there's no knowing for sure. \"Night modes\" might\n> > be completely different things too,\n>\n> If it's not HDR, then does it really belong to this control ?\n\nOur implementation of \"night mode\" is a form of HDR. But I can't speak\nfor other vendors. If their \"night mode\" isn't a form of HDR, then I'd\nexpect them not to implement this here. Short of tuning the whole \"HDR\nMode\" thing into a \"Special Vendor Imaging Mode\", though I'm not sure\nif we really want to go there just yet!\n\nGenerally I think a big problem in this area is that we have no clue\nwhat any other vendor will want to do, so it's hard to know how\nspecific or precise we can be. I appreciate this is a general\nlibcamera problem, and as soon as we get beyond the trivial \"set the\nexposure to X\" it becomes quite hard to know how to deal with any of\nit.\n\n>\n> > like we have this magic HDR mode\n> > in the imx708 sensor which just doesn't fit in here.\n>\n> What is that magic HDR mode ?\n\nThis is the in-sensor HDR mode, i.e. everything happens in the sensor.\nMultiple exposures are merged and tonemapped, and turned back into raw\nBayer to be sent to the Pi. This is where we need V4L2 to indicate\nspecial capabilities of its camera modes, but that's a whole other can\nof worms.\n\n>\n> > > > +\n> > > > +  - HdrChannel:\n> > > > +      type: int32_t\n> > > > +      description: |\n> > > > +        This value is reported back to the application so that it can discover\n> > > > +        whether this capture corresponds to the short or long exposure image (or\n> > > > +        any other image used by the HDR procedure).\n>\n> You mention short and long here, but there's also a medium value below.\n> If the implementation uses two exposures only, which ones will it be ?\n> This should be documented. Also, in the single exposure case, will the\n> channel be set to HdrChannelNone ? This should also be documented.\n> Similarly, in the HdrModeMultiExposure mode, what will be reported here,\n> given that all images will be the result of merging multiple exposures ?\n\nI expect other HDR implementations would be quite likely to use a\n\"medium channel\". So I don't mind dropping it... though I think at\nsome point someone would probably want to add it back. Maybe even us\none day! I can see arguments for \"very short\" and \"very long\" too...\n\nIn our implementation the HdrChannel is helpful because you can switch\nseemlessly between the \"off\", \"single-exposure\" and \"night\" modes. It\nobviously takes a few frames for each change to happen, but the\nHdrChannel tells you exactly when you get a frame in the mode you've\nrequested. I would certainly want to preserve this.\n\nActually we aren't reporting the HdrMode currently, though I think we\nclearly should. But knowing the long/short channel is still useful in\nthe \"multi-exposure\" mode. In some respects it's just that \"this is\nwhat the code does\", but It's certainly useful to see the \"long\"\nchannel image come through. When you see that, then you know the\nmerging has happened. Even knowing the HdrMode isn't enough - because\nthe first frame in the \"multi-exposure\" mode might not be the result\nof merging two frames (it might be made entirely from the short\nframe). There is some degree of overlap with the \"IQ stability\"\ncontrol here, so these questions need to be addressed really quite\nurgently.\n\n>\n> > > > +\n> > > > +      enum:\n> > > > +        - name: HdrChannelNone\n> > > > +          value: 0\n> > > > +          description: |\n> > > > +            This image does not correspond to any of the captures used to create\n> > > > +            an HDR image.\n> > > > +        - name: HdrChannelShort\n> > > > +          value: 1\n> > > > +          description: |\n> > > > +            This is a short exposure image.\n> > > > +        - name: HdrChannelMedium\n> > > > +          value: 2\n> > > > +          description: |\n> > > > +            This is a medium exposure image.\n> > > > +        - name: HdrChannelLong\n> > > > +          value: 3\n> > > > +          description: |\n> > > > +            This is a long exposure image.\n> > > > +        - name: HdrChannelNight\n> > > > +          value: 4\n> > > > +          description: |\n> > > > +            This frame has been used to produce a \"night mode\" image.\n>\n> This reminds me of the three-states boolean:\n> https://thedailywtf.com/articles/What_Is_Truth_0x3f_.\n>\n> > > What does it mean if a 'single' frame has been used to produce a night\n> > > mode image. Does that mean it's blended with previous frames? Or is this\n> > > frame only 'one' of a set?\n> >\n> > It's hard to say what this really means, as I think different vendors\n> > will do all kinds of different things. Unfortunately I suspect the HDR\n> > or \"night mode\" is likely to be very vendor specific, and we may find\n> > they work quite differently.\n> >\n> > In our case, all it really means is that the exposure for the frame\n> > was calculated by AEC/AGC running with its \"night mode\" settings. It\n> > will have been combined with previous \"night mode\" frames (if any).\n>\n> If I understand you correctly, this is single exposure HDR, but with AGC\n> running in night mode. Do you then need this value ? It seems to me that\n> you could report HdrChannelNone here, and applications would still know\n> that night mode is being used from the value of HdrMode.\n\nYes, I'm not sure. We aren't currently reporting back the value of\nHdrMode, though as I've said we need to do that. With that, reporting\nthe night mode channel as being the \"short\" channel feels quite\nnatural to me, because that's what it is!\n\n>\n> > > \"Frame has been used to produce\" makes it sound like the result of using\n> > > this frame will be output later and maybe this frame should be discarded\n> > > because it's not the \"final\" night mode image ?\n> >\n> > To some extent this is me using vague language because I can't say for\n> > sure what anyone else will do. Actually it does so happen that in our\n> > \"night mode\" you will need to discard a few frames to give the image\n> > merging time to work (you could take the very first image - but it\n> > will be noisy). But other vendors may not do it like this. Or they\n> > may. But probably not. Who knows!\n> >\n> > But I'm definitely open to rewording any of this if we can agree on\n> > the level of vaguess/specificness that's required...\n>\n> I want controls to be defined in an unambiguous way, for the benefit of\n> application developers who will then know what they get, and IPA\n> developers who will know what to implement. There's some leeway in the\n> sense that implementations can be given some latitude, but that latitude\n> needs to be documented with clear boundaries.\n>\n> As guessing what other platforms would do, I would rather be more\n> precise here and document the use case(s) you want to target. When we\n> will implement support for other platforms, we'll revisit it.\n\nIn principle I'm happy to document \"what the Pi does\". Though I worry\nthat it is unlikely that other platforms would want to do the same\nthings and use the same mechanisms. But maybe we wait just for them to\ncatch us up... (I'm guessing this might take a while) ?\n\nThanks!\nDavid\n\n>\n> > > > +\n> > > >    # ----------------------------------------------------------------------------\n> > > >    # Draft controls section\n> > > >\n>\n> --\n> Regards,\n>\n> Laurent Pinchart","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 27044BDCBD\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu, 19 Oct 2023 08:35:03 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6B20B6297F;\n\tThu, 19 Oct 2023 10:35:02 +0200 (CEST)","from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com\n\t[IPv6:2607:f8b0:4864:20::f31])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 28B0161DD0\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 19 Oct 2023 10:35:01 +0200 (CEST)","by mail-qv1-xf31.google.com with SMTP id\n\t6a1803df08f44-66d0252578aso42229066d6.0\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 19 Oct 2023 01:35:01 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1697704502;\n\tbh=qIjC1/nkx/ZI5yPzVCz7RTGsSy5tiCmiBvKxKJsX2iI=;\n\th=References:In-Reply-To:Date:To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=ICNuzoHpZQl1gTzTqLYaLgc2HR1nFtSsvD/AkiX1Ws7ykTi6joZZTZrvz8YnLXFd2\n\tFzNYbWiFrD/2hU93GkV/+68xhyGTYqCYq7cKVn32f5IghwBG2YJWr1TpEBJ+XvlW7n\n\txC1Dd8KIa+UCKSCtX9g1iSvqPpsQtu/ogU0a92yQvWFIVPtGWgyHs18PRiNhEMsScz\n\tyF3cgPYtUZLUgj25YSzwnYI0tayhfDAt9QhghUOJbjHp2bJ83znFJ2oSWWpDixuQef\n\tXZ0QbujmvqZGduduRRKwad4GPwNX1zoSG6w+Zc4f4sgPyvYzXXeW2Go2lBVbX6NbFK\n\tUevX9gZyYIvMw==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1697704499; x=1698309299;\n\tdarn=lists.libcamera.org; \n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to;\n\tbh=YbpKxpf14OeCrUZ/Xu1u8p6ocjEA+NO6251jwMxS0xs=;\n\tb=gtUps5nBPEwvcfJKtnIEMCEW2I+QFjZSTT0mO/Var+1JUqwazNo0k3UESUEVyvSIPi\n\tQpyi+u1dFlP3CK4jphuMf98B+NERt9fdNl7uAAShGKurTLSPbminqDJiIlr0Kb+MhGSN\n\t2Nm27VUpUU7T4TXJf4ifQbQRmTfg9ScahL48DIXPPfp+879dQjr1n+tLPStQa8hEa/qV\n\tgBSgfa8hI+7d7OWUJeHMGUnh30PFbTxpsE+HWzreJcwWiZ0uKir/mPKHxHN+JURL6HNV\n\twpaFP6RM+8G9FzJYh1O490KG1vItXhcu0b+TBZlrlxy3oRNzdX9j0aTzZsy90PgI6QKr\n\tOcgg=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"gtUps5nB\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1697704499; x=1698309299;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=YbpKxpf14OeCrUZ/Xu1u8p6ocjEA+NO6251jwMxS0xs=;\n\tb=uK7oyD3L2d3TGOEWlHESA0lWX/l/wYkcY07cEut+Qbd2xamPj9vEb1kF9HBMb4BHak\n\tm/3ajReAxm1OBncSbp7xzwfyxt6FX3BvCeSE3YhpWtrNMVs4YaSZeQwy0oagYJ0WdP8A\n\tzBFsSbEXp6AcgwRO8peytPid4O2ZfzC3s4bqYWBeTxK6mCNAWpaFGDAkKNZu/YZqfBUu\n\t9xF+d1y2a/Ej5U8BAc2eMW/mpAD6b2yZLlrweX/J5vK54rI8PivYmqTbeqeC1LyNhxOU\n\tSRAG07jxjsz1FOYdh7jksShg6KtycAdtA/k4+qZyKZSZKZwIVHQCimYmMPJCfHF5Uto8\n\tHTRg==","X-Gm-Message-State":"AOJu0YwSyzgK8xUgrFXnz4C6sPu5Cyl6x8xjK04wfnEpyH6G/fwFfvdZ\n\tsEEeLWQKx7xuV/ZCynJ5LRzMellE6z4j2a7F6rl0Y7FKQ0oEGAfG","X-Google-Smtp-Source":"AGHT+IHc8sFIayfL0tRSY2o5Wm5vO5YXpgrcLet6NYrFKKvevjkF78XyA9ActXD83D7EcokCUQK+V4Kzt5KGIUiSmI0=","X-Received":"by 2002:a05:6214:2267:b0:66d:1eb6:96c9 with SMTP id\n\tgs7-20020a056214226700b0066d1eb696c9mr1639420qvb.60.1697704499493;\n\tThu, 19 Oct 2023 01:34:59 -0700 (PDT)","MIME-Version":"1.0","References":"<20231005104133.51901-1-david.plowman@raspberrypi.com>\n\t<169701964499.2915094.17960336807330474858@ping.linuxembedded.co.uk>\n\t<CAHW6GY+P6yVG-HSGaZ=qTGY8vwi6oiAqsEQqq3yYjfjAh_cmGQ@mail.gmail.com>\n\t<20231018235427.GV1512@pendragon.ideasonboard.com>","In-Reply-To":"<20231018235427.GV1512@pendragon.ideasonboard.com>","Date":"Thu, 19 Oct 2023 09:34:48 +0100","Message-ID":"<CAHW6GY+u6y-7_PfNp3jhbRZiNoMd-t0KB7Q0yjGS86yKDAMWbw@mail.gmail.com>","To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","From":"David Plowman via libcamera-devel <libcamera-devel@lists.libcamera.org>","Reply-To":"David Plowman <david.plowman@raspberrypi.com>","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":28008,"web_url":"https://patchwork.libcamera.org/comment/28008/","msgid":"<20231019203707.GN14832@pendragon.ideasonboard.com>","date":"2023-10-19T20:37:07","subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hello David,\n\nOn Thu, Oct 19, 2023 at 09:34:48AM +0100, David Plowman wrote:\n> On Thu, 19 Oct 2023 at 00:54, Laurent Pinchart wrote:\n> > > > Quoting David Plowman via libcamera-devel (2023-10-05 11:41:33)\n> > > > > We add an HdrMode control (to enable and disable HDR processing)\n> > > > > and an HdrChannel, which indicates what kind of HDR frame (short, long\n> > > > > or otherwise) has just arrived.\n> > > > >\n> > > > > Currently the HdrMode supports the following values:\n> > > > >\n> > > > > * Off - no HDR processing at all.\n> > > > > * MultiExposureUnmerged - frames at multiple different exposures are\n> > > > >   produced, but not merged together. They are returned \"as is\".\n> > > > > * MultiExposure - frames at multiple different exposures are merged\n> > > > >   to create HDR images.\n> > > > > * SingleExposure - multiple frames all at the same exposure are\n> > > > >   merged to create HDR images.\n> > > > > * Night - multiple frames will be combined to create \"night mode\"\n> > > > >   images.\n> > > > >\n> > > > > Signed-off-by: David Plowman <david.plowman@raspberrypi.com>\n> > > > > Reviewed-by: Naushir Patuck <naush@raspberrypi.com>\n> > > > > ---\n> > > > >  src/libcamera/control_ids.yaml | 63 ++++++++++++++++++++++++++++++++++\n> > > > >  1 file changed, 63 insertions(+)\n> > > > >\n> > > > > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > > > > index f2e542f4..e3699e74 100644\n> > > > > --- a/src/libcamera/control_ids.yaml\n> > > > > +++ b/src/libcamera/control_ids.yaml\n> > > > > @@ -774,6 +774,69 @@ controls:\n> > > > >              Continuous AF is paused. No further state changes or lens movements\n> > > > >              will occur until the AfPauseResume control is sent.\n> > > > >\n> > > > > +  - HdrMode:\n> > > > > +      type: int32_t\n> > > > > +      description: |\n> > > > > +        Control to set the mode to be used for High Dynamic Range (HDR)\n> > > > > +        imaging.\n> > > > > +\n> > > > > +      enum:\n> > > > > +        - name: HdrModeOff\n> > > > > +          value: 0\n> > > > > +          description: |\n> > > > > +            HDR is not enabled.\n> >\n> > Sounds quite non controversial :-) Maybe s/not enabled/disabled/\n> \n> Sure, can change that.\n> \n> > > > > +        - name: HdrModeMultiExposureUnmerged\n> > > > > +          value: 1\n> > > > > +          description: |\n> > > > > +            Multiple exposures will be generated in an alternating fashion.\n> > > > > +            However, they will not be merged together and will be returned to\n> > > > > +            the application as they are. The expectation is that, if necessary,\n> > > > > +            the application can merge them to create HDR images for itself.\n> > > > > +        - name: HdrModeMultiExposure\n> > > > > +          value: 2\n> > > > > +          description: |\n> > > > > +            Multiple exposures will be generated and merged to create HDR\n> > > > > +            images.\n> > > > > +        - name: HdrModeSingleExposure\n> > > > > +          value: 3\n> > > > > +          description: |\n> > > > > +            Multiple frames all at a single exposure will be used to create HDR\n> > > > > +            images.\n> >\n> > If a platform supports both the multi-exposure and single-exposure\n> > modes, how will an application decide which one to use, what are the\n> > pros and cons ?\n> \n> I think it's a case of \"read the platform documentation\". A Pi 5 will\n> do both, but we very strongly recommend use of single-exposure mode.\n\nReading the platform documentation is what I would really like\ndevelopers *not* to do. It precludes writing portable applications, when\nlibcamera's goal is to abstract the platform as much as possible. This\nbeing said, we all know that we can't make all platforms look exactly\nthe same (unless we aim for the lowest common denominator of not\ncovering any real use case at all), so this is more of a design\nprinciple than a goal we want to fully achieve.\n\nAs discussed separately, I think we're trying to design a generic HDR\nAPI based on a single platform, and without having a real idea of how\napplications will want to use it. That's twice a recipe for certain\nfailure, and that's good news: we don't need to pretend we think we can\nget it right :-) Let's proceed with what we have, see how users like or\ndislike it (I'm sure you'll witness that on your forums), how future\nplatforms will compare, and we'll then fix the design mistakes.\n\n> > > > > +        - name: HdrModeNight\n> > > > > +          value: 4\n> > > > > +          description: |\n> > > > > +            Multiple frames will be combined to produce \"night mode\" images.\n> > > >\n> > > > MdrModeMultiExposureUnmerged has been documented to describe what that\n> > > > is doing. HdrModeMultiExposure, and HdrModeSingleExposure are more self\n> > > > explanatory\n> >\n> > It feels a bit like we're mixing different concepts in the same control,\n> > but I'm OK with it for now.\n> >\n> > What does bother me though is that there's no definition of HDR\n> > anywhere. I think that needs to be fixed, in order to clearly set the\n> > boundaries of what falls in the category of HDR-related controls, and\n> > what is entirely separate.\n> \n> I think that's a fair point, though it's quite hard to describe.\n> \n> For us, it's a process of producing multiple frames at either a single\n> or multiple different exposures, and then optionally processing them\n> using methods that may include merging/fusing frames, and various\n> types of tonemapping.\n> \n> But basically yeah, it's a real mixed bag of imaging techniques. We\n> want to bring to bear any functionality or tricks that we have in our\n> pipeline to accomplish certain outcomes, and we can probably expect\n> other vendors to take a similar view.\n> \n> > > > - but HdrModeNight might need more explaining.\n> > > >\n> > > > Does this just mean sequential images are stacked? Are there different\n> > > > exposures? Are there any other expectations that a user may need or\n> > > > assumptions they could get wrong?\n> > > >\n> > > > Or is it just 'Hey do anything you like to make pictures look better at\n> > > > night time or in low light conditions'?.\n> > >\n> > > It means this last thing. I don't feel I can judge what any other\n> > > vendor will do HDR-wise, so these modes are the most generic \"hooks\" I\n> > > can think of that stand a fair chance of vendors at large being able\n> > > to hang their HDR implementations off them.\n> > >\n> > > For night mode in particular - it's \"do whatever it is that you do\".\n> > > Though there's an implicit \"probably do it with some kind of frame\n> > > merging or HDR-like procedure\", because I think a number of vendors do\n> > > indeed do this. But there's no knowing for sure. \"Night modes\" might\n> > > be completely different things too,\n> >\n> > If it's not HDR, then does it really belong to this control ?\n> \n> Our implementation of \"night mode\" is a form of HDR. But I can't speak\n> for other vendors. If their \"night mode\" isn't a form of HDR, then I'd\n> expect them not to implement this here. Short of tuning the whole \"HDR\n> Mode\" thing into a \"Special Vendor Imaging Mode\", though I'm not sure\n> if we really want to go there just yet!\n> \n> Generally I think a big problem in this area is that we have no clue\n> what any other vendor will want to do, so it's hard to know how\n> specific or precise we can be. I appreciate this is a general\n> libcamera problem, and as soon as we get beyond the trivial \"set the\n> exposure to X\" it becomes quite hard to know how to deal with any of\n> it.\n> \n> > > like we have this magic HDR mode\n> > > in the imx708 sensor which just doesn't fit in here.\n> >\n> > What is that magic HDR mode ?\n> \n> This is the in-sensor HDR mode, i.e. everything happens in the sensor.\n> Multiple exposures are merged and tonemapped, and turned back into raw\n> Bayer to be sent to the Pi. This is where we need V4L2 to indicate\n> special capabilities of its camera modes, but that's a whole other can\n> of worms.\n> \n> > > > > +\n> > > > > +  - HdrChannel:\n> > > > > +      type: int32_t\n> > > > > +      description: |\n> > > > > +        This value is reported back to the application so that it can discover\n> > > > > +        whether this capture corresponds to the short or long exposure image (or\n> > > > > +        any other image used by the HDR procedure).\n> >\n> > You mention short and long here, but there's also a medium value below.\n> > If the implementation uses two exposures only, which ones will it be ?\n> > This should be documented. Also, in the single exposure case, will the\n> > channel be set to HdrChannelNone ? This should also be documented.\n> > Similarly, in the HdrModeMultiExposure mode, what will be reported here,\n> > given that all images will be the result of merging multiple exposures ?\n> \n> I expect other HDR implementations would be quite likely to use a\n> \"medium channel\". So I don't mind dropping it... though I think at\n> some point someone would probably want to add it back. Maybe even us\n> one day! I can see arguments for \"very short\" and \"very long\" too...\n> \n> In our implementation the HdrChannel is helpful because you can switch\n> seemlessly between the \"off\", \"single-exposure\" and \"night\" modes. It\n> obviously takes a few frames for each change to happen, but the\n> HdrChannel tells you exactly when you get a frame in the mode you've\n> requested. I would certainly want to preserve this.\n> \n> Actually we aren't reporting the HdrMode currently, though I think we\n> clearly should. But knowing the long/short channel is still useful in\n> the \"multi-exposure\" mode. In some respects it's just that \"this is\n> what the code does\", but It's certainly useful to see the \"long\"\n> channel image come through. When you see that, then you know the\n> merging has happened. Even knowing the HdrMode isn't enough - because\n> the first frame in the \"multi-exposure\" mode might not be the result\n> of merging two frames (it might be made entirely from the short\n> frame). There is some degree of overlap with the \"IQ stability\"\n> control here, so these questions need to be addressed really quite\n> urgently.\n> \n> > > > > +\n> > > > > +      enum:\n> > > > > +        - name: HdrChannelNone\n> > > > > +          value: 0\n> > > > > +          description: |\n> > > > > +            This image does not correspond to any of the captures used to create\n> > > > > +            an HDR image.\n> > > > > +        - name: HdrChannelShort\n> > > > > +          value: 1\n> > > > > +          description: |\n> > > > > +            This is a short exposure image.\n> > > > > +        - name: HdrChannelMedium\n> > > > > +          value: 2\n> > > > > +          description: |\n> > > > > +            This is a medium exposure image.\n> > > > > +        - name: HdrChannelLong\n> > > > > +          value: 3\n> > > > > +          description: |\n> > > > > +            This is a long exposure image.\n> > > > > +        - name: HdrChannelNight\n> > > > > +          value: 4\n> > > > > +          description: |\n> > > > > +            This frame has been used to produce a \"night mode\" image.\n> >\n> > This reminds me of the three-states boolean:\n> > https://thedailywtf.com/articles/What_Is_Truth_0x3f_.\n> >\n> > > > What does it mean if a 'single' frame has been used to produce a night\n> > > > mode image. Does that mean it's blended with previous frames? Or is this\n> > > > frame only 'one' of a set?\n> > >\n> > > It's hard to say what this really means, as I think different vendors\n> > > will do all kinds of different things. Unfortunately I suspect the HDR\n> > > or \"night mode\" is likely to be very vendor specific, and we may find\n> > > they work quite differently.\n> > >\n> > > In our case, all it really means is that the exposure for the frame\n> > > was calculated by AEC/AGC running with its \"night mode\" settings. It\n> > > will have been combined with previous \"night mode\" frames (if any).\n> >\n> > If I understand you correctly, this is single exposure HDR, but with AGC\n> > running in night mode. Do you then need this value ? It seems to me that\n> > you could report HdrChannelNone here, and applications would still know\n> > that night mode is being used from the value of HdrMode.\n> \n> Yes, I'm not sure. We aren't currently reporting back the value of\n> HdrMode, though as I've said we need to do that. With that, reporting\n> the night mode channel as being the \"short\" channel feels quite\n> natural to me, because that's what it is!\n\nIf a single channel is used, is it meaningful to report which channel\nthat is ?\n\n> > > > \"Frame has been used to produce\" makes it sound like the result of using\n> > > > this frame will be output later and maybe this frame should be discarded\n> > > > because it's not the \"final\" night mode image ?\n> > >\n> > > To some extent this is me using vague language because I can't say for\n> > > sure what anyone else will do. Actually it does so happen that in our\n> > > \"night mode\" you will need to discard a few frames to give the image\n> > > merging time to work (you could take the very first image - but it\n> > > will be noisy). But other vendors may not do it like this. Or they\n> > > may. But probably not. Who knows!\n> > >\n> > > But I'm definitely open to rewording any of this if we can agree on\n> > > the level of vaguess/specificness that's required...\n> >\n> > I want controls to be defined in an unambiguous way, for the benefit of\n> > application developers who will then know what they get, and IPA\n> > developers who will know what to implement. There's some leeway in the\n> > sense that implementations can be given some latitude, but that latitude\n> > needs to be documented with clear boundaries.\n> >\n> > As guessing what other platforms would do, I would rather be more\n> > precise here and document the use case(s) you want to target. When we\n> > will implement support for other platforms, we'll revisit it.\n> \n> In principle I'm happy to document \"what the Pi does\". Though I worry\n> that it is unlikely that other platforms would want to do the same\n> things and use the same mechanisms. But maybe we wait just for them to\n> catch us up... (I'm guessing this might take a while) ?\n\nOr to do something else entirely. We need more data points, whatever the\ndata will be, and we'll then revisit these controls.\n\n> > > > > +\n> > > > >    # ----------------------------------------------------------------------------\n> > > > >    # Draft controls section\n> > > > >","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5401DC3272\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu, 19 Oct 2023 20:37:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 602B36297F;\n\tThu, 19 Oct 2023 22:37:03 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2A6DD6055B\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 19 Oct 2023 22:37:01 +0200 (CEST)","from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi\n\t[213.243.189.158])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A655325A;\n\tThu, 19 Oct 2023 22:36:52 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1697747823;\n\tbh=BlGJdZJZ0ybuAr1CH9IVXE7R86go1VnDxVX25kaJLN8=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=zIg0dbZA3PtBxAWWrLixr1WLXu3XLfhhh+IV2qnlv6oV7T1AAaBr9rjfSDPS34k8Q\n\t3V9TuwLKRF1J+cklw4G/yanh3lCBtPKyuz5ciOGxDnjBhu3QQTngDYMjV1kV+v5v2w\n\tMWG0JPWfAc3Cu6VPkdSG0Kzc26BRH/vLE5/368GiVlasnRbKZRvVD4z8x6Z5YdiRkj\n\tJL9fuJ+msrPKouT5RrlOMWy3KHnRn79W106Tq7WGtIKAnV/b/C2rh+BR/hcyCZAjT2\n\tX97v4pGvcqK79hsI8kToYOg7I4aDh+DwRSZC4fKywGvqoYU+/nQdRivTqK6Ya9mSJA\n\t9weSmlP203k3Q==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1697747812;\n\tbh=BlGJdZJZ0ybuAr1CH9IVXE7R86go1VnDxVX25kaJLN8=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=eZ0x5Aaj/XluWpWPmUFSTMa4bi6T6TLEcS4TTmYvc5hZ7agmeZlzCgCvvS76hCztq\n\t1kSYpz1T+1vZP9ByhNJCzvGeIfiOJwZ7lT4PG2TlvJR+MEnTKA/ZGVbZl/l9th1LCr\n\tvT+N+xzISL4lG6bz3rzmM16FA6YvRDZUA+049rAo="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"eZ0x5Aaj\"; dkim-atps=neutral","Date":"Thu, 19 Oct 2023 23:37:07 +0300","To":"David Plowman <david.plowman@raspberrypi.com>","Message-ID":"<20231019203707.GN14832@pendragon.ideasonboard.com>","References":"<20231005104133.51901-1-david.plowman@raspberrypi.com>\n\t<169701964499.2915094.17960336807330474858@ping.linuxembedded.co.uk>\n\t<CAHW6GY+P6yVG-HSGaZ=qTGY8vwi6oiAqsEQqq3yYjfjAh_cmGQ@mail.gmail.com>\n\t<20231018235427.GV1512@pendragon.ideasonboard.com>\n\t<CAHW6GY+u6y-7_PfNp3jhbRZiNoMd-t0KB7Q0yjGS86yKDAMWbw@mail.gmail.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<CAHW6GY+u6y-7_PfNp3jhbRZiNoMd-t0KB7Q0yjGS86yKDAMWbw@mail.gmail.com>","Subject":"Re: [libcamera-devel] [PATCH v3] libcamera: controls: Add controls\n\tfor HDR","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","From":"Laurent Pinchart via libcamera-devel\n\t<libcamera-devel@lists.libcamera.org>","Reply-To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]