[{"id":19547,"web_url":"https://patchwork.libcamera.org/comment/19547/","msgid":"<CAO5uPHNapWZCjm+BzF0Mn+qq+aAeyzVhk1cio302jMv5Cvh0Ng@mail.gmail.com>","date":"2021-09-08T10:02:58","subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","submitter":{"id":63,"url":"https://patchwork.libcamera.org/api/people/63/","name":"Hirokazu Honda","email":"hiroh@chromium.org"},"content":"Hi Umang, thank you for the patch.\n\nOn Wed, Sep 8, 2021 at 6:50 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n>\n> After multi-planar support was introduced for jpeg encoding as well,\n> EncoderLibJpeg::encode() expects a vector of planes as the source of\n> framebuffer to be encoded. Currently, we are passing a contiguous buffer\n> which is treated as only one plane (instead of two, as thumbnail is NV12).\n>\n> Hence, split the thumbnail data into respective planes according to NV12.\n> This fixes a crash in encoding of thumbnails.\n>\n> Fixes: 894ca69f6043(\"android: jpeg: Support multi-planar buffers\")\n> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>\n> ---\n> Changes in v2:\n> - Fix basic split logic error.\n> - Add Jacopo's suggestion as a \\todo since it's a superior method to fix\n>   this.\n> ---\n>  src/android/jpeg/post_processor_jpeg.cpp | 18 ++++++++++++++----\n>  1 file changed, 14 insertions(+), 4 deletions(-)\n>\n> diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\n> index 68d74842..1bf507a4 100644\n> --- a/src/android/jpeg/post_processor_jpeg.cpp\n> +++ b/src/android/jpeg/post_processor_jpeg.cpp\n> @@ -66,13 +66,23 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n>         int ret = thumbnailEncoder_.configure(thCfg);\n>\n>         if (!rawThumbnail.empty() && !ret) {\n> +               thumbnail->resize(rawThumbnail.size());\n> +\n>                 /*\n> -                * \\todo Avoid value-initialization of all elements of the\n> -                * vector.\n\nI think this comment should not be removed.\n\n> +                * Split planes manually as the encoder expects a vector of\n> +                * planes.\n> +                *\n> +                * \\todo Pass a vector of planes directly to\n> +                * Thumbnailer::createThumbnailer above and remove the manual\n> +                * planes split from here.\n>                  */\n> -               thumbnail->resize(rawThumbnail.size());\n> +               std::vector<Span<uint8_t>> thumbnailPlanes;\n> +               unsigned int thumbnailSize = targetSize.height * targetSize.width;\n> +               thumbnailPlanes.push_back({ rawThumbnail.data(), thumbnailSize });\n> +               thumbnailPlanes.push_back({ rawThumbnail.data() + thumbnailSize,\n> +                                           thumbnailSize / 2 });\n\nShall PixelFormatInfo::planeSize() be used?\n\nWith nits,\nReviewed-by: Hirokazu Honda <hiroh@chromium.org>\n\n-Hiro\n>\n> -               int jpeg_size = thumbnailEncoder_.encode({ rawThumbnail },\n> +               int jpeg_size = thumbnailEncoder_.encode(thumbnailPlanes,\n>                                                          *thumbnail, {}, quality);\n>                 thumbnail->resize(jpeg_size);\n>\n> --\n> 2.31.0\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8E4EEBDC71\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed,  8 Sep 2021 10:03:13 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1EDBF6916E;\n\tWed,  8 Sep 2021 12:03:13 +0200 (CEST)","from mail-yb1-xb32.google.com (mail-yb1-xb32.google.com\n\t[IPv6:2607:f8b0:4864:20::b32])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C0FEB60503\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed,  8 Sep 2021 12:03:10 +0200 (CEST)","by mail-yb1-xb32.google.com with SMTP id c6so3061116ybm.10\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 08 Sep 2021 03:03:10 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"LVMP9R/3\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=nZxKrOuUklhyXysmEYS6LPwelGPgYJpyx8b5TVvQu4s=;\n\tb=LVMP9R/3/bo/jQcASDhX7HvUa1NuJJ98N9fiXvbGJBRmaMOOrldkNAuQD3aarlQuLV\n\tzbk5t8jVxnohZIVytZWRqU/MCqw5Yz5Yp0Nss4dhyYlANa3Y7+v0Zj/NcCPa+0uZ6zPd\n\tmV7ZclvEbeVaib/foZS4W4lcxqov0WnNNDH8I=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=nZxKrOuUklhyXysmEYS6LPwelGPgYJpyx8b5TVvQu4s=;\n\tb=U3drW795YAJuMQv4QtfjGcfQ4Ax3/WC+sfFV5CK9m/sJP2DgUjZA8Nt2KpoQtIkOyb\n\t3AXifBrZf8QAozy0ifQY1+QtgoFXftbm6XA0aDBG+PNqVQRZSZKGiIWkhStnEqQ1lgZY\n\tru7HBFLcuP671H+nxbaH86YQqlZHzU8G39VUmtvKkQ9hKDG0tB4kX5vdJMCMyCKvLxyW\n\tmewRXX6ND5W1X8Wcry4EWGuFZ/MIf80In6aeZSSG2VVX8WQGoCuSuf8oiDwjZaap4mcL\n\t6gq0o5LbEMrUZFuPBsAv++M82/mNBiIRWaeRxiN87kGJp5Yfx91qH2wki2QAuyJbH6tA\n\t4DVQ==","X-Gm-Message-State":"AOAM532TmvzzYHT034KX2atTfeSuZOuLU4IWM/Mr4XkchR/aATIinLSK\n\tgWFlkAJd7+mpXf3u+WFX+H/LAsb6aGl5zUpWk42lOqUUdaA=","X-Google-Smtp-Source":"ABdhPJwVRzGQboGZmifizJ4XKtZdAXQBl2l098dv8xsr3SO2MSkr+MCYS7MtnaeVavSTXlMKHYtS10ph9JdWD7L+4Cw=","X-Received":"by 2002:a25:2785:: with SMTP id\n\tn127mr3907867ybn.235.1631095389197; \n\tWed, 08 Sep 2021 03:03:09 -0700 (PDT)","MIME-Version":"1.0","References":"<20210908094949.111140-1-umang.jain@ideasonboard.com>","In-Reply-To":"<20210908094949.111140-1-umang.jain@ideasonboard.com>","From":"Hirokazu Honda <hiroh@chromium.org>","Date":"Wed, 8 Sep 2021 19:02:58 +0900","Message-ID":"<CAO5uPHNapWZCjm+BzF0Mn+qq+aAeyzVhk1cio302jMv5Cvh0Ng@mail.gmail.com>","To":"Umang Jain <umang.jain@ideasonboard.com>","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera devel <libcamera-devel@lists.libcamera.org>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":19563,"web_url":"https://patchwork.libcamera.org/comment/19563/","msgid":"<2abd52d2-4a67-ee89-7bc0-655bca4ab3ae@ideasonboard.com>","date":"2021-09-09T06:38:04","subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/people/86/","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"content":"Hi Hiro?\n\nOn 9/8/21 3:32 PM, Hirokazu Honda wrote:\n> Hi Umang, thank you for the patch.\n>\n> On Wed, Sep 8, 2021 at 6:50 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n>> After multi-planar support was introduced for jpeg encoding as well,\n>> EncoderLibJpeg::encode() expects a vector of planes as the source of\n>> framebuffer to be encoded. Currently, we are passing a contiguous buffer\n>> which is treated as only one plane (instead of two, as thumbnail is NV12).\n>>\n>> Hence, split the thumbnail data into respective planes according to NV12.\n>> This fixes a crash in encoding of thumbnails.\n>>\n>> Fixes: 894ca69f6043(\"android: jpeg: Support multi-planar buffers\")\n>> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>\n>> ---\n>> Changes in v2:\n>> - Fix basic split logic error.\n>> - Add Jacopo's suggestion as a \\todo since it's a superior method to fix\n>>    this.\n>> ---\n>>   src/android/jpeg/post_processor_jpeg.cpp | 18 ++++++++++++++----\n>>   1 file changed, 14 insertions(+), 4 deletions(-)\n>>\n>> diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\n>> index 68d74842..1bf507a4 100644\n>> --- a/src/android/jpeg/post_processor_jpeg.cpp\n>> +++ b/src/android/jpeg/post_processor_jpeg.cpp\n>> @@ -66,13 +66,23 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n>>          int ret = thumbnailEncoder_.configure(thCfg);\n>>\n>>          if (!rawThumbnail.empty() && !ret) {\n>> +               thumbnail->resize(rawThumbnail.size());\n>> +\n>>                  /*\n>> -                * \\todo Avoid value-initialization of all elements of the\n>> -                * vector.\n> I think this comment should not be removed.\n>\n>> +                * Split planes manually as the encoder expects a vector of\n>> +                * planes.\n>> +                *\n>> +                * \\todo Pass a vector of planes directly to\n>> +                * Thumbnailer::createThumbnailer above and remove the manual\n>> +                * planes split from here.\n>>                   */\n>> -               thumbnail->resize(rawThumbnail.size());\n>> +               std::vector<Span<uint8_t>> thumbnailPlanes;\n>> +               unsigned int thumbnailSize = targetSize.height * targetSize.width;\n>> +               thumbnailPlanes.push_back({ rawThumbnail.data(), thumbnailSize });\n>> +               thumbnailPlanes.push_back({ rawThumbnail.data() + thumbnailSize,\n>> +                                           thumbnailSize / 2 });\n> Shall PixelFormatInfo::planeSize() be used?\n\n\nUsed for what? The \" / 2\" part?\n\nAlso, Did you mean PixelFormatInfo::numPlanes() instead? I won't worry \ntoo much here, as the Thumbnail class is tightly bound to NV12. So that \nplanes are constant. See Thumbnailer::configure\n\nWhen we add support for more diverse thumbnailing for formats, I expect \nthis to be changed in a major way.\n\n\n>\n> With nits,\n> Reviewed-by: Hirokazu Honda <hiroh@chromium.org>\n>\n> -Hiro\n>> -               int jpeg_size = thumbnailEncoder_.encode({ rawThumbnail },\n>> +               int jpeg_size = thumbnailEncoder_.encode(thumbnailPlanes,\n>>                                                           *thumbnail, {}, quality);\n>>                  thumbnail->resize(jpeg_size);\n>>\n>> --\n>> 2.31.0\n>>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6817EBDB1D\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu,  9 Sep 2021 06:38:12 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B184C69169;\n\tThu,  9 Sep 2021 08:38:11 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 671946024A\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu,  9 Sep 2021 08:38:09 +0200 (CEST)","from [192.168.1.104] (unknown [103.251.226.149])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 81F1B466;\n\tThu,  9 Sep 2021 08:38:08 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"YEus9WsO\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1631169489;\n\tbh=Gz4/tstwWLwVINH8An6Qgwv9dpnT/7cl9qVE0mi/wHs=;\n\th=Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=YEus9WsOG/XEPpOphoXZbbm4lvUCGIqvrBCPnI2i3fvAbWODiA+f2yzGY6CzxBwCo\n\tCOLD38QLFHx9MaTpJ9uTdSjVGPDtXduV3yIRLfaXP/C+5Q+KyGY+6OXEknbYJ9IXMm\n\t4WAE8IBrqsShB0GvXd/4gJbEmxHRlfLL7/+llctM=","To":"Hirokazu Honda <hiroh@chromium.org>","References":"<20210908094949.111140-1-umang.jain@ideasonboard.com>\n\t<CAO5uPHNapWZCjm+BzF0Mn+qq+aAeyzVhk1cio302jMv5Cvh0Ng@mail.gmail.com>","From":"Umang Jain <umang.jain@ideasonboard.com>","Message-ID":"<2abd52d2-4a67-ee89-7bc0-655bca4ab3ae@ideasonboard.com>","Date":"Thu, 9 Sep 2021 12:08:04 +0530","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.10.2","MIME-Version":"1.0","In-Reply-To":"<CAO5uPHNapWZCjm+BzF0Mn+qq+aAeyzVhk1cio302jMv5Cvh0Ng@mail.gmail.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Transfer-Encoding":"7bit","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera devel <libcamera-devel@lists.libcamera.org>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":19564,"web_url":"https://patchwork.libcamera.org/comment/19564/","msgid":"<CAO5uPHN7b7ffPEkYXmrJx284FkAcA=0qQrh90FZhOe_LaU=8dQ@mail.gmail.com>","date":"2021-09-09T06:56:06","subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","submitter":{"id":63,"url":"https://patchwork.libcamera.org/api/people/63/","name":"Hirokazu Honda","email":"hiroh@chromium.org"},"content":"Hi Umang,\n\nOn Thu, Sep 9, 2021 at 3:38 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n>\n> Hi Hiro?\n>\n> On 9/8/21 3:32 PM, Hirokazu Honda wrote:\n> > Hi Umang, thank you for the patch.\n> >\n> > On Wed, Sep 8, 2021 at 6:50 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n> >> After multi-planar support was introduced for jpeg encoding as well,\n> >> EncoderLibJpeg::encode() expects a vector of planes as the source of\n> >> framebuffer to be encoded. Currently, we are passing a contiguous buffer\n> >> which is treated as only one plane (instead of two, as thumbnail is NV12).\n> >>\n> >> Hence, split the thumbnail data into respective planes according to NV12.\n> >> This fixes a crash in encoding of thumbnails.\n> >>\n> >> Fixes: 894ca69f6043(\"android: jpeg: Support multi-planar buffers\")\n> >> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>\n> >> ---\n> >> Changes in v2:\n> >> - Fix basic split logic error.\n> >> - Add Jacopo's suggestion as a \\todo since it's a superior method to fix\n> >>    this.\n> >> ---\n> >>   src/android/jpeg/post_processor_jpeg.cpp | 18 ++++++++++++++----\n> >>   1 file changed, 14 insertions(+), 4 deletions(-)\n> >>\n> >> diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\n> >> index 68d74842..1bf507a4 100644\n> >> --- a/src/android/jpeg/post_processor_jpeg.cpp\n> >> +++ b/src/android/jpeg/post_processor_jpeg.cpp\n> >> @@ -66,13 +66,23 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n> >>          int ret = thumbnailEncoder_.configure(thCfg);\n> >>\n> >>          if (!rawThumbnail.empty() && !ret) {\n> >> +               thumbnail->resize(rawThumbnail.size());\n> >> +\n> >>                  /*\n> >> -                * \\todo Avoid value-initialization of all elements of the\n> >> -                * vector.\n> > I think this comment should not be removed.\n> >\n> >> +                * Split planes manually as the encoder expects a vector of\n> >> +                * planes.\n> >> +                *\n> >> +                * \\todo Pass a vector of planes directly to\n> >> +                * Thumbnailer::createThumbnailer above and remove the manual\n> >> +                * planes split from here.\n> >>                   */\n> >> -               thumbnail->resize(rawThumbnail.size());\n> >> +               std::vector<Span<uint8_t>> thumbnailPlanes;\n> >> +               unsigned int thumbnailSize = targetSize.height * targetSize.width;\n> >> +               thumbnailPlanes.push_back({ rawThumbnail.data(), thumbnailSize });\n> >> +               thumbnailPlanes.push_back({ rawThumbnail.data() + thumbnailSize,\n> >> +                                           thumbnailSize / 2 });\n> > Shall PixelFormatInfo::planeSize() be used?\n>\n>\n> Used for what? The \" / 2\" part?\n>\n> Also, Did you mean PixelFormatInfo::numPlanes() instead? I won't worry\n> too much here, as the Thumbnail class is tightly bound to NV12. So that\n> planes are constant. See Thumbnailer::configure\n>\n> When we add support for more diverse thumbnailing for formats, I expect\n> this to be changed in a major way.\n>\n\nI would\nsize_t YPlaneSize = PixelFormatInfo(formats::NV12)::planeSize(targetSize, 0);\nsize_t UVPlaneSize = PixelFormatInfo(formats::NV12)::planeSize(targetSize, 1);\n thumbnailPlanes.push_back({ rawThumbnail.data(), YPlaneSize });\n thumbnailPlanes.push_back({ rawThumbnail.data() + YPlaneSize, UVPlaneSize });\n\nI don't have a strong opinion. Up to you.\n-Hiro\n>\n> >\n> > With nits,\n> > Reviewed-by: Hirokazu Honda <hiroh@chromium.org>\n> >\n> > -Hiro\n> >> -               int jpeg_size = thumbnailEncoder_.encode({ rawThumbnail },\n> >> +               int jpeg_size = thumbnailEncoder_.encode(thumbnailPlanes,\n> >>                                                           *thumbnail, {}, quality);\n> >>                  thumbnail->resize(jpeg_size);\n> >>\n> >> --\n> >> 2.31.0\n> >>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 54E87BDC71\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu,  9 Sep 2021 06:56:20 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 79CF06916E;\n\tThu,  9 Sep 2021 08:56:19 +0200 (CEST)","from mail-ej1-x632.google.com (mail-ej1-x632.google.com\n\t[IPv6:2a00:1450:4864:20::632])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 213C76024A\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu,  9 Sep 2021 08:56:18 +0200 (CEST)","by mail-ej1-x632.google.com with SMTP id me10so1449803ejb.11\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 08 Sep 2021 23:56:18 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"T23C+ZXW\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=ftEjUo1GFQZX50eAXLvbKUIDGY1dzZ5vZOkuQlu7gYA=;\n\tb=T23C+ZXWryoTIQ3ub7N5ckqt0a2uefxPIyeGQQkuf9jS8vx5G/nZidKTq5JuheoYWl\n\tgr65V4Ndm8jzC4hHGN0TrkZ8qK8jW9WmUvmUT6bU8b4Z+aYarkD/RYk5ZfmXYluZB1qW\n\tDiWYRpFfOuSClPhnRoJcGRBi4PPIBmyVvZLOE=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=ftEjUo1GFQZX50eAXLvbKUIDGY1dzZ5vZOkuQlu7gYA=;\n\tb=cuxzuiI4NNOTZT7n7Cj8KzVIfHVXaNOikLAMgiNpZLb2kjvdqtWMgv0tw98r1kftff\n\tCm/X+Zij441euvsr5MOjNAFOewEWbrxWwA0JxNnZfH0DK+WyjAGXZMn8+KMfMp/oWlGL\n\tPmG+6h6VZEEK0TxLGYCJ1ZnweA2XqxTLbyxk29PgfzYAES6a0ePqnj5DQa1H00G1yqLW\n\tc2ksoKX0qrVxxevPZh5RPO4p1O42gzkSiqyrfPSmIRRb9xEvCJl8fdU3eq+yhBW+i8Oi\n\tNSxpjMHs/m88XPGJ3sclc1NjY2UxZYW8KYEnijJ8Pj5fMGv3CK0SfOeYinNcZPrAaKbz\n\t52xw==","X-Gm-Message-State":"AOAM533svlepbiHCNHu3Srp9SpDojxt2S0A2mQvWeLo0WfRcKvgMw2WZ\n\tutKirHDcKMfuxpzFR4rR0al4/DdiuUEe9mU20URbsbXIbQUytQ==","X-Google-Smtp-Source":"ABdhPJxYnA99mZ63Q4vi/y1qijdrAKF39/D5jK1f3bqJwK4b7IFHl8izcD6LiWRTOjcRzilZf3K8VPkcckKqcWZVPsQ=","X-Received":"by 2002:a17:906:3146:: with SMTP id\n\te6mr1786837eje.296.1631170577659; \n\tWed, 08 Sep 2021 23:56:17 -0700 (PDT)","MIME-Version":"1.0","References":"<20210908094949.111140-1-umang.jain@ideasonboard.com>\n\t<CAO5uPHNapWZCjm+BzF0Mn+qq+aAeyzVhk1cio302jMv5Cvh0Ng@mail.gmail.com>\n\t<2abd52d2-4a67-ee89-7bc0-655bca4ab3ae@ideasonboard.com>","In-Reply-To":"<2abd52d2-4a67-ee89-7bc0-655bca4ab3ae@ideasonboard.com>","From":"Hirokazu Honda <hiroh@chromium.org>","Date":"Thu, 9 Sep 2021 15:56:06 +0900","Message-ID":"<CAO5uPHN7b7ffPEkYXmrJx284FkAcA=0qQrh90FZhOe_LaU=8dQ@mail.gmail.com>","To":"Umang Jain <umang.jain@ideasonboard.com>","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera devel <libcamera-devel@lists.libcamera.org>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":19566,"web_url":"https://patchwork.libcamera.org/comment/19566/","msgid":"<133072b7-81c4-5cf1-c212-7f7b89de6c8b@ideasonboard.com>","date":"2021-09-09T07:24:18","subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/people/86/","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"content":"Hi Hiro,\n\nOn 9/9/21 12:26 PM, Hirokazu Honda wrote:\n> Hi Umang,\n>\n> On Thu, Sep 9, 2021 at 3:38 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n>> Hi Hiro?\n>>\n>> On 9/8/21 3:32 PM, Hirokazu Honda wrote:\n>>> Hi Umang, thank you for the patch.\n>>>\n>>> On Wed, Sep 8, 2021 at 6:50 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n>>>> After multi-planar support was introduced for jpeg encoding as well,\n>>>> EncoderLibJpeg::encode() expects a vector of planes as the source of\n>>>> framebuffer to be encoded. Currently, we are passing a contiguous buffer\n>>>> which is treated as only one plane (instead of two, as thumbnail is NV12).\n>>>>\n>>>> Hence, split the thumbnail data into respective planes according to NV12.\n>>>> This fixes a crash in encoding of thumbnails.\n>>>>\n>>>> Fixes: 894ca69f6043(\"android: jpeg: Support multi-planar buffers\")\n>>>> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>\n>>>> ---\n>>>> Changes in v2:\n>>>> - Fix basic split logic error.\n>>>> - Add Jacopo's suggestion as a \\todo since it's a superior method to fix\n>>>>     this.\n>>>> ---\n>>>>    src/android/jpeg/post_processor_jpeg.cpp | 18 ++++++++++++++----\n>>>>    1 file changed, 14 insertions(+), 4 deletions(-)\n>>>>\n>>>> diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\n>>>> index 68d74842..1bf507a4 100644\n>>>> --- a/src/android/jpeg/post_processor_jpeg.cpp\n>>>> +++ b/src/android/jpeg/post_processor_jpeg.cpp\n>>>> @@ -66,13 +66,23 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n>>>>           int ret = thumbnailEncoder_.configure(thCfg);\n>>>>\n>>>>           if (!rawThumbnail.empty() && !ret) {\n>>>> +               thumbnail->resize(rawThumbnail.size());\n>>>> +\n>>>>                   /*\n>>>> -                * \\todo Avoid value-initialization of all elements of the\n>>>> -                * vector.\n>>> I think this comment should not be removed.\n>>>\n>>>> +                * Split planes manually as the encoder expects a vector of\n>>>> +                * planes.\n>>>> +                *\n>>>> +                * \\todo Pass a vector of planes directly to\n>>>> +                * Thumbnailer::createThumbnailer above and remove the manual\n>>>> +                * planes split from here.\n>>>>                    */\n>>>> -               thumbnail->resize(rawThumbnail.size());\n>>>> +               std::vector<Span<uint8_t>> thumbnailPlanes;\n>>>> +               unsigned int thumbnailSize = targetSize.height * targetSize.width;\n>>>> +               thumbnailPlanes.push_back({ rawThumbnail.data(), thumbnailSize });\n>>>> +               thumbnailPlanes.push_back({ rawThumbnail.data() + thumbnailSize,\n>>>> +                                           thumbnailSize / 2 });\n>>> Shall PixelFormatInfo::planeSize() be used?\n>>\n>> Used for what? The \" / 2\" part?\n>>\n>> Also, Did you mean PixelFormatInfo::numPlanes() instead? I won't worry\n>> too much here, as the Thumbnail class is tightly bound to NV12. So that\n>> planes are constant. See Thumbnailer::configure\n>>\n>> When we add support for more diverse thumbnailing for formats, I expect\n>> this to be changed in a major way.\n>>\n> I would\n> size_t YPlaneSize = PixelFormatInfo(formats::NV12)::planeSize(targetSize, 0);\n> size_t UVPlaneSize = PixelFormatInfo(formats::NV12)::planeSize(targetSize, 1);\n>   thumbnailPlanes.push_back({ rawThumbnail.data(), YPlaneSize });\n>   thumbnailPlanes.push_back({ rawThumbnail.data() + YPlaneSize, UVPlaneSize });\n\nOk, I see. These helpers are quite recent, and I wasn't that familiar \nwith these helpers.\n\nThese are more readable, I think. I'll wait for more reviews to pitch in \nmeanwhile.\n\n\n>\n> I don't have a strong opinion. Up to you.\n> -Hiro\n>>> With nits,\n>>> Reviewed-by: Hirokazu Honda <hiroh@chromium.org>\n>>>\n>>> -Hiro\n>>>> -               int jpeg_size = thumbnailEncoder_.encode({ rawThumbnail },\n>>>> +               int jpeg_size = thumbnailEncoder_.encode(thumbnailPlanes,\n>>>>                                                            *thumbnail, {}, quality);\n>>>>                   thumbnail->resize(jpeg_size);\n>>>>\n>>>> --\n>>>> 2.31.0\n>>>>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1721EBDC71\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu,  9 Sep 2021 07:24:25 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 94CC96916A;\n\tThu,  9 Sep 2021 09:24:24 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id ED3616024E\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu,  9 Sep 2021 09:24:22 +0200 (CEST)","from [192.168.1.104] (unknown [103.251.226.149])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0F1CE24F;\n\tThu,  9 Sep 2021 09:24:21 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"wGd6AqD9\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1631172262;\n\tbh=SPnbdaSSyu1kgUmu3v/uC0SOXSVdE1lofOFQL3AETcQ=;\n\th=Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=wGd6AqD9Tl02EP5Dlcl0J2wWhbj4HTHM7tvrVqSMfZ4B3HojLmQWAAMEOaLTlhOTA\n\tjjfXf2W7F6wnkeOPjKGP+55RvG2TTGGz1FhOaBe9FfSuxsL30XSdHRgbdPiGUFHXhF\n\t+4pRX7cjOLNaAeKGGu4cbcLbSV2F78HpII/0ZoX4=","To":"Hirokazu Honda <hiroh@chromium.org>","References":"<20210908094949.111140-1-umang.jain@ideasonboard.com>\n\t<CAO5uPHNapWZCjm+BzF0Mn+qq+aAeyzVhk1cio302jMv5Cvh0Ng@mail.gmail.com>\n\t<2abd52d2-4a67-ee89-7bc0-655bca4ab3ae@ideasonboard.com>\n\t<CAO5uPHN7b7ffPEkYXmrJx284FkAcA=0qQrh90FZhOe_LaU=8dQ@mail.gmail.com>","From":"Umang Jain <umang.jain@ideasonboard.com>","Message-ID":"<133072b7-81c4-5cf1-c212-7f7b89de6c8b@ideasonboard.com>","Date":"Thu, 9 Sep 2021 12:54:18 +0530","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.10.2","MIME-Version":"1.0","In-Reply-To":"<CAO5uPHN7b7ffPEkYXmrJx284FkAcA=0qQrh90FZhOe_LaU=8dQ@mail.gmail.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Transfer-Encoding":"7bit","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera devel <libcamera-devel@lists.libcamera.org>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":19573,"web_url":"https://patchwork.libcamera.org/comment/19573/","msgid":"<20210909100903.rsw7lnhqfx6xqmpq@uno.localdomain>","date":"2021-09-09T10:09:03","subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Umang,\n\nOn Thu, Sep 09, 2021 at 12:54:18PM +0530, Umang Jain wrote:\n> Hi Hiro,\n>\n> On 9/9/21 12:26 PM, Hirokazu Honda wrote:\n> > Hi Umang,\n> >\n> > On Thu, Sep 9, 2021 at 3:38 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n> > > Hi Hiro?\n> > >\n> > > On 9/8/21 3:32 PM, Hirokazu Honda wrote:\n> > > > Hi Umang, thank you for the patch.\n> > > >\n> > > > On Wed, Sep 8, 2021 at 6:50 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n> > > > > After multi-planar support was introduced for jpeg encoding as well,\n> > > > > EncoderLibJpeg::encode() expects a vector of planes as the source of\n> > > > > framebuffer to be encoded. Currently, we are passing a contiguous buffer\n> > > > > which is treated as only one plane (instead of two, as thumbnail is NV12).\n> > > > >\n> > > > > Hence, split the thumbnail data into respective planes according to NV12.\n> > > > > This fixes a crash in encoding of thumbnails.\n> > > > >\n> > > > > Fixes: 894ca69f6043(\"android: jpeg: Support multi-planar buffers\")\n> > > > > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>\n> > > > > ---\n> > > > > Changes in v2:\n> > > > > - Fix basic split logic error.\n> > > > > - Add Jacopo's suggestion as a \\todo since it's a superior method to fix\n> > > > >     this.\n> > > > > ---\n> > > > >    src/android/jpeg/post_processor_jpeg.cpp | 18 ++++++++++++++----\n> > > > >    1 file changed, 14 insertions(+), 4 deletions(-)\n> > > > >\n> > > > > diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\n> > > > > index 68d74842..1bf507a4 100644\n> > > > > --- a/src/android/jpeg/post_processor_jpeg.cpp\n> > > > > +++ b/src/android/jpeg/post_processor_jpeg.cpp\n> > > > > @@ -66,13 +66,23 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n> > > > >           int ret = thumbnailEncoder_.configure(thCfg);\n> > > > >\n> > > > >           if (!rawThumbnail.empty() && !ret) {\n> > > > > +               thumbnail->resize(rawThumbnail.size());\n> > > > > +\n> > > > >                   /*\n> > > > > -                * \\todo Avoid value-initialization of all elements of the\n> > > > > -                * vector.\n> > > > I think this comment should not be removed.\n\nBut please move it below the following paragraph to keep todo items at\nthe end of the comment block\n\n> > > >\n> > > > > +                * Split planes manually as the encoder expects a vector of\n> > > > > +                * planes.\n> > > > > +                *\n> > > > > +                * \\todo Pass a vector of planes directly to\n> > > > > +                * Thumbnailer::createThumbnailer above and remove the manual\n> > > > > +                * planes split from here.\n> > > > >                    */\n> > > > > -               thumbnail->resize(rawThumbnail.size());\n> > > > > +               std::vector<Span<uint8_t>> thumbnailPlanes;\n> > > > > +               unsigned int thumbnailSize = targetSize.height * targetSize.width;\n> > > > > +               thumbnailPlanes.push_back({ rawThumbnail.data(), thumbnailSize });\n> > > > > +               thumbnailPlanes.push_back({ rawThumbnail.data() + thumbnailSize,\n> > > > > +                                           thumbnailSize / 2 });\n> > > > Shall PixelFormatInfo::planeSize() be used?\n> > >\n> > > Used for what? The \" / 2\" part?\n> > >\n> > > Also, Did you mean PixelFormatInfo::numPlanes() instead? I won't worry\n> > > too much here, as the Thumbnail class is tightly bound to NV12. So that\n> > > planes are constant. See Thumbnailer::configure\n> > >\n> > > When we add support for more diverse thumbnailing for formats, I expect\n> > > this to be changed in a major way.\n> > >\n> > I would\n> > size_t YPlaneSize = PixelFormatInfo(formats::NV12)::planeSize(targetSize, 0);\n> > size_t UVPlaneSize = PixelFormatInfo(formats::NV12)::planeSize(targetSize, 1);\n> >   thumbnailPlanes.push_back({ rawThumbnail.data(), YPlaneSize });\n> >   thumbnailPlanes.push_back({ rawThumbnail.data() + YPlaneSize, UVPlaneSize });\n>\n> Ok, I see. These helpers are quite recent, and I wasn't that familiar with\n> these helpers.\n>\n> These are more readable, I think. I'll wait for more reviews to pitch in\n> meanwhile.\n\nWhatever you decide to use:\nReviewed-by: Jacopo Mondi <jacopo@jmondi.org>\n\nThanks\n  j\n>\n>\n> >\n> > I don't have a strong opinion. Up to you.\n> > -Hiro\n> > > > With nits,\n> > > > Reviewed-by: Hirokazu Honda <hiroh@chromium.org>\n> > > >\n> > > > -Hiro\n> > > > > -               int jpeg_size = thumbnailEncoder_.encode({ rawThumbnail },\n> > > > > +               int jpeg_size = thumbnailEncoder_.encode(thumbnailPlanes,\n> > > > >                                                            *thumbnail, {}, quality);\n> > > > >                   thumbnail->resize(jpeg_size);\n> > > > >\n> > > > > --\n> > > > > 2.31.0\n> > > > >","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B75A1BDB1D\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu,  9 Sep 2021 10:08:20 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 26F636916E;\n\tThu,  9 Sep 2021 12:08:20 +0200 (CEST)","from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net\n\t[217.70.183.195])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3438A6916B\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu,  9 Sep 2021 12:08:18 +0200 (CEST)","(Authenticated sender: jacopo@jmondi.org)\n\tby relay3-d.mail.gandi.net (Postfix) with ESMTPSA id 6F7D660005;\n\tThu,  9 Sep 2021 10:08:17 +0000 (UTC)"],"Date":"Thu, 9 Sep 2021 12:09:03 +0200","From":"Jacopo Mondi <jacopo@jmondi.org>","To":"Umang Jain <umang.jain@ideasonboard.com>","Message-ID":"<20210909100903.rsw7lnhqfx6xqmpq@uno.localdomain>","References":"<20210908094949.111140-1-umang.jain@ideasonboard.com>\n\t<CAO5uPHNapWZCjm+BzF0Mn+qq+aAeyzVhk1cio302jMv5Cvh0Ng@mail.gmail.com>\n\t<2abd52d2-4a67-ee89-7bc0-655bca4ab3ae@ideasonboard.com>\n\t<CAO5uPHN7b7ffPEkYXmrJx284FkAcA=0qQrh90FZhOe_LaU=8dQ@mail.gmail.com>\n\t<133072b7-81c4-5cf1-c212-7f7b89de6c8b@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<133072b7-81c4-5cf1-c212-7f7b89de6c8b@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera devel <libcamera-devel@lists.libcamera.org>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":19582,"web_url":"https://patchwork.libcamera.org/comment/19582/","msgid":"<a3e4a51c-bbcc-dc0e-ff8d-38a2361f97d5@ideasonboard.com>","date":"2021-09-09T14:37:28","subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/people/86/","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"content":"Hi Jacopo,\n\n\nOn 9/9/21 3:39 PM, Jacopo Mondi wrote:\n> Hi Umang,\n>\n> On Thu, Sep 09, 2021 at 12:54:18PM +0530, Umang Jain wrote:\n>> Hi Hiro,\n>>\n>> On 9/9/21 12:26 PM, Hirokazu Honda wrote:\n>>> Hi Umang,\n>>>\n>>> On Thu, Sep 9, 2021 at 3:38 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n>>>> Hi Hiro?\n>>>>\n>>>> On 9/8/21 3:32 PM, Hirokazu Honda wrote:\n>>>>> Hi Umang, thank you for the patch.\n>>>>>\n>>>>> On Wed, Sep 8, 2021 at 6:50 PM Umang Jain <umang.jain@ideasonboard.com> wrote:\n>>>>>> After multi-planar support was introduced for jpeg encoding as well,\n>>>>>> EncoderLibJpeg::encode() expects a vector of planes as the source of\n>>>>>> framebuffer to be encoded. Currently, we are passing a contiguous buffer\n>>>>>> which is treated as only one plane (instead of two, as thumbnail is NV12).\n>>>>>>\n>>>>>> Hence, split the thumbnail data into respective planes according to NV12.\n>>>>>> This fixes a crash in encoding of thumbnails.\n>>>>>>\n>>>>>> Fixes: 894ca69f6043(\"android: jpeg: Support multi-planar buffers\")\n>>>>>> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>\n>>>>>> ---\n>>>>>> Changes in v2:\n>>>>>> - Fix basic split logic error.\n>>>>>> - Add Jacopo's suggestion as a \\todo since it's a superior method to fix\n>>>>>>      this.\n>>>>>> ---\n>>>>>>     src/android/jpeg/post_processor_jpeg.cpp | 18 ++++++++++++++----\n>>>>>>     1 file changed, 14 insertions(+), 4 deletions(-)\n>>>>>>\n>>>>>> diff --git a/src/android/jpeg/post_processor_jpeg.cpp b/src/android/jpeg/post_processor_jpeg.cpp\n>>>>>> index 68d74842..1bf507a4 100644\n>>>>>> --- a/src/android/jpeg/post_processor_jpeg.cpp\n>>>>>> +++ b/src/android/jpeg/post_processor_jpeg.cpp\n>>>>>> @@ -66,13 +66,23 @@ void PostProcessorJpeg::generateThumbnail(const FrameBuffer &source,\n>>>>>>            int ret = thumbnailEncoder_.configure(thCfg);\n>>>>>>\n>>>>>>            if (!rawThumbnail.empty() && !ret) {\n>>>>>> +               thumbnail->resize(rawThumbnail.size());\n>>>>>> +\n>>>>>>                    /*\n>>>>>> -                * \\todo Avoid value-initialization of all elements of the\n>>>>>> -                * vector.\n>>>>> I think this comment should not be removed.\n> But please move it below the following paragraph to keep todo items at\n> the end of the comment block\n\n\nI am going to undo the deletion so it will stay as is. I'll make sure \nany \\todos are at end of comment block\n\n>\n>>>>>> +                * Split planes manually as the encoder expects a vector of\n>>>>>> +                * planes.\n>>>>>> +                *\n>>>>>> +                * \\todo Pass a vector of planes directly to\n>>>>>> +                * Thumbnailer::createThumbnailer above and remove the manual\n>>>>>> +                * planes split from here.\n>>>>>>                     */\n>>>>>> -               thumbnail->resize(rawThumbnail.size());\n>>>>>> +               std::vector<Span<uint8_t>> thumbnailPlanes;\n>>>>>> +               unsigned int thumbnailSize = targetSize.height * targetSize.width;\n>>>>>> +               thumbnailPlanes.push_back({ rawThumbnail.data(), thumbnailSize });\n>>>>>> +               thumbnailPlanes.push_back({ rawThumbnail.data() + thumbnailSize,\n>>>>>> +                                           thumbnailSize / 2 });\n>>>>> Shall PixelFormatInfo::planeSize() be used?\n>>>> Used for what? The \" / 2\" part?\n>>>>\n>>>> Also, Did you mean PixelFormatInfo::numPlanes() instead? I won't worry\n>>>> too much here, as the Thumbnail class is tightly bound to NV12. So that\n>>>> planes are constant. See Thumbnailer::configure\n>>>>\n>>>> When we add support for more diverse thumbnailing for formats, I expect\n>>>> this to be changed in a major way.\n>>>>\n>>> I would\n>>> size_t YPlaneSize = PixelFormatInfo(formats::NV12)::planeSize(targetSize, 0);\n>>> size_t UVPlaneSize = PixelFormatInfo(formats::NV12)::planeSize(targetSize, 1);\n>>>    thumbnailPlanes.push_back({ rawThumbnail.data(), YPlaneSize });\n>>>    thumbnailPlanes.push_back({ rawThumbnail.data() + YPlaneSize, UVPlaneSize });\n>> Ok, I see. These helpers are quite recent, and I wasn't that familiar with\n>> these helpers.\n>>\n>> These are more readable, I think. I'll wait for more reviews to pitch in\n>> meanwhile.\n> Whatever you decide to use:\n> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>\n\n\nThanks, I'll apply Hiro's suggestion and test once before pushing to master.\n\nThis is slightly urgent patch to merge since CTS will fail to run \nwithout this. So I'll merge it in a few minutes.\n\n>\n> Thanks\n>    j\n>>\n>>> I don't have a strong opinion. Up to you.\n>>> -Hiro\n>>>>> With nits,\n>>>>> Reviewed-by: Hirokazu Honda <hiroh@chromium.org>\n>>>>>\n>>>>> -Hiro\n>>>>>> -               int jpeg_size = thumbnailEncoder_.encode({ rawThumbnail },\n>>>>>> +               int jpeg_size = thumbnailEncoder_.encode(thumbnailPlanes,\n>>>>>>                                                             *thumbnail, {}, quality);\n>>>>>>                    thumbnail->resize(jpeg_size);\n>>>>>>\n>>>>>> --\n>>>>>> 2.31.0\n>>>>>>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 25433BDC71\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu,  9 Sep 2021 14:37:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 56BB36916E;\n\tThu,  9 Sep 2021 16:37:35 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D84EB6916B\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu,  9 Sep 2021 16:37:33 +0200 (CEST)","from [192.168.1.104] (unknown [103.251.226.149])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B8FA7883;\n\tThu,  9 Sep 2021 16:37:32 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"u4vb7dNw\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1631198253;\n\tbh=m87pnH/igWLTavzVMsa2S3TDGzuR7POcLM4TU8wMrtk=;\n\th=Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=u4vb7dNw+BYZnSidrXtSrLFfpth8lTHEWdYchK1WiGkK6bwkEfwby9YVMqTI8Qy7V\n\tyQPC6pNNQ/XDeZ2xl2TLbDj0sa89mPN8B6yEbiXBDCIKpzq9gGFGHeKoHwa5zAcFlH\n\tWLFI/NkO65RaOSvmAIsNulq04FDq64T0CpRVGhBs=","To":"Jacopo Mondi <jacopo@jmondi.org>","References":"<20210908094949.111140-1-umang.jain@ideasonboard.com>\n\t<CAO5uPHNapWZCjm+BzF0Mn+qq+aAeyzVhk1cio302jMv5Cvh0Ng@mail.gmail.com>\n\t<2abd52d2-4a67-ee89-7bc0-655bca4ab3ae@ideasonboard.com>\n\t<CAO5uPHN7b7ffPEkYXmrJx284FkAcA=0qQrh90FZhOe_LaU=8dQ@mail.gmail.com>\n\t<133072b7-81c4-5cf1-c212-7f7b89de6c8b@ideasonboard.com>\n\t<20210909100903.rsw7lnhqfx6xqmpq@uno.localdomain>","From":"Umang Jain <umang.jain@ideasonboard.com>","Message-ID":"<a3e4a51c-bbcc-dc0e-ff8d-38a2361f97d5@ideasonboard.com>","Date":"Thu, 9 Sep 2021 20:07:28 +0530","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.10.2","MIME-Version":"1.0","In-Reply-To":"<20210909100903.rsw7lnhqfx6xqmpq@uno.localdomain>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Transfer-Encoding":"7bit","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v2] android: jpeg: Split and pass the\n\tthumbnail planes to encoder","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera devel <libcamera-devel@lists.libcamera.org>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]