[{"id":5173,"web_url":"https://patchwork.libcamera.org/comment/5173/","msgid":"<20200611030526.GC13598@pendragon.ideasonboard.com>","date":"2020-06-11T03:05:26","subject":"Re: [libcamera-devel] [PATCH v2] qcam: dng_writer: Add support for\n\tIPU3 Bayer formats","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Niklas,\n\nThank you for the patch.\n\nOn Thu, Jun 11, 2020 at 03:45:36AM +0200, Niklas Söderlund wrote:\n> Add support for the Bayer formats produced on the IPU3. The format uses\n> a memory layout that is hard to repack and keep the 10-bit sample size,\n> therefore scale the samples to 16-bit when creating the scanlines.\n> \n> Signed-off-by: Niklas Söderlund \n> ---\n> src/qcam/dng_writer.cpp | 121 ++++++++++++++++++++++++++++++++++++++++\n> 1 file changed, 121 insertions(+)\n> \n> diff --git a/src/qcam/dng_writer.cpp b/src/qcam/dng_writer.cpp\n> index cbd8bed3e6d02269..330b169af25b7ac7 100644\n> --- a/src/qcam/dng_writer.cpp\n> +++ b/src/qcam/dng_writer.cpp\n> @@ -82,6 +82,103 @@ void thumbScanlineSBGGRxxP(const FormatInfo &info, void *output,\n> \t}\n> }\n> \n> +void packScanlineIPU3(void *output, const void *input, unsigned int width)\n> +{\n> +\tconst uint8_t *in = static_cast(input);\n> +\tuint16_t *out = static_cast(output);\n> +\n> +\t/*\n> +\t * Upscale the 10-bit format to 16-bit as it's not trivial to pack it\n> +\t * as 10-bit without gaps.\n> +\t *\n> +\t * \\todo Improve packing to keep the 10-bit sample size.\n> +\t */\n\nI think we could use an intermediate line buffer. When we'll address\nthat, not now :-)\n\n> +\tunsigned int x = 0;\n> +\twhile (true) {\n> +\t\tfor (unsigned int i = 0; i < 6; i++) {\n> +\t\t\t*out++ = (in[1] & 0x03) << 14 | (in[0] & 0xff) << 6;\n> +\t\t\tif (++x >= width)\n> +\t\t\t\tbreak;\n> +\n> +\t\t\t*out++ = (in[2] & 0x0f) << 12 | (in[1] & 0xfc) << 4;\n> +\t\t\tif (++x >= width)\n> +\t\t\t\tbreak;\n> +\n> +\t\t\t*out++ = (in[3] & 0x3f) << 10 | (in[2] & 0xf0) << 2;\n> +\t\t\tif (++x >= width)\n> +\t\t\t\tbreak;\n> +\n> +\t\t\t*out++ = (in[4] & 0xff) << 8 | (in[3] & 0xc0) << 0;\n> +\t\t\tif (++x >= width)\n> +\t\t\t\tbreak;\n> +\n> +\t\t\tin += 5;\n> +\t\t}\n> +\n> +\t\t*out++ = (in[1] & 0x03) << 14 | (in[0] & 0xff) << 6;\n\nYou will execute this even if the above for loop stopped due to EOL\nbeing reached. You could replace all the break by return to fix that.\n\nWhat an awful format... I wonder how they handle it at the hardware\nlevel, I suppose there's an efficient mechanism.\n\n> +\t\tif (++x >= width)\n> +\t\t\tbreak;\n> +\n> +\t\tin += 2;\n> +\t}\n> +}\n> +\n> +void thumbScanlineIPU3(const FormatInfo &info, void *output,\n> +\t\t const void *input, unsigned int width,\n> +\t\t unsigned int stride)\n> +{\n> +\tuint8_t *out = static_cast(output);\n> +\n> +\tfor (unsigned int x = 0; x < width; x++) {\n> +\t\tunsigned int pixel = x * 16;\n> +\t\tunsigned int block = pixel / 25;\n> +\t\tunsigned int pixelInBlock = pixel - block * 25;\n> +\n> +\t\t/*\n> +\t\t * If the pixel is the last in the block cheat a little and\n> +\t\t * move one pixel backward to avoid reading between two blocks\n> +\t\t * and having to deal with the padding bits.\n> +\t\t */\n> +\t\tif (pixelInBlock == 24)\n> +\t\t\tpixelInBlock--;\n\nI think that's fair enough. I wonder if we could switch (pixelInBlock)\nbelow instead of switch (pixelInBlock % 4), in order to handle case 24\nexplicitly, but if it would require any additional complexity, I think\nwe can live with this hack.\n\n> +\n> +\t\tconst uint8_t *in = static_cast(input)\n> +\t\t\t+ block * 32 + (pixelInBlock / 4) * 5;\n\nAligning + under = ?\n\n> +\n> +\t\tuint16_t val1, val2, val3, val4;\n> +\t\tswitch (pixelInBlock % 4) {\n> +\t\tcase 0:\n> +\t\t\tval1 = (in[1] & 0x03) << 14 | (in[0] & 0xff) << 6;\n> +\t\t\tval2 = (in[2] & 0x0f) << 12 | (in[1] & 0xfc) << 4;\n> +\t\t\tval3 = (in[stride + 1] & 0x03) << 14 | (in[stride + 0] & 0xff) << 6;\n> +\t\t\tval4 = (in[stride + 2] & 0x0f) << 12 | (in[stride + 1] & 0xfc) << 4;\n> +\t\t\tbreak;\n> +\t\tcase 1:\n> +\t\t\tval1 = (in[2] & 0x0f) << 12 | (in[1] & 0xfc) << 4;\n> +\t\t\tval2 = (in[3] & 0x3f) << 10 | (in[2] & 0xf0) << 2;\n> +\t\t\tval3 = (in[stride + 2] & 0x0f) << 12 | (in[stride + 1] & 0xfc) << 4;\n> +\t\t\tval4 = (in[stride + 3] & 0x3f) << 10 | (in[stride + 2] & 0xf0) << 2;\n> +\t\t\tbreak;\n> +\t\tcase 2:\n> +\t\t\tval1 = (in[3] & 0x3f) << 10 | (in[2] & 0xf0) << 2;\n> +\t\t\tval2 = (in[4] & 0xff) << 8 | (in[3] & 0xc0) << 0;\n> +\t\t\tval3 = (in[stride + 3] & 0x3f) << 10 | (in[stride + 2] & 0xf0) << 2;\n> +\t\t\tval4 = (in[stride + 4] & 0xff) << 8 | (in[stride + 3] & 0xc0) << 0;\n> +\t\t\tbreak;\n> +\t\tcase 3:\n> +\t\t\tval1 = (in[4] & 0xff) << 8 | (in[3] & 0xc0) << 0;\n> +\t\t\tval2 = (in[6] & 0x03) << 14 | (in[5] & 0xff) << 6;\n> +\t\t\tval3 = (in[stride + 4] & 0xff) << 8 | (in[stride + 3] & 0xc0) << 0;\n> +\t\t\tval4 = (in[stride + 6] & 0x03) << 14 | (in[stride + 5] & 0xff) << 6;\n> +\t\t}\n> +\n> +\t\tuint8_t value = (val1 + val2 + val3 + val4) >> 8;\n\nAs val[1234] are 16-bit values, shouldn't you shift right by 10 to\nconvert to 8-bit and average the four values ?\n\nReviewed-by: Laurent Pinchart \n\n> +\t\t*out++ = value;\n> +\t\t*out++ = value;\n> +\t\t*out++ = value;\n> +\t}\n> +}\n> +\n> static const std::map formatInfo = {\n> \t{ PixelFormat(DRM_FORMAT_SBGGR10, MIPI_FORMAT_MOD_CSI2_PACKED), {\n> \t\t.bitsPerSample = 10,\n> @@ -131,6 +228,30 @@ static const std::map formatInfo = {\n> \t\t.packScanline = packScanlineSBGGR12P,\n> \t\t.thumbScanline = thumbScanlineSBGGRxxP,\n> \t} },\n> +\t{ PixelFormat(DRM_FORMAT_SBGGR10, IPU3_FORMAT_MOD_PACKED), {\n> +\t\t.bitsPerSample = 16,\n> +\t\t.pattern = { CFAPatternBlue, CFAPatternGreen, CFAPatternGreen, CFAPatternRed },\n> +\t\t.packScanline = packScanlineIPU3,\n> +\t\t.thumbScanline = thumbScanlineIPU3,\n> +\t} },\n> +\t{ PixelFormat(DRM_FORMAT_SGBRG10, IPU3_FORMAT_MOD_PACKED), {\n> +\t\t.bitsPerSample = 16,\n> +\t\t.pattern = { CFAPatternGreen, CFAPatternBlue, CFAPatternRed, CFAPatternGreen },\n> +\t\t.packScanline = packScanlineIPU3,\n> +\t\t.thumbScanline = thumbScanlineIPU3,\n> +\t} },\n> +\t{ PixelFormat(DRM_FORMAT_SGRBG10, IPU3_FORMAT_MOD_PACKED), {\n> +\t\t.bitsPerSample = 16,\n> +\t\t.pattern = { CFAPatternGreen, CFAPatternRed, CFAPatternBlue, CFAPatternGreen },\n> +\t\t.packScanline = packScanlineIPU3,\n> +\t\t.thumbScanline = thumbScanlineIPU3,\n> +\t} },\n> +\t{ PixelFormat(DRM_FORMAT_SRGGB10, IPU3_FORMAT_MOD_PACKED), {\n> +\t\t.bitsPerSample = 16,\n> +\t\t.pattern = { CFAPatternRed, CFAPatternGreen, CFAPatternGreen, CFAPatternBlue },\n> +\t\t.packScanline = packScanlineIPU3,\n> +\t\t.thumbScanline = thumbScanlineIPU3,\n> +\t} },\n> };\n> \n> int DNGWriter::write(const char *filename, const Camera *camera,","headers":{"Return-Path":"","Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 15C8D600F7\n\tfor ;\n\tThu, 11 Jun 2020 05:05:48 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8DCA526A;\n\tThu, 11 Jun 2020 05:05:47 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"ZP/3JpZq\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1591844747;\n\tbh=tVIaDzPPyQhwMYhgDfjFYkyMod4T7SDgV71912tiNbo=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=ZP/3JpZq9p9KcSAo5EXaX5rowMo6mIGWkDyrf+Sqzeh2ngI0Ihsp6IdgsI6M9gwso\n\tVZCas0XwDCXs8RtD0CJQkSQgmr8GwJkMspb/njwRyyHFTy1g/gMbj44I8v5Q0Ifgim\n\tYKk1G0nCUKci0W/LxRxKhhjA2hpLT1jFFJpfdsiw=","Date":"Thu, 11 Jun 2020 06:05:26 +0300","From":"Laurent Pinchart ","To":"Niklas =?utf-8?q?S=C3=B6derlund?= ","Cc":"libcamera-devel@lists.libcamera.org","Message-ID":"<20200611030526.GC13598@pendragon.ideasonboard.com>","References":"<20200611014536.1683233-1-niklas.soderlund@ragnatech.se>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20200611014536.1683233-1-niklas.soderlund@ragnatech.se>","Subject":"Re: [libcamera-devel] [PATCH v2] qcam: dng_writer: Add support for\n\tIPU3 Bayer formats","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Thu, 11 Jun 2020 03:05:48 -0000"}}]