[{"id":11735,"web_url":"https://patchwork.libcamera.org/comment/11735/","msgid":"<20200730203428.GE6107@pendragon.ideasonboard.com>","date":"2020-07-30T20:34:28","subject":"Re: [libcamera-devel] [PATCH v4 16/19] libcamera: ipu3: imgu:\n\tCalculate ImgU pipe configuration","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Jacopo,\n\nThank you for the patch.\n\nOn Mon, Jul 20, 2020 at 12:47:33PM +0200, Jacopo Mondi wrote:\n> Instrument the ImgU component to dynamically calculate the image\n> manipulation pipeline intermediate sizes.\n> \n> To correctly configure the ImgU it is necessary to program the IF, BDS\n> and GDC sizes, which are currently fixed to the input frame size.\n> \n> The procedure used to calculate the intermediate sizes has been ported\n> from the pipe_config.py python script, available at:\n> https://github.com/intel/intel-ipu3-pipecfg\n> at revision:\n> 61e83f2f7606 (\"Add more information into README\")\n> \n> Define two structures (ImgUDevice::Pipe and ImgUDevice::PipeConfig)\n> to allow the pipeline handler to supply and retrieve configuration\n> parameters from the ImgU.\n> \n> Finally, add a new operation to the ImgUDevice that calculates\n> the pipe configuration parameters based on the requested input and\n> output sizes.\n> \n> Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>\n> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>\n\nAs commented on the previous version, I think there's quite a bit of\nroom for improvement in this code, including fairly low hanging fruits.\nI however don't see a reason to block the feature, but please don't\nhesitate to add improvements on top (if I don't beat you to it). In\nparticular, anything that we don't understand properly needs to be\nlogged, and we need to request more information.\n\nAcked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n\n> ---\n>  src/libcamera/pipeline/ipu3/imgu.cpp | 349 +++++++++++++++++++++++++++\n>  src/libcamera/pipeline/ipu3/imgu.h   |  20 ++\n>  2 files changed, 369 insertions(+)\n> \n> diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp\n> index 4bec4b2f1a76..17b749435ee2 100644\n> --- a/src/libcamera/pipeline/ipu3/imgu.cpp\n> +++ b/src/libcamera/pipeline/ipu3/imgu.cpp\n> @@ -7,6 +7,9 @@\n>  \n>  #include \"imgu.h\"\n>  \n> +#include <limits>\n> +#include <math.h>\n> +\n>  #include <linux/media-bus-format.h>\n>  \n>  #include <libcamera/formats.h>\n> @@ -14,11 +17,300 @@\n>  \n>  #include \"libcamera/internal/log.h\"\n>  #include \"libcamera/internal/media_device.h\"\n> +#include \"libcamera/internal/utils.h\"\n>  \n>  namespace libcamera {\n>  \n>  LOG_DECLARE_CATEGORY(IPU3)\n>  \n> +namespace {\n> +\n> +static constexpr unsigned int FILTER_H = 4;\n> +\n> +static constexpr unsigned int IF_ALIGN_W = 2;\n> +static constexpr unsigned int IF_ALIGN_H = 4;\n> +\n> +static constexpr unsigned int BDS_ALIGN_W = 2;\n> +static constexpr unsigned int BDS_ALIGN_H = 4;\n> +\n> +static constexpr unsigned int IF_CROP_MAX_W = 40;\n> +static constexpr unsigned int IF_CROP_MAX_H = 540;\n> +\n> +static constexpr float BDS_SF_MAX = 2.5;\n> +static constexpr float BDS_SF_MIN = 1.0;\n> +static constexpr float BDS_SF_STEP = 0.03125;\n> +\n> +/* BSD scaling factors: min=1, max=2.5, step=1/32 */\n> +const std::vector<float> bdsScalingFactors = {\n> +\t1, 1.03125, 1.0625, 1.09375, 1.125, 1.15625, 1.1875, 1.21875, 1.25,\n> +\t1.28125, 1.3125, 1.34375, 1.375, 1.40625, 1.4375, 1.46875, 1.5, 1.53125,\n> +\t1.5625, 1.59375, 1.625, 1.65625, 1.6875, 1.71875, 1.75, 1.78125, 1.8125,\n> +\t1.84375, 1.875, 1.90625, 1.9375, 1.96875, 2, 2.03125, 2.0625, 2.09375,\n> +\t2.125, 2.15625, 2.1875, 2.21875, 2.25, 2.28125, 2.3125, 2.34375, 2.375,\n> +\t2.40625, 2.4375, 2.46875, 2.5\n> +};\n> +\n> +/* GDC scaling factors: min=1, max=16, step=1/4 */\n> +const std::vector<float> gdcScalingFactors = {\n> +\t1, 1.25, 1.5, 1.75, 2, 2.25, 2.5, 2.75, 3, 3.25, 3.5, 3.75, 4, 4.25,\n> +\t4.5, 4.75, 5, 5.25, 5.5, 5.75, 6, 6.25, 6.5, 6.75, 7, 7.25, 7.5, 7.75,\n> +\t8, 8.25, 8.5, 8.75, 9, 9.25, 9.5, 9.75, 10, 10.25, 10.5, 10.75, 11,\n> +\t11.25, 11.5, 11.75, 12, 12.25, 12.5, 12.75, 13, 13.25, 13.5, 13.75, 14,\n> +\t14.25, 14.5, 14.75, 15, 15.25, 15.5, 15.75, 16,\n> +};\n> +\n> +std::vector<ImgUDevice::PipeConfig> pipeConfigs;\n> +\n> +struct FOV {\n> +\tfloat w;\n> +\tfloat h;\n> +\n> +\tbool isLarger(const FOV &other)\n> +\t{\n> +\t\tif (w > other.w)\n> +\t\t\treturn true;\n> +\t\tif (w == other.w && h > other.h)\n> +\t\t\treturn true;\n> +\t\treturn false;\n> +\t}\n> +};\n> +\n> +/* Approximate a scaling factor sf to the closest one available in a range. */\n> +float findScaleFactor(float sf, const std::vector<float> &range,\n> +\t\t      bool roundDown = false)\n> +{\n> +\tif (sf <= range[0])\n> +\t\treturn range[0];\n> +\tif (sf >= range[range.size() - 1])\n> +\t\treturn range[range.size() - 1];\n> +\n> +\n> +\tfloat bestDiff = std::numeric_limits<float>::max();\n> +\tunsigned int index = 0;\n> +\tfor (unsigned int i = 0; i < range.size(); ++i) {\n> +\t\tfloat diff = std::abs(sf - range[i]);\n> +\t\tif (diff < bestDiff) {\n> +\t\t\tbestDiff = diff;\n> +\t\t\tindex = i;\n> +\t\t}\n> +\t}\n> +\n> +\tif (roundDown && index > 0 && sf < range[index])\n> +\t\tindex--;\n> +\n> +\treturn range[index];\n> +}\n> +\n> +bool isSameRatio(const Size &in, const Size &out)\n> +{\n> +\tfloat inRatio = static_cast<float>(in.width) /\n> +\t\t\tstatic_cast<float>(in.height);\n> +\tfloat outRatio = static_cast<float>(out.width) /\n> +\t\t\t static_cast<float>(out.height);\n> +\n> +\tif (std::abs(inRatio - outRatio) > 0.1)\n> +\t\treturn false;\n> +\n> +\treturn true;\n> +}\n> +\n> +void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc,\n> +\t\t\tunsigned int bdsWidth, float bdsSF)\n> +{\n> +\tunsigned int minIFHeight = iif.height - IF_CROP_MAX_H;\n> +\tunsigned int minBDSHeight = gdc.height + FILTER_H * 2;\n> +\tunsigned int ifHeight;\n> +\tfloat bdsHeight;\n> +\n> +\tif (!isSameRatio(pipe->input, gdc)) {\n> +\t\tbool found = false;\n> +\t\tfloat estIFHeight = static_cast<float>(iif.width *  gdc.height) /\n> +\t\t\t\t    static_cast<float>(gdc.width);\n> +\t\testIFHeight = utils::clamp<float>(estIFHeight, minIFHeight, iif.height);\n> +\t\tifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H);\n> +\t\twhile (ifHeight >= minIFHeight &&\n> +\t\t       static_cast<float>(ifHeight) / bdsSF >= minBDSHeight) {\n> +\t\t\tbdsHeight = static_cast<float>(ifHeight) / bdsSF;\n> +\t\t\tif (std::fmod(bdsHeight, 1.0) == 0) {\n> +\t\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n> +\t\t\t\tif (!(bdsIntHeight % BDS_ALIGN_H)) {\n> +\t\t\t\t\tfound = true;\n> +\t\t\t\t\tbreak;\n> +\t\t\t\t}\n> +\t\t\t}\n> +\n> +\t\t\tifHeight -= IF_ALIGN_H;\n> +\t\t}\n> +\n> +\t\tifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H);\n> +\t\twhile (ifHeight <= iif.height &&\n> +\t\t       static_cast<float>(ifHeight) / bdsSF >= minBDSHeight) {\n> +\t\t\tbdsHeight = static_cast<float>(ifHeight) / bdsSF;\n> +\t\t\tif (std::fmod(bdsHeight, 1.0) == 0) {\n> +\t\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n> +\t\t\t\tif (!(bdsIntHeight % BDS_ALIGN_H)) {\n> +\t\t\t\t\tfound = true;\n> +\t\t\t\t\tbreak;\n> +\t\t\t\t}\n> +\t\t\t}\n> +\n> +\t\t\tifHeight += IF_ALIGN_H;\n> +\t\t}\n> +\n> +\t\tif (found) {\n> +\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n> +\t\t\tpipeConfigs.push_back({ bdsSF, { iif.width, ifHeight },\n> +\t\t\t\t\t\t{ bdsWidth, bdsIntHeight }, gdc });\n> +\t\t\treturn;\n> +\t\t}\n> +\t} else {\n> +\t\tifHeight = utils::alignUp(iif.height, IF_ALIGN_H);\n> +\t\twhile (ifHeight > minIFHeight &&\n> +\t\t       static_cast<float>(ifHeight) / bdsSF >= minBDSHeight) {\n> +\t\t\tbdsHeight = ifHeight / bdsSF;\n> +\t\t\tif (std::fmod(ifHeight, 1.0) == 0 && std::fmod(bdsHeight, 1.0) == 0) {\n> +\t\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n> +\n> +\t\t\t\tif (!(ifHeight % IF_ALIGN_H) &&\n> +\t\t\t\t    !(bdsIntHeight % BDS_ALIGN_H)) {\n> +\t\t\t\t\tpipeConfigs.push_back({ bdsSF, { iif.width, ifHeight },\n> +\t\t\t\t\t\t\t\t{ bdsWidth, bdsIntHeight }, gdc });\n> +\t\t\t\t}\n> +\t\t\t}\n> +\n> +\t\t\tifHeight -= IF_ALIGN_H;\n> +\t\t}\n> +\t}\n> +}\n> +\n> +void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc,\n> +\t\t  float bdsSF)\n> +{\n> +\tunsigned int minBDSWidth = gdc.width + FILTER_H * 2;\n> +\n> +\tfloat sf = bdsSF;\n> +\twhile (sf <= BDS_SF_MAX && sf >= BDS_SF_MIN) {\n> +\t\tfloat bdsWidth = static_cast<float>(iif.width) / sf;\n> +\n> +\t\tif (std::fmod(bdsWidth, 1.0) == 0) {\n> +\t\t\tunsigned int bdsIntWidth = static_cast<unsigned int>(bdsWidth);\n> +\t\t\tif (!(bdsIntWidth % BDS_ALIGN_W) && bdsWidth >= minBDSWidth)\n> +\t\t\t\tcalculateBDSHeight(pipe, iif, gdc, bdsIntWidth, sf);\n> +\t\t}\n> +\n> +\t\tsf += BDS_SF_STEP;\n> +\t}\n> +\n> +\tsf = bdsSF;\n> +\twhile (sf <= BDS_SF_MAX && sf >= BDS_SF_MIN) {\n> +\t\tfloat bdsWidth = static_cast<float>(iif.width) / sf;\n> +\n> +\t\tif (std::fmod(bdsWidth, 1.0) == 0) {\n> +\t\t\tunsigned int bdsIntWidth = static_cast<unsigned int>(bdsWidth);\n> +\t\t\tif (!(bdsIntWidth % BDS_ALIGN_W) && bdsWidth >= minBDSWidth)\n> +\t\t\t\tcalculateBDSHeight(pipe, iif, gdc, bdsIntWidth, sf);\n> +\t\t}\n> +\n> +\t\tsf -= BDS_SF_STEP;\n> +\t}\n> +}\n> +\n> +Size calculateGDC(ImgUDevice::Pipe *pipe)\n> +{\n> +\tconst Size &in = pipe->input;\n> +\tconst Size &main = pipe->output;\n> +\tconst Size &vf = pipe->viewfinder;\n> +\tSize gdc;\n> +\n> +\tif (!vf.isNull()) {\n> +\t\tgdc.width = main.width;\n> +\n> +\t\tfloat ratio = static_cast<float>(main.width * vf.height) /\n> +\t\t\t      static_cast<float>(vf.width);\n> +\t\tgdc.height = std::max(static_cast<float>(main.height), ratio);\n> +\n> +\t\treturn gdc;\n> +\t}\n> +\n> +\tif (!isSameRatio(in, main)) {\n> +\t\tgdc = main;\n> +\t\treturn gdc;\n> +\t}\n> +\n> +\tfloat totalSF = static_cast<float>(in.width) /\n> +\t\t\tstatic_cast<float>(main.width);\n> +\tfloat bdsSF = totalSF > 2 ? 2 : 1;\n> +\tfloat yuvSF = totalSF / bdsSF;\n> +\tfloat sf = findScaleFactor(yuvSF, gdcScalingFactors);\n> +\n> +\tgdc.width = static_cast<float>(main.width) * sf;\n> +\tgdc.height = static_cast<float>(main.height) * sf;\n> +\n> +\treturn gdc;\n> +}\n> +\n> +FOV calcFOV(const Size &in, const ImgUDevice::PipeConfig &pipe)\n> +{\n> +\tFOV fov{};\n> +\n> +\tfloat inW = static_cast<float>(in.width);\n> +\tfloat inH = static_cast<float>(in.height);\n> +\tfloat ifCropW = static_cast<float>(in.width - pipe.iif.width);\n> +\tfloat ifCropH = static_cast<float>(in.height - pipe.iif.height);\n> +\tfloat gdcCropW = static_cast<float>(pipe.bds.width - pipe.gdc.width) * pipe.bds_sf;\n> +\tfloat gdcCropH = static_cast<float>(pipe.bds.height - pipe.gdc.height) * pipe.bds_sf;\n> +\tfov.w = (inW - (ifCropW + gdcCropW)) / inW;\n> +\tfov.h = (inH - (ifCropH + gdcCropH)) / inH;\n> +\n> +\treturn fov;\n> +}\n> +\n> +} /* namespace */\n> +\n> +/**\n> + * \\struct PipeConfig\n> + * \\brief The ImgU pipe configuration parameters\n> + *\n> + * The ImgU image pipeline is composed of several hardware blocks that crop\n> + * and scale the input image to obtain the desired output sizes. The\n> + * scaling/cropping operations of those components is configured though the\n> + * V4L2 selection API and the V4L2 subdev API applied to the ImgU media entity.\n> + *\n> + * The configurable components in the pipeline are:\n> + * - IF: image feeder\n> + * - BDS: bayer downscaler\n> + * - GDC: geometric distorsion correction\n> + *\n> + * The IF crop rectangle is controlled by the V4L2_SEL_TGT_CROP selection target\n> + * applied to the ImgU media entity sink pad number 0. The BDS scaler is\n> + * controlled by the V4L2_SEL_TGT_COMPOSE target on the same pad, while the GDC\n> + * output size is configured with the VIDIOC_SUBDEV_S_FMT IOCTL, again on pad\n> + * number 0.\n> + *\n> + * The PipeConfig structure collects the sizes of each of those components\n> + * plus the BDS scaling factor used to calculate the field of view\n> + * of the final images.\n> + */\n> +\n> +/**\n> + * \\struct Pipe\n> + * \\brief Describe the ImgU requested configuration\n> + *\n> + * The ImgU unit processes images through several components, which have\n> + * to be properly configured inspecting the input image size and the desired\n> + * output sizes. This structure collects the ImgU input configuration and the\n> + * requested main output and viewfinder configurations.\n> + *\n> + * \\var Pipe::input\n> + * \\brief The input image size\n> + *\n> + * \\var Pipe::output\n> + * \\brief The requested main output size\n> + *\n> + * \\var Pipe::viewfinder\n> + * \\brief The requested viewfinder size\n> + */\n> +\n>  /**\n>   * \\brief Initialize components of the ImgU instance\n>   * \\param[in] mediaDevice The ImgU instance media device\n> @@ -74,6 +366,63 @@ int ImgUDevice::init(MediaDevice *media, unsigned int index)\n>  \treturn 0;\n>  }\n>  \n> +/**\n> + * \\brief Calculate the ImgU pipe configuration parameters\n> + * \\param[in] pipe The requested ImgU configuration\n> + * \\return An ImgUDevice::PipeConfig instance on success, an empty configuration\n> + * otherwise\n> + */\n> +ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe)\n> +{\n> +\tpipeConfigs.clear();\n> +\n> +\tLOG(IPU3, Debug) << \"Calculating pipe configuration for: \";\n> +\tLOG(IPU3, Debug) << \"input: \" << pipe->input.toString();\n> +\tLOG(IPU3, Debug) << \"main: \" << pipe->output.toString();\n> +\tLOG(IPU3, Debug) << \"vf: \" << pipe->viewfinder.toString();\n> +\n> +\tconst Size &in = pipe->input;\n> +\tSize gdc = calculateGDC(pipe);\n> +\n> +\tunsigned int ifWidth = utils::alignUp(in.width, IF_ALIGN_W);\n> +\tunsigned int ifHeight = in.height;\n> +\tunsigned int minIfWidth = in.width - IF_CROP_MAX_W;\n> +\tfloat bdsSF = static_cast<float>(in.width) /\n> +\t\t      static_cast<float>(gdc.width);\n> +\tfloat sf = findScaleFactor(bdsSF, bdsScalingFactors, true);\n> +\twhile (ifWidth >= minIfWidth) {\n> +\t\tSize iif{ ifWidth, ifHeight };\n> +\t\tcalculateBDS(pipe, iif, gdc, sf);\n> +\n> +\t\tifWidth -= IF_ALIGN_W;\n> +\t}\n> +\n> +\tif (pipeConfigs.size() == 0) {\n> +\t\tLOG(IPU3, Error) << \"Failed to calculate pipe configuration\";\n> +\t\treturn {};\n> +\t}\n> +\n> +\tFOV bestFov = calcFOV(pipe->input, pipeConfigs[0]);\n> +\tunsigned int bestIndex = 0;\n> +\tunsigned int p = 0;\n> +\tfor (auto pipeConfig : pipeConfigs) {\n> +\t\tFOV fov = calcFOV(pipe->input, pipeConfig);\n> +\t\tif (fov.isLarger(bestFov)) {\n> +\t\t\tbestFov = fov;\n> +\t\t\tbestIndex = p;\n> +\t\t}\n> +\n> +\t\t++p;\n> +\t}\n> +\n> +\tLOG(IPU3, Debug) << \"Computed pipe configuration: \";\n> +\tLOG(IPU3, Debug) << \"IF: \" << pipeConfigs[bestIndex].iif.toString();\n> +\tLOG(IPU3, Debug) << \"BDS: \" << pipeConfigs[bestIndex].bds.toString();\n> +\tLOG(IPU3, Debug) << \"GDC: \" << pipeConfigs[bestIndex].gdc.toString();\n> +\n> +\treturn pipeConfigs[bestIndex];\n> +}\n> +\n>  /**\n>   * \\brief Configure the ImgU unit input\n>   * \\param[in] size The ImgU input frame size\n> diff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h\n> index 23ec1ca1c6ae..0b2dd1f965c1 100644\n> --- a/src/libcamera/pipeline/ipu3/imgu.h\n> +++ b/src/libcamera/pipeline/ipu3/imgu.h\n> @@ -23,8 +23,28 @@ struct StreamConfiguration;\n>  class ImgUDevice\n>  {\n>  public:\n> +\tstruct PipeConfig {\n> +\t\tfloat bds_sf;\n> +\t\tSize iif;\n> +\t\tSize bds;\n> +\t\tSize gdc;\n> +\n> +\t\tbool isNull() const\n> +\t\t{\n> +\t\t\treturn iif.isNull() || bds.isNull() || gdc.isNull();\n> +\t\t}\n> +\t};\n> +\n> +\tstruct Pipe {\n> +\t\tSize input;\n> +\t\tSize output;\n> +\t\tSize viewfinder;\n> +\t};\n> +\n>  \tint init(MediaDevice *media, unsigned int index);\n>  \n> +\tPipeConfig calculatePipeConfig(Pipe *pipe);\n> +\n>  \tint configureInput(const Size &size, V4L2DeviceFormat *inputFormat);\n>  \n>  \tint configureOutput(const StreamConfiguration &cfg,","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id EEC53BD878\n\tfor <parsemail@patchwork.libcamera.org>;\n\tThu, 30 Jul 2020 20:34:44 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5A13561988;\n\tThu, 30 Jul 2020 22:34:44 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 014D3611A2\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tThu, 30 Jul 2020 22:34:42 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id CBADB9B1;\n\tThu, 30 Jul 2020 22:34:37 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"H0ZwGAXm\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1596141278;\n\tbh=LjudtbYFBagTUQyw3d3/hv8d7IHdt2JhVmDZiR3j4eQ=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=H0ZwGAXmS9C8hX+Jc5R432caEggeY/S68fVk3nQ3kJFEFvYjoNhvNuYorkywDoq4Q\n\tqMfodkZgp2fK1y1qr8UHd25RO1Mf52YeW+yOlYwj2ClMJzgMagLlJs4UQpnAG1n5GG\n\tervr6jIhP8/IN7gGDJrNgoL4GB8/HOzzlTXcEkDs=","Date":"Thu, 30 Jul 2020 23:34:28 +0300","From":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","To":"Jacopo Mondi <jacopo@jmondi.org>","Message-ID":"<20200730203428.GE6107@pendragon.ideasonboard.com>","References":"<20200720104736.19986-1-jacopo@jmondi.org>\n\t<20200720104736.19986-17-jacopo@jmondi.org>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200720104736.19986-17-jacopo@jmondi.org>","Subject":"Re: [libcamera-devel] [PATCH v4 16/19] libcamera: ipu3: imgu:\n\tCalculate ImgU pipe configuration","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":11744,"web_url":"https://patchwork.libcamera.org/comment/11744/","msgid":"<20200731092417.545cof5jk63g2eeo@uno.localdomain>","date":"2020-07-31T09:24:17","subject":"Re: [libcamera-devel] [PATCH v4 16/19] libcamera: ipu3: imgu:\n\tCalculate ImgU pipe configuration","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Laurent,\n\nOn Thu, Jul 30, 2020 at 11:34:28PM +0300, Laurent Pinchart wrote:\n> Hi Jacopo,\n>\n> Thank you for the patch.\n>\n> On Mon, Jul 20, 2020 at 12:47:33PM +0200, Jacopo Mondi wrote:\n> > Instrument the ImgU component to dynamically calculate the image\n> > manipulation pipeline intermediate sizes.\n> >\n> > To correctly configure the ImgU it is necessary to program the IF, BDS\n> > and GDC sizes, which are currently fixed to the input frame size.\n> >\n> > The procedure used to calculate the intermediate sizes has been ported\n> > from the pipe_config.py python script, available at:\n> > https://github.com/intel/intel-ipu3-pipecfg\n> > at revision:\n> > 61e83f2f7606 (\"Add more information into README\")\n> >\n> > Define two structures (ImgUDevice::Pipe and ImgUDevice::PipeConfig)\n> > to allow the pipeline handler to supply and retrieve configuration\n> > parameters from the ImgU.\n> >\n> > Finally, add a new operation to the ImgUDevice that calculates\n> > the pipe configuration parameters based on the requested input and\n> > output sizes.\n> >\n> > Reviewed-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>\n> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>\n>\n> As commented on the previous version, I think there's quite a bit of\n> room for improvement in this code, including fairly low hanging fruits.\n\nYou are right, I was scared to change this quite fragile\nimplementation and exausted by this series, but looking at the code\nafter a few weeks I realize I can at least try to make it a bit nicer\nto read, expecially the function used to calculate the BDS scaling\nfactor, which is quite dense.\n\nI'll also add a few comments, here and on patch #13, so I'll send out\na v5 anyway.\n\nThanks\n  j\n\n> I however don't see a reason to block the feature, but please don't\n> hesitate to add improvements on top (if I don't beat you to it). In\n> particular, anything that we don't understand properly needs to be\n> logged, and we need to request more information.\n>\n> Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n>\n> > ---\n> >  src/libcamera/pipeline/ipu3/imgu.cpp | 349 +++++++++++++++++++++++++++\n> >  src/libcamera/pipeline/ipu3/imgu.h   |  20 ++\n> >  2 files changed, 369 insertions(+)\n> >\n> > diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp\n> > index 4bec4b2f1a76..17b749435ee2 100644\n> > --- a/src/libcamera/pipeline/ipu3/imgu.cpp\n> > +++ b/src/libcamera/pipeline/ipu3/imgu.cpp\n> > @@ -7,6 +7,9 @@\n> >\n> >  #include \"imgu.h\"\n> >\n> > +#include <limits>\n> > +#include <math.h>\n> > +\n> >  #include <linux/media-bus-format.h>\n> >\n> >  #include <libcamera/formats.h>\n> > @@ -14,11 +17,300 @@\n> >\n> >  #include \"libcamera/internal/log.h\"\n> >  #include \"libcamera/internal/media_device.h\"\n> > +#include \"libcamera/internal/utils.h\"\n> >\n> >  namespace libcamera {\n> >\n> >  LOG_DECLARE_CATEGORY(IPU3)\n> >\n> > +namespace {\n> > +\n> > +static constexpr unsigned int FILTER_H = 4;\n> > +\n> > +static constexpr unsigned int IF_ALIGN_W = 2;\n> > +static constexpr unsigned int IF_ALIGN_H = 4;\n> > +\n> > +static constexpr unsigned int BDS_ALIGN_W = 2;\n> > +static constexpr unsigned int BDS_ALIGN_H = 4;\n> > +\n> > +static constexpr unsigned int IF_CROP_MAX_W = 40;\n> > +static constexpr unsigned int IF_CROP_MAX_H = 540;\n> > +\n> > +static constexpr float BDS_SF_MAX = 2.5;\n> > +static constexpr float BDS_SF_MIN = 1.0;\n> > +static constexpr float BDS_SF_STEP = 0.03125;\n> > +\n> > +/* BSD scaling factors: min=1, max=2.5, step=1/32 */\n> > +const std::vector<float> bdsScalingFactors = {\n> > +\t1, 1.03125, 1.0625, 1.09375, 1.125, 1.15625, 1.1875, 1.21875, 1.25,\n> > +\t1.28125, 1.3125, 1.34375, 1.375, 1.40625, 1.4375, 1.46875, 1.5, 1.53125,\n> > +\t1.5625, 1.59375, 1.625, 1.65625, 1.6875, 1.71875, 1.75, 1.78125, 1.8125,\n> > +\t1.84375, 1.875, 1.90625, 1.9375, 1.96875, 2, 2.03125, 2.0625, 2.09375,\n> > +\t2.125, 2.15625, 2.1875, 2.21875, 2.25, 2.28125, 2.3125, 2.34375, 2.375,\n> > +\t2.40625, 2.4375, 2.46875, 2.5\n> > +};\n> > +\n> > +/* GDC scaling factors: min=1, max=16, step=1/4 */\n> > +const std::vector<float> gdcScalingFactors = {\n> > +\t1, 1.25, 1.5, 1.75, 2, 2.25, 2.5, 2.75, 3, 3.25, 3.5, 3.75, 4, 4.25,\n> > +\t4.5, 4.75, 5, 5.25, 5.5, 5.75, 6, 6.25, 6.5, 6.75, 7, 7.25, 7.5, 7.75,\n> > +\t8, 8.25, 8.5, 8.75, 9, 9.25, 9.5, 9.75, 10, 10.25, 10.5, 10.75, 11,\n> > +\t11.25, 11.5, 11.75, 12, 12.25, 12.5, 12.75, 13, 13.25, 13.5, 13.75, 14,\n> > +\t14.25, 14.5, 14.75, 15, 15.25, 15.5, 15.75, 16,\n> > +};\n> > +\n> > +std::vector<ImgUDevice::PipeConfig> pipeConfigs;\n> > +\n> > +struct FOV {\n> > +\tfloat w;\n> > +\tfloat h;\n> > +\n> > +\tbool isLarger(const FOV &other)\n> > +\t{\n> > +\t\tif (w > other.w)\n> > +\t\t\treturn true;\n> > +\t\tif (w == other.w && h > other.h)\n> > +\t\t\treturn true;\n> > +\t\treturn false;\n> > +\t}\n> > +};\n> > +\n> > +/* Approximate a scaling factor sf to the closest one available in a range. */\n> > +float findScaleFactor(float sf, const std::vector<float> &range,\n> > +\t\t      bool roundDown = false)\n> > +{\n> > +\tif (sf <= range[0])\n> > +\t\treturn range[0];\n> > +\tif (sf >= range[range.size() - 1])\n> > +\t\treturn range[range.size() - 1];\n> > +\n> > +\n> > +\tfloat bestDiff = std::numeric_limits<float>::max();\n> > +\tunsigned int index = 0;\n> > +\tfor (unsigned int i = 0; i < range.size(); ++i) {\n> > +\t\tfloat diff = std::abs(sf - range[i]);\n> > +\t\tif (diff < bestDiff) {\n> > +\t\t\tbestDiff = diff;\n> > +\t\t\tindex = i;\n> > +\t\t}\n> > +\t}\n> > +\n> > +\tif (roundDown && index > 0 && sf < range[index])\n> > +\t\tindex--;\n> > +\n> > +\treturn range[index];\n> > +}\n> > +\n> > +bool isSameRatio(const Size &in, const Size &out)\n> > +{\n> > +\tfloat inRatio = static_cast<float>(in.width) /\n> > +\t\t\tstatic_cast<float>(in.height);\n> > +\tfloat outRatio = static_cast<float>(out.width) /\n> > +\t\t\t static_cast<float>(out.height);\n> > +\n> > +\tif (std::abs(inRatio - outRatio) > 0.1)\n> > +\t\treturn false;\n> > +\n> > +\treturn true;\n> > +}\n> > +\n> > +void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc,\n> > +\t\t\tunsigned int bdsWidth, float bdsSF)\n> > +{\n> > +\tunsigned int minIFHeight = iif.height - IF_CROP_MAX_H;\n> > +\tunsigned int minBDSHeight = gdc.height + FILTER_H * 2;\n> > +\tunsigned int ifHeight;\n> > +\tfloat bdsHeight;\n> > +\n> > +\tif (!isSameRatio(pipe->input, gdc)) {\n> > +\t\tbool found = false;\n> > +\t\tfloat estIFHeight = static_cast<float>(iif.width *  gdc.height) /\n> > +\t\t\t\t    static_cast<float>(gdc.width);\n> > +\t\testIFHeight = utils::clamp<float>(estIFHeight, minIFHeight, iif.height);\n> > +\t\tifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H);\n> > +\t\twhile (ifHeight >= minIFHeight &&\n> > +\t\t       static_cast<float>(ifHeight) / bdsSF >= minBDSHeight) {\n> > +\t\t\tbdsHeight = static_cast<float>(ifHeight) / bdsSF;\n> > +\t\t\tif (std::fmod(bdsHeight, 1.0) == 0) {\n> > +\t\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n> > +\t\t\t\tif (!(bdsIntHeight % BDS_ALIGN_H)) {\n> > +\t\t\t\t\tfound = true;\n> > +\t\t\t\t\tbreak;\n> > +\t\t\t\t}\n> > +\t\t\t}\n> > +\n> > +\t\t\tifHeight -= IF_ALIGN_H;\n> > +\t\t}\n> > +\n> > +\t\tifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H);\n> > +\t\twhile (ifHeight <= iif.height &&\n> > +\t\t       static_cast<float>(ifHeight) / bdsSF >= minBDSHeight) {\n> > +\t\t\tbdsHeight = static_cast<float>(ifHeight) / bdsSF;\n> > +\t\t\tif (std::fmod(bdsHeight, 1.0) == 0) {\n> > +\t\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n> > +\t\t\t\tif (!(bdsIntHeight % BDS_ALIGN_H)) {\n> > +\t\t\t\t\tfound = true;\n> > +\t\t\t\t\tbreak;\n> > +\t\t\t\t}\n> > +\t\t\t}\n> > +\n> > +\t\t\tifHeight += IF_ALIGN_H;\n> > +\t\t}\n> > +\n> > +\t\tif (found) {\n> > +\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n> > +\t\t\tpipeConfigs.push_back({ bdsSF, { iif.width, ifHeight },\n> > +\t\t\t\t\t\t{ bdsWidth, bdsIntHeight }, gdc });\n> > +\t\t\treturn;\n> > +\t\t}\n> > +\t} else {\n> > +\t\tifHeight = utils::alignUp(iif.height, IF_ALIGN_H);\n> > +\t\twhile (ifHeight > minIFHeight &&\n> > +\t\t       static_cast<float>(ifHeight) / bdsSF >= minBDSHeight) {\n> > +\t\t\tbdsHeight = ifHeight / bdsSF;\n> > +\t\t\tif (std::fmod(ifHeight, 1.0) == 0 && std::fmod(bdsHeight, 1.0) == 0) {\n> > +\t\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n> > +\n> > +\t\t\t\tif (!(ifHeight % IF_ALIGN_H) &&\n> > +\t\t\t\t    !(bdsIntHeight % BDS_ALIGN_H)) {\n> > +\t\t\t\t\tpipeConfigs.push_back({ bdsSF, { iif.width, ifHeight },\n> > +\t\t\t\t\t\t\t\t{ bdsWidth, bdsIntHeight }, gdc });\n> > +\t\t\t\t}\n> > +\t\t\t}\n> > +\n> > +\t\t\tifHeight -= IF_ALIGN_H;\n> > +\t\t}\n> > +\t}\n> > +}\n> > +\n> > +void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc,\n> > +\t\t  float bdsSF)\n> > +{\n> > +\tunsigned int minBDSWidth = gdc.width + FILTER_H * 2;\n> > +\n> > +\tfloat sf = bdsSF;\n> > +\twhile (sf <= BDS_SF_MAX && sf >= BDS_SF_MIN) {\n> > +\t\tfloat bdsWidth = static_cast<float>(iif.width) / sf;\n> > +\n> > +\t\tif (std::fmod(bdsWidth, 1.0) == 0) {\n> > +\t\t\tunsigned int bdsIntWidth = static_cast<unsigned int>(bdsWidth);\n> > +\t\t\tif (!(bdsIntWidth % BDS_ALIGN_W) && bdsWidth >= minBDSWidth)\n> > +\t\t\t\tcalculateBDSHeight(pipe, iif, gdc, bdsIntWidth, sf);\n> > +\t\t}\n> > +\n> > +\t\tsf += BDS_SF_STEP;\n> > +\t}\n> > +\n> > +\tsf = bdsSF;\n> > +\twhile (sf <= BDS_SF_MAX && sf >= BDS_SF_MIN) {\n> > +\t\tfloat bdsWidth = static_cast<float>(iif.width) / sf;\n> > +\n> > +\t\tif (std::fmod(bdsWidth, 1.0) == 0) {\n> > +\t\t\tunsigned int bdsIntWidth = static_cast<unsigned int>(bdsWidth);\n> > +\t\t\tif (!(bdsIntWidth % BDS_ALIGN_W) && bdsWidth >= minBDSWidth)\n> > +\t\t\t\tcalculateBDSHeight(pipe, iif, gdc, bdsIntWidth, sf);\n> > +\t\t}\n> > +\n> > +\t\tsf -= BDS_SF_STEP;\n> > +\t}\n> > +}\n> > +\n> > +Size calculateGDC(ImgUDevice::Pipe *pipe)\n> > +{\n> > +\tconst Size &in = pipe->input;\n> > +\tconst Size &main = pipe->output;\n> > +\tconst Size &vf = pipe->viewfinder;\n> > +\tSize gdc;\n> > +\n> > +\tif (!vf.isNull()) {\n> > +\t\tgdc.width = main.width;\n> > +\n> > +\t\tfloat ratio = static_cast<float>(main.width * vf.height) /\n> > +\t\t\t      static_cast<float>(vf.width);\n> > +\t\tgdc.height = std::max(static_cast<float>(main.height), ratio);\n> > +\n> > +\t\treturn gdc;\n> > +\t}\n> > +\n> > +\tif (!isSameRatio(in, main)) {\n> > +\t\tgdc = main;\n> > +\t\treturn gdc;\n> > +\t}\n> > +\n> > +\tfloat totalSF = static_cast<float>(in.width) /\n> > +\t\t\tstatic_cast<float>(main.width);\n> > +\tfloat bdsSF = totalSF > 2 ? 2 : 1;\n> > +\tfloat yuvSF = totalSF / bdsSF;\n> > +\tfloat sf = findScaleFactor(yuvSF, gdcScalingFactors);\n> > +\n> > +\tgdc.width = static_cast<float>(main.width) * sf;\n> > +\tgdc.height = static_cast<float>(main.height) * sf;\n> > +\n> > +\treturn gdc;\n> > +}\n> > +\n> > +FOV calcFOV(const Size &in, const ImgUDevice::PipeConfig &pipe)\n> > +{\n> > +\tFOV fov{};\n> > +\n> > +\tfloat inW = static_cast<float>(in.width);\n> > +\tfloat inH = static_cast<float>(in.height);\n> > +\tfloat ifCropW = static_cast<float>(in.width - pipe.iif.width);\n> > +\tfloat ifCropH = static_cast<float>(in.height - pipe.iif.height);\n> > +\tfloat gdcCropW = static_cast<float>(pipe.bds.width - pipe.gdc.width) * pipe.bds_sf;\n> > +\tfloat gdcCropH = static_cast<float>(pipe.bds.height - pipe.gdc.height) * pipe.bds_sf;\n> > +\tfov.w = (inW - (ifCropW + gdcCropW)) / inW;\n> > +\tfov.h = (inH - (ifCropH + gdcCropH)) / inH;\n> > +\n> > +\treturn fov;\n> > +}\n> > +\n> > +} /* namespace */\n> > +\n> > +/**\n> > + * \\struct PipeConfig\n> > + * \\brief The ImgU pipe configuration parameters\n> > + *\n> > + * The ImgU image pipeline is composed of several hardware blocks that crop\n> > + * and scale the input image to obtain the desired output sizes. The\n> > + * scaling/cropping operations of those components is configured though the\n> > + * V4L2 selection API and the V4L2 subdev API applied to the ImgU media entity.\n> > + *\n> > + * The configurable components in the pipeline are:\n> > + * - IF: image feeder\n> > + * - BDS: bayer downscaler\n> > + * - GDC: geometric distorsion correction\n> > + *\n> > + * The IF crop rectangle is controlled by the V4L2_SEL_TGT_CROP selection target\n> > + * applied to the ImgU media entity sink pad number 0. The BDS scaler is\n> > + * controlled by the V4L2_SEL_TGT_COMPOSE target on the same pad, while the GDC\n> > + * output size is configured with the VIDIOC_SUBDEV_S_FMT IOCTL, again on pad\n> > + * number 0.\n> > + *\n> > + * The PipeConfig structure collects the sizes of each of those components\n> > + * plus the BDS scaling factor used to calculate the field of view\n> > + * of the final images.\n> > + */\n> > +\n> > +/**\n> > + * \\struct Pipe\n> > + * \\brief Describe the ImgU requested configuration\n> > + *\n> > + * The ImgU unit processes images through several components, which have\n> > + * to be properly configured inspecting the input image size and the desired\n> > + * output sizes. This structure collects the ImgU input configuration and the\n> > + * requested main output and viewfinder configurations.\n> > + *\n> > + * \\var Pipe::input\n> > + * \\brief The input image size\n> > + *\n> > + * \\var Pipe::output\n> > + * \\brief The requested main output size\n> > + *\n> > + * \\var Pipe::viewfinder\n> > + * \\brief The requested viewfinder size\n> > + */\n> > +\n> >  /**\n> >   * \\brief Initialize components of the ImgU instance\n> >   * \\param[in] mediaDevice The ImgU instance media device\n> > @@ -74,6 +366,63 @@ int ImgUDevice::init(MediaDevice *media, unsigned int index)\n> >  \treturn 0;\n> >  }\n> >\n> > +/**\n> > + * \\brief Calculate the ImgU pipe configuration parameters\n> > + * \\param[in] pipe The requested ImgU configuration\n> > + * \\return An ImgUDevice::PipeConfig instance on success, an empty configuration\n> > + * otherwise\n> > + */\n> > +ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe)\n> > +{\n> > +\tpipeConfigs.clear();\n> > +\n> > +\tLOG(IPU3, Debug) << \"Calculating pipe configuration for: \";\n> > +\tLOG(IPU3, Debug) << \"input: \" << pipe->input.toString();\n> > +\tLOG(IPU3, Debug) << \"main: \" << pipe->output.toString();\n> > +\tLOG(IPU3, Debug) << \"vf: \" << pipe->viewfinder.toString();\n> > +\n> > +\tconst Size &in = pipe->input;\n> > +\tSize gdc = calculateGDC(pipe);\n> > +\n> > +\tunsigned int ifWidth = utils::alignUp(in.width, IF_ALIGN_W);\n> > +\tunsigned int ifHeight = in.height;\n> > +\tunsigned int minIfWidth = in.width - IF_CROP_MAX_W;\n> > +\tfloat bdsSF = static_cast<float>(in.width) /\n> > +\t\t      static_cast<float>(gdc.width);\n> > +\tfloat sf = findScaleFactor(bdsSF, bdsScalingFactors, true);\n> > +\twhile (ifWidth >= minIfWidth) {\n> > +\t\tSize iif{ ifWidth, ifHeight };\n> > +\t\tcalculateBDS(pipe, iif, gdc, sf);\n> > +\n> > +\t\tifWidth -= IF_ALIGN_W;\n> > +\t}\n> > +\n> > +\tif (pipeConfigs.size() == 0) {\n> > +\t\tLOG(IPU3, Error) << \"Failed to calculate pipe configuration\";\n> > +\t\treturn {};\n> > +\t}\n> > +\n> > +\tFOV bestFov = calcFOV(pipe->input, pipeConfigs[0]);\n> > +\tunsigned int bestIndex = 0;\n> > +\tunsigned int p = 0;\n> > +\tfor (auto pipeConfig : pipeConfigs) {\n> > +\t\tFOV fov = calcFOV(pipe->input, pipeConfig);\n> > +\t\tif (fov.isLarger(bestFov)) {\n> > +\t\t\tbestFov = fov;\n> > +\t\t\tbestIndex = p;\n> > +\t\t}\n> > +\n> > +\t\t++p;\n> > +\t}\n> > +\n> > +\tLOG(IPU3, Debug) << \"Computed pipe configuration: \";\n> > +\tLOG(IPU3, Debug) << \"IF: \" << pipeConfigs[bestIndex].iif.toString();\n> > +\tLOG(IPU3, Debug) << \"BDS: \" << pipeConfigs[bestIndex].bds.toString();\n> > +\tLOG(IPU3, Debug) << \"GDC: \" << pipeConfigs[bestIndex].gdc.toString();\n> > +\n> > +\treturn pipeConfigs[bestIndex];\n> > +}\n> > +\n> >  /**\n> >   * \\brief Configure the ImgU unit input\n> >   * \\param[in] size The ImgU input frame size\n> > diff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h\n> > index 23ec1ca1c6ae..0b2dd1f965c1 100644\n> > --- a/src/libcamera/pipeline/ipu3/imgu.h\n> > +++ b/src/libcamera/pipeline/ipu3/imgu.h\n> > @@ -23,8 +23,28 @@ struct StreamConfiguration;\n> >  class ImgUDevice\n> >  {\n> >  public:\n> > +\tstruct PipeConfig {\n> > +\t\tfloat bds_sf;\n> > +\t\tSize iif;\n> > +\t\tSize bds;\n> > +\t\tSize gdc;\n> > +\n> > +\t\tbool isNull() const\n> > +\t\t{\n> > +\t\t\treturn iif.isNull() || bds.isNull() || gdc.isNull();\n> > +\t\t}\n> > +\t};\n> > +\n> > +\tstruct Pipe {\n> > +\t\tSize input;\n> > +\t\tSize output;\n> > +\t\tSize viewfinder;\n> > +\t};\n> > +\n> >  \tint init(MediaDevice *media, unsigned int index);\n> >\n> > +\tPipeConfig calculatePipeConfig(Pipe *pipe);\n> > +\n> >  \tint configureInput(const Size &size, V4L2DeviceFormat *inputFormat);\n> >\n> >  \tint configureOutput(const StreamConfiguration &cfg,\n>\n> --\n> Regards,\n>\n> Laurent Pinchart","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C7865BD878\n\tfor <parsemail@patchwork.libcamera.org>;\n\tFri, 31 Jul 2020 09:20:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2F69661988;\n\tFri, 31 Jul 2020 11:20:40 +0200 (CEST)","from relay11.mail.gandi.net (relay11.mail.gandi.net\n\t[217.70.178.231])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0072C60395\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri, 31 Jul 2020 11:20:38 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay11.mail.gandi.net (Postfix) with ESMTPSA id 3D53B100007;\n\tFri, 31 Jul 2020 09:20:37 +0000 (UTC)"],"Date":"Fri, 31 Jul 2020 11:24:17 +0200","From":"Jacopo Mondi <jacopo@jmondi.org>","To":"Laurent Pinchart <laurent.pinchart@ideasonboard.com>","Message-ID":"<20200731092417.545cof5jk63g2eeo@uno.localdomain>","References":"<20200720104736.19986-1-jacopo@jmondi.org>\n\t<20200720104736.19986-17-jacopo@jmondi.org>\n\t<20200730203428.GE6107@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200730203428.GE6107@pendragon.ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v4 16/19] libcamera: ipu3: imgu:\n\tCalculate ImgU pipe configuration","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]