Patch Detail
Show a patch.
GET /api/1.1/patches/9098/?format=api
{ "id": 9098, "url": "https://patchwork.libcamera.org/api/1.1/patches/9098/?format=api", "web_url": "https://patchwork.libcamera.org/patch/9098/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/1.1/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20200731153320.58107-1-jacopo@jmondi.org>", "date": "2020-07-31T15:33:01", "name": "[libcamera-devel,v5,00/19] ipu3: rework pipe confiuguration", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "d62b0b789637b66a14d22fc78588598bf3d95abd", "submitter": { "id": 3, "url": "https://patchwork.libcamera.org/api/1.1/people/3/?format=api", "name": "Jacopo Mondi", "email": "jacopo@jmondi.org" }, "delegate": { "id": 15, "url": "https://patchwork.libcamera.org/api/1.1/users/15/?format=api", "username": "jmondi", "first_name": "Jacopo", "last_name": "Mondi", "email": "jacopo@jmondi.org" }, "mbox": "https://patchwork.libcamera.org/patch/9098/mbox/", "series": [], "comments": "https://patchwork.libcamera.org/api/patches/9098/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/9098/checks/", "tags": {}, "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B5D11BD86F\n\tfor <parsemail@patchwork.libcamera.org>;\n\tFri, 31 Jul 2020 15:29:48 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 42A7461A14;\n\tFri, 31 Jul 2020 17:29:48 +0200 (CEST)", "from relay10.mail.gandi.net (relay10.mail.gandi.net\n\t[217.70.178.230])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 40F9B60398\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri, 31 Jul 2020 17:29:46 +0200 (CEST)", "from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233])\n\t(Authenticated sender: jacopo@jmondi.org)\n\tby relay10.mail.gandi.net (Postfix) with ESMTPSA id B4A02240004;\n\tFri, 31 Jul 2020 15:29:45 +0000 (UTC)" ], "From": "Jacopo Mondi <jacopo@jmondi.org>", "To": "libcamera-devel@lists.libcamera.org", "Date": "Fri, 31 Jul 2020 17:33:01 +0200", "Message-Id": "<20200731153320.58107-1-jacopo@jmondi.org>", "X-Mailer": "git-send-email 2.27.0", "MIME-Version": "1.0", "Subject": "[libcamera-devel] [PATCH v5 00/19] ipu3: rework pipe confiuguration", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "7bit", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "I have addressed minor comments and added a few more details in comment and\ndocumentation here and there.\n\nI have simplified a bit the ImgU pipe configuration by removing not-required\ncasts, by making the code a bit less dense, and with a small rename.\n\nThe diff between v4 and v5 is so minimal I reported the full diff below.\n\nI tested results against the IPU3 pipe configuration tool by instrumenting\na version of it to run on a set of known values and compare the results between\nits calculation and libcamera:\nhttps://jmondi.org/cgit/intel-ipu3-pipe-config/commit/?id=78bc59f8bfe8c14908f612008a62edc621660cf3\n\nAll patches reviewed but I'm not pushing yet as I would like Kieran's ack, as\nhe's working on JPEG and moving the ground under his feet while developing seems\nnot nice.\n\n\nThanks\n j\n\nJacopo Mondi (19):\n libcamera: ipu3: Rename mbusCodesToInfo\n libcamera: utils: Add alignUp and alignDown functions\n libcamera: geometry: Add isNull() function to Rectangle class\n libcamera: ipu3: Remove streams from generateConfiguration\n libcamera: ipu3: Make sure the config is valid\n libcamera: ipu3: cio2: Report format and sizes\n libcamera: ipu3: Do not overwrite StreamConfiguration\n libcamera: ipu3: Report StreamFormats\n libcamera: ipu3: Remove initialization of Size\n libcamera: ipu3: Validate the stream combination\n libcamera: camera: Zero streams before validate()\n libcamera: ipu3: cio2: Add a const sensor() method\n libcamera: ipu3: Adjust and assign streams in validate()\n libcamera: ipu3: Remove streams from IPU3CameraConfiguration\n libcamera: ipu3: Remove camera_ from IPU3CameraConfiguration\n libcamera: ipu3: imgu: Calculate ImgU pipe configuration\n libcamera: ipu3: Validate the pipe configuration\n libcamera: ipu3: Configure ImgU with the computed parameters\n libcamera: ipu3: imgu: Rename configureInput()\n\n include/libcamera/geometry.h | 1 +\n include/libcamera/internal/utils.h | 10 +\n src/libcamera/camera.cpp | 4 +-\n src/libcamera/geometry.cpp | 6 +\n src/libcamera/pipeline/ipu3/cio2.cpp | 54 +++-\n src/libcamera/pipeline/ipu3/cio2.h | 6 +\n src/libcamera/pipeline/ipu3/imgu. | 0\n src/libcamera/pipeline/ipu3/imgu.cpp | 386 +++++++++++++++++++++++-\n src/libcamera/pipeline/ipu3/imgu.h | 22 +-\n src/libcamera/pipeline/ipu3/ipu3.cpp | 424 +++++++++++++--------------\n src/libcamera/utils.cpp | 16 +\n test/geometry.cpp | 14 +\n test/utils.cpp | 11 +\n 13 files changed, 714 insertions(+), 240 deletions(-)\n create mode 100644 src/libcamera/pipeline/ipu3/imgu.\n\n--\n2.27.0", "diff": "diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp\nindex a3ea3eabe318..cf26980c75f1 100644\n--- a/src/libcamera/pipeline/ipu3/imgu.cpp\n+++ b/src/libcamera/pipeline/ipu3/imgu.cpp\n@@ -25,6 +25,14 @@ LOG_DECLARE_CATEGORY(IPU3)\n\n namespace {\n\n+/*\n+ * The procedure to calculate the ImgU pipe configuration has been ported\n+ * from the pipe_config.py python script, available at:\n+ * https://github.com/intel/intel-ipu3-pipecfg\n+ * at revision: 61e83f2f7606 (\"Add more information into README\")\n+ */\n+\n static constexpr unsigned int FILTER_H = 4;\n\n static constexpr unsigned int IF_ALIGN_W = 2;\n@@ -103,10 +111,8 @@ float findScaleFactor(float sf, const std::vector<float> &range,\n\n bool isSameRatio(const Size &in, const Size &out)\n {\n-\tfloat inRatio = static_cast<float>(in.width) /\n-\t\t\tstatic_cast<float>(in.height);\n-\tfloat outRatio = static_cast<float>(out.width) /\n-\t\t\t static_cast<float>(out.height);\n+\tfloat inRatio = static_cast<float>(in.width) / in.height;\n+\tfloat outRatio = static_cast<float>(out.width) / out.height;\n\n \tif (std::abs(inRatio - outRatio) > 0.1)\n \t\treturn false;\n@@ -123,16 +129,18 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc\n \tfloat bdsHeight;\n\n \tif (!isSameRatio(pipe->input, gdc)) {\n-\t\tbool found = false;\n-\t\tfloat estIFHeight = static_cast<float>(iif.width * gdc.height) /\n+\t\tfloat estIFHeight = (iif.width * gdc.height) /\n \t\t\t\t static_cast<float>(gdc.width);\n \t\testIFHeight = utils::clamp<float>(estIFHeight, minIFHeight, iif.height);\n+\t\tbool found = false;\n+\n \t\tifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H);\n-\t\twhile (ifHeight >= minIFHeight &&\n-\t\t static_cast<float>(ifHeight) / bdsSF >= minBDSHeight) {\n-\t\t\tbdsHeight = static_cast<float>(ifHeight) / bdsSF;\n+\t\twhile (ifHeight >= minIFHeight && ifHeight / bdsSF >= minBDSHeight) {\n+\n+\t\t\tbdsHeight = ifHeight / bdsSF;\n \t\t\tif (std::fmod(bdsHeight, 1.0) == 0) {\n \t\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n+\n \t\t\t\tif (!(bdsIntHeight % BDS_ALIGN_H)) {\n \t\t\t\t\tfound = true;\n \t\t\t\t\tbreak;\n@@ -143,11 +151,12 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc\n \t\t}\n\n \t\tifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H);\n-\t\twhile (ifHeight <= iif.height &&\n-\t\t static_cast<float>(ifHeight) / bdsSF >= minBDSHeight) {\n-\t\t\tbdsHeight = static_cast<float>(ifHeight) / bdsSF;\n+\t\twhile (ifHeight <= iif.height && ifHeight / bdsSF >= minBDSHeight) {\n+\n+\t\t\tbdsHeight = ifHeight / bdsSF;\n \t\t\tif (std::fmod(bdsHeight, 1.0) == 0) {\n \t\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n+\n \t\t\t\tif (!(bdsIntHeight % BDS_ALIGN_H)) {\n \t\t\t\t\tfound = true;\n \t\t\t\t\tbreak;\n@@ -159,14 +168,15 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc\n\n \t\tif (found) {\n \t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n+\n \t\t\tpipeConfigs.push_back({ bdsSF, { iif.width, ifHeight },\n \t\t\t\t\t\t{ bdsWidth, bdsIntHeight }, gdc });\n \t\t\treturn;\n \t\t}\n \t} else {\n \t\tifHeight = utils::alignUp(iif.height, IF_ALIGN_H);\n-\t\twhile (ifHeight > minIFHeight &&\n-\t\t static_cast<float>(ifHeight) / bdsSF >= minBDSHeight) {\n+\t\twhile (ifHeight > minIFHeight && ifHeight / bdsSF >= minBDSHeight) {\n+\n \t\t\tbdsHeight = ifHeight / bdsSF;\n \t\t\tif (std::fmod(ifHeight, 1.0) == 0 && std::fmod(bdsHeight, 1.0) == 0) {\n \t\t\t\tunsigned int bdsIntHeight = static_cast<unsigned int>(bdsHeight);\n@@ -183,8 +193,7 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc\n \t}\n }\n\n-void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc,\n-\t\t float bdsSF)\n+void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc, float bdsSF)\n {\n \tunsigned int minBDSWidth = gdc.width + FILTER_H * 2;\n\n@@ -218,15 +227,14 @@ void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc,\n Size calculateGDC(ImgUDevice::Pipe *pipe)\n {\n \tconst Size &in = pipe->input;\n-\tconst Size &main = pipe->output;\n+\tconst Size &main = pipe->main;\n \tconst Size &vf = pipe->viewfinder;\n \tSize gdc;\n\n \tif (!vf.isNull()) {\n \t\tgdc.width = main.width;\n\n-\t\tfloat ratio = static_cast<float>(main.width * vf.height) /\n-\t\t\t static_cast<float>(vf.width);\n+\t\tfloat ratio = (main.width * vf.height) / static_cast<float>(vf.width);\n \t\tgdc.height = std::max(static_cast<float>(main.height), ratio);\n\n \t\treturn gdc;\n@@ -237,14 +245,13 @@ Size calculateGDC(ImgUDevice::Pipe *pipe)\n \t\treturn gdc;\n \t}\n\n-\tfloat totalSF = static_cast<float>(in.width) /\n-\t\t\tstatic_cast<float>(main.width);\n+\tfloat totalSF = static_cast<float>(in.width) / main.width;\n \tfloat bdsSF = totalSF > 2 ? 2 : 1;\n \tfloat yuvSF = totalSF / bdsSF;\n \tfloat sf = findScaleFactor(yuvSF, gdcScalingFactors);\n\n-\tgdc.width = static_cast<float>(main.width) * sf;\n-\tgdc.height = static_cast<float>(main.height) * sf;\n+\tgdc.width = main.width * sf;\n+\tgdc.height = main.height * sf;\n\n \treturn gdc;\n }\n@@ -259,6 +266,7 @@ FOV calcFOV(const Size &in, const ImgUDevice::PipeConfig &pipe)\n \tfloat ifCropH = static_cast<float>(in.height - pipe.iif.height);\n \tfloat gdcCropW = static_cast<float>(pipe.bds.width - pipe.gdc.width) * pipe.bds_sf;\n \tfloat gdcCropH = static_cast<float>(pipe.bds.height - pipe.gdc.height) * pipe.bds_sf;\n+\n \tfov.w = (inW - (ifCropW + gdcCropW)) / inW;\n \tfov.h = (inH - (ifCropH + gdcCropH)) / inH;\n\n@@ -304,11 +312,11 @@ FOV calcFOV(const Size &in, const ImgUDevice::PipeConfig &pipe)\n * \\var Pipe::input\n * \\brief The input image size\n *\n- * \\var Pipe::output\n+ * \\var Pipe::main\n * \\brief The requested main output size\n *\n * \\var Pipe::viewfinder\n- * \\brief The requested viewfinder size\n+ * \\brief The requested viewfinder output size\n */\n\n /**\n@@ -378,7 +386,7 @@ ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe)\n\n \tLOG(IPU3, Debug) << \"Calculating pipe configuration for: \";\n \tLOG(IPU3, Debug) << \"input: \" << pipe->input.toString();\n-\tLOG(IPU3, Debug) << \"main: \" << pipe->output.toString();\n+\tLOG(IPU3, Debug) << \"main: \" << pipe->main.toString();\n \tLOG(IPU3, Debug) << \"vf: \" << pipe->viewfinder.toString();\n\n \tconst Size &in = pipe->input;\n@@ -387,9 +395,9 @@ ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe)\n \tunsigned int ifWidth = utils::alignUp(in.width, IF_ALIGN_W);\n \tunsigned int ifHeight = in.height;\n \tunsigned int minIfWidth = in.width - IF_CROP_MAX_W;\n-\tfloat bdsSF = static_cast<float>(in.width) /\n-\t\t static_cast<float>(gdc.width);\n+\tfloat bdsSF = static_cast<float>(in.width) / gdc.width;\n \tfloat sf = findScaleFactor(bdsSF, bdsScalingFactors, true);\n+\n \twhile (ifWidth >= minIfWidth) {\n \t\tSize iif{ ifWidth, ifHeight };\n \t\tcalculateBDS(pipe, iif, gdc, sf);\ndiff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h\nindex ff8dab7d645c..c73ac5a5a37c 100644\n--- a/src/libcamera/pipeline/ipu3/imgu.h\n+++ b/src/libcamera/pipeline/ipu3/imgu.h\n@@ -37,7 +37,7 @@ public:\n\n \tstruct Pipe {\n \t\tSize input;\n-\t\tSize output;\n+\t\tSize main;\n \t\tSize viewfinder;\n \t};\n\ndiff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\nindex 06afc7d79dac..161a88da8497 100644\n--- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n@@ -225,11 +225,12 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n \t\t\t * margins from the CIO2 output frame size.\n \t\t\t *\n \t\t\t * The ImgU outputs needs to be strictly smaller than\n-\t\t\t * the CIO2 output frame and rounded down to 64\n-\t\t\t * pixels in width and 32 pixels in height. This comes\n-\t\t\t * from inspecting the pipe configuration script results\n-\t\t\t * and the available suggested configurations in the\n-\t\t\t * ChromeOS BSP .xml camera tuning files.\n+\t\t\t * the CIO2 output frame and rounded down to 64 pixels\n+\t\t\t * in width and 32 pixels in height. This assumption\n+\t\t\t * comes from inspecting the pipe configuration script\n+\t\t\t * results and the available suggested configurations in\n+\t\t\t * the ChromeOS BSP .xml camera tuning files and shall\n+\t\t\t * be validated.\n \t\t\t *\n \t\t\t * \\todo Clarify what are the hardware constraints\n \t\t\t * that require this alignements, if any. It might\n@@ -267,9 +268,9 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n \t\t\t\tcfg->setStream(const_cast<Stream *>(&data_->outStream_));\n \t\t\t\tmainOutputAvailable = false;\n\n-\t\t\t\tpipe.output = cfg->size;\n+\t\t\t\tpipe.main = cfg->size;\n \t\t\t\tif (yuvCount == 1)\n-\t\t\t\t\tpipe.viewfinder = pipe.output;\n+\t\t\t\t\tpipe.viewfinder = pipe.main;\n\n \t\t\t\tLOG(IPU3, Debug) << \"Assigned \" << cfg->toString()\n \t\t\t\t\t\t << \" to the main output\";\n@@ -333,8 +334,8 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera,\n \t\t\t * to the ImgU maximum output size) and aligned down to\n \t\t\t * the required frame margin.\n \t\t\t *\n-\t\t\t * The alignment constraints should be clarified\n-\t\t\t * as per the todo item in validate().\n+\t\t\t * \\todo Clarify the alignment constraints as exaplained\n+\t\t\t * in validate()\n \t\t\t */\n \t\t\tsize = sensorResolution.boundedTo(IMGU_OUTPUT_MAX_SIZE);\n \t\t\tsize.width = utils::alignDown(size.width - 1,\n", "prefixes": [ "libcamera-devel", "v5", "00/19" ] }