[{"id":19375,"web_url":"https://patchwork.libcamera.org/comment/19375/","msgid":"<5fdcc63d-15f9-340a-bf27-dee9f1885a07@ideasonboard.com>","date":"2021-09-03T13:44:14","subject":"Re: [libcamera-devel] [PATCH v2] ipa: ipu3: Replace\n\tipa::ipu3::algorithms::Ipu3AwbCell","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"On 02/09/2021 17:06, Jean-Michel Hautbois wrote:\n> The intel-ipu3.h public interface from the kernel does not define how to\n> parse the statistics for a cell. This had to be identified by a process\n> of reverse engineering, and later identifying the structures from\n> https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/refs/heads/master/hal/intel/include/ia_imaging/awb_public.h\n> leading to our custom definition of struct Ipu3AwbCell.\n> \n> To improve the kernel interface, a proposal has been made to the\n> linux-kernel [0] to incorporate the memory layout for each cell into the\n> intel-ipu3 header directly.\n> \n> [0]\n> https://lore.kernel.org/linux-media/20210831185140.77400-1-jeanmichel.hautbois@ideasonboard.com/\n> \n> Update our local copy of the intel-ipu3.h to match the proposal and\n> change the AGC and AWB algorithms to reference that structure directly,\n> allowing us to remove the deprecated custom Ipu3AwbCell definition.\n> \n> Signed-off-by: Jean-Michel Hautbois \n> ---\n> include/linux/intel-ipu3.h | 38 +++++++++++++++++++++++++--------\n> src/ipa/ipu3/algorithms/agc.cpp | 7 +++---\n> src/ipa/ipu3/algorithms/awb.cpp | 10 ++++-----\n> src/ipa/ipu3/algorithms/awb.h | 10 ---------\n> 4 files changed, 38 insertions(+), 27 deletions(-)\n> \n> diff --git a/include/linux/intel-ipu3.h b/include/linux/intel-ipu3.h\n> index ee0e6d0e..0633e04e 100644\n> --- a/include/linux/intel-ipu3.h\n> +++ b/include/linux/intel-ipu3.h\n> @@ -59,20 +59,40 @@ struct ipu3_uapi_grid_config {\n> \t__u16 y_end;\n> } __attribute__((packed));\n> \n> +/**\n> + * struct ipu3_uapi_awb_set_item - Memory layout for each cell in AWB\n> + *\n> + * @Gr_avg:\tGreen average for red lines in the cell.\n> + * @R_avg:\tRed average in the cell.\n> + * @B_avg:\tBlue average in the cell.\n> + * @Gb_avg:\tGreen average for blue lines in the cell.\n> + * @sat_ratio: Saturation ratio in the cell.\n> + * @padding0: Unused byte for padding.\n> + * @padding1: Unused byte for padding.\n> + * @padding2: Unused byte for padding.\n> + */\n> +struct ipu3_uapi_awb_set_item {\n> +\tunsigned char Gr_avg;\n> +\tunsigned char R_avg;\n> +\tunsigned char B_avg;\n> +\tunsigned char Gb_avg;\n> +\tunsigned char sat_ratio;\n> +\tunsigned char padding0;\n> +\tunsigned char padding1;\n> +\tunsigned char padding2;\n> +} __attribute__((packed));\n> +\n> /*\n> * The grid based data is divided into \"slices\" called set, each slice of setX\n> * refers to ipu3_uapi_grid_config width * height_per_slice.\n> */\n> #define IPU3_UAPI_AWB_MAX_SETS\t\t\t\t60\n> -/* Based on grid size 80 * 60 and cell size 16 x 16 */\n> -#define IPU3_UAPI_AWB_SET_SIZE\t\t\t\t1280\n> -#define IPU3_UAPI_AWB_MD_ITEM_SIZE\t\t\t8\n> -#define IPU3_UAPI_AWB_SPARE_FOR_BUBBLES \\\n> -\t(IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \\\n> -\t IPU3_UAPI_AWB_MD_ITEM_SIZE)\n> +#define AWB_PUBLIC_NUM_OF_ITEMS_IN_SET\t\t\t160\n> +/* Based on max grid height + Spare for bubbles */\n> +#define AWB_PUBLIC_NUM_OF_SETS_IN_BUFFER IPU3_UAPI_AWB_MAX_SETS + \\\n> +\t(IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES)\n> #define IPU3_UAPI_AWB_MAX_BUFFER_SIZE \\\n> -\t(IPU3_UAPI_AWB_MAX_SETS * \\\n> -\t (IPU3_UAPI_AWB_SET_SIZE + IPU3_UAPI_AWB_SPARE_FOR_BUBBLES))\n> + AWB_PUBLIC_NUM_OF_SETS_IN_BUFFER * AWB_PUBLIC_NUM_OF_ITEMS_IN_SET\n> \n> \n> /**\n> @@ -82,7 +102,7 @@ struct ipu3_uapi_grid_config {\n> *\t\tthe average values for each color channel.\n> */\n> struct ipu3_uapi_awb_raw_buffer {\n> -\t__u8 meta_data[IPU3_UAPI_AWB_MAX_BUFFER_SIZE]\n> +\tstruct ipu3_uapi_awb_set_item meta_data[IPU3_UAPI_AWB_MAX_BUFFER_SIZE]\n> \t\t__attribute__((aligned(32)));\n> } __attribute__((packed));\n\nI'll somewhat ignore all those changes, as they will be reviewed on\nlinux-media.\n\nAs long as we track any changes that occur in the upstreaming process\nhere, it's fine.\n\n\n> \n> diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\n> index 5ff50f4a..36f648be 100644\n> --- a/src/ipa/ipu3/algorithms/agc.cpp\n> +++ b/src/ipa/ipu3/algorithms/agc.cpp\n> @@ -96,9 +96,10 @@ void Agc::processBrightness(const ipu3_uapi_stats_3a *stats,\n> \t\t\t * We observed a bit shift which makes the value 160 to be 32 in the stats grid.\n> \t\t\t * Use the one passed at init time.\n> \t\t\t */\n> -\t\t\tif (stats->awb_raw_buffer.meta_data[i + 4 + j * grid.width] == 0) {\n> -\t\t\t\tuint8_t Gr = stats->awb_raw_buffer.meta_data[i + 0 + j * grid.width];\n> -\t\t\t\tuint8_t Gb = stats->awb_raw_buffer.meta_data[i + 3 + j * grid.width];\n> +\t\t\tconst ipu3_uapi_awb_set_item *currentCell = &stats->awb_raw_buffer.meta_data[i + 4 + j * grid.width];\n\n\nstats->awb_raw_buffer.meta_data has gone from an array of u8, where you\nindex it by\n\t\"i + 4 + j * grid.width\"\n\nto an array of struct ipu3_uapi_awb_set_item yet it's still indexed here by\n\t\"i + 4 + j * grid.width\"\n\nThat doesn't sound right to me.. It will be reading different offsets\nthan before... Does it work ?\n\n\n> +\t\t\tif (currentCell->sat_ratio == 0) {\n> +\t\t\t\tuint8_t Gr = currentCell->Gr_avg;\n> +\t\t\t\tuint8_t Gb = currentCell->Gb_avg;\n\nOtherwise, that looks cleaner and more readable though ;-)\n\n> \t\t\t\thist[(Gr + Gb) / 2]++;\n> \t\t\t\tcount++;\n> \t\t\t}\n> diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp\n> index 06b66bbb..7c0b7639 100644\n> --- a/src/ipa/ipu3/algorithms/awb.cpp\n> +++ b/src/ipa/ipu3/algorithms/awb.cpp\n> @@ -207,14 +207,14 @@ void Awb::generateAwbStats(const ipu3_uapi_stats_3a *stats,\n> \t\t\tcellPosition *= 8;\n> \n> \t\t\t/* Cast the initial IPU3 structure to simplify the reading */\n> -\t\t\tIpu3AwbCell *currentCell = reinterpret_cast(const_cast(&stats->awb_raw_buffer.meta_data[cellPosition]));\n> -\t\t\tif (currentCell->satRatio == 0) {\n> +\t\t\tconst ipu3_uapi_awb_set_item *currentCell = &stats->awb_raw_buffer.meta_data[cellPosition];\n\nAnd again here - I presume cellPosition would need to change given the\nchange in type of meta_data...\n\n\n> +\t\t\tif (currentCell->sat_ratio == 0) {\n> \t\t\t\t/* The cell is not saturated, use the current cell */\n> \t\t\t\tawbStats_[awbRegionPosition].counted++;\n> -\t\t\t\tuint32_t greenValue = currentCell->greenRedAvg + currentCell->greenBlueAvg;\n> +\t\t\t\tuint32_t greenValue = currentCell->Gr_avg + currentCell->Gb_avg;\n> \t\t\t\tawbStats_[awbRegionPosition].sum.green += greenValue / 2;\n> -\t\t\t\tawbStats_[awbRegionPosition].sum.red += currentCell->redAvg;\n> -\t\t\t\tawbStats_[awbRegionPosition].sum.blue += currentCell->blueAvg;\n> +\t\t\t\tawbStats_[awbRegionPosition].sum.red += currentCell->R_avg;\n> +\t\t\t\tawbStats_[awbRegionPosition].sum.blue += currentCell->B_avg;\n> \t\t\t}\n> \t\t}\n> \t}\n> diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h\n> index 23f3e872..a9320b4c 100644\n> --- a/src/ipa/ipu3/algorithms/awb.h\n> +++ b/src/ipa/ipu3/algorithms/awb.h\n> @@ -23,16 +23,6 @@ namespace ipa::ipu3::algorithms {\n> static constexpr uint32_t kAwbStatsSizeX = 16;\n> static constexpr uint32_t kAwbStatsSizeY = 12;\n> \n> -/* \\todo Move the cell layout into intel-ipu3.h kernel header */\n> -struct Ipu3AwbCell {\n> -\tunsigned char greenRedAvg;\n> -\tunsigned char redAvg;\n> -\tunsigned char blueAvg;\n> -\tunsigned char greenBlueAvg;\n> -\tunsigned char satRatio;\n> -\tunsigned char padding[3];\n> -} __attribute__((packed));\n> -\n> struct Accumulator {\n> \tunsigned int counted;\n> \tunsigned int total;\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5F2B0BDC71\n\tfor ;\n\tFri, 3 Sep 2021 13:44:19 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D42F06916A;\n\tFri, 3 Sep 2021 15:44:18 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9F2F769167\n\tfor ;\n\tFri, 3 Sep 2021 15:44:17 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 14339BBE;\n\tFri, 3 Sep 2021 15:44:17 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"gcZbGtOC\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1630676657;\n\tbh=je8sXiM0ruBlMQyumjQ4f4IWkbMjlOi3Mird6FnyS/M=;\n\th=To:References:From:Subject:Date:In-Reply-To:From;\n\tb=gcZbGtOCCq2lueQ9B7FRpm64YTtsbAWLTPypVC0s4o5eeinMXoYDHdZRapb0k+xRX\n\ttwdYYs3YXp9ABOiZCV/6IaX9hVWYeKV/5b80zSODqQPaQ4fdlP91C6RSIZFsU5D7aP\n\t25GtW2EQw7kp64KPr5UAkhZ2AoM6jkMLS+qfor0k=","To":"Jean-Michel Hautbois ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20210902160609.90886-1-jeanmichel.hautbois@ideasonboard.com>","From":"Kieran Bingham ","Message-ID":"<5fdcc63d-15f9-340a-bf27-dee9f1885a07@ideasonboard.com>","Date":"Fri, 3 Sep 2021 14:44:14 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.11.0","MIME-Version":"1.0","In-Reply-To":"<20210902160609.90886-1-jeanmichel.hautbois@ideasonboard.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-GB","Content-Transfer-Encoding":"8bit","Subject":"Re: [libcamera-devel] [PATCH v2] ipa: ipu3: Replace\n\tipa::ipu3::algorithms::Ipu3AwbCell","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]