[{"id":15897,"web_url":"https://patchwork.libcamera.org/comment/15897/","msgid":"<20210325104519.2gbf6khuo5v56ya6@uno.localdomain>","date":"2021-03-25T10:45:19","subject":"Re: [libcamera-devel] [PATCH 1/3] android: CameraDevice: Validate\n\tcrop_rotate_scale_degrees in configuration","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Hiro,\n\nOn Thu, Mar 25, 2021 at 02:19:29PM +0900, Hirokazu Honda wrote:\n> Libcamera doesn't handle |crop_rotate_scale_degrees| in\n> camera3_stream at all. This adds the validation of the requested\n> |crop_rotate_scale_degrees| in configuration, but still not\n> handle the specified values.\n>\n> Signed-off-by: Hirokazu Honda \n> ---\n> src/android/camera_device.cpp | 35 +++++++++++++++++++++++++++++++++++\n> 1 file changed, 35 insertions(+)\n>\n> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> index 72a89258..1414bfef 100644\n> --- a/src/android/camera_device.cpp\n> +++ b/src/android/camera_device.cpp\n> @@ -256,6 +256,38 @@ void sortCamera3StreamConfigs(std::vector &unsortedConfigs,\n> \tunsortedConfigs = sortedConfigs;\n> }\n>\n> +/*\n> + * verifyCropRotateScaleDegrees returns where |crop_rotate_scale_degrees| in\n\ndouble space at beginning of the line\n> + * all camera3_stream in stream_list are valid.\n\nAnd usage of |variable| is a bit alien to libcamera :)\n\n> + */\n> +bool verifyCropRotateScaleDegrees(const camera3_stream_configuration_t &stream_list)\n\njust verifyCropRotate() ?\nOr better validateCropRotate() ?\n\n> +{\n> +\tif (stream_list.num_streams == 0)\n> +\t\treturn true;\n\nEmtpy line ?\nAlso, is this worth being checked in configureStreams() ?\n\n> +\tconst int crop_rotate_scale_degrees =\n\nWe use camelCase for variables\n\n> +\t\tstream_list.streams[0]->crop_rotate_scale_degrees;\n> +\tfor (unsigned int i = 0; i < stream_list.num_streams; ++i) {\n> +\t\tconst camera3_stream_t &stream = *stream_list.streams[i];\n> +\t\tif (CAMERA3_STREAM_ROTATION_0 > stream.crop_rotate_scale_degrees ||\n> +\t\t CAMERA3_STREAM_ROTATION_270 < stream.crop_rotate_scale_degrees) {\n\nThese are the possible values for crop_rotate_scale_degrees\n\n/**\n * camera3_stream_rotation_t:\n *\n * The required counterclockwise rotation of camera stream.\n */\ntypedef enum camera3_stream_rotation {\n /* No rotation */\n CAMERA3_STREAM_ROTATION_0 = 0,\n\n /* Rotate by 90 degree counterclockwise */\n CAMERA3_STREAM_ROTATION_90 = 1,\n\n /* Rotate by 180 degree counterclockwise */\n CAMERA3_STREAM_ROTATION_180 = 2,\n\n /* Rotate by 270 degree counterclockwise */\n CAMERA3_STREAM_ROTATION_270 = 3\n} camera3_stream_rotation_t;\n\nThe filed is documented as\n\n /**\n * This should be one of the camera3_stream_rotation_t values except for\n * CAMERA3_STREAM_ROTATION_180.\n * When setting to CAMERA3_STREAM_ROTATION_90 or CAMERA3_STREAM_ROTATION_270, HAL would crop,\n * rotate the frame by the specified degrees clockwise and scale it up to original size.\n * In Chrome OS, it's possible to have a portrait activity run in a landscape screen with\n * landscape-mounted camera. The activity would show stretched or rotated preview because it\n * does not expect to receive landscape preview frames. To solve this problem, we ask HAL to\n * crop, rotate and scale the frames and modify CameraCharacteristics.SENSOR_ORIENTATION\n * accordingly to imitate a portrait camera.\n * Setting it to CAMERA3_STREAM_ROTATION_0 means no crop-rotate-scale would be performed.\n * |cros_rotate_scale_degrees| in all camera3_stream_t of a configure_streams() call must be\n * identical. The HAL should return -EINVAL if the degrees are not the same for all the streams.\n */\n\nI think checking for != ROTATION_180 would be enough, but in\ncamera3_stream_t the field as type int, not\ncamera3_stream_rotation_t, so do we need to make sure it is 'valid'\nin the [0, 3] interval ? It doesn't hurt though\n\nHandling rotation properly will be quite an headache as we'll have to\ncompensate the properties::Rotation value from the Camera. I admit\nit's not yet clear in my mind how that will have to be done, but we'll\nsee. I guess we cannot rely on the sensor's flips but this will have\nto go through the YUV post-processor (also to handle the crop-scale\npart)\n\n> +\t\t\tLOG(HAL, Error) << \"invalid crop_rotate_scale_degrees: \"\n> +\t\t\t\t\t<< stream.crop_rotate_scale_degrees;\n\nInvalid\n\n> +\t\t\treturn false;\n> +\t\t}\n> +\t\tif (stream.crop_rotate_scale_degrees == CAMERA3_STREAM_ROTATION_180) {\n> +\t\t\tLOG(HAL, Error) << \"crop_rotate_scale_degrees should \"\n> +\t\t\t\t\t<< \"not be CAMERA3_STREAM_ROTATION_180\";\n> +\t\t\treturn false;\n> +\t\t}\n> +\t\tif (crop_rotate_scale_degrees != stream.crop_rotate_scale_degrees) {\n> +\t\t\tLOG(HAL, Error) << \"crop_rotate_scale_degrees in all \"\n> +\t\t\t\t\t<< \"streams are not identical\";\n> +\t\t\treturn false;\n> +\t\t}\n> +\t}\n\nEmpty line ?\n\n> +\treturn true;\n> +}\n> +\n> } /* namespace */\n>\n> /*\n> @@ -1566,6 +1598,9 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n> \t\trunning_ = false;\n> \t}\n>\n> +\tif (!verifyCropRotateScaleDegrees(*stream_list))\n> +\t\treturn -EINVAL;\n> +\n> \t/*\n> \t * Generate an empty configuration, and construct a StreamConfiguration\n> \t * for each camera3_stream to add to it.\n> --\n> 2.31.0.291.g576ba9dcdaf-goog\n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 013CFBDC66\n\tfor ;\n\tThu, 25 Mar 2021 10:44:48 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4E7E168D6C;\n\tThu, 25 Mar 2021 11:44:48 +0100 (CET)","from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net\n\t[217.70.183.200])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7E45368D58\n\tfor ;\n\tThu, 25 Mar 2021 11:44:46 +0100 (CET)","from uno.localdomain (93-61-96-190.ip145.fastwebnet.it\n\t[93.61.96.190]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay7-d.mail.gandi.net (Postfix) with ESMTPSA id DD2D220016;\n\tThu, 25 Mar 2021 10:44:45 +0000 (UTC)"],"X-Originating-IP":"93.61.96.190","Date":"Thu, 25 Mar 2021 11:45:19 +0100","From":"Jacopo Mondi ","To":"Hirokazu Honda ","Message-ID":"<20210325104519.2gbf6khuo5v56ya6@uno.localdomain>","References":"<20210325051931.3748204-1-hiroh@chromium.org>\n\t<20210325051931.3748204-2-hiroh@chromium.org>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210325051931.3748204-2-hiroh@chromium.org>","Subject":"Re: [libcamera-devel] [PATCH 1/3] android: CameraDevice: Validate\n\tcrop_rotate_scale_degrees in configuration","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15977,"web_url":"https://patchwork.libcamera.org/comment/15977/","msgid":"","date":"2021-03-28T22:37:16","subject":"Re: [libcamera-devel] [PATCH 1/3] android: CameraDevice: Validate\n\tcrop_rotate_scale_degrees in configuration","submitter":{"id":63,"url":"https://patchwork.libcamera.org/api/people/63/","name":"Hirokazu Honda","email":"hiroh@chromium.org"},"content":"On Thu, Mar 25, 2021 at 7:44 PM Jacopo Mondi wrote:\n>\n> Hi Hiro,\n>\n> On Thu, Mar 25, 2021 at 02:19:29PM +0900, Hirokazu Honda wrote:\n> > Libcamera doesn't handle |crop_rotate_scale_degrees| in\n> > camera3_stream at all. This adds the validation of the requested\n> > |crop_rotate_scale_degrees| in configuration, but still not\n> > handle the specified values.\n> >\n> > Signed-off-by: Hirokazu Honda \n> > ---\n> > src/android/camera_device.cpp | 35 +++++++++++++++++++++++++++++++++++\n> > 1 file changed, 35 insertions(+)\n> >\n> > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > index 72a89258..1414bfef 100644\n> > --- a/src/android/camera_device.cpp\n> > +++ b/src/android/camera_device.cpp\n> > @@ -256,6 +256,38 @@ void sortCamera3StreamConfigs(std::vector &unsortedConfigs,\n> > unsortedConfigs = sortedConfigs;\n> > }\n> >\n> > +/*\n> > + * verifyCropRotateScaleDegrees returns where |crop_rotate_scale_degrees| in\n>\n> double space at beginning of the line\n> > + * all camera3_stream in stream_list are valid.\n>\n> And usage of |variable| is a bit alien to libcamera :)\n>\n> > + */\n> > +bool verifyCropRotateScaleDegrees(const camera3_stream_configuration_t &stream_list)\n>\n> just verifyCropRotate() ?\n> Or better validateCropRotate() ?\n>\n> > +{\n> > + if (stream_list.num_streams == 0)\n> > + return true;\n>\n> Emtpy line ?\n> Also, is this worth being checked in configureStreams() ?\n>\n> > + const int crop_rotate_scale_degrees =\n>\n> We use camelCase for variables\n>\n> > + stream_list.streams[0]->crop_rotate_scale_degrees;\n> > + for (unsigned int i = 0; i < stream_list.num_streams; ++i) {\n> > + const camera3_stream_t &stream = *stream_list.streams[i];\n> > + if (CAMERA3_STREAM_ROTATION_0 > stream.crop_rotate_scale_degrees ||\n> > + CAMERA3_STREAM_ROTATION_270 < stream.crop_rotate_scale_degrees) {\n>\n> These are the possible values for crop_rotate_scale_degrees\n>\n> /**\n> * camera3_stream_rotation_t:\n> *\n> * The required counterclockwise rotation of camera stream.\n> */\n> typedef enum camera3_stream_rotation {\n> /* No rotation */\n> CAMERA3_STREAM_ROTATION_0 = 0,\n>\n> /* Rotate by 90 degree counterclockwise */\n> CAMERA3_STREAM_ROTATION_90 = 1,\n>\n> /* Rotate by 180 degree counterclockwise */\n> CAMERA3_STREAM_ROTATION_180 = 2,\n>\n> /* Rotate by 270 degree counterclockwise */\n> CAMERA3_STREAM_ROTATION_270 = 3\n> } camera3_stream_rotation_t;\n>\n> The filed is documented as\n>\n> /**\n> * This should be one of the camera3_stream_rotation_t values except for\n> * CAMERA3_STREAM_ROTATION_180.\n> * When setting to CAMERA3_STREAM_ROTATION_90 or CAMERA3_STREAM_ROTATION_270, HAL would crop,\n> * rotate the frame by the specified degrees clockwise and scale it up to original size.\n> * In Chrome OS, it's possible to have a portrait activity run in a landscape screen with\n> * landscape-mounted camera. The activity would show stretched or rotated preview because it\n> * does not expect to receive landscape preview frames. To solve this problem, we ask HAL to\n> * crop, rotate and scale the frames and modify CameraCharacteristics.SENSOR_ORIENTATION\n> * accordingly to imitate a portrait camera.\n> * Setting it to CAMERA3_STREAM_ROTATION_0 means no crop-rotate-scale would be performed.\n> * |cros_rotate_scale_degrees| in all camera3_stream_t of a configure_streams() call must be\n> * identical. The HAL should return -EINVAL if the degrees are not the same for all the streams.\n> */\n>\n> I think checking for != ROTATION_180 would be enough, but in\n> camera3_stream_t the field as type int, not\n> camera3_stream_rotation_t, so do we need to make sure it is 'valid'\n> in the [0, 3] interval ? It doesn't hurt though\n>\n\nYeah, since the type is int, it can be any value.\nIn fact, chromeos camera test has a test to set an invalid value and\nchecks if the configuration fails.\nSo we should keep this check.\n\n> Handling rotation properly will be quite an headache as we'll have to\n> compensate the properties::Rotation value from the Camera. I admit\n> it's not yet clear in my mind how that will have to be done, but we'll\n> see. I guess we cannot rely on the sensor's flips but this will have\n> to go through the YUV post-processor (also to handle the crop-scale\n> part)\n>\n\nMy intuition is to flip (rotate) on a sensor, and cropping and scaling\nis done in the YUV post-processor.\nIf a sensor is not reliable, all should be done in the YUV post processor.\n\nBest Regards,\n-Hiro\n\n> > + LOG(HAL, Error) << \"invalid crop_rotate_scale_degrees: \"\n> > + << stream.crop_rotate_scale_degrees;\n>\n> Invalid\n>\n> > + return false;\n> > + }\n> > + if (stream.crop_rotate_scale_degrees == CAMERA3_STREAM_ROTATION_180) {\n> > + LOG(HAL, Error) << \"crop_rotate_scale_degrees should \"\n> > + << \"not be CAMERA3_STREAM_ROTATION_180\";\n> > + return false;\n> > + }\n> > + if (crop_rotate_scale_degrees != stream.crop_rotate_scale_degrees) {\n> > + LOG(HAL, Error) << \"crop_rotate_scale_degrees in all \"\n> > + << \"streams are not identical\";\n> > + return false;\n> > + }\n> > + }\n>\n> Empty line ?\n>\n> > + return true;\n> > +}\n> > +\n> > } /* namespace */\n> >\n> > /*\n> > @@ -1566,6 +1598,9 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n> > running_ = false;\n> > }\n> >\n> > + if (!verifyCropRotateScaleDegrees(*stream_list))\n> > + return -EINVAL;\n> > +\n> > /*\n> > * Generate an empty configuration, and construct a StreamConfiguration\n> > * for each camera3_stream to add to it.\n> > --\n> > 2.31.0.291.g576ba9dcdaf-goog\n> > _______________________________________________\n> > libcamera-devel mailing list\n> > libcamera-devel@lists.libcamera.org\n> > https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 678AEC32EA\n\tfor ;\n\tSun, 28 Mar 2021 22:37:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DCABF6877F;\n\tMon, 29 Mar 2021 00:37:27 +0200 (CEST)","from mail-ed1-x533.google.com (mail-ed1-x533.google.com\n\t[IPv6:2a00:1450:4864:20::533])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3AE63602D7\n\tfor ;\n\tMon, 29 Mar 2021 00:37:27 +0200 (CEST)","by mail-ed1-x533.google.com with SMTP id o19so12210917edc.3\n\tfor ;\n\tSun, 28 Mar 2021 15:37:27 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=chromium.org header.i=@chromium.org\n\theader.b=\"eJInUSI9\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org;\n\ts=google; \n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=2lu7rFjue8rBCpq/3A1qNwXrbXHh/qxATEP2TOJ4A74=;\n\tb=eJInUSI9aMSHOgFt6BdWbK7Waqnj7RFuL2L+tMdt+ZfZGcjgWkkXoWZEX0IOD4fBYw\n\t2sx+E+/JZhzgwXT1KASUwc9F5+dfJg53jvWk/sRTJYjWSj7xneHZ8u1AZQOnhbxT9pcL\n\t6EqrVKFylpYjFaVUCidYboiC78VLG/hBd4RG0=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=2lu7rFjue8rBCpq/3A1qNwXrbXHh/qxATEP2TOJ4A74=;\n\tb=S9vdRSsXZz8/2O1U/KM8p6292NV9DCuAh/HJQ+2I0bPhOpqt3ZqFZc1p1ERpI+5BYm\n\tkhoweWRaGKQ1MS80NQgj4A8VplVCdz0jFCw4hE6E6HxQmxgWa10W7luA5kA+sxzPJ0UW\n\tV/KomHEMB64eFvAWjP9kghtDHvPRovkx8lWvTxZpw5wv4EFH7jNUw+qt6CIdeq6v+RuU\n\tJ7YDpmgUWXiwwU/9NwBe//jqcMGRB1mrFmSI6MLYtIff08dlSKw1JiH4/l1AOR7Vj8sb\n\tVwJdYp203wt7+CNTDjealxwMi6FAnW/d53YUcnEfnQrucrCqFsrsedfomjhC0fCqWVjd\n\tvcbQ==","X-Gm-Message-State":"AOAM530Z3QcBcatk8pOAaN8XLh7jZ4vPgm/0lLy8UDlewC1CrxE5OBcC\n\tJjOr428iBjV+mzs+ZJL0nV2Rtnp3Syhf1Ru8g7yvfDGDUXY=","X-Google-Smtp-Source":"ABdhPJwgJMt7tmlUVOivdlB4q8QNF/PStRHb05GWl0xSv/+iyYpZl4hGhAAl/d41jyM58QX9wRso+bQWZTVbW4jMptU=","X-Received":"by 2002:aa7:d4cb:: with SMTP id\n\tt11mr25540455edr.202.1616971046828; \n\tSun, 28 Mar 2021 15:37:26 -0700 (PDT)","MIME-Version":"1.0","References":"<20210325051931.3748204-1-hiroh@chromium.org>\n\t<20210325051931.3748204-2-hiroh@chromium.org>\n\t<20210325104519.2gbf6khuo5v56ya6@uno.localdomain>","In-Reply-To":"<20210325104519.2gbf6khuo5v56ya6@uno.localdomain>","From":"Hirokazu Honda ","Date":"Mon, 29 Mar 2021 07:37:16 +0900","Message-ID":"","To":"Jacopo Mondi ","Subject":"Re: [libcamera-devel] [PATCH 1/3] android: CameraDevice: Validate\n\tcrop_rotate_scale_degrees in configuration","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]