[{"id":12264,"web_url":"https://patchwork.libcamera.org/comment/12264/","msgid":"<b9b80bbf-8f68-bf9d-3265-14e6cdf0f2dc@ideasonboard.com>","date":"2020-09-02T13:41:44","subject":"Re: [libcamera-devel] [RFC 5/7] android: camera_device: Rework\n\tCameraStream handling","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Jacopo,\n\nOn 02/09/2020 14:08, Jacopo Mondi wrote:\n> The CameraDevice::streams_ vector of CameraStream instances is\n> currently mostly accessed by index. The current implementation\n> creates all the CameraStream during the first loop that inspects the\n> camera3_stream instances, and the update the index of the\n> StreamConfiguration associated with the CameraStream during a second\n> loop that inspects MJPEG streams. A third loop creates the JPEG encoder\n> associated with CameraStreams that produce MJPEG format.\n> \n> As the index-based association is hard to follow and rather fragile,\n> rework the creation and handling of CameraStream:\n\nI recall it was necessary to split operations between things\npre-validate, and post-validate though..\n\n\n> \n> 1) Make the StreamConfiguration index a constructor parameter and a\n>    private struct member.  This disallow the creation of CameraStream\n>    without a StreamConfiguration index assigned.\n\nI think your earlier index patches now make that possible (so I like this)\n\n> \n> 2) Create CameraStream only after the associated StreamConfiguration\n>    has been identified. The first loop creates CameraStream for non-JPEG\n>    streams, the second for the JPEG ones after having identified the\n>    associated StreamConfiguration. Since we have just created the\n>    CameraStream, create the JPEG encoder at the same time instead of\n>    deferring it.\n> \n> This change removes most accesses by index to the CameraDevice::streams_\n> vector.\n> \n> No functional changes intended, but this change aims to make the code\n> easier to follow and more robust.\n\nSounds good - Need to check in detail, as earlier this is where I\nthought I saw a key functional change... diggging....\n\nNope - the change I had in mind was good...\n\n> \n> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>\n\nI really like this.\n\n\nReviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>\n\n> ---\n>  src/android/camera_device.cpp | 73 +++++++++++++++++++----------------\n>  src/android/camera_device.h   | 18 +++++----\n>  2 files changed, 50 insertions(+), 41 deletions(-)\n> \n> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> index a917404016e7..15dc12556cf5 100644\n> --- a/src/android/camera_device.cpp\n> +++ b/src/android/camera_device.cpp\n> @@ -168,8 +168,8 @@ MappedCamera3Buffer::MappedCamera3Buffer(const buffer_handle_t camera3buffer,\n>  \t}\n>  }\n>  \n> -CameraStream::CameraStream(PixelFormat f, Size s)\n> -\t: index(-1), format(f), size(s), jpeg(nullptr)\n> +CameraStream::CameraStream(PixelFormat f, Size s, unsigned int i)\n> +\t: format(f), size(s), jpeg(nullptr), index_(i)\n>  {\n>  }\n>  \n> @@ -1212,30 +1212,28 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n>  \t\tif (!format.isValid())\n>  \t\t\treturn -EINVAL;\n>  \n> -\t\tstreams_.emplace_back(format, size);\n> -\t\tstream->priv = static_cast<void *>(&streams_[i]);\n> -\n>  \t\t/* Defer handling of MJPEG streams until all others are known. */\n>  \t\tif (stream->format == HAL_PIXEL_FORMAT_BLOB)\n>  \t\t\tcontinue;\n>  \n>  \t\tStreamConfiguration streamConfiguration;\n> -\n>  \t\tstreamConfiguration.size = size;\n>  \t\tstreamConfiguration.pixelFormat = format;\n>  \n> -\t\tstreams_[i].index = config_->addConfiguration(streamConfiguration);\n> +\t\tunsigned int index = config_->addConfiguration(streamConfiguration);\n> +\t\tCameraStream &cameraStream = streams_.emplace_back(format, size, index);\n> +\t\tstream->priv = static_cast<void *>(&cameraStream);\n\nah yes, this is better indeed.\n\nThat index from addConfiguration really helps out.\n\n\n>  \t}\n>  \n>  \t/* Now handle MJPEG streams, adding a new stream if required. */\n>  \tfor (unsigned int i = 0; i < stream_list->num_streams; ++i) {\n>  \t\tcamera3_stream_t *stream = stream_list->streams[i];\n> -\t\tbool match = false;\n> -\n>  \t\tif (stream->format != HAL_PIXEL_FORMAT_BLOB)\n>  \t\t\tcontinue;\n>  \n> -\t\t/* Search for a compatible stream */\n> +\t\t/* Search for a compatible stream in the non-JPEG ones. */\n> +\t\tbool match = false;\n> +\t\tunsigned int index;\n>  \t\tfor (unsigned int j = 0; j < config_->size(); j++) {\n>  \t\t\tStreamConfiguration &cfg = config_->at(j);\n>  \n> @@ -1243,13 +1241,15 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n>  \t\t\t * \\todo The PixelFormat must also be compatible with\n>  \t\t\t * the encoder.\n>  \t\t\t */\n> -\t\t\tif (cfg.size == streams_[i].size) {\n> -\t\t\t\tLOG(HAL, Info) << \"Stream \" << i\n> -\t\t\t\t\t       << \" using libcamera stream \" << j;\n> +\t\t\tif (cfg.size.width != stream->width ||\n> +\t\t\t    cfg.size.height != stream->height)\n> +\t\t\t\tcontinue;\n>  \n> -\t\t\t\tmatch = true;\n> -\t\t\t\tstreams_[i].index = j;\n> -\t\t\t}\n> +\t\t\tLOG(HAL, Info) << \"Stream \" << i\n> +\t\t\t\t       << \" using libcamera stream \" << j;\n> +\n> +\t\t\tindex = j;\n> +\t\t\tmatch = true;\n>  \t\t}\n>  \n>  \t\t/*\n\n\nSigh - this is where I hate e-mail based reviews. This is missing what\nwould be reallly helpful context right here ;-)\n\n\n> @@ -1272,7 +1272,26 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n>  \t\t\tLOG(HAL, Info) << \"Adding \" << streamConfiguration.toString()\n>  \t\t\t\t       << \" for MJPEG support\";\n>  \n> -\t\t\tstreams_[i].index = config_->addConfiguration(streamConfiguration);\n> +\t\t\tindex = config_->addConfiguration(streamConfiguration);\n> +\t\t}\n> +\n> +\t\tStreamConfiguration &cfg = config_->at(index);\n> +\t\tPixelFormat format = formats::MJPEG;\n\nI think your previous patches have rendered this format field unused?\n\n(But I 'like' having it ..., and I think it still has use/value when we\ndeal with other stream conversions later ... we'll see)\n\n> +\t\tSize size = cfg.size;\n> +\n> +\t\tCameraStream &cameraStream = streams_.emplace_back(format, size, index);\n> +\t\tstream->priv = static_cast<void *>(&cameraStream);\n> +\n> +\t\t/*\n> +\t\t * Construct a software encoder for MJPEG streams from the\n> +\t\t * chosen libcamera source stream.\n> +\t\t */\n> +\t\tcameraStream.jpeg = new EncoderLibJpeg();\n> +\t\tint ret = cameraStream.jpeg->configure(cfg);\n> +\t\tif (ret) {\n> +\t\t\tLOG(HAL, Error)\n> +\t\t\t\t<< \"Failed to configure encoder\";\n> +\t\t\treturn ret;\n\nNice, I'm not sure why I had the encoder so late before, but I think\nit's fine/better here.\n\nMaybe it was so we only created the encoders after we've validated the\nconfigurations ?\n\nBut maybe it's not a problem though.\n\n\n\n>  \t\t}\n>  \t}\n>  \n> @@ -1296,24 +1315,10 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n>  \tfor (unsigned int i = 0; i < stream_list->num_streams; ++i) {\n>  \t\tcamera3_stream_t *stream = stream_list->streams[i];\n>  \t\tCameraStream *cameraStream = &streams_[i];\n> -\t\tStreamConfiguration &cfg = config_->at(cameraStream->index);\n> +\t\tStreamConfiguration &cfg = config_->at(cameraStream->index());\n>  \n>  \t\t/* Use the bufferCount confirmed by the validation process. */\n>  \t\tstream->max_buffers = cfg.bufferCount;\n> -\n> -\t\t/*\n> -\t\t * Construct a software encoder for MJPEG streams from the\n> -\t\t * chosen libcamera source stream.\n> -\t\t */\n> -\t\tif (cameraStream->format == formats::MJPEG) {\n> -\t\t\tcameraStream->jpeg = new EncoderLibJpeg();\n> -\t\t\tint ret = cameraStream->jpeg->configure(cfg);\n> -\t\t\tif (ret) {\n> -\t\t\t\tLOG(HAL, Error)\n> -\t\t\t\t\t<< \"Failed to configure encoder\";\n> -\t\t\t\treturn ret;\n> -\t\t\t}\n> -\t\t}\n>  \t}\n>  \n>  \t/*\n> @@ -1427,7 +1432,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n>  \t\t}\n>  \t\tdescriptor->frameBuffers.emplace_back(buffer);\n>  \n> -\t\tStreamConfiguration *streamConfiguration = &config_->at(cameraStream->index);\n> +\t\tStreamConfiguration *streamConfiguration = &config_->at(cameraStream->index());\n>  \t\tStream *stream = streamConfiguration->stream();\n>  \n>  \t\trequest->addBuffer(stream, buffer);\n> @@ -1482,7 +1487,7 @@ void CameraDevice::requestComplete(Request *request)\n>  \t\t\tcontinue;\n>  \t\t}\n>  \n> -\t\tStreamConfiguration *streamConfiguration = &config_->at(cameraStream->index);\n> +\t\tStreamConfiguration *streamConfiguration = &config_->at(cameraStream->index());\n>  \t\tStream *stream = streamConfiguration->stream();\n>  \t\tFrameBuffer *buffer = request->findBuffer(stream);\n>  \t\tif (!buffer) {\n> diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> index 230e89b011e6..975c312c1c12 100644\n> --- a/src/android/camera_device.h\n> +++ b/src/android/camera_device.h\n> @@ -28,20 +28,24 @@\n>  class CameraMetadata;\n>  \n>  struct CameraStream {\n> -\tCameraStream(libcamera::PixelFormat, libcamera::Size);\n> +public:\n> +\tCameraStream(libcamera::PixelFormat, libcamera::Size, unsigned int i);\n>  \t~CameraStream();\n>  \n> -\t/*\n> -\t * The index of the libcamera StreamConfiguration as added during\n> -\t * configureStreams(). A single libcamera Stream may be used to deliver\n> -\t * one or more streams to the Android framework.\n> -\t */\n> -\tunsigned int index;\n> +\tunsigned int index() const { return index_; }\n>  \n>  \tlibcamera::PixelFormat format;\n>  \tlibcamera::Size size;\n>  \n>  \tEncoder *jpeg;\n> +\n> +private:\n> +\t/*\n> +\t * The index of the libcamera StreamConfiguration as added during\n> +\t * configureStreams(). A single libcamera Stream may be used to deliver\n> +\t * one or more streams to the Android framework.\n> +\t */\n> +\tunsigned int index_;\n>  };\n>  \n>  class CameraDevice : protected libcamera::Loggable\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 70559BE174\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed,  2 Sep 2020 13:41:50 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 10E5462984;\n\tWed,  2 Sep 2020 15:41:50 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E9FB960374\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed,  2 Sep 2020 15:41:48 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D9E819CC;\n\tWed,  2 Sep 2020 15:41:46 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"vHugy1r/\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1599054107;\n\tbh=1kuhhb8udGUspewl3p9tPTryAkHeZOWqeo+I1Ne1nPk=;\n\th=Reply-To:Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=vHugy1r/4MnEdySXNTVahHXbLHkKItzFHuwrZFZwDVxzLuVWAcyoa2RuDlwR+T6Gl\n\tFn/F5FqCli+4wr0alz/Mp4li1qfICiKIKKAqWa+t1pqBvuMxj8cZ0MG41C5vZbn8Zr\n\tU2k1UGvJFqVMN5EWQiArAMVCdF8zz823sVd1eSWA=","To":"Jacopo Mondi <jacopo@jmondi.org>, libcamera-devel@lists.libcamera.org","References":"<20200902130846.55910-1-jacopo@jmondi.org>\n\t<20200902130846.55910-6-jacopo@jmondi.org>","From":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<b9b80bbf-8f68-bf9d-3265-14e6cdf0f2dc@ideasonboard.com>","Date":"Wed, 2 Sep 2020 14:41:44 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20200902130846.55910-6-jacopo@jmondi.org>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [RFC 5/7] android: camera_device: Rework\n\tCameraStream handling","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Reply-To":"kieran.bingham@ideasonboard.com","Cc":"tfiga@google.com, hiroh@google.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":12266,"web_url":"https://patchwork.libcamera.org/comment/12266/","msgid":"<20200902135721.bb2o7vx4egv7euol@uno.localdomain>","date":"2020-09-02T13:57:21","subject":"Re: [libcamera-devel] [RFC 5/7] android: camera_device: Rework\n\tCameraStream handling","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Kieran,\n\nOn Wed, Sep 02, 2020 at 02:41:44PM +0100, Kieran Bingham wrote:\n> Hi Jacopo,\n>\n> On 02/09/2020 14:08, Jacopo Mondi wrote:\n> > The CameraDevice::streams_ vector of CameraStream instances is\n> > currently mostly accessed by index. The current implementation\n> > creates all the CameraStream during the first loop that inspects the\n> > camera3_stream instances, and the update the index of the\n> > StreamConfiguration associated with the CameraStream during a second\n> > loop that inspects MJPEG streams. A third loop creates the JPEG encoder\n> > associated with CameraStreams that produce MJPEG format.\n> >\n> > As the index-based association is hard to follow and rather fragile,\n> > rework the creation and handling of CameraStream:\n>\n> I recall it was necessary to split operations between things\n> pre-validate, and post-validate though..\n>\n>\n> >\n> > 1) Make the StreamConfiguration index a constructor parameter and a\n> >    private struct member.  This disallow the creation of CameraStream\n> >    without a StreamConfiguration index assigned.\n>\n> I think your earlier index patches now make that possible (so I like this)\n>\n> >\n> > 2) Create CameraStream only after the associated StreamConfiguration\n> >    has been identified. The first loop creates CameraStream for non-JPEG\n> >    streams, the second for the JPEG ones after having identified the\n> >    associated StreamConfiguration. Since we have just created the\n> >    CameraStream, create the JPEG encoder at the same time instead of\n> >    deferring it.\n> >\n> > This change removes most accesses by index to the CameraDevice::streams_\n> > vector.\n> >\n> > No functional changes intended, but this change aims to make the code\n> > easier to follow and more robust.\n>\n> Sounds good - Need to check in detail, as earlier this is where I\n> thought I saw a key functional change... diggging....\n>\n> Nope - the change I had in mind was good...\n>\n> >\n> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>\n>\n> I really like this.\n>\n>\n> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>\n\nThanks!\n\n>\n> > ---\n> >  src/android/camera_device.cpp | 73 +++++++++++++++++++----------------\n> >  src/android/camera_device.h   | 18 +++++----\n> >  2 files changed, 50 insertions(+), 41 deletions(-)\n> >\n> > diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\n> > index a917404016e7..15dc12556cf5 100644\n> > --- a/src/android/camera_device.cpp\n> > +++ b/src/android/camera_device.cpp\n> > @@ -168,8 +168,8 @@ MappedCamera3Buffer::MappedCamera3Buffer(const buffer_handle_t camera3buffer,\n> >  \t}\n> >  }\n> >\n> > -CameraStream::CameraStream(PixelFormat f, Size s)\n> > -\t: index(-1), format(f), size(s), jpeg(nullptr)\n> > +CameraStream::CameraStream(PixelFormat f, Size s, unsigned int i)\n> > +\t: format(f), size(s), jpeg(nullptr), index_(i)\n> >  {\n> >  }\n> >\n> > @@ -1212,30 +1212,28 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n> >  \t\tif (!format.isValid())\n> >  \t\t\treturn -EINVAL;\n> >\n> > -\t\tstreams_.emplace_back(format, size);\n> > -\t\tstream->priv = static_cast<void *>(&streams_[i]);\n> > -\n> >  \t\t/* Defer handling of MJPEG streams until all others are known. */\n> >  \t\tif (stream->format == HAL_PIXEL_FORMAT_BLOB)\n> >  \t\t\tcontinue;\n> >\n> >  \t\tStreamConfiguration streamConfiguration;\n> > -\n> >  \t\tstreamConfiguration.size = size;\n> >  \t\tstreamConfiguration.pixelFormat = format;\n> >\n> > -\t\tstreams_[i].index = config_->addConfiguration(streamConfiguration);\n> > +\t\tunsigned int index = config_->addConfiguration(streamConfiguration);\n> > +\t\tCameraStream &cameraStream = streams_.emplace_back(format, size, index);\n> > +\t\tstream->priv = static_cast<void *>(&cameraStream);\n>\n> ah yes, this is better indeed.\n>\n> That index from addConfiguration really helps out.\n>\n>\n> >  \t}\n> >\n> >  \t/* Now handle MJPEG streams, adding a new stream if required. */\n> >  \tfor (unsigned int i = 0; i < stream_list->num_streams; ++i) {\n> >  \t\tcamera3_stream_t *stream = stream_list->streams[i];\n> > -\t\tbool match = false;\n> > -\n> >  \t\tif (stream->format != HAL_PIXEL_FORMAT_BLOB)\n> >  \t\t\tcontinue;\n> >\n> > -\t\t/* Search for a compatible stream */\n> > +\t\t/* Search for a compatible stream in the non-JPEG ones. */\n> > +\t\tbool match = false;\n> > +\t\tunsigned int index;\n> >  \t\tfor (unsigned int j = 0; j < config_->size(); j++) {\n> >  \t\t\tStreamConfiguration &cfg = config_->at(j);\n> >\n> > @@ -1243,13 +1241,15 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n> >  \t\t\t * \\todo The PixelFormat must also be compatible with\n> >  \t\t\t * the encoder.\n> >  \t\t\t */\n> > -\t\t\tif (cfg.size == streams_[i].size) {\n> > -\t\t\t\tLOG(HAL, Info) << \"Stream \" << i\n> > -\t\t\t\t\t       << \" using libcamera stream \" << j;\n> > +\t\t\tif (cfg.size.width != stream->width ||\n> > +\t\t\t    cfg.size.height != stream->height)\n> > +\t\t\t\tcontinue;\n> >\n> > -\t\t\t\tmatch = true;\n> > -\t\t\t\tstreams_[i].index = j;\n> > -\t\t\t}\n> > +\t\t\tLOG(HAL, Info) << \"Stream \" << i\n> > +\t\t\t\t       << \" using libcamera stream \" << j;\n> > +\n> > +\t\t\tindex = j;\n> > +\t\t\tmatch = true;\n> >  \t\t}\n> >\n> >  \t\t/*\n>\n>\n> Sigh - this is where I hate e-mail based reviews. This is missing what\n> would be reallly helpful context right here ;-)\n>\n>\n> > @@ -1272,7 +1272,26 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n> >  \t\t\tLOG(HAL, Info) << \"Adding \" << streamConfiguration.toString()\n> >  \t\t\t\t       << \" for MJPEG support\";\n> >\n> > -\t\t\tstreams_[i].index = config_->addConfiguration(streamConfiguration);\n> > +\t\t\tindex = config_->addConfiguration(streamConfiguration);\n> > +\t\t}\n> > +\n> > +\t\tStreamConfiguration &cfg = config_->at(index);\n> > +\t\tPixelFormat format = formats::MJPEG;\n>\n> I think your previous patches have rendered this format field unused?\n>\n> (But I 'like' having it ..., and I think it still has use/value when we\n> deal with other stream conversions later ... we'll see)\n\nYes indeed, we can use formats::MJPEG directly as CameraStream\nconstructor paramter.\n\n>\n> > +\t\tSize size = cfg.size;\n> > +\n> > +\t\tCameraStream &cameraStream = streams_.emplace_back(format, size, index);\n> > +\t\tstream->priv = static_cast<void *>(&cameraStream);\n> > +\n> > +\t\t/*\n> > +\t\t * Construct a software encoder for MJPEG streams from the\n> > +\t\t * chosen libcamera source stream.\n> > +\t\t */\n> > +\t\tcameraStream.jpeg = new EncoderLibJpeg();\n> > +\t\tint ret = cameraStream.jpeg->configure(cfg);\n> > +\t\tif (ret) {\n\n                        Leaking the encoder ?\n\n> > +\t\t\tLOG(HAL, Error)\n> > +\t\t\t\t<< \"Failed to configure encoder\";\n> > +\t\t\treturn ret;\n>\n> Nice, I'm not sure why I had the encoder so late before, but I think\n> it's fine/better here.\n>\n> Maybe it was so we only created the encoders after we've validated the\n> configurations ?\n\nWhich makes sense, and actually I wonder if I'm now leaking the\nEncoder. and yes I am in the error path here above.\n\nAfter it gets assinged to the CameraStream instance we should be safe,\nas long as all CameraStream instances are correctly deleted (see the\ncomment below).\n\n>\n> But maybe it's not a problem though.\n>\n>\n>\n> >  \t\t}\n> >  \t}\n> >\n> > @@ -1296,24 +1315,10 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list)\n> >  \tfor (unsigned int i = 0; i < stream_list->num_streams; ++i) {\n> >  \t\tcamera3_stream_t *stream = stream_list->streams[i];\n> >  \t\tCameraStream *cameraStream = &streams_[i];\n\nPlease note this can also be replaced with\n\n\t\tCameraStream *cameraStream = static_cast<CameraStream *>(stream->priv);\n\nSo all the index-based accesses to streams_ are gone, which I think\nit's nice. So the only use of streams_ is now to store the CameraStream\ninstances so that they can easily be destroyed during the next call to\nconfigureStreams() with streams_.clear() (or at CameraDevice\ndestruction time).\n\nI wonder if we can disallow future index-based accesses more strictly than\njust pointing that out at code review time..\n\nThanks\n  j\n\n> > -\t\tStreamConfiguration &cfg = config_->at(cameraStream->index);\n> > +\t\tStreamConfiguration &cfg = config_->at(cameraStream->index());\n> >\n> >  \t\t/* Use the bufferCount confirmed by the validation process. */\n> >  \t\tstream->max_buffers = cfg.bufferCount;\n> > -\n> > -\t\t/*\n> > -\t\t * Construct a software encoder for MJPEG streams from the\n> > -\t\t * chosen libcamera source stream.\n> > -\t\t */\n> > -\t\tif (cameraStream->format == formats::MJPEG) {\n> > -\t\t\tcameraStream->jpeg = new EncoderLibJpeg();\n> > -\t\t\tint ret = cameraStream->jpeg->configure(cfg);\n> > -\t\t\tif (ret) {\n> > -\t\t\t\tLOG(HAL, Error)\n> > -\t\t\t\t\t<< \"Failed to configure encoder\";\n> > -\t\t\t\treturn ret;\n> > -\t\t\t}\n> > -\t\t}\n> >  \t}\n> >\n> >  \t/*\n> > @@ -1427,7 +1432,7 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques\n> >  \t\t}\n> >  \t\tdescriptor->frameBuffers.emplace_back(buffer);\n> >\n> > -\t\tStreamConfiguration *streamConfiguration = &config_->at(cameraStream->index);\n> > +\t\tStreamConfiguration *streamConfiguration = &config_->at(cameraStream->index());\n> >  \t\tStream *stream = streamConfiguration->stream();\n> >\n> >  \t\trequest->addBuffer(stream, buffer);\n> > @@ -1482,7 +1487,7 @@ void CameraDevice::requestComplete(Request *request)\n> >  \t\t\tcontinue;\n> >  \t\t}\n> >\n> > -\t\tStreamConfiguration *streamConfiguration = &config_->at(cameraStream->index);\n> > +\t\tStreamConfiguration *streamConfiguration = &config_->at(cameraStream->index());\n> >  \t\tStream *stream = streamConfiguration->stream();\n> >  \t\tFrameBuffer *buffer = request->findBuffer(stream);\n> >  \t\tif (!buffer) {\n> > diff --git a/src/android/camera_device.h b/src/android/camera_device.h\n> > index 230e89b011e6..975c312c1c12 100644\n> > --- a/src/android/camera_device.h\n> > +++ b/src/android/camera_device.h\n> > @@ -28,20 +28,24 @@\n> >  class CameraMetadata;\n> >\n> >  struct CameraStream {\n> > -\tCameraStream(libcamera::PixelFormat, libcamera::Size);\n> > +public:\n> > +\tCameraStream(libcamera::PixelFormat, libcamera::Size, unsigned int i);\n> >  \t~CameraStream();\n> >\n> > -\t/*\n> > -\t * The index of the libcamera StreamConfiguration as added during\n> > -\t * configureStreams(). A single libcamera Stream may be used to deliver\n> > -\t * one or more streams to the Android framework.\n> > -\t */\n> > -\tunsigned int index;\n> > +\tunsigned int index() const { return index_; }\n> >\n> >  \tlibcamera::PixelFormat format;\n> >  \tlibcamera::Size size;\n> >\n> >  \tEncoder *jpeg;\n> > +\n> > +private:\n> > +\t/*\n> > +\t * The index of the libcamera StreamConfiguration as added during\n> > +\t * configureStreams(). A single libcamera Stream may be used to deliver\n> > +\t * one or more streams to the Android framework.\n> > +\t */\n> > +\tunsigned int index_;\n> >  };\n> >\n> >  class CameraDevice : protected libcamera::Loggable\n> >\n>\n> --\n> Regards\n> --\n> Kieran","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 13AAEBF019\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed,  2 Sep 2020 13:53:38 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9013362984;\n\tWed,  2 Sep 2020 15:53:37 +0200 (CEST)","from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net\n\t[217.70.183.198])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 567D860374\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed,  2 Sep 2020 15:53:36 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 6DBE9C0012;\n\tWed,  2 Sep 2020 13:53:35 +0000 (UTC)"],"X-Originating-IP":"93.34.118.233","Date":"Wed, 2 Sep 2020 15:57:21 +0200","From":"Jacopo Mondi <jacopo@jmondi.org>","To":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Message-ID":"<20200902135721.bb2o7vx4egv7euol@uno.localdomain>","References":"<20200902130846.55910-1-jacopo@jmondi.org>\n\t<20200902130846.55910-6-jacopo@jmondi.org>\n\t<b9b80bbf-8f68-bf9d-3265-14e6cdf0f2dc@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<b9b80bbf-8f68-bf9d-3265-14e6cdf0f2dc@ideasonboard.com>","Subject":"Re: [libcamera-devel] [RFC 5/7] android: camera_device: Rework\n\tCameraStream handling","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Cc":"tfiga@google.com, libcamera-devel@lists.libcamera.org, hiroh@google.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]