From patchwork Thu Oct 14 17:41:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14137 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AFA6EC323E for ; Thu, 14 Oct 2021 17:41:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8A33068F56; Thu, 14 Oct 2021 19:41:26 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9C9E768F4E for ; Thu, 14 Oct 2021 19:41:24 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id C93F9FF808; Thu, 14 Oct 2021 17:41:23 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:41:53 +0200 Message-Id: <20211014174208.50509-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 01/16] libcamera: ipu3: Use the optimal sensor size X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As reported by commit 7208e70211a6 ("libcamera: ipu3: Always use sensor full frame size") the current implementation of the IPU3 pipeline handler always uses the sensor resolution as the ImgU input frame size in order to work around an issue with the ImgU configuration procedure. Now that the frame selection policy has been modified in the CIO2Device class implementation to comply with the requirements of the ImgU configuration script we can remove the workaround and select the most opportune sensor size to feed the ImgU with. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/ipu3/ipu3.cpp | 41 ++++++++++++++++------------ 1 file changed, 23 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 262b9a23703e..8aab5041b3ea 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -229,7 +229,16 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() status = Adjusted; } - /* Validate the requested stream configuration */ + /* + * Validate the requested stream configuration and select the sensor + * format by collecting the maximum RAW stream width and height and + * picking the closest larger match. + * + * If no RAW stream is requested use the one of the largest YUV stream, + * plus margin pixels for the IF and BDS rectangle to downscale. + * + * \todo Clarify the IF and BDS margins requirements. + */ unsigned int rawCount = 0; unsigned int yuvCount = 0; Size maxYuvSize; @@ -240,7 +249,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { rawCount++; - rawSize = cfg.size; + rawSize.expandTo(cfg.size); } else { yuvCount++; maxYuvSize.expandTo(cfg.size); @@ -269,24 +278,20 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() /* * Generate raw configuration from CIO2. * - * \todo The image sensor frame size should be selected to optimize - * operations based on the sizes of the requested streams. However such - * a selection makes the pipeline configuration procedure fail for small - * resolutions (for example: 640x480 with OV5670) and causes the capture - * operations to stall for some stream size combinations (see the - * commit message of the patch that introduced this comment for more - * failure examples). + * The output YUV streams will be limited in size to the maximum frame + * size requested for the RAW stream, if present. * - * Until the sensor frame size calculation criteria are clarified, when - * capturing from ImgU always use the largest possible size which - * guarantees better results at the expense of the frame rate and CSI-2 - * bus bandwidth. When only a raw stream is requested use the requested - * size instead, as the ImgU is not involved. + * If no raw stream is requested generate a size as large as the maximum + * requested YUV size aligned to the ImgU constraints and bound by the + * sensor's maximum resolution. See + * https://bugs.libcamera.org/show_bug.cgi?id=32 */ - if (!yuvCount) - cio2Configuration_ = data_->cio2_.generateConfiguration(rawSize); - else - cio2Configuration_ = data_->cio2_.generateConfiguration({}); + if (rawSize.isNull()) + rawSize = maxYuvSize.expandedTo({40, 540}) + .grownBy({ IMGU_OUTPUT_WIDTH_MARGIN, + IMGU_OUTPUT_HEIGHT_MARGIN }) + .boundedTo(data_->cio2_.sensor()->resolution()); + cio2Configuration_ = data_->cio2_.generateConfiguration(rawSize); if (!cio2Configuration_.pixelFormat.isValid()) return Invalid; From patchwork Thu Oct 14 17:41:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14138 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B4779C323E for ; Thu, 14 Oct 2021 17:41:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B516968F59; Thu, 14 Oct 2021 19:41:26 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DF22E68F4D for ; Thu, 14 Oct 2021 19:41:25 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id C5AEBFF80B; Thu, 14 Oct 2021 17:41:24 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:41:54 +0200 Message-Id: <20211014174208.50509-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 02/16] libcamera: ipu3: Centralize ImgU sizes definition X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The definition of several constants that describe the ImgU characteristics are spread between two files: ipu3.cpp and imgu.cpp. As the ipu3.cpp uses definitions from the imgu.cpp file, in order to remove the usage of magic numbers, it is required to move the definitions to a common header file where they are accessible to the other .cpp modules. Move all the definitions of the ImgU sizes and alignments to the ImgUDevice class as static constexpr and update their users accordingly. Cosmetic changes, no functional changes intended. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Jean-Michel Hautbois Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/ipu3/imgu.cpp | 86 +++++++++++----------------- src/libcamera/pipeline/ipu3/imgu.h | 23 ++++++++ src/libcamera/pipeline/ipu3/ipu3.cpp | 48 +++++++--------- 3 files changed, 80 insertions(+), 77 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp index 3e1ef645ec93..3ef0ef144301 100644 --- a/src/libcamera/pipeline/ipu3/imgu.cpp +++ b/src/libcamera/pipeline/ipu3/imgu.cpp @@ -34,22 +34,6 @@ namespace { * at revision: 243d13446e44 ("Fix some bug for some resolutions") */ -static constexpr unsigned int FILTER_W = 4; -static constexpr unsigned int FILTER_H = 4; - -static constexpr unsigned int IF_ALIGN_W = 2; -static constexpr unsigned int IF_ALIGN_H = 4; - -static constexpr unsigned int BDS_ALIGN_W = 2; -static constexpr unsigned int BDS_ALIGN_H = 4; - -static constexpr unsigned int IF_CROP_MAX_W = 40; -static constexpr unsigned int IF_CROP_MAX_H = 540; - -static constexpr float BDS_SF_MAX = 2.5; -static constexpr float BDS_SF_MIN = 1.0; -static constexpr float BDS_SF_STEP = 0.03125; - /* BSD scaling factors: min=1, max=2.5, step=1/32 */ const std::vector bdsScalingFactors = { 1, 1.03125, 1.0625, 1.09375, 1.125, 1.15625, 1.1875, 1.21875, 1.25, @@ -124,8 +108,8 @@ bool isSameRatio(const Size &in, const Size &out) void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc, unsigned int bdsWidth, float bdsSF) { - unsigned int minIFHeight = iif.height - IF_CROP_MAX_H; - unsigned int minBDSHeight = gdc.height + FILTER_H * 2; + unsigned int minIFHeight = iif.height - ImgUDevice::kIFMaxCropHeight; + unsigned int minBDSHeight = gdc.height + ImgUDevice::kFilterHeight * 2; unsigned int ifHeight; float bdsHeight; @@ -135,7 +119,7 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc static_cast(gdc.width); estIFHeight = std::clamp(estIFHeight, minIFHeight, iif.height); - ifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H); + ifHeight = utils::alignUp(estIFHeight, ImgUDevice::kIFAlignHeight); while (ifHeight >= minIFHeight && ifHeight <= iif.height && ifHeight / bdsSF >= minBDSHeight) { @@ -143,17 +127,17 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc if (std::fmod(height, 1.0) == 0) { unsigned int bdsIntHeight = static_cast(height); - if (!(bdsIntHeight % BDS_ALIGN_H)) { + if (!(bdsIntHeight % ImgUDevice::kBDSAlignHeight)) { foundIfHeight = ifHeight; bdsHeight = height; break; } } - ifHeight -= IF_ALIGN_H; + ifHeight -= ImgUDevice::kIFAlignHeight; } - ifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H); + ifHeight = utils::alignUp(estIFHeight, ImgUDevice::kIFAlignHeight); while (ifHeight >= minIFHeight && ifHeight <= iif.height && ifHeight / bdsSF >= minBDSHeight) { @@ -161,14 +145,14 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc if (std::fmod(height, 1.0) == 0) { unsigned int bdsIntHeight = static_cast(height); - if (!(bdsIntHeight % BDS_ALIGN_H)) { + if (!(bdsIntHeight % ImgUDevice::kBDSAlignHeight)) { foundIfHeight = ifHeight; bdsHeight = height; break; } } - ifHeight += IF_ALIGN_H; + ifHeight += ImgUDevice::kIFAlignHeight; } if (foundIfHeight) { @@ -179,32 +163,32 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc return; } } else { - ifHeight = utils::alignUp(iif.height, IF_ALIGN_H); + ifHeight = utils::alignUp(iif.height, ImgUDevice::kIFAlignHeight); while (ifHeight >= minIFHeight && ifHeight / bdsSF >= minBDSHeight) { bdsHeight = ifHeight / bdsSF; if (std::fmod(ifHeight, 1.0) == 0 && std::fmod(bdsHeight, 1.0) == 0) { unsigned int bdsIntHeight = static_cast(bdsHeight); - if (!(ifHeight % IF_ALIGN_H) && - !(bdsIntHeight % BDS_ALIGN_H)) { + if (!(ifHeight % ImgUDevice::kIFAlignHeight) && + !(bdsIntHeight % ImgUDevice::kBDSAlignHeight)) { pipeConfigs.push_back({ bdsSF, { iif.width, ifHeight }, { bdsWidth, bdsIntHeight }, gdc }); } } - ifHeight -= IF_ALIGN_H; + ifHeight -= ImgUDevice::kIFAlignHeight; } } } void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc, float bdsSF) { - unsigned int minBDSWidth = gdc.width + FILTER_W * 2; - unsigned int minBDSHeight = gdc.height + FILTER_H * 2; + unsigned int minBDSWidth = gdc.width + ImgUDevice::kFilterWidth * 2; + unsigned int minBDSHeight = gdc.height + ImgUDevice::kFilterHeight * 2; float sf = bdsSF; - while (sf <= BDS_SF_MAX && sf >= BDS_SF_MIN) { + while (sf <= ImgUDevice::kBDSSfMax && sf >= ImgUDevice::kBDSSfMin) { float bdsWidth = static_cast(iif.width) / sf; float bdsHeight = static_cast(iif.height) / sf; @@ -212,16 +196,16 @@ void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc, floa std::fmod(bdsHeight, 1.0) == 0) { unsigned int bdsIntWidth = static_cast(bdsWidth); unsigned int bdsIntHeight = static_cast(bdsHeight); - if (!(bdsIntWidth % BDS_ALIGN_W) && bdsWidth >= minBDSWidth && - !(bdsIntHeight % BDS_ALIGN_H) && bdsHeight >= minBDSHeight) + if (!(bdsIntWidth % ImgUDevice::kBDSAlignWidth) && bdsWidth >= minBDSWidth && + !(bdsIntHeight % ImgUDevice::kBDSAlignHeight) && bdsHeight >= minBDSHeight) calculateBDSHeight(pipe, iif, gdc, bdsIntWidth, sf); } - sf += BDS_SF_STEP; + sf += ImgUDevice::kBDSSfStep; } sf = bdsSF; - while (sf <= BDS_SF_MAX && sf >= BDS_SF_MIN) { + while (sf <= ImgUDevice::kBDSSfMax && sf >= ImgUDevice::kBDSSfMin) { float bdsWidth = static_cast(iif.width) / sf; float bdsHeight = static_cast(iif.height) / sf; @@ -229,12 +213,12 @@ void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc, floa std::fmod(bdsHeight, 1.0) == 0) { unsigned int bdsIntWidth = static_cast(bdsWidth); unsigned int bdsIntHeight = static_cast(bdsHeight); - if (!(bdsIntWidth % BDS_ALIGN_W) && bdsWidth >= minBDSWidth && - !(bdsIntHeight % BDS_ALIGN_H) && bdsHeight >= minBDSHeight) + if (!(bdsIntWidth % ImgUDevice::kBDSAlignWidth) && bdsWidth >= minBDSWidth && + !(bdsIntHeight % ImgUDevice::kBDSAlignHeight) && bdsHeight >= minBDSHeight) calculateBDSHeight(pipe, iif, gdc, bdsIntWidth, sf); } - sf -= BDS_SF_STEP; + sf -= ImgUDevice::kBDSSfStep; } } @@ -412,7 +396,7 @@ ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe) * \todo Filter out all resolutions < IF_CROP_MAX. * See https://bugs.libcamera.org/show_bug.cgi?id=32 */ - if (in.width < IF_CROP_MAX_W || in.height < IF_CROP_MAX_H) { + if (in.width < ImgUDevice::kIFMaxCropWidth || in.height < ImgUDevice::kIFMaxCropHeight) { LOG(IPU3, Error) << "Input resolution " << in.toString() << " not supported"; return {}; @@ -424,25 +408,25 @@ ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe) float sf = findScaleFactor(bdsSF, bdsScalingFactors, true); /* Populate the configurations vector by scaling width and height. */ - unsigned int ifWidth = utils::alignUp(in.width, IF_ALIGN_W); - unsigned int ifHeight = utils::alignUp(in.height, IF_ALIGN_H); - unsigned int minIfWidth = in.width - IF_CROP_MAX_W; - unsigned int minIfHeight = in.height - IF_CROP_MAX_H; + unsigned int ifWidth = utils::alignUp(in.width, ImgUDevice::kIFAlignWidth); + unsigned int ifHeight = utils::alignUp(in.height, ImgUDevice::kIFAlignHeight); + unsigned int minIfWidth = in.width - ImgUDevice::kIFMaxCropWidth; + unsigned int minIfHeight = in.height - ImgUDevice::kIFMaxCropHeight; while (ifWidth >= minIfWidth) { while (ifHeight >= minIfHeight) { Size iif{ ifWidth, ifHeight }; calculateBDS(pipe, iif, gdc, sf); - ifHeight -= IF_ALIGN_H; + ifHeight -= ImgUDevice::kIFAlignHeight; } - ifWidth -= IF_ALIGN_W; + ifWidth -= ImgUDevice::kIFAlignWidth; } /* Repeat search by scaling width first. */ - ifWidth = utils::alignUp(in.width, IF_ALIGN_W); - ifHeight = utils::alignUp(in.height, IF_ALIGN_H); - minIfWidth = in.width - IF_CROP_MAX_W; - minIfHeight = in.height - IF_CROP_MAX_H; + ifWidth = utils::alignUp(in.width, ImgUDevice::kIFAlignWidth); + ifHeight = utils::alignUp(in.height, ImgUDevice::kIFAlignHeight); + minIfWidth = in.width - ImgUDevice::kIFMaxCropWidth; + minIfHeight = in.height - ImgUDevice::kIFMaxCropHeight; while (ifHeight >= minIfHeight) { /* * \todo This procedure is probably broken: @@ -451,10 +435,10 @@ ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe) while (ifWidth >= minIfWidth) { Size iif{ ifWidth, ifHeight }; calculateBDS(pipe, iif, gdc, sf); - ifWidth -= IF_ALIGN_W; + ifWidth -= ImgUDevice::kIFAlignWidth; } - ifHeight -= IF_ALIGN_H; + ifHeight -= ImgUDevice::kIFAlignHeight; } if (pipeConfigs.size() == 0) { diff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h index 9d4915116087..2b28d912676a 100644 --- a/src/libcamera/pipeline/ipu3/imgu.h +++ b/src/libcamera/pipeline/ipu3/imgu.h @@ -23,6 +23,29 @@ struct StreamConfiguration; class ImgUDevice { public: + static constexpr unsigned int kFilterWidth = 4; + static constexpr unsigned int kFilterHeight = 4; + + static constexpr unsigned int kIFAlignWidth = 2; + static constexpr unsigned int kIFAlignHeight = 4; + + static constexpr unsigned int kIFMaxCropWidth = 40; + static constexpr unsigned int kIFMaxCropHeight = 540; + + static constexpr unsigned int kBDSAlignWidth = 2; + static constexpr unsigned int kBDSAlignHeight = 4; + + static constexpr float kBDSSfMax = 2.5; + static constexpr float kBDSSfMin = 1.0; + static constexpr float kBDSSfStep = 0.03125; + + static constexpr Size kOutputMinSize = { 2, 2 }; + static constexpr Size kOutputMaxSize = { 4480, 34004 }; + static constexpr unsigned int kOutputAlignWidth = 64; + static constexpr unsigned int kOutputAlignHeight = 4; + static constexpr unsigned int kOutputMarginWidth = 64; + static constexpr unsigned int kOutputMarginHeight = 32; + struct PipeConfig { float bds_sf; Size iif; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 8aab5041b3ea..76fe9edb7dbc 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -41,12 +41,6 @@ LOG_DEFINE_CATEGORY(IPU3) static constexpr unsigned int IPU3_BUFFER_COUNT = 4; static constexpr unsigned int IPU3_MAX_STREAMS = 3; -static const Size IMGU_OUTPUT_MIN_SIZE = { 2, 2 }; -static const Size IMGU_OUTPUT_MAX_SIZE = { 4480, 34004 }; -static constexpr unsigned int IMGU_OUTPUT_WIDTH_ALIGN = 64; -static constexpr unsigned int IMGU_OUTPUT_HEIGHT_ALIGN = 4; -static constexpr unsigned int IMGU_OUTPUT_WIDTH_MARGIN = 64; -static constexpr unsigned int IMGU_OUTPUT_HEIGHT_MARGIN = 32; static constexpr Size IPU3ViewfinderSize(1280, 720); static const ControlInfoMap::Map IPU3Controls = { @@ -287,10 +281,12 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() * https://bugs.libcamera.org/show_bug.cgi?id=32 */ if (rawSize.isNull()) - rawSize = maxYuvSize.expandedTo({40, 540}) - .grownBy({ IMGU_OUTPUT_WIDTH_MARGIN, - IMGU_OUTPUT_HEIGHT_MARGIN }) + rawSize = maxYuvSize.expandedTo({ ImgUDevice::kIFMaxCropWidth, + ImgUDevice::kIFMaxCropHeight }) + .grownBy({ ImgUDevice::kOutputMarginWidth, + ImgUDevice::kOutputMarginHeight }) .boundedTo(data_->cio2_.sensor()->resolution()); + cio2Configuration_ = data_->cio2_.generateConfiguration(rawSize); if (!cio2Configuration_.pixelFormat.isValid()) return Invalid; @@ -345,19 +341,19 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() */ unsigned int limit; limit = utils::alignDown(cio2Configuration_.size.width - 1, - IMGU_OUTPUT_WIDTH_MARGIN); + ImgUDevice::kOutputMarginWidth); cfg->size.width = std::clamp(cfg->size.width, - IMGU_OUTPUT_MIN_SIZE.width, + ImgUDevice::kOutputMinSize.width, limit); limit = utils::alignDown(cio2Configuration_.size.height - 1, - IMGU_OUTPUT_HEIGHT_MARGIN); + ImgUDevice::kOutputMarginHeight); cfg->size.height = std::clamp(cfg->size.height, - IMGU_OUTPUT_MIN_SIZE.height, + ImgUDevice::kOutputMinSize.height, limit); - cfg->size.alignDownTo(IMGU_OUTPUT_WIDTH_ALIGN, - IMGU_OUTPUT_HEIGHT_ALIGN); + cfg->size.alignDownTo(ImgUDevice::kOutputAlignWidth, + ImgUDevice::kOutputAlignHeight); cfg->pixelFormat = formats::NV12; cfg->bufferCount = IPU3_BUFFER_COUNT; @@ -443,13 +439,13 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, * \todo Clarify the alignment constraints as explained * in validate() */ - size = sensorResolution.boundedTo(IMGU_OUTPUT_MAX_SIZE) + size = sensorResolution.boundedTo(ImgUDevice::kOutputMaxSize) .shrunkBy({ 1, 1 }) - .alignedDownTo(IMGU_OUTPUT_WIDTH_MARGIN, - IMGU_OUTPUT_HEIGHT_MARGIN); + .alignedDownTo(ImgUDevice::kOutputMarginWidth, + ImgUDevice::kOutputMarginHeight); pixelFormat = formats::NV12; bufferCount = IPU3_BUFFER_COUNT; - streamFormats[pixelFormat] = { { IMGU_OUTPUT_MIN_SIZE, size } }; + streamFormats[pixelFormat] = { { ImgUDevice::kOutputMinSize, size } }; break; @@ -474,11 +470,11 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, * to the ImgU output constraints. */ size = sensorResolution.boundedTo(IPU3ViewfinderSize) - .alignedDownTo(IMGU_OUTPUT_WIDTH_ALIGN, - IMGU_OUTPUT_HEIGHT_ALIGN); + .alignedDownTo(ImgUDevice::kOutputAlignWidth, + ImgUDevice::kOutputAlignHeight); pixelFormat = formats::NV12; bufferCount = IPU3_BUFFER_COUNT; - streamFormats[pixelFormat] = { { IMGU_OUTPUT_MIN_SIZE, size } }; + streamFormats[pixelFormat] = { { ImgUDevice::kOutputMinSize, size } }; break; } @@ -1001,16 +997,16 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) /* The strictly smaller size than the sensor resolution, aligned to margins. */ Size minSize = sensor->resolution().shrunkBy({ 1, 1 }) - .alignedDownTo(IMGU_OUTPUT_WIDTH_MARGIN, - IMGU_OUTPUT_HEIGHT_MARGIN); + .alignedDownTo(ImgUDevice::kOutputMarginWidth, + ImgUDevice::kOutputMarginHeight); /* * Either the smallest margin-aligned size larger than the viewfinder * size or the adjusted sensor resolution. */ minSize = IPU3ViewfinderSize.grownBy({ 1, 1 }) - .alignedUpTo(IMGU_OUTPUT_WIDTH_MARGIN, - IMGU_OUTPUT_HEIGHT_MARGIN) + .alignedUpTo(ImgUDevice::kOutputMarginWidth, + ImgUDevice::kOutputMarginHeight) .boundedTo(minSize); /* From patchwork Thu Oct 14 17:41:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14139 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D8851C323E for ; Thu, 14 Oct 2021 17:41:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B57D568F58; Thu, 14 Oct 2021 19:41:29 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 29F4D68F55 for ; Thu, 14 Oct 2021 19:41:27 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 0DB2EFF805; Thu, 14 Oct 2021 17:41:25 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:41:55 +0200 Message-Id: <20211014174208.50509-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 03/16] libcamera: ipu3: Rationalize constant expressions names X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Following the previous patch that moved all the ImgU-related contants in the ImgUDevice class namespace and that aligned their naming scheme to the 'kNameOfConstant' scheme, apply the same changes to the other components of the IPU3 pipeline handler. Cosmetic change, no functional changes intended. Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- src/libcamera/pipeline/ipu3/cio2.cpp | 6 +++--- src/libcamera/pipeline/ipu3/cio2.h | 2 +- src/libcamera/pipeline/ipu3/ipu3.cpp | 22 +++++++++++----------- 3 files changed, 15 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index dc62ab197acb..59dda56bdd3d 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -234,7 +234,7 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const cfg.size = sensorFormat.size; cfg.pixelFormat = mbusCodesToPixelFormat.at(sensorFormat.mbus_code); - cfg.bufferCount = CIO2_BUFFER_COUNT; + cfg.bufferCount = kBufferCount; return cfg; } @@ -338,11 +338,11 @@ int CIO2Device::exportBuffers(unsigned int count, int CIO2Device::start() { - int ret = output_->exportBuffers(CIO2_BUFFER_COUNT, &buffers_); + int ret = output_->exportBuffers(kBufferCount, &buffers_); if (ret < 0) return ret; - ret = output_->importBuffers(CIO2_BUFFER_COUNT); + ret = output_->importBuffers(kBufferCount); if (ret) LOG(IPU3, Error) << "Failed to import CIO2 buffers"; diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index 5fffe921f213..ba8f0052c5b3 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -30,7 +30,7 @@ struct StreamConfiguration; class CIO2Device { public: - static constexpr unsigned int CIO2_BUFFER_COUNT = 4; + static constexpr unsigned int kBufferCount = 4; CIO2Device(); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 76fe9edb7dbc..11ac55c02681 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -39,10 +39,6 @@ namespace libcamera { LOG_DEFINE_CATEGORY(IPU3) -static constexpr unsigned int IPU3_BUFFER_COUNT = 4; -static constexpr unsigned int IPU3_MAX_STREAMS = 3; -static constexpr Size IPU3ViewfinderSize(1280, 720); - static const ControlInfoMap::Map IPU3Controls = { { &controls::draft::PipelineDepth, ControlInfo(2, 3) }, }; @@ -93,6 +89,9 @@ private: class IPU3CameraConfiguration : public CameraConfiguration { public: + static constexpr unsigned int kBufferCount = 4; + static constexpr unsigned int kMaxStreams = 3; + IPU3CameraConfiguration(IPU3CameraData *data); Status validate() override; @@ -119,6 +118,7 @@ class PipelineHandlerIPU3 : public PipelineHandler { public: static constexpr unsigned int V4L2_CID_IPU3_PIPE_MODE = 0x009819c1; + static constexpr Size kViewfinderSize{ 1280, 720 }; enum IPU3PipeModes { IPU3PipeModeVideo = 0, @@ -218,8 +218,8 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() combinedTransform_ = combined; /* Cap the number of entries to the available streams. */ - if (config_.size() > IPU3_MAX_STREAMS) { - config_.resize(IPU3_MAX_STREAMS); + if (config_.size() > kMaxStreams) { + config_.resize(kMaxStreams); status = Adjusted; } @@ -356,7 +356,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() ImgUDevice::kOutputAlignHeight); cfg->pixelFormat = formats::NV12; - cfg->bufferCount = IPU3_BUFFER_COUNT; + cfg->bufferCount = kBufferCount; cfg->stride = info.stride(cfg->size.width, 0, 1); cfg->frameSize = info.frameSize(cfg->size, 1); @@ -444,7 +444,7 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, .alignedDownTo(ImgUDevice::kOutputMarginWidth, ImgUDevice::kOutputMarginHeight); pixelFormat = formats::NV12; - bufferCount = IPU3_BUFFER_COUNT; + bufferCount = IPU3CameraConfiguration::kBufferCount; streamFormats[pixelFormat] = { { ImgUDevice::kOutputMinSize, size } }; break; @@ -469,11 +469,11 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, * capped to the maximum sensor resolution and aligned * to the ImgU output constraints. */ - size = sensorResolution.boundedTo(IPU3ViewfinderSize) + size = sensorResolution.boundedTo(kViewfinderSize) .alignedDownTo(ImgUDevice::kOutputAlignWidth, ImgUDevice::kOutputAlignHeight); pixelFormat = formats::NV12; - bufferCount = IPU3_BUFFER_COUNT; + bufferCount = IPU3CameraConfiguration::kBufferCount; streamFormats[pixelFormat] = { { ImgUDevice::kOutputMinSize, size } }; break; @@ -1004,7 +1004,7 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) * Either the smallest margin-aligned size larger than the viewfinder * size or the adjusted sensor resolution. */ - minSize = IPU3ViewfinderSize.grownBy({ 1, 1 }) + minSize = kViewfinderSize.grownBy({ 1, 1 }) .alignedUpTo(ImgUDevice::kOutputMarginWidth, ImgUDevice::kOutputMarginHeight) .boundedTo(minSize); From patchwork Thu Oct 14 17:41:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14140 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6A468C323E for ; Thu, 14 Oct 2021 17:41:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2896668F5E; Thu, 14 Oct 2021 19:41:30 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0F5E268F4E for ; Thu, 14 Oct 2021 19:41:28 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 2CAA4FF808; Thu, 14 Oct 2021 17:41:26 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:41:56 +0200 Message-Id: <20211014174208.50509-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 04/16] libcamera: ipu3: Split controls init/update X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to prepare for updating the Camera controls limits when a new camera configuration is applied, split the initControls() function in two: - updateControls() to actually compute controls values - initControls() to initialize the sensor configuration and call updateControls Update the functions documentation accordingly. No functional changes intended. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain --- src/libcamera/pipeline/ipu3/ipu3.cpp | 33 ++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 11ac55c02681..cd71cce42421 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -148,6 +148,7 @@ private: } int initControls(IPU3CameraData *data); + int updateControls(IPU3CameraData *data); int registerCameras(); int allocateBuffers(Camera *camera); @@ -927,9 +928,11 @@ bool PipelineHandlerIPU3::match(DeviceEnumerator *enumerator) * \brief Initialize the camera controls * \param[in] data The camera data * - * Initialize the camera controls as the union of the static pipeline handler - * controls (IPU3Controls) and controls created dynamically from the sensor - * capabilities. + * Initialize the camera controls by calculating controls which the pipeline + * is reponsible for and merge them with the controls computed by the IPA. + * + * This function needs data->ipaControls_ to be initialized by the IPA init() + * function at camera creation time. Always call this function after IPA init(). * * \return 0 on success or a negative error code otherwise */ @@ -950,8 +953,30 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) if (ret) return ret; + return updateControls(data); +} + +/** + * \brief Update the camera controls + * \param[in] data The camera data + * + * Compute the camera controls by calculating controls which the pipeline + * is reponsible for and merge them with the controls computed by the IPA. + * + * This function needs data->ipaControls_ to be refreshed when a new + * configuration is applied to the camera by the IPA configure() function. + * + * Always call this function after IPA configure() to make sure to have a + * properly refreshed IPA controls list. + * + * \return 0 on success or a negative error code otherwise + */ +int PipelineHandlerIPU3::updateControls(IPU3CameraData *data) +{ + CameraSensor *sensor = data->cio2_.sensor(); IPACameraSensorInfo sensorInfo{}; - ret = sensor->sensorInfo(&sensorInfo); + + int ret = sensor->sensorInfo(&sensorInfo); if (ret) return ret; From patchwork Thu Oct 14 17:41:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14141 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C609EC323E for ; Thu, 14 Oct 2021 17:41:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A0D7068F56; Thu, 14 Oct 2021 19:41:32 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 573BF68541 for ; Thu, 14 Oct 2021 19:41:29 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 36F5EFF805; Thu, 14 Oct 2021 17:41:28 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:41:57 +0200 Message-Id: <20211014174208.50509-6-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 05/16] ipa: ipu3: Update camera controls in configure() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a new CameraConfiguration is applied to the Camera the IPA is configured as well, using the newly applied sensor configuration and its updated V4L2 controls. Also update the Camera controls at IPA::configure() time by re-computing the controls::ExposureTime and controls::FrameDurationLimits limits and update the controls on the pipeline handler side after having configured the IPA. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/ipu3.mojom | 3 +- src/ipa/ipu3/ipu3.cpp | 82 +++++++++++++++++++--------- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +- 3 files changed, 60 insertions(+), 29 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index 2045ce909a88..2f254ed4193e 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -45,7 +45,8 @@ interface IPAIPU3Interface { start() => (int32 ret); stop(); - configure(IPAConfigInfo configInfo) => (int32 ret); + configure(IPAConfigInfo configInfo) + => (int32 ret, libcamera.ControlInfoMap ipaControls); mapBuffers(array buffers); unmapBuffers(array ids); diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 6d9bbf391cb2..388f1902bcab 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -171,13 +171,17 @@ public: int start() override; void stop() override {} - int configure(const IPAConfigInfo &configInfo) override; + int configure(const IPAConfigInfo &configInfo, + ControlInfoMap *ipaControls) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; void processEvent(const IPU3Event &event) override; private: + void updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls); void processControls(unsigned int frame, const ControlList &controls); void fillParams(unsigned int frame, ipu3_uapi_params *params); void parseStatistics(unsigned int frame, @@ -212,36 +216,30 @@ private: struct IPAContext context_; }; -/** - * Initialize the IPA module and its controls. + +/* + * Compute camera controls using the sensor information and the sensor + * v4l2 controls. * - * This function receives the camera sensor information from the pipeline - * handler, computes the limits of the controls it handles and returns - * them in the \a ipaControls output parameter. + * Some of the camera controls are computed by the pipeline handler, some others + * by the IPA module which is in charge of handling, for example, the exposure + * time and the frame duration. + * + * This function computes: + * - controls::ExposureTime + * - controls::FrameDurationLimits */ -int IPAIPU3::init(const IPASettings &settings, - const IPACameraSensorInfo &sensorInfo, - const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) +void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) { - camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); - if (camHelper_ == nullptr) { - LOG(IPAIPU3, Error) - << "Failed to create camera sensor helper for " - << settings.sensorModel; - return -ENODEV; - } - - /* Initialize Controls. */ ControlInfoMap::Map controls{}; /* - * Compute exposure time limits. - * - * Initialize the control using the line length and pixel rate of the - * current configuration converted to microseconds. Use the - * V4L2_CID_EXPOSURE control to get exposure min, max and default and - * convert it from lines to microseconds. + * Compute exposure time limits by using line length and pixel rate + * converted to microseconds. Use the V4L2_CID_EXPOSURE control to get + * exposure min, max and default and convert it from lines to + * microseconds. */ double lineDuration = sensorInfo.lineLength / (sensorInfo.pixelRate / 1e6); const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; @@ -279,6 +277,27 @@ int IPAIPU3::init(const IPASettings &settings, frameDurations[2]); *ipaControls = ControlInfoMap(std::move(controls), controls::controls); +} + +/** + * Initialize the IPA module and its controls. + * + * This function receives the camera sensor information from the pipeline + * handler, computes the limits of the controls it handles and returns + * them in the \a ipaControls output parameter. + */ +int IPAIPU3::init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); + if (camHelper_ == nullptr) { + LOG(IPAIPU3, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } /* Construct our Algorithms */ algorithms_.push_back(std::make_unique()); @@ -286,6 +305,9 @@ int IPAIPU3::init(const IPASettings &settings, algorithms_.push_back(std::make_unique()); algorithms_.push_back(std::make_unique()); + /* Initialize controls. */ + updateControls(sensorInfo, sensorControls, ipaControls); + return 0; } @@ -365,7 +387,8 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) << (int)bdsGrid.height << " << " << (int)bdsGrid.block_height_log2 << ")"; } -int IPAIPU3::configure(const IPAConfigInfo &configInfo) +int IPAIPU3::configure(const IPAConfigInfo &configInfo, + ControlInfoMap *ipaControls) { if (configInfo.sensorControls.empty()) { LOG(IPAIPU3, Error) << "No sensor controls provided"; @@ -374,6 +397,10 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) sensorInfo_ = configInfo.sensorInfo; + /* + * Compute the sensor V4L2 controls to be used by the algorithms and + * to be set on the sensor. + */ ctrls_ = configInfo.sensorControls; const auto itExp = ctrls_.find(V4L2_CID_EXPOSURE); @@ -415,6 +442,9 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) return ret; } + /* Update the camera controls using the new sensor settings. */ + updateControls(sensorInfo_, ctrls_, ipaControls); + return 0; } diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index cd71cce42421..8a1aaf02db7c 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -660,14 +660,14 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) configInfo.bdsOutputSize = config->imguConfig().bds; configInfo.iif = config->imguConfig().iif; - ret = data->ipa_->configure(configInfo); + ret = data->ipa_->configure(configInfo, &data->ipaControls_); if (ret) { LOG(IPU3, Error) << "Failed to configure IPA: " << strerror(-ret); return ret; } - return 0; + return updateControls(data); } int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, From patchwork Thu Oct 14 17:41:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14142 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 06367C324D for ; Thu, 14 Oct 2021 17:41:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CEA3568F53; Thu, 14 Oct 2021 19:41:32 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8557C68F4D for ; Thu, 14 Oct 2021 19:41:30 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 81AAFFF805; Thu, 14 Oct 2021 17:41:29 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:41:58 +0200 Message-Id: <20211014174208.50509-7-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 06/16] android: capabilities: Collect per-stream frame durations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Collect the per-stream frame durations while building the list of supported stream formats and resolutions. In order to get an updated list of controls it is necessary to apply to the Camera the configuration we're testing, which was so far only validated. The per-configuration durations will be used to populate the Android ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS static metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 39 ++++++++++++++++++++++++++--- src/android/camera_capabilities.h | 2 ++ 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index baeedc11500a..1583a0a09cee 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -619,7 +619,32 @@ int CameraCapabilities::initializeStreamConfigurations() } for (const Size &res : resolutions) { - streamConfigurations_.push_back({ res, androidFormat }); + /* + * Configure the Camera with the collected format and + * resolution to get an updated list of controls. + * + * \todo Avoid the need to configure the camera when + * redesigning the configuration API. + */ + cfg.size = res; + int ret = camera_->configure(cameraConfig.get()); + if (ret) + return ret; + + const ControlInfoMap &controls = camera_->controls(); + const auto frameDurations = controls.find( + &controls::FrameDurationLimits); + if (frameDurations == controls.end()) { + LOG(HAL, Error) + << "Camera does not report frame durations"; + return -EINVAL; + } + + int64_t minFrameDuration = frameDurations->second.min().get() * 1000; + int64_t maxFrameDuration = frameDurations->second.max().get() * 1000; + streamConfigurations_.push_back({ + res, androidFormat, minFrameDuration, maxFrameDuration, + }); /* * If the format is HAL_PIXEL_FORMAT_YCbCr_420_888 @@ -631,10 +656,18 @@ int CameraCapabilities::initializeStreamConfigurations() * * \todo Support JPEG streams produced by the camera * natively. + * + * \todo HAL_PIXEL_FORMAT_BLOB is a 'stalling' format, + * its duration should take into account the time + * required for the YUV to JPEG encoding. For now + * use the same frame durations as collected for + * the YUV/RGB streams. */ if (androidFormat == HAL_PIXEL_FORMAT_YCbCr_420_888) { - streamConfigurations_.push_back( - { res, HAL_PIXEL_FORMAT_BLOB }); + streamConfigurations_.push_back({ + res, HAL_PIXEL_FORMAT_BLOB, + minFrameDuration, maxFrameDuration, + }); maxJpegSize = std::max(maxJpegSize, res); } } diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index a12596993ee5..6e55ddab445e 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -43,6 +43,8 @@ private: struct Camera3StreamConfiguration { libcamera::Size resolution; int androidFormat; + int64_t minFrameDurationNsec; + int64_t maxFrameDurationNsec; }; bool validateManualSensorCapability(); From patchwork Thu Oct 14 17:41:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14143 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9E079C323E for ; Thu, 14 Oct 2021 17:41:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 50BB268F64; Thu, 14 Oct 2021 19:41:33 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AC43368F4D for ; Thu, 14 Oct 2021 19:41:31 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id A7971FF80A; Thu, 14 Oct 2021 17:41:30 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:41:59 +0200 Message-Id: <20211014174208.50509-8-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 07/16] android: capabilities: Initialize camera state when building properties X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that building the list of supported stream configuration requires applying a configuration to the Camera, re-initialize the camera controls by applying a configuration generated for the Viewfinder stream role before building the list of static metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 1583a0a09cee..a98e6099bb77 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -394,11 +394,14 @@ int CameraCapabilities::initialize(std::shared_ptr camera, } ret = initializeStreamConfigurations(); - camera_->release(); - if (ret) + if (ret) { + camera_->release(); return ret; + } - return initializeStaticMetadata(); + ret = initializeStaticMetadata(); + camera_->release(); + return ret; } std::vector @@ -696,6 +699,25 @@ int CameraCapabilities::initializeStaticMetadata() return -EINVAL; } + /* + * Generate and apply a new configuration for the Viewfinder role to + * collect control limits and properties from a known state. + */ + std::unique_ptr cameraConfig = + camera_->generateConfiguration({ StreamRole::Viewfinder }); + if (!cameraConfig) { + LOG(HAL, Error) << "Failed to generate camera configuration"; + staticMetadata_.reset(); + return -ENODEV; + } + + int ret = camera_->configure(cameraConfig.get()); + if (ret) { + LOG(HAL, Error) << "Failed to initialize the camera state"; + staticMetadata_.reset(); + return ret; + } + const ControlInfoMap &controlsInfo = camera_->controls(); const ControlList &properties = camera_->properties(); From patchwork Thu Oct 14 17:42:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14144 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8E782C323E for ; Thu, 14 Oct 2021 17:41:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 308B668541; Thu, 14 Oct 2021 19:41:37 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE6A668F63 for ; Thu, 14 Oct 2021 19:41:32 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id D45D9FF808; Thu, 14 Oct 2021 17:41:31 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:42:00 +0200 Message-Id: <20211014174208.50509-9-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 08/16] android: capabilties: Assume controls::FrameDurationLimits is supported X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As we now collect the per-stream frame durations at initializeStreamConfigurations() times, the Camera is now guaranteed to support the controls::FrameDurationLimits control. Remove the check for its presence when populating the ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES static metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 85 ++++++++++++++--------------- 1 file changed, 41 insertions(+), 44 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index a98e6099bb77..e22bc35d420c 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -882,55 +882,52 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_MODES, aeAvailableModes); - int64_t minFrameDurationNsec = -1; - int64_t maxFrameDurationNsec = -1; + /* Initialize the AE frame duration limits. */ const auto frameDurationsInfo = controlsInfo.find(&controls::FrameDurationLimits); - if (frameDurationsInfo != controlsInfo.end()) { - minFrameDurationNsec = frameDurationsInfo->second.min().get() * 1000; - maxFrameDurationNsec = frameDurationsInfo->second.max().get() * 1000; + int64_t minFrameDurationNsec = frameDurationsInfo->second.min().get() * 1000; + int64_t maxFrameDurationNsec = frameDurationsInfo->second.max().get() * 1000; - /* - * Adjust the minimum frame duration to comply with Android - * requirements. The camera service mandates all preview/record - * streams to have a minimum frame duration < 33,366 milliseconds - * (see MAX_PREVIEW_RECORD_DURATION_NS in the camera service - * implementation). - * - * If we're close enough (+ 500 useconds) to that value, round - * the minimum frame duration of the camera to an accepted - * value. - */ - static constexpr int64_t MAX_PREVIEW_RECORD_DURATION_NS = 1e9 / 29.97; - if (minFrameDurationNsec > MAX_PREVIEW_RECORD_DURATION_NS && - minFrameDurationNsec < MAX_PREVIEW_RECORD_DURATION_NS + 500000) - minFrameDurationNsec = MAX_PREVIEW_RECORD_DURATION_NS - 1000; + /* + * Adjust the minimum frame duration to comply with Android + * requirements. The camera service mandates all preview/record + * streams to have a minimum frame duration < 33,366 milliseconds + * (see MAX_PREVIEW_RECORD_DURATION_NS in the camera service + * implementation). + * + * If we're close enough (+ 500 useconds) to that value, round + * the minimum frame duration of the camera to an accepted + * value. + */ + static constexpr int64_t MAX_PREVIEW_RECORD_DURATION_NS = 1e9 / 29.97; + if (minFrameDurationNsec > MAX_PREVIEW_RECORD_DURATION_NS && + minFrameDurationNsec < MAX_PREVIEW_RECORD_DURATION_NS + 500000) + minFrameDurationNsec = MAX_PREVIEW_RECORD_DURATION_NS - 1000; - /* - * The AE routine frame rate limits are computed using the frame - * duration limits, as libcamera clips the AE routine to the - * frame durations. - */ - int32_t maxFps = std::round(1e9 / minFrameDurationNsec); - int32_t minFps = std::round(1e9 / maxFrameDurationNsec); - minFps = std::max(1, minFps); + /* + * The AE routine frame rate limits are computed using the frame + * duration limits, as libcamera clips the AE routine to the + * frame durations. + */ + int32_t maxFps = std::round(1e9 / minFrameDurationNsec); + int32_t minFps = std::round(1e9 / maxFrameDurationNsec); + minFps = std::max(1, minFps); - /* - * Force rounding errors so that we have the proper frame - * durations for when we reuse these variables later - */ - minFrameDurationNsec = 1e9 / maxFps; - maxFrameDurationNsec = 1e9 / minFps; + /* + * Force rounding errors so that we have the proper frame + * durations for when we reuse these variables later + */ + minFrameDurationNsec = 1e9 / maxFps; + maxFrameDurationNsec = 1e9 / minFps; - /* - * Register to the camera service {min, max} and {max, max} - * intervals as requested by the metadata documentation. - */ - int32_t availableAeFpsTarget[] = { - minFps, maxFps, maxFps, maxFps - }; - staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES, - availableAeFpsTarget); - } + /* + * Register to the camera service {min, max} and {max, max} + * intervals as requested by the metadata documentation. + */ + int32_t availableAeFpsTarget[] = { + minFps, maxFps, maxFps, maxFps + }; + staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES, + availableAeFpsTarget); std::vector aeCompensationRange = { 0, 0, From patchwork Thu Oct 14 17:42:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14145 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8A6DAC323E for ; Thu, 14 Oct 2021 17:41:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5679268F4E; Thu, 14 Oct 2021 19:41:38 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 114D268F4F for ; Thu, 14 Oct 2021 19:41:34 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 109ADFF805; Thu, 14 Oct 2021 17:41:32 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:42:01 +0200 Message-Id: <20211014174208.50509-10-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 09/16] android: capabilities: Use per-configuration durations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the per-configuration stream durations as collected during initializeStreamConfigurations() to populate the ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_OUTPUT static metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index e22bc35d420c..bf6135ccaf02 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1291,19 +1291,16 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STALL_DURATIONS, availableStallDurations); - /* Use the minimum frame duration for all the YUV/RGB formats. */ - if (minFrameDurationNsec > 0) { - std::vector minFrameDurations; - minFrameDurations.reserve(streamConfigurations_.size() * 4); - for (const auto &entry : streamConfigurations_) { - minFrameDurations.push_back(entry.androidFormat); - minFrameDurations.push_back(entry.resolution.width); - minFrameDurations.push_back(entry.resolution.height); - minFrameDurations.push_back(minFrameDurationNsec); - } - staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, - minFrameDurations); + std::vector minFrameDurations; + minFrameDurations.reserve(streamConfigurations_.size() * 4); + for (const auto &entry : streamConfigurations_) { + minFrameDurations.push_back(entry.androidFormat); + minFrameDurations.push_back(entry.resolution.width); + minFrameDurations.push_back(entry.resolution.height); + minFrameDurations.push_back(entry.minFrameDurationNsec); } + staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, + minFrameDurations); uint8_t croppingType = ANDROID_SCALER_CROPPING_TYPE_CENTER_ONLY; staticMetadata_->addEntry(ANDROID_SCALER_CROPPING_TYPE, croppingType); From patchwork Thu Oct 14 17:42:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14146 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4C0BEC323E for ; Thu, 14 Oct 2021 17:41:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 29CC068F5A; Thu, 14 Oct 2021 19:41:40 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DA42B68F4F for ; Thu, 14 Oct 2021 19:41:35 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 3B297FF805; Thu, 14 Oct 2021 17:41:34 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:42:02 +0200 Message-Id: <20211014174208.50509-11-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 10/16] android: capabilties: Correctly populate STALL durations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We currently hardcode 2560x1920@30FPS as the only stalling frame duration. This is of course not correct, and all the required information to properly populate the ANDROID_SCALER_AVAILABLE_STALL_DURATIONS static metadata are available from initializeStaticMetadata(). Use the collected stalling durations and sizes to properly popoulate the static property. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index bf6135ccaf02..b92af5deac08 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1285,12 +1285,6 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS, availableStreamConfigurations); - std::vector availableStallDurations = { - ANDROID_SCALER_AVAILABLE_FORMATS_BLOB, 2560, 1920, 33333333, - }; - staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STALL_DURATIONS, - availableStallDurations); - std::vector minFrameDurations; minFrameDurations.reserve(streamConfigurations_.size() * 4); for (const auto &entry : streamConfigurations_) { @@ -1302,6 +1296,19 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, minFrameDurations); + std::vector availableStallDurations; + for (const auto &entry : streamConfigurations_) { + if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB) + continue; + + availableStallDurations.push_back(entry.androidFormat); + availableStallDurations.push_back(entry.resolution.width); + availableStallDurations.push_back(entry.resolution.height); + availableStallDurations.push_back(entry.minFrameDurationNsec); + } + staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STALL_DURATIONS, + availableStallDurations); + uint8_t croppingType = ANDROID_SCALER_CROPPING_TYPE_CENTER_ONLY; staticMetadata_->addEntry(ANDROID_SCALER_CROPPING_TYPE, croppingType); From patchwork Thu Oct 14 17:42:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14147 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 71283C324D for ; Thu, 14 Oct 2021 17:41:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 47D4268F69; Thu, 14 Oct 2021 19:41:40 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 194E068F4F for ; Thu, 14 Oct 2021 19:41:37 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 1018EFF805; Thu, 14 Oct 2021 17:41:35 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:42:03 +0200 Message-Id: <20211014174208.50509-12-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 11/16] android: capabilities: Collect absolute max frame durations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" While building the list of supported stream configurations also collect the absolute max frame durations to be used to populate the sensor maximum frame duration. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 9 ++++++--- src/android/camera_capabilities.h | 1 + 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index b92af5deac08..43ef2ee6ec74 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -385,6 +385,7 @@ int CameraCapabilities::initialize(std::shared_ptr camera, orientation_ = orientation; facing_ = facing; rawStreamAvailable_ = false; + maxFrameDuration_ = 0; /* Acquire the camera and initialize available stream configurations. */ int ret = camera_->acquire(); @@ -673,6 +674,9 @@ int CameraCapabilities::initializeStreamConfigurations() }); maxJpegSize = std::max(maxJpegSize, res); } + + maxFrameDuration_ = std::max(maxFrameDuration_, + maxFrameDuration); } /* @@ -1154,9 +1158,8 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE, timestampSource); - if (maxFrameDurationNsec > 0) - staticMetadata_->addEntry(ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, - maxFrameDurationNsec); + staticMetadata_->addEntry(ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, + maxFrameDuration_); /* Statistics static metadata. */ uint8_t faceDetectMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index 6e55ddab445e..2cf97ae80095 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -71,6 +71,7 @@ private: int facing_; int orientation_; bool rawStreamAvailable_; + int64_t maxFrameDuration_; camera_metadata_enum_android_info_supported_hardware_level hwLevel_; std::set capabilities_; From patchwork Thu Oct 14 17:42:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14148 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3A458C323E for ; Thu, 14 Oct 2021 17:41:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D3E4768F56; Thu, 14 Oct 2021 19:41:40 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 61E5768F5A for ; Thu, 14 Oct 2021 19:41:38 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 43228FF805; Thu, 14 Oct 2021 17:41:37 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:42:04 +0200 Message-Id: <20211014174208.50509-13-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 12/16] android: Filter preview streams on FPS X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Register as preview streams only streams capable of producing at least 30 FPS. This requirement comes from inspecting the existing HAL implementation on Intel IPU3 platform and from inspecting the CTS RecordingTests results. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 43ef2ee6ec74..bf82bbf7424e 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1279,6 +1279,23 @@ int CameraCapabilities::initializeStaticMetadata() std::vector availableStreamConfigurations; availableStreamConfigurations.reserve(streamConfigurations_.size() * 4); for (const auto &entry : streamConfigurations_) { + /* + * Filter out YUV streams not capable of running at 30 FPS. + * + * This requirement comes from CTS RecordingTest failures most + * probably related to a requirement of the camcoder video + * recording profile. Inspecting the Intel IPU3 HAL + * implementation confirms this but no reference has been found + * in the metadata documentation. + * + * Calculate FPS as CTS does: see + * Camera2SurfaceViewTestCase.java:getSuitableFpsRangeForDuration() + */ + unsigned int fps = static_cast + (floor(1e9 / entry.minFrameDurationNsec + 0.05f)); + if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) + continue; + availableStreamConfigurations.push_back(entry.androidFormat); availableStreamConfigurations.push_back(entry.resolution.width); availableStreamConfigurations.push_back(entry.resolution.height); @@ -1291,6 +1308,11 @@ int CameraCapabilities::initializeStaticMetadata() std::vector minFrameDurations; minFrameDurations.reserve(streamConfigurations_.size() * 4); for (const auto &entry : streamConfigurations_) { + unsigned int fps = static_cast + (floor(1e9 / entry.minFrameDurationNsec + 0.05f)); + if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) + continue; + minFrameDurations.push_back(entry.androidFormat); minFrameDurations.push_back(entry.resolution.width); minFrameDurations.push_back(entry.resolution.height); From patchwork Thu Oct 14 17:42:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14149 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A32E2C323E for ; Thu, 14 Oct 2021 17:41:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8255368F57; Thu, 14 Oct 2021 19:41:42 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7AF1A68F4F for ; Thu, 14 Oct 2021 19:41:39 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 77276FF805; Thu, 14 Oct 2021 17:41:38 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:42:05 +0200 Message-Id: <20211014174208.50509-14-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 13/16] android: capabilities: Print output stream list X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a debug statement to print out the list of collected output stream and their characteristics. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index bf82bbf7424e..ebe51ef206dd 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1301,6 +1301,12 @@ int CameraCapabilities::initializeStaticMetadata() availableStreamConfigurations.push_back(entry.resolution.height); availableStreamConfigurations.push_back( ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_OUTPUT); + + LOG(HAL, Debug) + << "Output Stream: " << utils::hex(entry.androidFormat) + << " (" << entry.resolution.toString() << ")[" + << entry.minFrameDurationNsec << "]" + << "@" << fps; } staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS, availableStreamConfigurations); From patchwork Thu Oct 14 17:42:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14150 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C22F3C323E for ; Thu, 14 Oct 2021 17:41:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A337268F64; Thu, 14 Oct 2021 19:41:43 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E45568F6B for ; Thu, 14 Oct 2021 19:41:40 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 9C84CFF805; Thu, 14 Oct 2021 17:41:39 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:42:06 +0200 Message-Id: <20211014174208.50509-15-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 14/16] android: Populate streams and duration in the same loop X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS and ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS static metadata are populated by looping on the streamConfigurations_ vector. Unify them in a single loop to avoid repeating it. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index ebe51ef206dd..b7c3fdbd715d 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1277,7 +1277,9 @@ int CameraCapabilities::initializeStaticMetadata() maxZoom); std::vector availableStreamConfigurations; + std::vector minFrameDurations; availableStreamConfigurations.reserve(streamConfigurations_.size() * 4); + minFrameDurations.reserve(streamConfigurations_.size() * 4); for (const auto &entry : streamConfigurations_) { /* * Filter out YUV streams not capable of running at 30 FPS. @@ -1296,12 +1298,19 @@ int CameraCapabilities::initializeStaticMetadata() if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) continue; + /* Stream configuration map. */ availableStreamConfigurations.push_back(entry.androidFormat); availableStreamConfigurations.push_back(entry.resolution.width); availableStreamConfigurations.push_back(entry.resolution.height); availableStreamConfigurations.push_back( ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_OUTPUT); + /* Per-stream durations. */ + minFrameDurations.push_back(entry.androidFormat); + minFrameDurations.push_back(entry.resolution.width); + minFrameDurations.push_back(entry.resolution.height); + minFrameDurations.push_back(entry.minFrameDurationNsec); + LOG(HAL, Debug) << "Output Stream: " << utils::hex(entry.androidFormat) << " (" << entry.resolution.toString() << ")[" @@ -1311,19 +1320,6 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS, availableStreamConfigurations); - std::vector minFrameDurations; - minFrameDurations.reserve(streamConfigurations_.size() * 4); - for (const auto &entry : streamConfigurations_) { - unsigned int fps = static_cast - (floor(1e9 / entry.minFrameDurationNsec + 0.05f)); - if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) - continue; - - minFrameDurations.push_back(entry.androidFormat); - minFrameDurations.push_back(entry.resolution.width); - minFrameDurations.push_back(entry.resolution.height); - minFrameDurations.push_back(entry.minFrameDurationNsec); - } staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, minFrameDurations); From patchwork Thu Oct 14 17:42:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14151 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0AC22C324D for ; Thu, 14 Oct 2021 17:41:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D4B1468F68; Thu, 14 Oct 2021 19:41:43 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A6DF668F57 for ; Thu, 14 Oct 2021 19:41:41 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id C4781FF805; Thu, 14 Oct 2021 17:41:40 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:42:07 +0200 Message-Id: <20211014174208.50509-16-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 15/16] android: capabilties: Fix ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As reported by the CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES documentation in the Android developer reference: "For devices advertising any color filter arrangement other than NIR, or devices not advertising color filter arrangement, this list will always include (min, max) and (max, max) where min <= 15 and max = the maximum output frame rate of the maximum YUV_420_888 output size." Collect the higher FPS of the larger YUV stream and use it with the minimum FPS rate the camera can produce to populate the ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES static metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 75 +++++++++++------------------ 1 file changed, 28 insertions(+), 47 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index b7c3fdbd715d..f2f984c564bb 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -886,53 +886,6 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_MODES, aeAvailableModes); - /* Initialize the AE frame duration limits. */ - const auto frameDurationsInfo = controlsInfo.find(&controls::FrameDurationLimits); - int64_t minFrameDurationNsec = frameDurationsInfo->second.min().get() * 1000; - int64_t maxFrameDurationNsec = frameDurationsInfo->second.max().get() * 1000; - - /* - * Adjust the minimum frame duration to comply with Android - * requirements. The camera service mandates all preview/record - * streams to have a minimum frame duration < 33,366 milliseconds - * (see MAX_PREVIEW_RECORD_DURATION_NS in the camera service - * implementation). - * - * If we're close enough (+ 500 useconds) to that value, round - * the minimum frame duration of the camera to an accepted - * value. - */ - static constexpr int64_t MAX_PREVIEW_RECORD_DURATION_NS = 1e9 / 29.97; - if (minFrameDurationNsec > MAX_PREVIEW_RECORD_DURATION_NS && - minFrameDurationNsec < MAX_PREVIEW_RECORD_DURATION_NS + 500000) - minFrameDurationNsec = MAX_PREVIEW_RECORD_DURATION_NS - 1000; - - /* - * The AE routine frame rate limits are computed using the frame - * duration limits, as libcamera clips the AE routine to the - * frame durations. - */ - int32_t maxFps = std::round(1e9 / minFrameDurationNsec); - int32_t minFps = std::round(1e9 / maxFrameDurationNsec); - minFps = std::max(1, minFps); - - /* - * Force rounding errors so that we have the proper frame - * durations for when we reuse these variables later - */ - minFrameDurationNsec = 1e9 / maxFps; - maxFrameDurationNsec = 1e9 / minFps; - - /* - * Register to the camera service {min, max} and {max, max} - * intervals as requested by the metadata documentation. - */ - int32_t availableAeFpsTarget[] = { - minFps, maxFps, maxFps, maxFps - }; - staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES, - availableAeFpsTarget); - std::vector aeCompensationRange = { 0, 0, }; @@ -1278,8 +1231,12 @@ int CameraCapabilities::initializeStaticMetadata() std::vector availableStreamConfigurations; std::vector minFrameDurations; + int maxYUVFps = 0; + Size maxYUVSize; + availableStreamConfigurations.reserve(streamConfigurations_.size() * 4); minFrameDurations.reserve(streamConfigurations_.size() * 4); + for (const auto &entry : streamConfigurations_) { /* * Filter out YUV streams not capable of running at 30 FPS. @@ -1298,6 +1255,16 @@ int CameraCapabilities::initializeStaticMetadata() if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) continue; + /* + * Collect the FPS of the maximum YUV output size to populate + * AE_AVAILABLE_TARGET_FPS_RANGE + */ + if (entry.androidFormat == HAL_PIXEL_FORMAT_YCbCr_420_888 && + entry.resolution > maxYUVSize) { + maxYUVSize = entry.resolution; + maxYUVFps = fps; + } + /* Stream configuration map. */ availableStreamConfigurations.push_back(entry.androidFormat); availableStreamConfigurations.push_back(entry.resolution.width); @@ -1323,6 +1290,20 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, minFrameDurations); + /* + * Register to the camera service {min, max} and {max, max} with + * 'max' being the larger YUV stream maximum frame rate and 'min' being + * the globally minimum frame rate rounded to the next largest integer + * as the camera service expects the camera maximum frame duration to be + * smaller than 10^9 / minFps. + */ + int32_t minFps = std::ceil(1e9 / maxFrameDuration_); + int32_t availableAeFpsTarget[] = { + minFps, maxYUVFps, maxYUVFps, maxYUVFps, + }; + staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES, + availableAeFpsTarget); + std::vector availableStallDurations; for (const auto &entry : streamConfigurations_) { if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB) From patchwork Thu Oct 14 17:42:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 14152 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 69297C323E for ; Thu, 14 Oct 2021 17:41:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1EBDB68F5F; Thu, 14 Oct 2021 19:41:44 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 96C6368F61 for ; Thu, 14 Oct 2021 19:41:42 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id CF0F3FF809; Thu, 14 Oct 2021 17:41:41 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Thu, 14 Oct 2021 19:42:08 +0200 Message-Id: <20211014174208.50509-17-jacopo@jmondi.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211014174208.50509-1-jacopo@jmondi.org> References: <20211014174208.50509-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 16/16] android: capabilities: Cap frame rate to 30 FPS X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Limit the reported minumum frame duration to 30 FPS. The reason to do is to bring the libcamra HAL in par with the Intel HAL implementation on IPU3 platform, where 30FPS is the frame rate used to perform quality tuning in the closed-source IPA module and has been validated as the most efficient rate for the power/performace budget. This change bring into the HAL a platform specific constraints, which might be opportune for most platforms but should rather be configurable by system integrators. Record that with a \todo entry. Also record that, even if we report a lower frame rate, we currently do not limit what the camera actually produce. Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/android/camera_capabilities.cpp | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index f2f984c564bb..f357902ec459 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -646,6 +646,30 @@ int CameraCapabilities::initializeStreamConfigurations() int64_t minFrameDuration = frameDurations->second.min().get() * 1000; int64_t maxFrameDuration = frameDurations->second.max().get() * 1000; + + /* + * Cap min frame duration to 30 FPS. + * + * 30 frames per second has been validated as the most + * opportune frame rate for quality tuning, and power + * vs performances budget on Intel IPU3-based + * Chromebooks. + * + * \todo This is a platform-specific decision that needs + * to be abstracted and delegated to the configuration + * file. + * + * \todo libcamera only allows to control frame duration + * through the per-request controls::FrameDuration + * control. If we cap the durations here, we should be + * capable of configuring the camera to operate at such + * duration without requiring to have the FrameDuration + * control to be specified for each Request. Defer this + * to the in-development configuration API rework. + */ + if (minFrameDuration < 1e9 / 30.0) + minFrameDuration = 1e9 / 30.0; + streamConfigurations_.push_back({ res, androidFormat, minFrameDuration, maxFrameDuration, });