From patchwork Tue Sep 7 19:40:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13732 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A1132BDB1D for ; Tue, 7 Sep 2021 19:40:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7C2E369174; Tue, 7 Sep 2021 21:40:30 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 79B536916A for ; Tue, 7 Sep 2021 21:40:28 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 89DB8240004; Tue, 7 Sep 2021 19:40:27 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:40:51 +0200 Message-Id: <20210907194107.803730-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 01/17] libcamera: ipu3: Use the optimal sensor size X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As reported by commit 7208e70211a6 ("libcamera: ipu3: Always use sensor full frame size") the current implementation of the IPU3 pipeline handler always uses the sensor resolution as the ImgU input frame size in order to work around an issue with the ImgU configuration procedure. Now that the frame selection policy has been modified in the CIO2Device class implementation to comply with the requirements of the ImgU configuration script we can remove the workaround and select the most opportune sensor size to feed the ImgU with. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain --- src/libcamera/pipeline/ipu3/ipu3.cpp | 41 ++++++++++++++++------------ 1 file changed, 23 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index c287bf86e79a..291338288685 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -229,7 +229,16 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() status = Adjusted; } - /* Validate the requested stream configuration */ + /* + * Validate the requested stream configuration and select the sensor + * format by collecting the maximum RAW stream width and height and + * picking the closest larger match. + * + * If no RAW stream is requested use the one of the largest YUV stream, + * plus margin pixels for the IF and BDS rectangle to downscale. + * + * \todo Clarify the IF and BDS margins requirements. + */ unsigned int rawCount = 0; unsigned int yuvCount = 0; Size maxYuvSize; @@ -240,7 +249,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { rawCount++; - rawSize = cfg.size; + rawSize.expandTo(cfg.size); } else { yuvCount++; maxYuvSize.expandTo(cfg.size); @@ -269,24 +278,20 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() /* * Generate raw configuration from CIO2. * - * \todo The image sensor frame size should be selected to optimize - * operations based on the sizes of the requested streams. However such - * a selection makes the pipeline configuration procedure fail for small - * resolutions (for example: 640x480 with OV5670) and causes the capture - * operations to stall for some stream size combinations (see the - * commit message of the patch that introduced this comment for more - * failure examples). + * The output YUV streams will be limited in size to the + * maximum frame size requested for the RAW stream, if present. * - * Until the sensor frame size calculation criteria are clarified, when - * capturing from ImgU always use the largest possible size which - * guarantees better results at the expense of the frame rate and CSI-2 - * bus bandwidth. When only a raw stream is requested use the requested - * size instead, as the ImgU is not involved. + * If no raw stream is requested generate a size as large as the maximum + * requested YUV size aligned to the ImgU constraints and bound by the + * sensor's maximum resolution. See + * https://bugs.libcamera.org/show_bug.cgi?id=32 */ - if (!yuvCount) - cio2Configuration_ = data_->cio2_.generateConfiguration(rawSize); - else - cio2Configuration_ = data_->cio2_.generateConfiguration({}); + if (rawSize.isNull()) + rawSize = maxYuvSize.alignedUpTo(40, 540) + .alignedUpTo(IMGU_OUTPUT_WIDTH_MARGIN, + IMGU_OUTPUT_HEIGHT_MARGIN) + .boundedTo(data_->cio2_.sensor()->resolution()); + cio2Configuration_ = data_->cio2_.generateConfiguration(rawSize); if (!cio2Configuration_.pixelFormat.isValid()) return Invalid; From patchwork Tue Sep 7 19:40:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13733 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8D31CBDB1D for ; Tue, 7 Sep 2021 19:40:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9EA2269177; Tue, 7 Sep 2021 21:40:30 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F390F6916F for ; Tue, 7 Sep 2021 21:40:28 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 7E7DF240005; Tue, 7 Sep 2021 19:40:28 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:40:52 +0200 Message-Id: <20210907194107.803730-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 02/17] libcamera: ipu3: Centralize ImgU sizes definition X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The definition of several constants that describe the ImgU characteristics are spread between two files: ipu3.cpp and imgu.cpp. As the ipu3.cpp uses definitions from the imgu.cpp one, in order to remove the usage of magic numbers, it is required to move the definitions to a common header file where they are accessible to the other .cpp modules. Move all the definitions of the ImgU sizes and alignment in a dedicated namespace in imgu.h and update their users accordingly. Cosmetic changes, no functional changes intended. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Jean-Michel Hautbois --- src/libcamera/pipeline/ipu3/imgu.cpp | 86 +++++++++++----------------- src/libcamera/pipeline/ipu3/imgu.h | 27 +++++++++ src/libcamera/pipeline/ipu3/ipu3.cpp | 47 +++++++-------- 3 files changed, 83 insertions(+), 77 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/imgu.cpp b/src/libcamera/pipeline/ipu3/imgu.cpp index 317e482a1498..441ff1b0705c 100644 --- a/src/libcamera/pipeline/ipu3/imgu.cpp +++ b/src/libcamera/pipeline/ipu3/imgu.cpp @@ -34,22 +34,6 @@ namespace { * at revision: 243d13446e44 ("Fix some bug for some resolutions") */ -static constexpr unsigned int FILTER_W = 4; -static constexpr unsigned int FILTER_H = 4; - -static constexpr unsigned int IF_ALIGN_W = 2; -static constexpr unsigned int IF_ALIGN_H = 4; - -static constexpr unsigned int BDS_ALIGN_W = 2; -static constexpr unsigned int BDS_ALIGN_H = 4; - -static constexpr unsigned int IF_CROP_MAX_W = 40; -static constexpr unsigned int IF_CROP_MAX_H = 540; - -static constexpr float BDS_SF_MAX = 2.5; -static constexpr float BDS_SF_MIN = 1.0; -static constexpr float BDS_SF_STEP = 0.03125; - /* BSD scaling factors: min=1, max=2.5, step=1/32 */ const std::vector bdsScalingFactors = { 1, 1.03125, 1.0625, 1.09375, 1.125, 1.15625, 1.1875, 1.21875, 1.25, @@ -124,8 +108,8 @@ bool isSameRatio(const Size &in, const Size &out) void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc, unsigned int bdsWidth, float bdsSF) { - unsigned int minIFHeight = iif.height - IF_CROP_MAX_H; - unsigned int minBDSHeight = gdc.height + FILTER_H * 2; + unsigned int minIFHeight = iif.height - IMGU::IF_CROP_MAX_H; + unsigned int minBDSHeight = gdc.height + IMGU::FILTER_H * 2; unsigned int ifHeight; float bdsHeight; @@ -135,7 +119,7 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc static_cast(gdc.width); estIFHeight = std::clamp(estIFHeight, minIFHeight, iif.height); - ifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H); + ifHeight = utils::alignUp(estIFHeight, IMGU::IF_ALIGN_H); while (ifHeight >= minIFHeight && ifHeight <= iif.height && ifHeight / bdsSF >= minBDSHeight) { @@ -143,17 +127,17 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc if (std::fmod(height, 1.0) == 0) { unsigned int bdsIntHeight = static_cast(height); - if (!(bdsIntHeight % BDS_ALIGN_H)) { + if (!(bdsIntHeight % IMGU::BDS_ALIGN_H)) { foundIfHeight = ifHeight; bdsHeight = height; break; } } - ifHeight -= IF_ALIGN_H; + ifHeight -= IMGU::IF_ALIGN_H; } - ifHeight = utils::alignUp(estIFHeight, IF_ALIGN_H); + ifHeight = utils::alignUp(estIFHeight, IMGU::IF_ALIGN_H); while (ifHeight >= minIFHeight && ifHeight <= iif.height && ifHeight / bdsSF >= minBDSHeight) { @@ -161,14 +145,14 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc if (std::fmod(height, 1.0) == 0) { unsigned int bdsIntHeight = static_cast(height); - if (!(bdsIntHeight % BDS_ALIGN_H)) { + if (!(bdsIntHeight % IMGU::BDS_ALIGN_H)) { foundIfHeight = ifHeight; bdsHeight = height; break; } } - ifHeight += IF_ALIGN_H; + ifHeight += IMGU::IF_ALIGN_H; } if (foundIfHeight) { @@ -179,32 +163,32 @@ void calculateBDSHeight(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc return; } } else { - ifHeight = utils::alignUp(iif.height, IF_ALIGN_H); + ifHeight = utils::alignUp(iif.height, IMGU::IF_ALIGN_H); while (ifHeight >= minIFHeight && ifHeight / bdsSF >= minBDSHeight) { bdsHeight = ifHeight / bdsSF; if (std::fmod(ifHeight, 1.0) == 0 && std::fmod(bdsHeight, 1.0) == 0) { unsigned int bdsIntHeight = static_cast(bdsHeight); - if (!(ifHeight % IF_ALIGN_H) && - !(bdsIntHeight % BDS_ALIGN_H)) { + if (!(ifHeight % IMGU::IF_ALIGN_H) && + !(bdsIntHeight % IMGU::BDS_ALIGN_H)) { pipeConfigs.push_back({ bdsSF, { iif.width, ifHeight }, { bdsWidth, bdsIntHeight }, gdc }); } } - ifHeight -= IF_ALIGN_H; + ifHeight -= IMGU::IF_ALIGN_H; } } } void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc, float bdsSF) { - unsigned int minBDSWidth = gdc.width + FILTER_W * 2; - unsigned int minBDSHeight = gdc.height + FILTER_H * 2; + unsigned int minBDSWidth = gdc.width + IMGU::FILTER_W * 2; + unsigned int minBDSHeight = gdc.height + IMGU::FILTER_H * 2; float sf = bdsSF; - while (sf <= BDS_SF_MAX && sf >= BDS_SF_MIN) { + while (sf <= IMGU::BDS_SF_MAX && sf >= IMGU::BDS_SF_MIN) { float bdsWidth = static_cast(iif.width) / sf; float bdsHeight = static_cast(iif.height) / sf; @@ -212,16 +196,16 @@ void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc, floa std::fmod(bdsHeight, 1.0) == 0) { unsigned int bdsIntWidth = static_cast(bdsWidth); unsigned int bdsIntHeight = static_cast(bdsHeight); - if (!(bdsIntWidth % BDS_ALIGN_W) && bdsWidth >= minBDSWidth && - !(bdsIntHeight % BDS_ALIGN_H) && bdsHeight >= minBDSHeight) + if (!(bdsIntWidth % IMGU::BDS_ALIGN_W) && bdsWidth >= minBDSWidth && + !(bdsIntHeight % IMGU::BDS_ALIGN_H) && bdsHeight >= minBDSHeight) calculateBDSHeight(pipe, iif, gdc, bdsIntWidth, sf); } - sf += BDS_SF_STEP; + sf += IMGU::BDS_SF_STEP; } sf = bdsSF; - while (sf <= BDS_SF_MAX && sf >= BDS_SF_MIN) { + while (sf <= IMGU::BDS_SF_MAX && sf >= IMGU::BDS_SF_MIN) { float bdsWidth = static_cast(iif.width) / sf; float bdsHeight = static_cast(iif.height) / sf; @@ -229,12 +213,12 @@ void calculateBDS(ImgUDevice::Pipe *pipe, const Size &iif, const Size &gdc, floa std::fmod(bdsHeight, 1.0) == 0) { unsigned int bdsIntWidth = static_cast(bdsWidth); unsigned int bdsIntHeight = static_cast(bdsHeight); - if (!(bdsIntWidth % BDS_ALIGN_W) && bdsWidth >= minBDSWidth && - !(bdsIntHeight % BDS_ALIGN_H) && bdsHeight >= minBDSHeight) + if (!(bdsIntWidth % IMGU::BDS_ALIGN_W) && bdsWidth >= minBDSWidth && + !(bdsIntHeight % IMGU::BDS_ALIGN_H) && bdsHeight >= minBDSHeight) calculateBDSHeight(pipe, iif, gdc, bdsIntWidth, sf); } - sf -= BDS_SF_STEP; + sf -= IMGU::BDS_SF_STEP; } } @@ -412,7 +396,7 @@ ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe) * \todo Filter out all resolutions < IF_CROP_MAX. * See https://bugs.libcamera.org/show_bug.cgi?id=32 */ - if (in.width < IF_CROP_MAX_W || in.height < IF_CROP_MAX_H) { + if (in.width < IMGU::IF_CROP_MAX_W || in.height < IMGU::IF_CROP_MAX_H) { LOG(IPU3, Error) << "Input resolution " << in.toString() << " not supported"; return {}; @@ -424,25 +408,25 @@ ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe) float sf = findScaleFactor(bdsSF, bdsScalingFactors, true); /* Populate the configurations vector by scaling width and height. */ - unsigned int ifWidth = utils::alignUp(in.width, IF_ALIGN_W); - unsigned int ifHeight = utils::alignUp(in.height, IF_ALIGN_H); - unsigned int minIfWidth = in.width - IF_CROP_MAX_W; - unsigned int minIfHeight = in.height - IF_CROP_MAX_H; + unsigned int ifWidth = utils::alignUp(in.width, IMGU::IF_ALIGN_W); + unsigned int ifHeight = utils::alignUp(in.height, IMGU::IF_ALIGN_H); + unsigned int minIfWidth = in.width - IMGU::IF_CROP_MAX_W; + unsigned int minIfHeight = in.height - IMGU::IF_CROP_MAX_H; while (ifWidth >= minIfWidth) { while (ifHeight >= minIfHeight) { Size iif{ ifWidth, ifHeight }; calculateBDS(pipe, iif, gdc, sf); - ifHeight -= IF_ALIGN_H; + ifHeight -= IMGU::IF_ALIGN_H; } - ifWidth -= IF_ALIGN_W; + ifWidth -= IMGU::IF_ALIGN_W; } /* Repeat search by scaling width first. */ - ifWidth = utils::alignUp(in.width, IF_ALIGN_W); - ifHeight = utils::alignUp(in.height, IF_ALIGN_H); - minIfWidth = in.width - IF_CROP_MAX_W; - minIfHeight = in.height - IF_CROP_MAX_H; + ifWidth = utils::alignUp(in.width, IMGU::IF_ALIGN_W); + ifHeight = utils::alignUp(in.height, IMGU::IF_ALIGN_H); + minIfWidth = in.width - IMGU::IF_CROP_MAX_W; + minIfHeight = in.height - IMGU::IF_CROP_MAX_H; while (ifHeight >= minIfHeight) { /* * \todo This procedure is probably broken: @@ -451,10 +435,10 @@ ImgUDevice::PipeConfig ImgUDevice::calculatePipeConfig(Pipe *pipe) while (ifWidth >= minIfWidth) { Size iif{ ifWidth, ifHeight }; calculateBDS(pipe, iif, gdc, sf); - ifWidth -= IF_ALIGN_W; + ifWidth -= IMGU::IF_ALIGN_W; } - ifHeight -= IF_ALIGN_H; + ifHeight -= IMGU::IF_ALIGN_H; } if (pipeConfigs.size() == 0) { diff --git a/src/libcamera/pipeline/ipu3/imgu.h b/src/libcamera/pipeline/ipu3/imgu.h index 9d4915116087..df64cbaba5a7 100644 --- a/src/libcamera/pipeline/ipu3/imgu.h +++ b/src/libcamera/pipeline/ipu3/imgu.h @@ -15,6 +15,33 @@ namespace libcamera { +namespace IMGU { + +static constexpr unsigned int FILTER_W = 4; +static constexpr unsigned int FILTER_H = 4; + +static constexpr unsigned int IF_ALIGN_W = 2; +static constexpr unsigned int IF_ALIGN_H = 4; + +static constexpr unsigned int IF_CROP_MAX_W = 40; +static constexpr unsigned int IF_CROP_MAX_H = 540; + +static constexpr unsigned int BDS_ALIGN_W = 2; +static constexpr unsigned int BDS_ALIGN_H = 4; + +static constexpr float BDS_SF_MAX = 2.5; +static constexpr float BDS_SF_MIN = 1.0; +static constexpr float BDS_SF_STEP = 0.03125; + +static const Size OUTPUT_MIN_SIZE = { 2, 2 }; +static const Size OUTPUT_MAX_SIZE = { 4480, 34004 }; +static constexpr unsigned int OUTPUT_WIDTH_ALIGN = 64; +static constexpr unsigned int OUTPUT_HEIGHT_ALIGN = 4; +static constexpr unsigned int OUTPUT_WIDTH_MARGIN = 64; +static constexpr unsigned int OUTPUT_HEIGHT_MARGIN = 32; + +} /* namespace IMGU */ + class FrameBuffer; class MediaDevice; class Size; diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 291338288685..89a05fab69ad 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -41,12 +41,6 @@ LOG_DEFINE_CATEGORY(IPU3) static constexpr unsigned int IPU3_BUFFER_COUNT = 4; static constexpr unsigned int IPU3_MAX_STREAMS = 3; -static const Size IMGU_OUTPUT_MIN_SIZE = { 2, 2 }; -static const Size IMGU_OUTPUT_MAX_SIZE = { 4480, 34004 }; -static constexpr unsigned int IMGU_OUTPUT_WIDTH_ALIGN = 64; -static constexpr unsigned int IMGU_OUTPUT_HEIGHT_ALIGN = 4; -static constexpr unsigned int IMGU_OUTPUT_WIDTH_MARGIN = 64; -static constexpr unsigned int IMGU_OUTPUT_HEIGHT_MARGIN = 32; static constexpr Size IPU3ViewfinderSize(1280, 720); static const ControlInfoMap::Map IPU3Controls = { @@ -287,9 +281,10 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() * https://bugs.libcamera.org/show_bug.cgi?id=32 */ if (rawSize.isNull()) - rawSize = maxYuvSize.alignedUpTo(40, 540) - .alignedUpTo(IMGU_OUTPUT_WIDTH_MARGIN, - IMGU_OUTPUT_HEIGHT_MARGIN) + rawSize = maxYuvSize.alignedUpTo(IMGU::IF_CROP_MAX_W, + IMGU::IF_CROP_MAX_H) + .alignedUpTo(IMGU::OUTPUT_WIDTH_MARGIN, + IMGU::OUTPUT_HEIGHT_MARGIN) .boundedTo(data_->cio2_.sensor()->resolution()); cio2Configuration_ = data_->cio2_.generateConfiguration(rawSize); if (!cio2Configuration_.pixelFormat.isValid()) @@ -345,19 +340,19 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate() */ unsigned int limit; limit = utils::alignDown(cio2Configuration_.size.width - 1, - IMGU_OUTPUT_WIDTH_MARGIN); + IMGU::OUTPUT_WIDTH_MARGIN); cfg->size.width = std::clamp(cfg->size.width, - IMGU_OUTPUT_MIN_SIZE.width, + IMGU::OUTPUT_MIN_SIZE.width, limit); limit = utils::alignDown(cio2Configuration_.size.height - 1, - IMGU_OUTPUT_HEIGHT_MARGIN); + IMGU::OUTPUT_HEIGHT_MARGIN); cfg->size.height = std::clamp(cfg->size.height, - IMGU_OUTPUT_MIN_SIZE.height, + IMGU::OUTPUT_MIN_SIZE.height, limit); - cfg->size.alignDownTo(IMGU_OUTPUT_WIDTH_ALIGN, - IMGU_OUTPUT_HEIGHT_ALIGN); + cfg->size.alignDownTo(IMGU::OUTPUT_WIDTH_ALIGN, + IMGU::OUTPUT_HEIGHT_ALIGN); cfg->pixelFormat = formats::NV12; cfg->bufferCount = IPU3_BUFFER_COUNT; @@ -443,14 +438,14 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, * \todo Clarify the alignment constraints as explained * in validate() */ - size = sensorResolution.boundedTo(IMGU_OUTPUT_MAX_SIZE); + size = sensorResolution.boundedTo(IMGU::OUTPUT_MAX_SIZE); size.width = utils::alignDown(size.width - 1, - IMGU_OUTPUT_WIDTH_MARGIN); + IMGU::OUTPUT_WIDTH_MARGIN); size.height = utils::alignDown(size.height - 1, - IMGU_OUTPUT_HEIGHT_MARGIN); + IMGU::OUTPUT_HEIGHT_MARGIN); pixelFormat = formats::NV12; bufferCount = IPU3_BUFFER_COUNT; - streamFormats[pixelFormat] = { { IMGU_OUTPUT_MIN_SIZE, size } }; + streamFormats[pixelFormat] = { { IMGU::OUTPUT_MIN_SIZE, size } }; break; @@ -475,11 +470,11 @@ CameraConfiguration *PipelineHandlerIPU3::generateConfiguration(Camera *camera, * to the ImgU output constraints. */ size = sensorResolution.boundedTo(IPU3ViewfinderSize) - .alignedDownTo(IMGU_OUTPUT_WIDTH_ALIGN, - IMGU_OUTPUT_HEIGHT_ALIGN); + .alignedDownTo(IMGU::OUTPUT_WIDTH_ALIGN, + IMGU::OUTPUT_HEIGHT_ALIGN); pixelFormat = formats::NV12; bufferCount = IPU3_BUFFER_COUNT; - streamFormats[pixelFormat] = { { IMGU_OUTPUT_MIN_SIZE, size } }; + streamFormats[pixelFormat] = { { IMGU::OUTPUT_MIN_SIZE, size } }; break; } @@ -1003,8 +998,8 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) /* The strictly smaller size than the sensor resolution, aligned to margins. */ Size minSize = Size(sensor->resolution().width - 1, sensor->resolution().height - 1) - .alignedDownTo(IMGU_OUTPUT_WIDTH_MARGIN, - IMGU_OUTPUT_HEIGHT_MARGIN); + .alignedDownTo(IMGU::OUTPUT_WIDTH_MARGIN, + IMGU::OUTPUT_HEIGHT_MARGIN); /* * Either the smallest margin-aligned size larger than the viewfinder @@ -1012,8 +1007,8 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) */ minSize = Size(IPU3ViewfinderSize.width + 1, IPU3ViewfinderSize.height + 1) - .alignedUpTo(IMGU_OUTPUT_WIDTH_MARGIN, - IMGU_OUTPUT_HEIGHT_MARGIN) + .alignedUpTo(IMGU::OUTPUT_WIDTH_MARGIN, + IMGU::OUTPUT_HEIGHT_MARGIN) .boundedTo(minSize); /* From patchwork Tue Sep 7 19:40:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13734 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6AD41BDB1D for ; Tue, 7 Sep 2021 19:40:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 470846917E; Tue, 7 Sep 2021 21:40:34 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E94C36916A for ; Tue, 7 Sep 2021 21:40:29 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 25BCA240005; Tue, 7 Sep 2021 19:40:28 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:40:53 +0200 Message-Id: <20210907194107.803730-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 03/17] libcamera: ipu3: Split controls init/update X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to prepare for updating the Camera controls limits when a new camera configuration is applied, split the initControls() function in two: - updateControls() to actually compute controls values - initControls() to initialize the sensor configuration and call updateControls Update the functions documentation accordingly. No functional changes intended. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain --- src/libcamera/pipeline/ipu3/ipu3.cpp | 33 ++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 89a05fab69ad..1c383336e45f 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -148,6 +148,7 @@ private: } int initControls(IPU3CameraData *data); + int updateControls(IPU3CameraData *data); int registerCameras(); int allocateBuffers(Camera *camera); @@ -927,9 +928,11 @@ bool PipelineHandlerIPU3::match(DeviceEnumerator *enumerator) * \brief Initialize the camera controls * \param[in] data The camera data * - * Initialize the camera controls as the union of the static pipeline handler - * controls (IPU3Controls) and controls created dynamically from the sensor - * capabilities. + * Initialize the camera controls by calculating controls which the pipeline + * is reponsible for and merge them with the controls computed by the IPA. + * + * This function needs data->ipaControls_ to be initialized by the IPA init() + * function at camera creation time. Always call this function after IPA init(). * * \return 0 on success or a negative error code otherwise */ @@ -950,8 +953,30 @@ int PipelineHandlerIPU3::initControls(IPU3CameraData *data) if (ret) return ret; + return updateControls(data); +} + +/** + * \brief Update the camera controls + * \param[in] data The camera data + * + * Compute the camera controls by calculating controls which the pipeline + * is reponsible for and merge them with the controls computed by the IPA. + * + * This function needs data->ipaControls_ to be refreshed when a new + * configuration is applied to the camera by the IPA configure() function. + * + * Always call this function after IPA configure() to make sure to have a + * properly refreshed IPA controls list. + * + * \return 0 on success or a negative error code otherwise + */ +int PipelineHandlerIPU3::updateControls(IPU3CameraData *data) +{ + CameraSensor *sensor = data->cio2_.sensor(); IPACameraSensorInfo sensorInfo{}; - ret = sensor->sensorInfo(&sensorInfo); + + int ret = sensor->sensorInfo(&sensorInfo); if (ret) return ret; From patchwork Tue Sep 7 19:40:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13735 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 014C3BDB1D for ; Tue, 7 Sep 2021 19:40:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6414569182; Tue, 7 Sep 2021 21:40:34 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E6F2169170 for ; Tue, 7 Sep 2021 21:40:30 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 1F9F1240006; Tue, 7 Sep 2021 19:40:29 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:40:54 +0200 Message-Id: <20210907194107.803730-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 04/17] ipa: ipu3: Update camera controls in configure() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a new CameraConfiguration is applied to the Camera the IPA is configured as well, using the newly applied sensor configuration and its updated V4L2 controls. Also update the Camera controls at IPA::configure() time by re-computing the controls::ExposureTime and controls::FrameDurationLimits limits and update the controls on the pipeline handler side after having configured the IPA. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/ipu3.mojom | 3 +- src/ipa/ipu3/ipu3.cpp | 82 +++++++++++++++++++--------- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +- 3 files changed, 60 insertions(+), 29 deletions(-) diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index d561c2244907..714f58ab8a42 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -45,7 +45,8 @@ interface IPAIPU3Interface { start() => (int32 ret); stop(); - configure(IPAConfigInfo configInfo) => (int32 ret); + configure(IPAConfigInfo configInfo) + => (int32 ret, libcamera.ControlInfoMap ipaControls); mapBuffers(array buffers); unmapBuffers(array ids); diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index c925cf642611..ec75b3ffa305 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -156,13 +156,17 @@ public: int start() override; void stop() override {} - int configure(const IPAConfigInfo &configInfo) override; + int configure(const IPAConfigInfo &configInfo, + ControlInfoMap *ipaControls) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; void processEvent(const IPU3Event &event) override; private: + void updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls); void processControls(unsigned int frame, const ControlList &controls); void fillParams(unsigned int frame, ipu3_uapi_params *params); void parseStatistics(unsigned int frame, @@ -197,36 +201,30 @@ private: struct IPAContext context_; }; -/** - * Initialize the IPA module and its controls. + +/* + * Compute camera controls using the sensor information and the sensor + * v4l2 controls. * - * This function receives the camera sensor information from the pipeline - * handler, computes the limits of the controls it handles and returns - * them in the \a ipaControls output parameter. + * Some of the camera controls are computed by the pipeline handler, some others + * by the IPA module which is in charge of handling, in example, the exposure + * time and the frame duration. + * + * This functions computes: + * - controls::ExposureTime + * - controls::FrameDurationLimits */ -int IPAIPU3::init(const IPASettings &settings, - const IPACameraSensorInfo &sensorInfo, - const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) +void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) { - camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); - if (camHelper_ == nullptr) { - LOG(IPAIPU3, Error) - << "Failed to create camera sensor helper for " - << settings.sensorModel; - return -ENODEV; - } - - /* Initialize Controls. */ ControlInfoMap::Map controls{}; /* - * Compute exposure time limits. - * - * Initialize the control using the line length and pixel rate of the - * current configuration converted to microseconds. Use the - * V4L2_CID_EXPOSURE control to get exposure min, max and default and - * convert it from lines to microseconds. + * Compute exposure time limits by using line length and pixel rate + * converted to microseconds. Use the V4L2_CID_EXPOSURE control to get + * exposure min, max and default and convert it from lines to + * microseconds. */ double lineDuration = sensorInfo.lineLength / (sensorInfo.pixelRate / 1e6); const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; @@ -264,12 +262,36 @@ int IPAIPU3::init(const IPASettings &settings, frameDurations[2]); *ipaControls = ControlInfoMap(std::move(controls), controls::controls); +} + +/** + * Initialize the IPA module and its controls. + * + * This function receives the camera sensor information from the pipeline + * handler, computes the limits of the controls it handles and returns + * them in the \a ipaControls output parameter. + */ +int IPAIPU3::init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + camHelper_ = CameraSensorHelperFactory::create(settings.sensorModel); + if (camHelper_ == nullptr) { + LOG(IPAIPU3, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } /* Construct our Algorithms */ algorithms_.push_back(std::make_unique()); algorithms_.push_back(std::make_unique()); algorithms_.push_back(std::make_unique()); + /* Initialize controls. */ + updateControls(sensorInfo, sensorControls, ipaControls); + return 0; } @@ -335,7 +357,8 @@ void IPAIPU3::calculateBdsGrid(const Size &bdsOutputSize) << (int)bdsGrid.height << " << " << (int)bdsGrid.block_height_log2 << ")"; } -int IPAIPU3::configure(const IPAConfigInfo &configInfo) +int IPAIPU3::configure(const IPAConfigInfo &configInfo, + ControlInfoMap *ipaControls) { if (configInfo.entityControls.empty()) { LOG(IPAIPU3, Error) << "No controls provided"; @@ -344,6 +367,10 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) sensorInfo_ = configInfo.sensorInfo; + /* + * Compute the sensor V4L2 controls to be used by the algorithms and + * to be set on the sensor. + */ ctrls_ = configInfo.entityControls.at(0); const auto itExp = ctrls_.find(V4L2_CID_EXPOSURE); @@ -385,6 +412,9 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo) return ret; } + /* Update the camera controls using the new sensor settings. */ + updateControls(sensorInfo_, ctrls_, ipaControls); + return 0; } diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 1c383336e45f..de0adffc965f 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -660,14 +660,14 @@ int PipelineHandlerIPU3::configure(Camera *camera, CameraConfiguration *c) configInfo.bdsOutputSize = config->imguConfig().bds; configInfo.iif = config->imguConfig().iif; - ret = data->ipa_->configure(configInfo); + ret = data->ipa_->configure(configInfo, &data->ipaControls_); if (ret) { LOG(IPU3, Error) << "Failed to configure IPA: " << strerror(-ret); return ret; } - return 0; + return updateControls(data); } int PipelineHandlerIPU3::exportFrameBuffers(Camera *camera, Stream *stream, From patchwork Tue Sep 7 19:40:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13736 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 91BC4BDB1D for ; Tue, 7 Sep 2021 19:40:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B63A86917D; Tue, 7 Sep 2021 21:40:34 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0890469171 for ; Tue, 7 Sep 2021 21:40:32 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 1BA6D240004; Tue, 7 Sep 2021 19:40:30 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:40:55 +0200 Message-Id: <20210907194107.803730-6-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 05/17] android: capabilities: Collect per-stream frame durations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Collect the per-stream frame durations while building the list of supported stream formats and resolutions. In order to get an updated list of controls it is necessary to apply to the Camera the configuration we're testing, which was so far only validated. The per-configuration durations will be used to populate the Android ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS static metadata. Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 36 ++++++++++++++++++++++++++--- src/android/camera_capabilities.h | 2 ++ 2 files changed, 35 insertions(+), 3 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index e92bca42779a..4e82f12a904e 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -608,7 +608,29 @@ int CameraCapabilities::initializeStreamConfigurations() } for (const Size &res : resolutions) { - streamConfigurations_.push_back({ res, androidFormat }); + /* + * Configure the Camera with the collected format and + * resolution to get an updated list of controls. + */ + cfg.size = res; + int ret = camera_->configure(cameraConfig.get()); + if (ret) + return ret; + + const ControlInfoMap &controls = camera_->controls(); + const auto frameDurations = controls.find( + &controls::FrameDurationLimits); + if (frameDurations == controls.end()) { + LOG(HAL, Error) + << "Camera does not report frame durations"; + return -EINVAL; + } + + int64_t minFrameDuration = frameDurations->second.min().get() * 1000; + int64_t maxFrameDuration = frameDurations->second.max().get() * 1000; + streamConfigurations_.push_back({ + res, androidFormat, minFrameDuration, maxFrameDuration, + }); /* * If the format is HAL_PIXEL_FORMAT_YCbCr_420_888 @@ -620,10 +642,18 @@ int CameraCapabilities::initializeStreamConfigurations() * * \todo Support JPEG streams produced by the camera * natively. + * + * \todo HAL_PIXEL_FORMAT_BLOB is a 'stalling' format, + * its duration should take into account the time + * required for the YUV to JPEG encoding. For now + * use the same frame durations as collected for + * the YUV/RGB streams. */ if (androidFormat == HAL_PIXEL_FORMAT_YCbCr_420_888) { - streamConfigurations_.push_back( - { res, HAL_PIXEL_FORMAT_BLOB }); + streamConfigurations_.push_back({ + res, HAL_PIXEL_FORMAT_BLOB, + minFrameDuration, maxFrameDuration, + }); maxJpegSize = std::max(maxJpegSize, res); } } diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index a12596993ee5..6e55ddab445e 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -43,6 +43,8 @@ private: struct Camera3StreamConfiguration { libcamera::Size resolution; int androidFormat; + int64_t minFrameDurationNsec; + int64_t maxFrameDurationNsec; }; bool validateManualSensorCapability(); From patchwork Tue Sep 7 19:40:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13737 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 19228BDB1D for ; Tue, 7 Sep 2021 19:40:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6B17969173; Tue, 7 Sep 2021 21:40:35 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 19EDC6916F for ; Tue, 7 Sep 2021 21:40:33 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 3490F240004; Tue, 7 Sep 2021 19:40:31 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:40:56 +0200 Message-Id: <20210907194107.803730-7-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 06/17] android: capabilities: Initialize camera state when building properties X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that building the list of supported stream configuration requires applying a configuration to the Camera, re-initialize the camera controls by applying a configuration generated for the Viewfinder stream role before building the list of static metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 4e82f12a904e..9bfc7ebc6334 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -394,11 +394,14 @@ int CameraCapabilities::initialize(std::shared_ptr camera, } ret = initializeStreamConfigurations(); - camera_->release(); - if (ret) + if (ret) { + camera_->release(); return ret; + } - return initializeStaticMetadata(); + ret = initializeStaticMetadata(); + camera_->release(); + return ret; } std::vector @@ -682,6 +685,19 @@ int CameraCapabilities::initializeStaticMetadata() return -EINVAL; } + /* + * Generate and apply a new configuration for the Viewfinder role to + * collect control limits and properties from a known state. + */ + std::unique_ptr cameraConfig = + camera_->generateConfiguration({ StreamRole::Viewfinder }); + int ret = camera_->configure(cameraConfig.get()); + if (ret) { + LOG(HAL, Error) << "Failed to initialize the camera state"; + staticMetadata_.reset(); + return ret; + } + const ControlInfoMap &controlsInfo = camera_->controls(); const ControlList &properties = camera_->properties(); From patchwork Tue Sep 7 19:40:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13738 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E33A0BDB1D for ; Tue, 7 Sep 2021 19:40:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ADEE869187; Tue, 7 Sep 2021 21:40:37 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0E3B169173 for ; Tue, 7 Sep 2021 21:40:34 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 3C825240004; Tue, 7 Sep 2021 19:40:33 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:40:57 +0200 Message-Id: <20210907194107.803730-8-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 07/17] android: capabilties: Assume controls::FrameDurationLimits is supported X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As we now collect the per-stream frame durations at initializeStreamConfigurations() times, the Camera is now guaranteed to support the controls::FrameDurationLimits control. Remove the check for its presence when populating the ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES static metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 85 ++++++++++++++--------------- 1 file changed, 41 insertions(+), 44 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 9bfc7ebc6334..ee2883ab2821 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -862,55 +862,52 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_MODES, aeAvailableModes); - int64_t minFrameDurationNsec = -1; - int64_t maxFrameDurationNsec = -1; + /* Initialize the AE frame duration limits. */ const auto frameDurationsInfo = controlsInfo.find(&controls::FrameDurationLimits); - if (frameDurationsInfo != controlsInfo.end()) { - minFrameDurationNsec = frameDurationsInfo->second.min().get() * 1000; - maxFrameDurationNsec = frameDurationsInfo->second.max().get() * 1000; + int64_t minFrameDurationNsec = frameDurationsInfo->second.min().get() * 1000; + int64_t maxFrameDurationNsec = frameDurationsInfo->second.max().get() * 1000; - /* - * Adjust the minimum frame duration to comply with Android - * requirements. The camera service mandates all preview/record - * streams to have a minimum frame duration < 33,366 milliseconds - * (see MAX_PREVIEW_RECORD_DURATION_NS in the camera service - * implementation). - * - * If we're close enough (+ 500 useconds) to that value, round - * the minimum frame duration of the camera to an accepted - * value. - */ - static constexpr int64_t MAX_PREVIEW_RECORD_DURATION_NS = 1e9 / 29.97; - if (minFrameDurationNsec > MAX_PREVIEW_RECORD_DURATION_NS && - minFrameDurationNsec < MAX_PREVIEW_RECORD_DURATION_NS + 500000) - minFrameDurationNsec = MAX_PREVIEW_RECORD_DURATION_NS - 1000; + /* + * Adjust the minimum frame duration to comply with Android + * requirements. The camera service mandates all preview/record + * streams to have a minimum frame duration < 33,366 milliseconds + * (see MAX_PREVIEW_RECORD_DURATION_NS in the camera service + * implementation). + * + * If we're close enough (+ 500 useconds) to that value, round + * the minimum frame duration of the camera to an accepted + * value. + */ + static constexpr int64_t MAX_PREVIEW_RECORD_DURATION_NS = 1e9 / 29.97; + if (minFrameDurationNsec > MAX_PREVIEW_RECORD_DURATION_NS && + minFrameDurationNsec < MAX_PREVIEW_RECORD_DURATION_NS + 500000) + minFrameDurationNsec = MAX_PREVIEW_RECORD_DURATION_NS - 1000; - /* - * The AE routine frame rate limits are computed using the frame - * duration limits, as libcamera clips the AE routine to the - * frame durations. - */ - int32_t maxFps = std::round(1e9 / minFrameDurationNsec); - int32_t minFps = std::round(1e9 / maxFrameDurationNsec); - minFps = std::max(1, minFps); + /* + * The AE routine frame rate limits are computed using the frame + * duration limits, as libcamera clips the AE routine to the + * frame durations. + */ + int32_t maxFps = std::round(1e9 / minFrameDurationNsec); + int32_t minFps = std::round(1e9 / maxFrameDurationNsec); + minFps = std::max(1, minFps); - /* - * Force rounding errors so that we have the proper frame - * durations for when we reuse these variables later - */ - minFrameDurationNsec = 1e9 / maxFps; - maxFrameDurationNsec = 1e9 / minFps; + /* + * Force rounding errors so that we have the proper frame + * durations for when we reuse these variables later + */ + minFrameDurationNsec = 1e9 / maxFps; + maxFrameDurationNsec = 1e9 / minFps; - /* - * Register to the camera service {min, max} and {max, max} - * intervals as requested by the metadata documentation. - */ - int32_t availableAeFpsTarget[] = { - minFps, maxFps, maxFps, maxFps - }; - staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES, - availableAeFpsTarget); - } + /* + * Register to the camera service {min, max} and {max, max} + * intervals as requested by the metadata documentation. + */ + int32_t availableAeFpsTarget[] = { + minFps, maxFps, maxFps, maxFps + }; + staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES, + availableAeFpsTarget); std::vector aeCompensationRange = { 0, 0, From patchwork Tue Sep 7 19:40:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13739 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3E84DBEFBE for ; Tue, 7 Sep 2021 19:40:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DB0856918C; Tue, 7 Sep 2021 21:40:37 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0A3B669185 for ; Tue, 7 Sep 2021 21:40:35 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 342D3240004; Tue, 7 Sep 2021 19:40:34 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:40:58 +0200 Message-Id: <20210907194107.803730-9-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 08/17] android: capabilities: Use per-configuration durations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the per-configuration stream durations as collected during initializeStreamConfigurations() to populate the ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_OUTPUT static metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index ee2883ab2821..5d8293843f57 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1271,19 +1271,16 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STALL_DURATIONS, availableStallDurations); - /* Use the minimum frame duration for all the YUV/RGB formats. */ - if (minFrameDurationNsec > 0) { - std::vector minFrameDurations; - minFrameDurations.reserve(streamConfigurations_.size() * 4); - for (const auto &entry : streamConfigurations_) { - minFrameDurations.push_back(entry.androidFormat); - minFrameDurations.push_back(entry.resolution.width); - minFrameDurations.push_back(entry.resolution.height); - minFrameDurations.push_back(minFrameDurationNsec); - } - staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, - minFrameDurations); + std::vector minFrameDurations; + minFrameDurations.reserve(streamConfigurations_.size() * 4); + for (const auto &entry : streamConfigurations_) { + minFrameDurations.push_back(entry.androidFormat); + minFrameDurations.push_back(entry.resolution.width); + minFrameDurations.push_back(entry.resolution.height); + minFrameDurations.push_back(entry.minFrameDurationNsec); } + staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, + minFrameDurations); uint8_t croppingType = ANDROID_SCALER_CROPPING_TYPE_CENTER_ONLY; staticMetadata_->addEntry(ANDROID_SCALER_CROPPING_TYPE, croppingType); From patchwork Tue Sep 7 19:40:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13740 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ADB82BDB1D for ; Tue, 7 Sep 2021 19:40:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3E2B569185; Tue, 7 Sep 2021 21:40:38 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F292569171 for ; Tue, 7 Sep 2021 21:40:35 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 2E660240002; Tue, 7 Sep 2021 19:40:34 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:40:59 +0200 Message-Id: <20210907194107.803730-10-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 09/17] android: capabilties: Correctly populate STALL durations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We currently hardcode 2560x1920@30FPS as the only stalling frame duration. This is of course not correct, and all the required information to properly populate the ANDROID_SCALER_AVAILABLE_STALL_DURATIONS static metadata are available from initializeStaticMetadata(). Use the collected stalling durations and sizes to properly popoulate the static property. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 5d8293843f57..526127176678 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1265,12 +1265,6 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS, availableStreamConfigurations); - std::vector availableStallDurations = { - ANDROID_SCALER_AVAILABLE_FORMATS_BLOB, 2560, 1920, 33333333, - }; - staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STALL_DURATIONS, - availableStallDurations); - std::vector minFrameDurations; minFrameDurations.reserve(streamConfigurations_.size() * 4); for (const auto &entry : streamConfigurations_) { @@ -1282,6 +1276,19 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, minFrameDurations); + std::vector availableStallDurations; + for (const auto &entry : streamConfigurations_) { + if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB) + continue; + + availableStallDurations.push_back(entry.androidFormat); + availableStallDurations.push_back(entry.resolution.width); + availableStallDurations.push_back(entry.resolution.height); + availableStallDurations.push_back(entry.minFrameDurationNsec); + } + staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STALL_DURATIONS, + availableStallDurations); + uint8_t croppingType = ANDROID_SCALER_CROPPING_TYPE_CENTER_ONLY; staticMetadata_->addEntry(ANDROID_SCALER_CROPPING_TYPE, croppingType); From patchwork Tue Sep 7 19:41:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13741 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 45635BDB1D for ; Tue, 7 Sep 2021 19:40:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E147C69172; Tue, 7 Sep 2021 21:40:38 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EBF1F60251 for ; Tue, 7 Sep 2021 21:40:36 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 2707D240002; Tue, 7 Sep 2021 19:40:35 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:41:00 +0200 Message-Id: <20210907194107.803730-11-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/17] android: capabilities: Collect absolute max frame durations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" While building the list of supported stream configurations also collect the absolute max frame durations to be used to populate the sensor maximum frame duration. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 9 ++++++--- src/android/camera_capabilities.h | 1 + 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 526127176678..c6791b9d6238 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -385,6 +385,7 @@ int CameraCapabilities::initialize(std::shared_ptr camera, orientation_ = orientation; facing_ = facing; rawStreamAvailable_ = false; + maxFrameDuration_ = 0; /* Acquire the camera and initialize available stream configurations. */ int ret = camera_->acquire(); @@ -659,6 +660,9 @@ int CameraCapabilities::initializeStreamConfigurations() }); maxJpegSize = std::max(maxJpegSize, res); } + + maxFrameDuration_ = std::max(maxFrameDuration_, + maxFrameDuration); } /* @@ -1134,9 +1138,8 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE, timestampSource); - if (maxFrameDurationNsec > 0) - staticMetadata_->addEntry(ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, - maxFrameDurationNsec); + staticMetadata_->addEntry(ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, + maxFrameDuration_); /* Statistics static metadata. */ uint8_t faceDetectMode = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index 6e55ddab445e..2cf97ae80095 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -71,6 +71,7 @@ private: int facing_; int orientation_; bool rawStreamAvailable_; + int64_t maxFrameDuration_; camera_metadata_enum_android_info_supported_hardware_level hwLevel_; std::set capabilities_; From patchwork Tue Sep 7 19:41:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13742 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E9DD1BDB1D for ; Tue, 7 Sep 2021 19:40:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ACDD56917C; Tue, 7 Sep 2021 21:40:41 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D382F6917C for ; Tue, 7 Sep 2021 21:40:37 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 17402240002; Tue, 7 Sep 2021 19:40:36 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:41:01 +0200 Message-Id: <20210907194107.803730-12-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 11/17] android: Filter preview streams on FPS X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Register as preview streams only streams capable of producing at least 30 FPS. This requirement comes from inspecting the existing HAL implementation on Intel IPU3 platform. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain --- src/android/camera_capabilities.cpp | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index c6791b9d6238..e5cfe67f9b73 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1259,6 +1259,21 @@ int CameraCapabilities::initializeStaticMetadata() std::vector availableStreamConfigurations; availableStreamConfigurations.reserve(streamConfigurations_.size() * 4); for (const auto &entry : streamConfigurations_) { + /* + * Filter out YUV streams not capable of running at 30 FPS. + * + * This requirement comes from inspecting the Intel IPU3 + * HAL implementation but no reference has been found in the + * metadata documentation. + * + * Calculate FPS as CTS does: see + * Camera2SurfaceViewTestCase.java:getSuitableFpsRangeForDuration() + */ + unsigned int fps = static_cast + (floor(1e9 / entry.minFrameDurationNsec + 0.05f)); + if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) + continue; + availableStreamConfigurations.push_back(entry.androidFormat); availableStreamConfigurations.push_back(entry.resolution.width); availableStreamConfigurations.push_back(entry.resolution.height); @@ -1271,6 +1286,11 @@ int CameraCapabilities::initializeStaticMetadata() std::vector minFrameDurations; minFrameDurations.reserve(streamConfigurations_.size() * 4); for (const auto &entry : streamConfigurations_) { + unsigned int fps = static_cast + (floor(1e9 / entry.minFrameDurationNsec + 0.05f)); + if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) + continue; + minFrameDurations.push_back(entry.androidFormat); minFrameDurations.push_back(entry.resolution.width); minFrameDurations.push_back(entry.resolution.height); From patchwork Tue Sep 7 19:41:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13743 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9E882BDB1D for ; Tue, 7 Sep 2021 19:40:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 55CA169178; Tue, 7 Sep 2021 21:40:42 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D59FF69177 for ; Tue, 7 Sep 2021 21:40:38 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 09EED240004; Tue, 7 Sep 2021 19:40:37 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:41:02 +0200 Message-Id: <20210907194107.803730-13-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 12/17] android: capabilities: Print output stream list X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a debug statement to print out the list of collected output stream and their characteristics. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index e5cfe67f9b73..ba800b545b9d 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1279,6 +1279,13 @@ int CameraCapabilities::initializeStaticMetadata() availableStreamConfigurations.push_back(entry.resolution.height); availableStreamConfigurations.push_back( ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_OUTPUT); + + LOG(HAL, Debug) + << "Output Stream: " << utils::hex(entry.androidFormat) + << " (" << entry.resolution.width << "x" + << entry.resolution.height << ")[" + << entry.minFrameDurationNsec << "]" + << "@" << fps; } staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS, availableStreamConfigurations); From patchwork Tue Sep 7 19:41:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13744 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D9397BDB1D for ; Tue, 7 Sep 2021 19:40:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B5A7069185; Tue, 7 Sep 2021 21:40:43 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CC1756916D for ; Tue, 7 Sep 2021 21:40:39 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 00D3C240002; Tue, 7 Sep 2021 19:40:38 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:41:03 +0200 Message-Id: <20210907194107.803730-14-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 13/17] android: Populate streams and duration in the same loop X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS and ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS static metadata are populated by looping on the streamConfigurations_ vector. Unify them in a single loop to avoid repeating it. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Umang Jain Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index ba800b545b9d..194ede866806 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -1257,7 +1257,9 @@ int CameraCapabilities::initializeStaticMetadata() maxZoom); std::vector availableStreamConfigurations; + std::vector minFrameDurations; availableStreamConfigurations.reserve(streamConfigurations_.size() * 4); + minFrameDurations.reserve(streamConfigurations_.size() * 4); for (const auto &entry : streamConfigurations_) { /* * Filter out YUV streams not capable of running at 30 FPS. @@ -1274,12 +1276,19 @@ int CameraCapabilities::initializeStaticMetadata() if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) continue; + /* Stream configuration map. */ availableStreamConfigurations.push_back(entry.androidFormat); availableStreamConfigurations.push_back(entry.resolution.width); availableStreamConfigurations.push_back(entry.resolution.height); availableStreamConfigurations.push_back( ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_OUTPUT); + /* Per-stream durations. */ + minFrameDurations.push_back(entry.androidFormat); + minFrameDurations.push_back(entry.resolution.width); + minFrameDurations.push_back(entry.resolution.height); + minFrameDurations.push_back(entry.minFrameDurationNsec); + LOG(HAL, Debug) << "Output Stream: " << utils::hex(entry.androidFormat) << " (" << entry.resolution.width << "x" @@ -1287,22 +1296,10 @@ int CameraCapabilities::initializeStaticMetadata() << entry.minFrameDurationNsec << "]" << "@" << fps; } + staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS, availableStreamConfigurations); - std::vector minFrameDurations; - minFrameDurations.reserve(streamConfigurations_.size() * 4); - for (const auto &entry : streamConfigurations_) { - unsigned int fps = static_cast - (floor(1e9 / entry.minFrameDurationNsec + 0.05f)); - if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) - continue; - - minFrameDurations.push_back(entry.androidFormat); - minFrameDurations.push_back(entry.resolution.width); - minFrameDurations.push_back(entry.resolution.height); - minFrameDurations.push_back(entry.minFrameDurationNsec); - } staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, minFrameDurations); From patchwork Tue Sep 7 19:41:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13745 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 54354BDB1D for ; Tue, 7 Sep 2021 19:40:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0876B69189; Tue, 7 Sep 2021 21:40:44 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8EDCB6916F for ; Tue, 7 Sep 2021 21:40:40 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id EFDDB240002; Tue, 7 Sep 2021 19:40:39 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:41:04 +0200 Message-Id: <20210907194107.803730-15-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 14/17] android: capabilties: Fix ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As reported by the CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES documentation in the Android developer reference: "For devices advertising any color filter arrangement other than NIR, or devices not advertising color filter arrangement, this list will always include (min, max) and (max, max) where min <= 15 and max = the maximum output frame rate of the maximum YUV_420_888 output size." Collect the higher FPS of the larger YUV stream and use it with the minimum FPS rate the camera can produce to populate the ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES static metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 75 +++++++++++------------------ 1 file changed, 28 insertions(+), 47 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 194ede866806..deaec3282fd7 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -866,53 +866,6 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_MODES, aeAvailableModes); - /* Initialize the AE frame duration limits. */ - const auto frameDurationsInfo = controlsInfo.find(&controls::FrameDurationLimits); - int64_t minFrameDurationNsec = frameDurationsInfo->second.min().get() * 1000; - int64_t maxFrameDurationNsec = frameDurationsInfo->second.max().get() * 1000; - - /* - * Adjust the minimum frame duration to comply with Android - * requirements. The camera service mandates all preview/record - * streams to have a minimum frame duration < 33,366 milliseconds - * (see MAX_PREVIEW_RECORD_DURATION_NS in the camera service - * implementation). - * - * If we're close enough (+ 500 useconds) to that value, round - * the minimum frame duration of the camera to an accepted - * value. - */ - static constexpr int64_t MAX_PREVIEW_RECORD_DURATION_NS = 1e9 / 29.97; - if (minFrameDurationNsec > MAX_PREVIEW_RECORD_DURATION_NS && - minFrameDurationNsec < MAX_PREVIEW_RECORD_DURATION_NS + 500000) - minFrameDurationNsec = MAX_PREVIEW_RECORD_DURATION_NS - 1000; - - /* - * The AE routine frame rate limits are computed using the frame - * duration limits, as libcamera clips the AE routine to the - * frame durations. - */ - int32_t maxFps = std::round(1e9 / minFrameDurationNsec); - int32_t minFps = std::round(1e9 / maxFrameDurationNsec); - minFps = std::max(1, minFps); - - /* - * Force rounding errors so that we have the proper frame - * durations for when we reuse these variables later - */ - minFrameDurationNsec = 1e9 / maxFps; - maxFrameDurationNsec = 1e9 / minFps; - - /* - * Register to the camera service {min, max} and {max, max} - * intervals as requested by the metadata documentation. - */ - int32_t availableAeFpsTarget[] = { - minFps, maxFps, maxFps, maxFps - }; - staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES, - availableAeFpsTarget); - std::vector aeCompensationRange = { 0, 0, }; @@ -1258,8 +1211,12 @@ int CameraCapabilities::initializeStaticMetadata() std::vector availableStreamConfigurations; std::vector minFrameDurations; + int maxYUVFps = 0; + Size maxYUVSize; + availableStreamConfigurations.reserve(streamConfigurations_.size() * 4); minFrameDurations.reserve(streamConfigurations_.size() * 4); + for (const auto &entry : streamConfigurations_) { /* * Filter out YUV streams not capable of running at 30 FPS. @@ -1276,6 +1233,16 @@ int CameraCapabilities::initializeStaticMetadata() if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30) continue; + /* + * Collect the FPS of the maximum YUV output size to populate + * AE_AVAILABLE_TARGET_FPS_RANGE + */ + if (entry.androidFormat == HAL_PIXEL_FORMAT_YCbCr_420_888 && + entry.resolution > maxYUVSize) { + maxYUVSize = entry.resolution; + maxYUVFps = fps; + } + /* Stream configuration map. */ availableStreamConfigurations.push_back(entry.androidFormat); availableStreamConfigurations.push_back(entry.resolution.width); @@ -1303,6 +1270,20 @@ int CameraCapabilities::initializeStaticMetadata() staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS, minFrameDurations); + /* + * Register to the camera service {min, max} and {max, max} with + * 'max' being the larger YUV stream maximum frame rate and 'min' being + * the globally minimum frame rate rounded to the next largest integer + * as the camera service expects the camera maximum frame duration to be + * smaller than 10^9 / minFps. + */ + int32_t minFps = std::ceil(1e9 / maxFrameDuration_); + int32_t availableAeFpsTarget[] = { + minFps, maxYUVFps, maxYUVFps, maxYUVFps, + }; + staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES, + availableAeFpsTarget); + std::vector availableStallDurations; for (const auto &entry : streamConfigurations_) { if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB) From patchwork Tue Sep 7 19:41:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13746 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AA55BBEFBE for ; Tue, 7 Sep 2021 19:40:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 645476917D; Tue, 7 Sep 2021 21:40:44 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 270086916F for ; Tue, 7 Sep 2021 21:40:41 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id B55F2240002; Tue, 7 Sep 2021 19:40:40 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:41:05 +0200 Message-Id: <20210907194107.803730-16-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 15/17] android: capabilities: Cap frame rate to 30 FPS X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Limit the reported minumum frame duration to 30 FPS. The reason to do is to bring the libcamra HAL in par with the Intel HAL implementation on IPU3 platform, where 30FPS is the frame rate used to perform quality tuning in the closed-source IPA module and has been validated as the most efficient rate for the power/performace budget. This change bring into the HAL a platform specific constraints, which might be opportune for most platforms but should rather be configurable by system integrators. Record that with a \todo entry. Also record that, even if we report a lower frame rate, we currently do not limit what the camera actually produce. Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- src/android/camera_capabilities.cpp | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index deaec3282fd7..cff1e30a9567 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -632,6 +632,29 @@ int CameraCapabilities::initializeStreamConfigurations() int64_t minFrameDuration = frameDurations->second.min().get() * 1000; int64_t maxFrameDuration = frameDurations->second.max().get() * 1000; + + /* + * Cap min frame duration to 30 FPS. + * + * 30 frames per second has been validated as the most + * opportune frame rate for quality tuning, and power + * vs performances budget on Intel IPU3. + * + * \todo This is a platform-specific decision that needs + * to be abstracted and delegated to the configuration + * file. + * + * \todo libcamera only allows to control frame duration + * through the per-request controls::FrameDuration + * control. If we cap the durations here, we should be + * capable of configuring the camera to operate at such + * duration without requiring to have the FrameDuration + * control to be specified for each Request. Defer this + * to the in-development configuration API rework. + */ + if (minFrameDuration < 1e9 / 30.0) + minFrameDuration = 1e9 / 30.0; + streamConfigurations_.push_back({ res, androidFormat, minFrameDuration, maxFrameDuration, }); From patchwork Tue Sep 7 19:41:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13747 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 17FAEBDB1D for ; Tue, 7 Sep 2021 19:40:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C4DF36917C; Tue, 7 Sep 2021 21:40:44 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B13A66917D for ; Tue, 7 Sep 2021 21:40:41 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 4D9AB240002; Tue, 7 Sep 2021 19:40:41 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:41:06 +0200 Message-Id: <20210907194107.803730-17-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 16/17] Revert "android: capabilities: Cap frame rate to 30 FPS" X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This reverts commit 257df463ad0e0862c211b03742140a99dd8cc0ef. --- src/android/camera_capabilities.cpp | 23 ----------------------- 1 file changed, 23 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index cff1e30a9567..deaec3282fd7 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -632,29 +632,6 @@ int CameraCapabilities::initializeStreamConfigurations() int64_t minFrameDuration = frameDurations->second.min().get() * 1000; int64_t maxFrameDuration = frameDurations->second.max().get() * 1000; - - /* - * Cap min frame duration to 30 FPS. - * - * 30 frames per second has been validated as the most - * opportune frame rate for quality tuning, and power - * vs performances budget on Intel IPU3. - * - * \todo This is a platform-specific decision that needs - * to be abstracted and delegated to the configuration - * file. - * - * \todo libcamera only allows to control frame duration - * through the per-request controls::FrameDuration - * control. If we cap the durations here, we should be - * capable of configuring the camera to operate at such - * duration without requiring to have the FrameDuration - * control to be specified for each Request. Defer this - * to the in-development configuration API rework. - */ - if (minFrameDuration < 1e9 / 30.0) - minFrameDuration = 1e9 / 30.0; - streamConfigurations_.push_back({ res, androidFormat, minFrameDuration, maxFrameDuration, }); From patchwork Tue Sep 7 19:41:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 13748 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7CF18BDB1D for ; Tue, 7 Sep 2021 19:40:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3E9B169184; Tue, 7 Sep 2021 21:40:45 +0200 (CEST) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A5806917F for ; Tue, 7 Sep 2021 21:40:42 +0200 (CEST) Received: (Authenticated sender: jacopo@jmondi.org) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id D76E9240002; Tue, 7 Sep 2021 19:40:41 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Tue, 7 Sep 2021 21:41:07 +0200 Message-Id: <20210907194107.803730-18-jacopo@jmondi.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210907194107.803730-1-jacopo@jmondi.org> References: <20210907194107.803730-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 17/17] ipa: ipu3: Cap frame duration to 30 FPS X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Limit the IPU3 frame rate to 30 FPS. The reason to do is to bring the IPU3 IPA in par with the Intel HAL implementation on IPU3 platform, where 30FPS is the frame rate used to perform quality tuning in the closed-source IPA module and has been validated as the most efficient rate for the power/performace budget. Compute the vertical blanking to maintain such frame rate and configure the sensor with that. Signed-off-by: Jacopo Mondi Reviewed-by: Paul Elder --- src/ipa/ipu3/ipu3.cpp | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index ec75b3ffa305..3480c828ea62 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -183,7 +183,7 @@ private: IPACameraSensorInfo sensorInfo_; /* Camera sensor controls. */ - uint32_t defVBlank_; + uint32_t vBlank_; uint32_t exposure_; uint32_t minExposure_; uint32_t maxExposure_; @@ -257,10 +257,39 @@ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); } + /* + * Cap minimum frame duration to 30FPS. + * + * 30 FPS has been validated in the closed source Intel 3A module as the + * most opportune frame rate for quality tuning, and power + * vs performances budget on Intel IPU3. + * + * Reduce the maximum achievable frame rate to 30 FPS and compute the + * vertical blanking to maintain that rate. + */ + int64_t *minFrameDuration = &frameDurations[0]; + if (*minFrameDuration < 1e6 / 30.0) + *minFrameDuration = 1e6 / 30.0; + controls[&controls::FrameDurationLimits] = ControlInfo(frameDurations[0], frameDurations[1], frameDurations[2]); + /* + * Adjust the vertical blanking to obtain the desired frame duration. + * + * Assume a fixed line length as horizontal blanking is seldom + * controllable. + * + * \todo Support making this overridable by the application through + * controls::FrameDuration. + * + * \todo Clamp exposure to frame duration. + */ + vBlank_ = *minFrameDuration * (sensorInfo.pixelRate / 1000000U); + vBlank_ /= lineLength; + vBlank_ -= sensorInfo.outputSize.height; + *ipaControls = ControlInfoMap(std::move(controls), controls::controls); } @@ -399,8 +428,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, maxGain_ = itGain->second.max().get(); gain_ = minGain_; - defVBlank_ = itVBlank->second.def().get(); - /* Clean context at configuration */ context_ = {}; @@ -511,8 +538,7 @@ void IPAIPU3::parseStatistics(unsigned int frame, setControls(frame); - /* \todo Use VBlank value calculated from each frame exposure. */ - int64_t frameDuration = sensorInfo_.lineLength * (defVBlank_ + sensorInfo_.outputSize.height) / + int64_t frameDuration = sensorInfo_.lineLength * (vBlank_ + sensorInfo_.outputSize.height) / (sensorInfo_.pixelRate / 1e6); ctrls.set(controls::FrameDuration, frameDuration); @@ -534,6 +560,7 @@ void IPAIPU3::setControls(unsigned int frame) ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure_)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain_)); + ctrls.set(V4L2_CID_VBLANK, static_cast(vBlank_)); op.controls = ctrls; queueFrameAction.emit(frame, op);