From patchwork Thu Sep 8 18:48:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xavier Roumegue X-Patchwork-Id: 17356 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A1D43C327F for ; Thu, 8 Sep 2022 18:49:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 58795620EF; Thu, 8 Sep 2022 20:49:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662662971; bh=m2ynyg7TqtczJaJAHD82Ze/daVvJ7iGwmFeJmDIk1vw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=BrHZ1HPjl84FL6ls3/4OufHPWfkUkLf7BT9zfpNwavAYhGivkYilB+MDFLSryBDrK 4s0oIVl+wSYOLoGxxl8ZNOZsCXaDyeigUlm0XQY3XMdn6NBLwJhTBa/KtyRW97e/Ar Jv1wXCuKigJWuwh29trjdaC4BVNsIo0+q1939kcOKQ6KDJSn9kQrLCbf1u/aT++uU2 6X0ol07xJqhxyjGDdgxClCRHS+QNYmltU2+EWTZ9iXleyUZITj686RvYx6Fv/LKzv2 vGbN5328apDLaGrGVl0RumzPQrJjOx1oN2700rw295GIhHmNgFdBHp5dpBWn/OvaCg P8UhPiJIFc7Ig== Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60060.outbound.protection.outlook.com [40.107.6.60]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4FFF9620D7 for ; Thu, 8 Sep 2022 20:49:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=NXP1.onmicrosoft.com header.i=@NXP1.onmicrosoft.com header.b="SceOCB9z"; dkim-atps=neutral ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KXBw07QCESiu1Ieya0wIOwDZeExXRZpn/mcSWQzuiY2ESBlxxj0n375FJwbceKFhldii8VN3hapIWzP8hhqvLvwaP3zCmCzE6sjWgvRaGKsH2+8lvEdJKPu/qB3veGwBM6burSrkWPb90ki9NKn3G3/p0z0syRYC/ggo5JXwv5ocLWqVOJxAwdVmKUsg76cgIreLwE14956Om/bojqAnPJOdCYcu9qxhmelKgF+jAGz8NJ+hdUlap3AUFy9Oc8U8hb4YKE1iRDhDlf6TGLvwBzdq+WvgGUIx6NWJ3VJWPDWS3q2p2d6L85CzQ/aq2GwWQILftmPJGfw4DUqaXt077A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=I0tyVzsP57ZnCFW+9pOhH2unE9htnH3rC/5FYI2a+ss=; b=ctSAn7mM+7Doh5G3TH4A3cLR6ti9EbRyQORjRriVMaPwdOhHeLapwuCeFDrZzVpO6FbG5FBSUofjGkOLmHQMljZSA0mmtnZt14wnpHIB4SD/pyGycW+9td0jIBzePY9+LMk6brxJU3DuXt4f1BRiqKdmjt6C9z8qio4rBScMx6zgNPMvtN5B+wG9uqf+uRu3skYo4jK8WpOPLPdF5DsSARwUKln4qLcja7VEJSur7338l81lSXg1v5IniEHVs+NLgTbf1d8yGn7zU+BE9FwJutGRDx+Fr9KIhdd0LDVtQRm+XtdLV+lTh1VHEIqeGxEYzT9lUvjSsdeL52aEmTgByg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oss.nxp.com; dmarc=pass action=none header.from=oss.nxp.com; dkim=pass header.d=oss.nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NXP1.onmicrosoft.com; s=selector2-NXP1-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I0tyVzsP57ZnCFW+9pOhH2unE9htnH3rC/5FYI2a+ss=; b=SceOCB9zpltKja/o3rLVVZQPQKTg0H1Yz51J19AOk0kjAtN+Ce6p/sMxEZJSDOU7VbUOb5hKFs8hYo9tnjjqbV9ZC3ucHn5w/pYV1gFFxlcqLKFVkhAy28w89nwd5YGnyeF7lZSGu6dmmhxpMzOJ7rmh6HeUKNN/Q4kiNYEa5oI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=oss.nxp.com; Received: from PAXPR04MB8703.eurprd04.prod.outlook.com (2603:10a6:102:21e::22) by AS8PR04MB8852.eurprd04.prod.outlook.com (2603:10a6:20b:42f::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.19; Thu, 8 Sep 2022 18:49:22 +0000 Received: from PAXPR04MB8703.eurprd04.prod.outlook.com ([fe80::485:adba:7081:715a]) by PAXPR04MB8703.eurprd04.prod.outlook.com ([fe80::485:adba:7081:715a%3]) with mapi id 15.20.5612.019; Thu, 8 Sep 2022 18:49:22 +0000 To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 20:48:50 +0200 Message-Id: <20220908184850.1874303-15-xavier.roumegue@oss.nxp.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220908184850.1874303-1-xavier.roumegue@oss.nxp.com> References: <20220908184850.1874303-1-xavier.roumegue@oss.nxp.com> X-ClientProxiedBy: PR3P250CA0001.EURP250.PROD.OUTLOOK.COM (2603:10a6:102:57::6) To PAXPR04MB8703.eurprd04.prod.outlook.com (2603:10a6:102:21e::22) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PAXPR04MB8703:EE_|AS8PR04MB8852:EE_ X-MS-Office365-Filtering-Correlation-Id: 65958e69-c80f-438b-40cf-08da91cad848 X-MS-Exchange-SharedMailbox-RoutingAgent-Processed: True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +CotIftFeazBBQxG9dKl42Umv3StQac4kOkImFma1iiEuZivkyGYeeg0Yri7Szv1mkPybxONj9SWmSqmniw/SupwwsvjKih42XdeGTIgnKFx0bofHanK3NQs8oLHek/c4c/DfldaPFGaa049b+C+q1vQa9wfiFm+1q0hm6+vzOyTfTE88p0/3i7dNiu6HXqhlAGVvXGI+s97JRwERes8THNjyw5nA5CLYsiDpQjQUeb+HSspzjzmf5Ww9hBhKQFuAxilDx3Q/OSsvSJ68WI3oa1z6xo0Pheriim2quB/fBYPXYTF97D2SPSYL6lWqcBtJFOCna1U86/pyWnNRXiXNaThK/L+2f+HdA3QFBQpbKoMFVUEoz0m+Vkp2tc69nRgCzquVRjuZy1CWKasbBfmwHbmmUpDtWygWB8EsB467SKJ06umvX1VXfm9n8McBZDb4cSps7IKjagXtiRHYLj/H0YAQSdzavjoGrFXUf+ahLa6WlCLSPwnk9egn0SgsDeYX9CzYcLsOIBLciODJGOvA/n3crjzP3bruMEmAmINWE1P7x/vf1Z9AKvD8T/dOEDPt3tA4IqdbFEqXl/qna/3bvv9xffjQowXUNN+SsOoIIWSP9JHw0SV1IeYAlgd/eDjHseydVBjvKDZ52cuYOCtgTrZ0+8Y3G9mM/ncBTauZ4ZYqzoYd83H38YkwCphwSSH5bAVg25CPxnCOAJEWfgbIJKuK8ZkMF0jBzYeDGBuOBs580fFMZqK8n9GQQbnOfb+nDitoYK/B2shmEosLvmSMQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAXPR04MB8703.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(39860400002)(346002)(376002)(396003)(136003)(366004)(38100700002)(66556008)(66476007)(66946007)(8936002)(30864003)(44832011)(5660300002)(4326008)(8676002)(6916009)(316002)(41300700001)(186003)(1076003)(2616005)(2906002)(478600001)(6666004)(86362001)(6512007)(52116002)(6506007)(83380400001)(6486002)(309714004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: OFZU2NbpIk1qnfPMJJHbw+yLg6cFFF4fdKQ/HKEW8JXaB6t8MO5eluMYSyrMb1gPN6lmXsmNDWTPqxGrbe+yI7LhJG4Cq87WV3Kyi2WA0BcSdoEPCuqXgaBMGvmmwN/7Tgg4fsEPCGtzkczVayVipCDkF1NFdUkuV3UMxxqRclxS/OFQyVjTk4ANBOV7wpjOLteU2wjZk8LpnYWVb/HscqFMygFIuiE/8t1dpRNy8q7rdwawB7lB/HgN2wz6geaIwS847VA2WU5QYxax801y4pL1mIZfA4jhZipg2iahpVsOoMBtv2MrPw9rFS5ocqBNYynLZzxIxjRUJmZl2a3QVkrJfaWFRV4QTpf/P5rdxHxKE2C8CMgVdjJFWJtyqJvH5Vikb8r4bx0nZVld7xCmvYfM2FdQCqvcnEhFF0OP1wIZ/UBBvuM7km2YMpQdHsZWUaW0etxDW6Iijca338ROu1N/DLeQK+NKx8r0q10GkHlI+EqMURfYSqsPQrRfbU3fWKMz/VbuUbMW81xqLvJXZH0xPi47Vkd5biX10CRcWNrtVVmUkI9Kr8+naH/qTwh2/SmKW8UXcoqMvn3N6KGxpn/O6AOfrjdhBteGdfuEsGGyUVHJiL+zS2qjaUk0yfXjNOJE0dNr8Vb0e7+1sY9S4fHw90zs1jIwPA43FST9v/05JAv+1HbjvQuCyB2ggX5cEwyXJdpRSVjZFK/uKqZ8DDEsJOszHrtNqTIVeMCnfGeL51cj6rHSpnV7IxvHAUT3sbpuKOMFZiVaqQHqaskISL/cwaHpHemWwysNxBvBhGiP1dFV6Qf/LcYVJDGtFA0pDV0T41sAY+zqX0vJfAmLzzi1y2ujKCIB9khA/9OORxWUHwbD4rQhozsTSNn3hxniLLPAb4GKaEdwH3v4e+up0dkWXq3K40sNstjtgFjX9K0BckpVYfv9m+8ZeWMpkCvTWkPjNzDE7JIldZuRcNWfnFDv1WVBUMdU+3xYjR8YDG+WwSLBe3G+FNNzXcNEdU14pDPKFXb8o63Pp4TCE5OE64g3e0usWvESEKEfLXxV/V9+kpm9gbZjP6oKIa3/2+T2WkSI8/Xl87B24k9z+E2hS4zVyHwPOe4iS/Jb5V/cMUMQ3U9NP5UWiBlDcM16mUZq6nUXIRTFqf1wQ6ssie2Al3j0rrr7LMjqaZio+QFKUiGoxjy+5B/gpU4B23tXa6RMb99JHcN85/vT2XvBY0w0F+IckChWvhb7ha1wi/28zAVenZrWAy2z4aWrFij2UGTJJQaognpeVQx6C3fwkNWULEqk2x+o7+3QrPQHl5Xh2wICQn92hRpFuQhr9ruvKNBNOz8t1Vx0VSwkFUSXXSzd19lu0rn4YYIR6fC7s/vtWtwAIPsIfbR8HJfSXcZdnI0tAqLxC1I2ZoiEnXnVBcPagTt8hoKTJmzdWNeURsPKUTucz8Bn5+df4sSOwpivFgKWYtA6Q8ZaU2uAJPXa8APQMJlPIAhhknufuV2hNn3XNQe7lNN6IliU9CSpqi/I5zMNAW2Vd/goaUq52WGgv2AIOsn1LsHRovfHosdA5LTfe1d6rfpqaFhWLB51LQhrrTFAjCo4zivAFYt1ADYmP2+459WYHMOWXmsqalXJYTUCS1O6x5D/ZksJXNmPRAYt10y2pP8RFQmgYg1jkjX3dK6BNA== X-OriginatorOrg: oss.nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 65958e69-c80f-438b-40cf-08da91cad848 X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB8703.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Sep 2022 18:49:22.5943 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3IpoPMeeSm3gY/5ZWj81EB/7ya9lWcbVDrCHDadBxgrVN8py7tJIKkLzDFUFAIoPxgsDEFHHikiVGK1L7mjWLA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB8852 Subject: [libcamera-devel] [PATCH 14/14] libcamera: pipeline: rkisp1: Add converter support X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Xavier Roumegue via libcamera-devel From: Xavier Roumegue Reply-To: Xavier Roumegue Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds a converter, if any present in the system, on each streams (self and main paths). In case a configuration file is successfully loaded, the converter use is getting compulsory, such as a dewarping map is unconditionally applied. Otherwise, the converter is only used if the stream configuration requires it. Signed-off-by: Xavier Roumegue --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 126 +++--- src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 374 +++++++++++++++++- src/libcamera/pipeline/rkisp1/rkisp1_path.h | 54 ++- 3 files changed, 485 insertions(+), 69 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c1522ca6..6bdf5a3a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2019, Google Inc. + * Copyright 2022 NXP * * rkisp1.cpp - Pipeline handler for Rockchip ISP1 */ @@ -194,6 +195,7 @@ private: Camera *activeCamera_; const MediaPad *ispSink_; + MediaDevice *converter_; }; RkISP1Frames::RkISP1Frames(PipelineHandler *pipe) @@ -449,57 +451,44 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() bool mainPathAvailable = true; bool selfPathAvailable = data_->selfPath_; + const std::array cameraStatus = { Valid, Adjusted }; + for (unsigned int index : order) { StreamConfiguration &cfg = config_[index]; - /* Try to match stream without adjusting configuration. */ - if (mainPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->mainPath_->validate(&tryCfg) == Valid) { - mainPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->mainPathStream_)); - continue; - } - } + for (auto &_status : cameraStatus) { + Status pipeStatus; - if (selfPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->selfPath_->validate(&tryCfg) == Valid) { - selfPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->selfPathStream_)); - continue; - } - } + if (mainPathAvailable) { + StreamConfiguration tryCfg = cfg; + pipeStatus = data_->mainPath_->validate(&tryCfg); - /* Try to match stream allowing adjusting configuration. */ - if (mainPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->mainPath_->validate(&tryCfg) == Adjusted) { - mainPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->mainPathStream_)); - status = Adjusted; - continue; + if (pipeStatus == _status) { + mainPathAvailable = false; + cfg = tryCfg; + cfg.setStream(const_cast(&data_->mainPathStream_)); + status = _status; + break; + } } - } - if (selfPathAvailable) { - StreamConfiguration tryCfg = cfg; - if (data_->selfPath_->validate(&tryCfg) == Adjusted) { - selfPathAvailable = false; - cfg = tryCfg; - cfg.setStream(const_cast(&data_->selfPathStream_)); - status = Adjusted; - continue; + if (selfPathAvailable) { + StreamConfiguration tryCfg = cfg; + pipeStatus = data_->selfPath_->validate(&tryCfg); + + if (pipeStatus == _status) { + selfPathAvailable = false; + cfg = tryCfg; + cfg.setStream(const_cast(&data_->selfPathStream_)); + status = _status; + break; + } } + /* All paths rejected configuration. */ + LOG(RkISP1, Debug) << "Camera configuration not supported " + << cfg.toString(); + return Invalid; } - - /* All paths rejected configuraiton. */ - LOG(RkISP1, Debug) << "Camera configuration not supported " - << cfg.toString(); - return Invalid; } /* Select the sensor format. */ @@ -680,15 +669,21 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) std::map streamConfig; - for (const StreamConfiguration &cfg : *config) { + for (StreamConfiguration &cfg : *config) { + size_t idx = 0; + StreamConfiguration internalCfg; if (cfg.stream() == &data->mainPathStream_) { + idx = 0; ret = mainPath_.configure(cfg, format); - streamConfig[0] = IPAStream(cfg.pixelFormat, - cfg.size); + internalCfg = mainPath_.internalStream(); + streamConfig[idx] = IPAStream(internalCfg.pixelFormat, + internalCfg.size); } else if (hasSelfPath_) { + idx = 1; ret = selfPath_.configure(cfg, format); - streamConfig[1] = IPAStream(cfg.pixelFormat, - cfg.size); + internalCfg = selfPath_.internalStream(); + streamConfig[idx] = IPAStream(internalCfg.pixelFormat, + internalCfg.size); } else { return -ENODEV; } @@ -754,6 +749,20 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) data->selfPathStream_.configuration().bufferCount, }); + if (data->mainPath_->isEnabled()) { + ret = data->mainPath_->allocateBuffers( + data->mainPathStream_.configuration().bufferCount); + if (ret < 0) + goto error; + } + + if (hasSelfPath_ && data->selfPath_->isEnabled()) { + ret = data->selfPath_->allocateBuffers( + data->selfPathStream_.configuration().bufferCount); + if (ret < 0) + goto error; + } + ret = param_->allocateBuffers(maxCount, ¶mBuffers_); if (ret < 0) goto error; @@ -813,6 +822,12 @@ int PipelineHandlerRkISP1::freeBuffers(Camera *camera) if (stat_->releaseBuffers()) LOG(RkISP1, Error) << "Failed to release stat buffers"; + if (hasSelfPath_ && data->selfPath_->isEnabled()) + data->selfPath_->releaseBuffers(); + + if (data->mainPath_->isEnabled()) + data->mainPath_->releaseBuffers(); + return 0; } @@ -1055,6 +1070,19 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) hasSelfPath_ = !!media_->getEntityByName("rkisp1_selfpath"); + /* Seek for a converter */ + for (auto converterName : ConverterFactory::names()) { + LOG(RkISP1, Debug) + << "Trying " << converterName << " converter"; + DeviceMatch converterMatch(converterName); + converter_ = acquireMediaDevice(enumerator, converterMatch); + if (converter_) { + LOG(RkISP1, Debug) + << "Get support for " << converterName << " converter"; + break; + } + } + /* Create the V4L2 subdevices we will need. */ isp_ = V4L2Subdevice::fromEntityName(media_, "rkisp1_isp"); if (isp_->open() < 0) @@ -1086,10 +1114,10 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) return false; /* Locate and open the ISP main and self paths. */ - if (!mainPath_.init(media_)) + if (!mainPath_.init(media_, converter_)) return false; - if (hasSelfPath_ && !selfPath_.init(media_)) + if (hasSelfPath_ && !selfPath_.init(media_, converter_)) return false; mainPath_.bufferReady().connect(this, &PipelineHandlerRkISP1::bufferReady); diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp index 2d38f0fb..c19a1e69 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2020, Google Inc. + * Copyright 2022 NXP * * rkisp1path.cpp - Rockchip ISP1 path helper */ @@ -28,7 +29,46 @@ RkISP1Path::RkISP1Path(const char *name, const Span &formats, { } -bool RkISP1Path::init(MediaDevice *media) +void RkISP1Path::initConverter(MediaDevice *mediaConverter) +{ + hasConverter_ = false; + useConverter_ = false; + + if (!mediaConverter) + return; + + converter_ = ConverterFactory::create(mediaConverter); + + if (!converter_->isValid()) { + LOG(RkISP1, Warning) + << "Failed to create converter, disabling format conversion"; + converter_.reset(); + } else { + char const *configFromEnv = utils::secure_getenv("LIBCAMERA_RKISP1_CONVERTER_FILE"); + + if (configFromEnv && *configFromEnv != '\0') { + int nrMappings; + LOG(RkISP1, Debug) + << "Getting pipeline converter filename as " << std::string(configFromEnv); + nrMappings = converter_->loadConfiguration(std::string(configFromEnv)); + if (nrMappings < 0) { + LOG(RkISP1, Error) + << "Error while reading converter configuration file"; + } else { + LOG(RkISP1, Debug) + << nrMappings << " mapping(s) loaded"; + if (nrMappings > 0) + /* We want to force the converter use to apply the mapping */ + useConverter_ = true; + } + } + converter_->inputBufferReady.connect(this, &RkISP1Path::pathConverterInputDone); + converter_->outputBufferReady.connect(this, &RkISP1Path::pathConverterOutputDone); + hasConverter_ = true; + } +} + +bool RkISP1Path::init(MediaDevice *media, MediaDevice *mediaConverter) { std::string resizer = std::string("rkisp1_resizer_") + name_ + "path"; std::string video = std::string("rkisp1_") + name_ + "path"; @@ -45,10 +85,13 @@ bool RkISP1Path::init(MediaDevice *media) if (!link_) return false; + initConverter(mediaConverter); + + video_->bufferReady.connect(this, &RkISP1Path::pathBufferReady); + return true; } - -StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution) +StreamConfiguration RkISP1Path::generateNativeConfiguration(const Size &resolution) { Size maxResolution = maxResolution_.boundedToAspectRatio(resolution) .boundedTo(resolution); @@ -67,7 +110,165 @@ StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution) return cfg; } -CameraConfiguration::Status RkISP1Path::validate(StreamConfiguration *cfg) +StreamConfiguration RkISP1Path::generateConfiguration(const Size &resolution) +{ + StreamConfiguration cfg = generateNativeConfiguration(resolution); + StreamFormats formats = cfg.formats(); + std::map> fullFormats; + + for (const PixelFormat &fmt : formats.pixelformats()) { + Configuration config; + SizeRange szRange; + + config.inputFormat = fmt; + config.inputSizes = formats.range(fmt); + + if (!useConverter_) { + config.outputFormats.push_back(fmt); + config.outputSizes = config.inputSizes; + config.withConverter = false; + configs_.push_back(config); + LOG(RkISP1, Debug) + << "Pushing native format " << fmt + << " resolution range " << config.inputSizes; + fullFormats[fmt].push_back(config.inputSizes); + } + + if (hasConverter_) { + config.outputFormats = converter_->formats(config.inputFormat); + if (config.outputFormats.empty()) + continue; + szRange.max = converter_->sizes(config.inputSizes.max).max; + szRange.min = converter_->sizes(config.inputSizes.min).min; + config.outputSizes = szRange; + config.withConverter = true; + configs_.push_back(config); + for (auto &_fmt : config.outputFormats) { + fullFormats[_fmt].push_back(config.outputSizes); + LOG(RkISP1, Debug) + << "Pushing converted format " << _fmt + << " resolution range " << config.outputSizes; + } + } + } + /* + * Sort the sizes and merge any consecutive overlapping ranges. + * Imported from src/libcamera/pipeline/simple.cpp + * + * TODO: Apply policy of converter use or not.. We likely want to force + * the converter use in case there is a mapping defined in the + * configuration file if any... if so, shall we filter out resolution + * not defined in the configuration file.. or should this policy be let + * to the application + * */ + + for (auto &[format, sizes] : fullFormats) { + std::sort(sizes.begin(), sizes.end(), + [](SizeRange &a, SizeRange &b) { + return a.min < b.min; + }); + + auto cur = sizes.begin(); + auto next = cur; + + while (++next != sizes.end()) { + if (cur->max.width >= next->min.width && + cur->max.height >= next->min.height) + cur->max = next->max; + else if (++cur != next) + *cur = *next; + } + + sizes.erase(++cur, sizes.end()); + } + + StreamConfiguration fullCfg{ StreamFormats{ fullFormats } }; + fullCfg.pixelFormat = fullFormats.begin()->first; + fullCfg.size = fullFormats.begin()->second[0].max; + fullCfg.bufferCount = cfg.bufferCount; + + return fullCfg; +} + +CameraConfiguration::Status RkISP1Path::getPipeConfiguration( + StreamConfiguration &streamCfg, + const RkISP1Path::Configuration **cfg) const +{ + CameraConfiguration::Status _status = CameraConfiguration::Adjusted; + + LOG(RkISP1, Debug) + << "Looking for " + << streamCfg.pixelFormat << "/" << streamCfg.size + << " output configuration"; + /* + * Select the fallback configuration + */ + for (auto &_cfg : configs_) { + if (_cfg.withConverter == useConverter_) { + *cfg = &_cfg; + break; + } + } + + PixelFormat pixelFormat = (*cfg)->outputFormats.front(); + Size size = streamCfg.size; + size.boundTo((*cfg)->outputSizes.max); + size.expandTo((*cfg)->outputSizes.min); + + /* Unless the converter use is enforced through a loaded configuration, + * prefer a configuration without the converter if possible + */ + std::vector converterUse; + if (!useConverter_) + converterUse.push_back(false); + if (hasConverter_) + converterUse.push_back(true); + + for (auto withConverter : converterUse) { + for (auto &_cfg : configs_) { + auto &outFmts = _cfg.outputFormats; + if (withConverter != _cfg.withConverter) + continue; + + if ((std::find(outFmts.begin(), + outFmts.end(), + streamCfg.pixelFormat)) == outFmts.end()) + continue; + + *cfg = &_cfg; + pixelFormat = streamCfg.pixelFormat; + + if (_cfg.outputSizes.contains(streamCfg.size)) { + _status = CameraConfiguration::Valid; + goto done; + } + + size.boundTo((*cfg)->outputSizes.max); + size.expandTo((*cfg)->outputSizes.min); + } + } + + streamCfg.pixelFormat = pixelFormat; + streamCfg.size = size; + +done: + if ((*cfg)->withConverter) { + std::tie(streamCfg.stride, streamCfg.frameSize) = + converter_->strideAndFrameSize(streamCfg.pixelFormat, + streamCfg.size); + if (streamCfg.stride == 0) + return CameraConfiguration::Invalid; + } + + LOG(RkISP1, Debug) + << "Chosen configuration: " + << (*cfg)->inputFormat << "/" << (*cfg)->inputSizes + << " --> " << streamCfg.pixelFormat << "/" << streamCfg.size + << ((*cfg)->withConverter ? " with" : " without") << " converter"; + return _status; +} + +CameraConfiguration::Status RkISP1Path::nativeValidate(StreamConfiguration *cfg) { const StreamConfiguration reqCfg = *cfg; CameraConfiguration::Status status = CameraConfiguration::Valid; @@ -101,7 +302,39 @@ CameraConfiguration::Status RkISP1Path::validate(StreamConfiguration *cfg) return status; } -int RkISP1Path::configure(const StreamConfiguration &config, +CameraConfiguration::Status RkISP1Path::validate(StreamConfiguration *cfg) +{ + StreamConfiguration tryCfg = *cfg; + StreamConfiguration internalCfg; + const Configuration *pipeCfg; + CameraConfiguration::Status pipeStatus; + + pipeStatus = getPipeConfiguration(tryCfg, &pipeCfg); + if (pipeStatus == CameraConfiguration::Invalid) + return CameraConfiguration::Invalid; + + if (!pipeCfg->withConverter) { + tryCfg = *cfg; + pipeStatus = nativeValidate(&tryCfg); + if (pipeStatus == CameraConfiguration::Invalid) + return CameraConfiguration::Invalid; + internalCfg = tryCfg; + } else { + internalCfg = tryCfg; + internalCfg.pixelFormat = pipeCfg->inputFormat; + auto _status = nativeValidate(&internalCfg); + if (_status == CameraConfiguration::Invalid) + return CameraConfiguration::Invalid; + } + + *cfg = tryCfg; + internalStream_ = internalCfg; + pipeConfig_ = pipeCfg; + + return pipeStatus; +} + +int RkISP1Path::nativeConfigure(const StreamConfiguration &config, const V4L2SubdeviceFormat &inputFormat) { int ret; @@ -165,6 +398,117 @@ int RkISP1Path::configure(const StreamConfiguration &config, return 0; } +int RkISP1Path::configure(const StreamConfiguration &config, + const V4L2SubdeviceFormat &inputFormat) +{ + int ret; + LOG(RkISP1, Debug) + << "Configuring " << name_ << " path " + << internalStream_.pixelFormat << "/" << internalStream_.size + << " --> " << config.pixelFormat << "/" << config.size + << (pipeConfig_->withConverter ? " with" : " without") << " converter"; + + ret = nativeConfigure(internalStream_, inputFormat); + if (ret) + return ret; + + if (pipeConfig_->withConverter) { + StreamConfiguration cfg = config; + std::vector> outputCfgs; + outputCfgs.push_back(cfg); + ret = converter_->configure(internalStream_, outputCfgs); + } + + return ret; +} + +int RkISP1Path::exportBuffers(unsigned int bufferCount, + std::vector> *buffers) +{ + if (pipeConfig_->withConverter) + return converter_->exportBuffers(0, bufferCount, buffers); + else + return video_->exportBuffers(bufferCount, buffers); +} + +int RkISP1Path::allocateBuffers(unsigned int bufferCount) +{ + if (pipeConfig_->withConverter) { + int ret = video_->allocateBuffers(bufferCount, &converterBuffers_); + if (ret < 0) + return ret; + if ((unsigned int)ret != bufferCount) + LOG(RkISP1, Warning) + << "Requested " << bufferCount << " but got " << ret << " buffers"; + + for (std::unique_ptr &buffer : converterBuffers_) + availableConverterBuffers_.push(buffer.get()); + } else { + return video_->importBuffers(bufferCount); + } + + return 0; +} + +void RkISP1Path::releaseBuffers() +{ + while (!availableConverterBuffers_.empty()) + availableConverterBuffers_.pop(); + + converterBuffers_.clear(); + + video_->releaseBuffers(); +} + +void RkISP1Path::pathBufferReady(FrameBuffer *buffer) +{ + Request *request = buffer->request(); + + if (request) { + internalBufferReady_.emit(buffer); + } else { + auto iter = converterBuffersMapping_.find(buffer); + if (iter != converterBuffersMapping_.end()) { + converter_->queueBuffer(buffer, iter->second); + converterBuffersMapping_.erase(iter); + } else { + LOG(RkISP1, Error) + << "Final buffer associated to converted buffer not found on " + << name_ << " path"; + } + } +} + +void RkISP1Path::pathConverterInputDone(FrameBuffer *buffer) +{ + availableConverterBuffers_.push(buffer); +} + +void RkISP1Path::pathConverterOutputDone(FrameBuffer *buffer) +{ + internalBufferReady_.emit(buffer); +} + +int RkISP1Path::queueBuffer(FrameBuffer *buffer) +{ + FrameBuffer *_buffer; + + if (pipeConfig_->withConverter) { + if (availableConverterBuffers_.empty()) { + LOG(RkISP1, Error) + << "converter buffer underrun on " << name_ << " path"; + return -ENOMEM; + } + _buffer = availableConverterBuffers_.front(); + availableConverterBuffers_.pop(); + converterBuffersMapping_[_buffer] = buffer; + } else { + _buffer = buffer; + } + + return video_->queueBuffer(_buffer); +} + int RkISP1Path::start() { int ret; @@ -172,20 +516,23 @@ int RkISP1Path::start() if (running_) return -EBUSY; - /* \todo Make buffer count user configurable. */ - ret = video_->importBuffers(RKISP1_BUFFER_COUNT); - if (ret) - return ret; - ret = video_->streamOn(); if (ret) { LOG(RkISP1, Error) << "Failed to start " << name_ << " path"; - - video_->releaseBuffers(); return ret; } + if (pipeConfig_->withConverter) { + ret = converter_->start(); + if (ret) { + LOG(RkISP1, Error) + << "Failed to start converter on " << name_ << " path"; + stop(); + return ret; + } + } + running_ = true; return 0; @@ -199,7 +546,8 @@ void RkISP1Path::stop() if (video_->streamOff()) LOG(RkISP1, Warning) << "Failed to stop " << name_ << " path"; - video_->releaseBuffers(); + if (pipeConfig_->withConverter) + converter_->stop(); running_ = false; } diff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h index f3f1ae39..0da3594f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h +++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2020, Google Inc. + * Copyright 2022 NXP * * rkisp1path.h - Rockchip ISP1 path helper */ @@ -8,6 +9,7 @@ #pragma once #include +#include #include #include @@ -17,9 +19,11 @@ #include #include +#include "libcamera/internal/converter.h" #include "libcamera/internal/media_object.h" #include "libcamera/internal/v4l2_videodevice.h" + namespace libcamera { class MediaDevice; @@ -33,7 +37,7 @@ public: RkISP1Path(const char *name, const Span &formats, const Size &minResolution, const Size &maxResolution); - bool init(MediaDevice *media); + bool init(MediaDevice *media, MediaDevice *mediaConverter = nullptr); int setEnabled(bool enable) { return link_->setEnabled(enable); } bool isEnabled() const { return link_->flags() & MEDIA_LNK_FL_ENABLED; } @@ -45,18 +49,31 @@ public: const V4L2SubdeviceFormat &inputFormat); int exportBuffers(unsigned int bufferCount, - std::vector> *buffers) - { - return video_->exportBuffers(bufferCount, buffers); - } + std::vector> *buffers); + + int allocateBuffers(unsigned int bufferCount); + void releaseBuffers(); int start(); void stop(); - int queueBuffer(FrameBuffer *buffer) { return video_->queueBuffer(buffer); } - Signal &bufferReady() { return video_->bufferReady; } + int queueBuffer(FrameBuffer *buffer); + Signal &bufferReady() { return internalBufferReady_; } + + StreamConfiguration internalStream() const + { + return internalStream_; + } private: + struct Configuration { + SizeRange inputSizes; + PixelFormat inputFormat; + std::vector outputFormats; + SizeRange outputSizes; + bool withConverter; + }; + static constexpr unsigned int RKISP1_BUFFER_COUNT = 4; const char *name_; @@ -65,10 +82,33 @@ private: const Span formats_; const Size minResolution_; const Size maxResolution_; + CameraConfiguration::Status getPipeConfiguration( + StreamConfiguration &streamCfg, + const RkISP1Path::Configuration **cfg) const; + + StreamConfiguration generateNativeConfiguration(const Size &resolution); + CameraConfiguration::Status nativeValidate(StreamConfiguration *cfg); + int nativeConfigure(const StreamConfiguration &config, + const V4L2SubdeviceFormat &inputFormat); std::unique_ptr resizer_; std::unique_ptr video_; MediaLink *link_; + + void initConverter(MediaDevice *mediaConverter); + std::unique_ptr converter_; + std::vector configs_; + StreamConfiguration internalStream_; + const Configuration *pipeConfig_; + bool hasConverter_; + bool useConverter_; + Signal internalBufferReady_; + std::vector> converterBuffers_; + std::queue availableConverterBuffers_; + std::map converterBuffersMapping_; + void pathBufferReady(FrameBuffer *buffer); + void pathConverterInputDone(FrameBuffer *buffer); + void pathConverterOutputDone(FrameBuffer *buffer); }; class RkISP1MainPath : public RkISP1Path