{"id":21535,"url":"https://patchwork.libcamera.org/api/patches/21535/?format=json","web_url":"https://patchwork.libcamera.org/patch/21535/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20241008101918.10414-2-umang.jain@ideasonboard.com>","date":"2024-10-08T10:19:18","name":"[v5,1/1] libcamera: rkisp1: Integrate SensorConfiguration support","commit_ref":"047d647452c44b610c58dc4425b95d7933cabdc8","pull_url":null,"state":"accepted","archived":false,"hash":"bc2ba0a27727b0075651c79d5fe8ad66e7a09fd7","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/people/86/?format=json","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/21535/mbox/","series":[{"id":4664,"url":"https://patchwork.libcamera.org/api/series/4664/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4664","date":"2024-10-08T10:19:17","name":"rkisp1: Integrate sensorConfiguration","version":5,"mbox":"https://patchwork.libcamera.org/series/4664/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/21535/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/21535/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 53EE2BD80A\n\tfor ;\n\tTue, 8 Oct 2024 10:19:38 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id F0B17618CB;\n\tTue, 8 Oct 2024 12:19:37 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CD98A618CB\n\tfor ;\n\tTue, 8 Oct 2024 12:19:33 +0200 (CEST)","from localhost.localdomain (unknown\n\t[IPv6:2405:201:2015:f873:55f8:639e:8e9f:12ec])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 5286E514;\n\tTue, 8 Oct 2024 12:17:56 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"QYV2P7yK\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1728382677;\n\tbh=kfJoFrPDdtL0G87mu7JW1XnOTWksO4XFXdsgBgje7QU=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=QYV2P7yK7Y4xg9G5UkNJrx6MMzWJQQQVO1Q9YJ8/hYN1ZVGOg/AkILwWR6Qo3Fvgp\n\twaoyTH7tfKD42Eb2EcC11pLZsfqCtr/tYnQv6wxYb4jB7BKJRM9A+7uIL/dW3eyNo2\n\ty8BTJaWNvMubvSBHS2CpV9g5GBoMZ4r23+YU/r5o=","From":"Umang Jain ","To":"libcamera-devel@lists.libcamera.org","Cc":"Jacopo Mondi ,\n\tKieran Bingham ,\n\tLaurent Pinchart ,\n\tPaul Elder ,\n\tUmang Jain ","Subject":"[PATCH v5 1/1] libcamera: rkisp1: Integrate SensorConfiguration\n\tsupport","Date":"Tue, 8 Oct 2024 15:49:18 +0530","Message-ID":"<20241008101918.10414-2-umang.jain@ideasonboard.com>","X-Mailer":"git-send-email 2.45.0","In-Reply-To":"<20241008101918.10414-1-umang.jain@ideasonboard.com>","References":"<20241008101918.10414-1-umang.jain@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Integrate the RkISP1 pipeline handler to support sensor configuration\nprovided by applications through CameraConfiguration::sensorConfig.\n\nThe SensorConfiguration must be validated on both RkISP1Path (mainPath\nand selfPath), so the parameters of RkISP1Path::validate() have been\nupdated to include sensorConfig.\n\nThe camera configuration will be marked as invalid when the sensor\nconfiguration is supplied, if:\n- Invalid sensor configuration (SensorConfiguration::isValid())\n- Bit depth not supported by RkISP1 pipeline\n- RAW stream configuration size doesn't matches sensor configuration\n output size\n- Sensor configuration output size is larger than maximum supported\n sensor configuration on RkISP1 pipeline\n- No matching sensor configuration output size supplied by the sensor\n\nSigned-off-by: Umang Jain \n---\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 42 +++++++++++++---\n src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 48 +++++++++++++++++--\n src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +\n 3 files changed, 80 insertions(+), 12 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\nindex c02c7cf3..42961c12 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n@@ -447,11 +447,12 @@ bool RkISP1CameraConfiguration::fitsAllPaths(const StreamConfiguration &cfg)\n \tStreamConfiguration config;\n \n \tconfig = cfg;\n-\tif (data_->mainPath_->validate(sensor, &config) != Valid)\n+\tif (data_->mainPath_->validate(sensor, sensorConfig, &config) != Valid)\n \t\treturn false;\n \n \tconfig = cfg;\n-\tif (data_->selfPath_ && data_->selfPath_->validate(sensor, &config) != Valid)\n+\tif (data_->selfPath_ &&\n+\t data_->selfPath_->validate(sensor, sensorConfig, &config) != Valid)\n \t\treturn false;\n \n \treturn true;\n@@ -468,6 +469,27 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \n \tstatus = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace);\n \n+\t/*\n+\t * Make sure that if a sensor configuration has been requested it\n+\t * is valid.\n+\t */\n+\tif (sensorConfig) {\n+\t\tif (!sensorConfig->isValid()) {\n+\t\t\tLOG(RkISP1, Error)\n+\t\t\t\t<< \"Invalid sensor configuration request\";\n+\n+\t\t\treturn Invalid;\n+\t\t}\n+\n+\t\tunsigned int bitDepth = sensorConfig->bitDepth;\n+\t\tif (bitDepth != 8 && bitDepth != 10 && bitDepth != 12) {\n+\t\t\tLOG(RkISP1, Error)\n+\t\t\t\t<< \"Invalid sensor configuration bit depth\";\n+\n+\t\t\treturn Invalid;\n+\t\t}\n+\t}\n+\n \t/* Cap the number of entries to the available streams. */\n \tif (config_.size() > pathCount) {\n \t\tconfig_.resize(pathCount);\n@@ -514,7 +536,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \t\t/* Try to match stream without adjusting configuration. */\n \t\tif (mainPathAvailable) {\n \t\t\tStreamConfiguration tryCfg = cfg;\n-\t\t\tif (data_->mainPath_->validate(sensor, &tryCfg) == Valid) {\n+\t\t\tif (data_->mainPath_->validate(sensor, sensorConfig, &tryCfg) == Valid) {\n \t\t\t\tmainPathAvailable = false;\n \t\t\t\tcfg = tryCfg;\n \t\t\t\tcfg.setStream(const_cast(&data_->mainPathStream_));\n@@ -524,7 +546,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \n \t\tif (selfPathAvailable) {\n \t\t\tStreamConfiguration tryCfg = cfg;\n-\t\t\tif (data_->selfPath_->validate(sensor, &tryCfg) == Valid) {\n+\t\t\tif (data_->selfPath_->validate(sensor, sensorConfig, &tryCfg) == Valid) {\n \t\t\t\tselfPathAvailable = false;\n \t\t\t\tcfg = tryCfg;\n \t\t\t\tcfg.setStream(const_cast(&data_->selfPathStream_));\n@@ -535,7 +557,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \t\t/* Try to match stream allowing adjusting configuration. */\n \t\tif (mainPathAvailable) {\n \t\t\tStreamConfiguration tryCfg = cfg;\n-\t\t\tif (data_->mainPath_->validate(sensor, &tryCfg) == Adjusted) {\n+\t\t\tif (data_->mainPath_->validate(sensor, sensorConfig, &tryCfg) == Adjusted) {\n \t\t\t\tmainPathAvailable = false;\n \t\t\t\tcfg = tryCfg;\n \t\t\t\tcfg.setStream(const_cast(&data_->mainPathStream_));\n@@ -546,7 +568,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \n \t\tif (selfPathAvailable) {\n \t\t\tStreamConfiguration tryCfg = cfg;\n-\t\t\tif (data_->selfPath_->validate(sensor, &tryCfg) == Adjusted) {\n+\t\t\tif (data_->selfPath_->validate(sensor, sensorConfig, &tryCfg) == Adjusted) {\n \t\t\t\tselfPathAvailable = false;\n \t\t\t\tcfg = tryCfg;\n \t\t\t\tcfg.setStream(const_cast(&data_->selfPathStream_));\n@@ -723,7 +745,13 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c)\n \tV4L2SubdeviceFormat format = config->sensorFormat();\n \tLOG(RkISP1, Debug) << \"Configuring sensor with \" << format;\n \n-\tret = sensor->setFormat(&format, config->combinedTransform());\n+\tif (config->sensorConfig)\n+\t\tret = sensor->applyConfiguration(*config->sensorConfig,\n+\t\t\t\t\t\t config->combinedTransform(),\n+\t\t\t\t\t\t &format);\n+\telse\n+\t\tret = sensor->setFormat(&format, config->combinedTransform());\n+\n \tif (ret < 0)\n \t\treturn ret;\n \ndiff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\nindex da8d25c3..3b5bea96 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n@@ -251,8 +251,10 @@ RkISP1Path::generateConfiguration(const CameraSensor *sensor, const Size &size,\n \treturn cfg;\n }\n \n-CameraConfiguration::Status RkISP1Path::validate(const CameraSensor *sensor,\n-\t\t\t\t\t\t StreamConfiguration *cfg)\n+CameraConfiguration::Status\n+RkISP1Path::validate(const CameraSensor *sensor,\n+\t\t std::optional &sensorConfig,\n+\t\t StreamConfiguration *cfg)\n {\n \tconst std::vector &mbusCodes = sensor->mbusCodes();\n \tSize resolution = filterSensorResolution(sensor);\n@@ -282,9 +284,14 @@ CameraConfiguration::Status RkISP1Path::validate(const CameraSensor *sensor,\n \t\t\t\tcontinue;\n \n \t\t\t/*\n-\t\t\t * Store the raw format with the highest bits per pixel\n-\t\t\t * for later usage.\n+\t\t\t * If the bits per pixel is supplied from the sensor\n+\t\t\t * configuration, choose a raw format that complies with\n+\t\t\t * it. Otherwise, store the raw format with the highest\n+\t\t\t * bits per pixel for later usage.\n \t\t\t */\n+\t\t\tif (sensorConfig && info.bitsPerPixel != sensorConfig->bitDepth)\n+\t\t\t\tcontinue;\n+\n \t\t\tif (info.bitsPerPixel > rawBitsPerPixel) {\n \t\t\t\trawBitsPerPixel = info.bitsPerPixel;\n \t\t\t\trawFormat = format;\n@@ -297,6 +304,9 @@ CameraConfiguration::Status RkISP1Path::validate(const CameraSensor *sensor,\n \t\t}\n \t}\n \n+\tif (sensorConfig && !found)\n+\t\treturn CameraConfiguration::Invalid;\n+\n \tbool isRaw = PixelFormatInfo::info(cfg->pixelFormat).colourEncoding ==\n \t\t PixelFormatInfo::ColourEncodingRAW;\n \n@@ -319,11 +329,39 @@ CameraConfiguration::Status RkISP1Path::validate(const CameraSensor *sensor,\n \t\t * size.\n \t\t */\n \t\tuint32_t mbusCode = formatToMediaBus.at(cfg->pixelFormat);\n+\t\tSize rawSize = sensorConfig ? sensorConfig->outputSize\n+\t\t\t\t\t : cfg->size;\n+\n \t\tV4L2SubdeviceFormat sensorFormat =\n-\t\t\tsensor->getFormat({ mbusCode }, cfg->size);\n+\t\t\tsensor->getFormat({ mbusCode }, rawSize);\n+\n+\t\tif (sensorConfig &&\n+\t\t sensorConfig->outputSize != sensorFormat.size)\n+\t\t\treturn CameraConfiguration::Invalid;\n \n \t\tminResolution = sensorFormat.size;\n \t\tmaxResolution = sensorFormat.size;\n+\t} else if (sensorConfig) {\n+\t\t/*\n+\t\t * We have already ensured 'rawFormat' has the matching bit\n+\t\t * depth with sensorConfig.bitDepth hence, only validate the\n+\t\t * sensorConfig's output size here.\n+\t\t */\n+\t\tSize sensorSize = sensorConfig->outputSize;\n+\n+\t\tif (sensorSize > resolution)\n+\t\t\treturn CameraConfiguration::Invalid;\n+\n+\t\tuint32_t mbusCode = formatToMediaBus.at(rawFormat);\n+\t\tV4L2SubdeviceFormat sensorFormat =\n+\t\t\tsensor->getFormat({ mbusCode }, sensorSize);\n+\n+\t\tif (sensorFormat.size != sensorSize)\n+\t\t\treturn CameraConfiguration::Invalid;\n+\n+\t\tminResolution = minResolution_.expandedToAspectRatio(sensorSize);\n+\t\tmaxResolution = maxResolution_.boundedTo(sensorSize)\n+\t\t\t\t\t .boundedToAspectRatio(sensorSize);\n \t} else {\n \t\t/*\n \t\t * Adjust the size based on the sensor resolution and absolute\ndiff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\nindex 777fcae7..ce9a5666 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n@@ -27,6 +27,7 @@ namespace libcamera {\n class CameraSensor;\n class MediaDevice;\n class V4L2Subdevice;\n+class SensorConfiguration;\n struct StreamConfiguration;\n struct V4L2SubdeviceFormat;\n \n@@ -44,6 +45,7 @@ public:\n \t\t\t\t\t\t const Size &resolution,\n \t\t\t\t\t\t StreamRole role);\n \tCameraConfiguration::Status validate(const CameraSensor *sensor,\n+\t\t\t\t\t std::optional &sensorConfig,\n \t\t\t\t\t StreamConfiguration *cfg);\n \n \tint configure(const StreamConfiguration &config,\n","prefixes":["v5","1/1"]}