{"id":21499,"url":"https://patchwork.libcamera.org/api/1.1/patches/21499/?format=json","web_url":"https://patchwork.libcamera.org/patch/21499/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20241003135741.60869-3-umang.jain@ideasonboard.com>","date":"2024-10-03T13:57:40","name":"[v2,2/2] libcamera: rkisp1: Integrate SensorConfiguration support","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"01669f95951093f9df6def2b7cae5f5eeb430b2b","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/1.1/people/86/?format=json","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/21499/mbox/","series":[{"id":4653,"url":"https://patchwork.libcamera.org/api/1.1/series/4653/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4653","date":"2024-10-03T13:57:38","name":"rkisp1: Integrate Sensorconfiguration","version":2,"mbox":"https://patchwork.libcamera.org/series/4653/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/21499/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/21499/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1DD49C3263\n\tfor ;\n\tThu, 3 Oct 2024 13:57:56 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8DF2063524;\n\tThu, 3 Oct 2024 15:57:55 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 54A5562C92\n\tfor ;\n\tThu, 3 Oct 2024 15:57:51 +0200 (CEST)","from umang.jain (unknown\n\t[IPv6:2405:201:2015:f873:55d7:c02e:b2eb:ee3f])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B27FB593;\n\tThu, 3 Oct 2024 15:56:16 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"TxUFZDfE\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1727963777;\n\tbh=jIhGs+F6oISbMbPQuV9fo+m7xKLJwScfPlhZ5AG+wsM=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=TxUFZDfEwJd6BJHo5BeKfD7usbGo3BM4N8+otWiLNbaTUwc39Ni24bQlLLIPP58D8\n\tokIdxZzSHSRx+qLqneEzRAeccotO23kVIqOkaCU0bPrS5TbzlJw02K3y7UwWB67wLu\n\tWvzMdNtUbmdVxyl/uPhr/kA3p9/VbgKVqbFZ9ZVk=","From":"Umang Jain ","To":"libcamera-devel@lists.libcamera.org","Cc":"Jacopo Mondi ,\n\tKieran Bingham ,\n\tLaurent Pinchart ,\n\tPaul Elder ,\n\tUmang Jain ","Subject":"[PATCH v2 2/2] libcamera: rkisp1: Integrate SensorConfiguration\n\tsupport","Date":"Thu, 3 Oct 2024 19:27:40 +0530","Message-ID":"<20241003135741.60869-3-umang.jain@ideasonboard.com>","X-Mailer":"git-send-email 2.45.2","In-Reply-To":"<20241003135741.60869-1-umang.jain@ideasonboard.com>","References":"<20241003135741.60869-1-umang.jain@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Integrate the RkISP1 pipeline handler to support sensor configuration\nprovided by applications through CameraConfiguration::sensorConfig.\n\nThe SensorConfiguration must be validated on both RkISP1Path (mainPath\nand selfPath), so the parameters of RkISP1Path::validate() have been\nupdated to include sensorConfig.\n\nFinally, if the sensor configuration fails to apply, the camera\nconfiguration validation will be marked as invalid.\n\nSigned-off-by: Umang Jain \n---\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 68 ++++++++++++++++---\n src/libcamera/pipeline/rkisp1/rkisp1_path.cpp | 45 ++++++++++--\n src/libcamera/pipeline/rkisp1/rkisp1_path.h | 2 +\n 3 files changed, 100 insertions(+), 15 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\nindex 6cee7893..95e70603 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n@@ -447,11 +447,12 @@ bool RkISP1CameraConfiguration::fitsAllPaths(const StreamConfiguration &cfg)\n \tStreamConfiguration config;\n \n \tconfig = cfg;\n-\tif (data_->mainPath_->validate(sensor, &config) != Valid)\n+\tif (data_->mainPath_->validate(sensor, sensorConfig, &config) != Valid)\n \t\treturn false;\n \n \tconfig = cfg;\n-\tif (data_->selfPath_ && data_->selfPath_->validate(sensor, &config) != Valid)\n+\tif (data_->selfPath_ &&\n+\t data_->selfPath_->validate(sensor, sensorConfig, &config) != Valid)\n \t\treturn false;\n \n \treturn true;\n@@ -468,6 +469,27 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \n \tstatus = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace);\n \n+\t/*\n+\t * Make sure that if a sensor configuration has been requested it\n+\t * is valid.\n+\t */\n+\tif (sensorConfig && !sensorConfig->isValid()) {\n+\t\tLOG(RkISP1, Error)\n+\t\t\t<< \"Invalid sensor configuration request\";\n+\n+\t\treturn Invalid;\n+\t}\n+\n+\tif (sensorConfig) {\n+\t\tstd::optional bitDepth = sensorConfig->bitDepth;\n+\t\tif (bitDepth != 8 && bitDepth != 10 && bitDepth != 12) {\n+\t\t\tLOG(RkISP1, Error)\n+\t\t\t\t<< \"Invalid sensor configuration bit depth\";\n+\n+\t\t\treturn Invalid;\n+\t\t}\n+\t}\n+\n \t/* Cap the number of entries to the available streams. */\n \tif (config_.size() > pathCount) {\n \t\tconfig_.resize(pathCount);\n@@ -513,7 +535,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \t\t/* Try to match stream without adjusting configuration. */\n \t\tif (mainPathAvailable) {\n \t\t\tStreamConfiguration tryCfg = cfg;\n-\t\t\tif (data_->mainPath_->validate(sensor, &tryCfg) == Valid) {\n+\t\t\tif (data_->mainPath_->validate(sensor, sensorConfig, &tryCfg) == Valid) {\n \t\t\t\tmainPathAvailable = false;\n \t\t\t\tcfg = tryCfg;\n \t\t\t\tcfg.setStream(const_cast(&data_->mainPathStream_));\n@@ -523,7 +545,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \n \t\tif (selfPathAvailable) {\n \t\t\tStreamConfiguration tryCfg = cfg;\n-\t\t\tif (data_->selfPath_->validate(sensor, &tryCfg) == Valid) {\n+\t\t\tif (data_->selfPath_->validate(sensor, sensorConfig, &tryCfg) == Valid) {\n \t\t\t\tselfPathAvailable = false;\n \t\t\t\tcfg = tryCfg;\n \t\t\t\tcfg.setStream(const_cast(&data_->selfPathStream_));\n@@ -534,7 +556,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \t\t/* Try to match stream allowing adjusting configuration. */\n \t\tif (mainPathAvailable) {\n \t\t\tStreamConfiguration tryCfg = cfg;\n-\t\t\tif (data_->mainPath_->validate(sensor, &tryCfg) == Adjusted) {\n+\t\t\tif (data_->mainPath_->validate(sensor, sensorConfig, &tryCfg) == Adjusted) {\n \t\t\t\tmainPathAvailable = false;\n \t\t\t\tcfg = tryCfg;\n \t\t\t\tcfg.setStream(const_cast(&data_->mainPathStream_));\n@@ -545,7 +567,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \n \t\tif (selfPathAvailable) {\n \t\t\tStreamConfiguration tryCfg = cfg;\n-\t\t\tif (data_->selfPath_->validate(sensor, &tryCfg) == Adjusted) {\n+\t\t\tif (data_->selfPath_->validate(sensor, sensorConfig, &tryCfg) == Adjusted) {\n \t\t\t\tselfPathAvailable = false;\n \t\t\t\tcfg = tryCfg;\n \t\t\t\tcfg.setStream(const_cast(&data_->selfPathStream_));\n@@ -562,26 +584,47 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n \n \t/* Select the sensor format. */\n \tPixelFormat rawFormat;\n+\tSize rawSize;\n \tSize maxSize;\n \n \tfor (const StreamConfiguration &cfg : config_) {\n-\t\tif (cfg.pixelFormat.isRaw())\n+\t\tif (cfg.pixelFormat.isRaw()) {\n \t\t\trawFormat = cfg.pixelFormat;\n+\t\t\trawSize = cfg.size;\n+\t\t}\n \n \t\tmaxSize = std::max(maxSize, cfg.size);\n \t}\n \n+\tif (rawFormat.isValid() && sensorConfig) {\n+\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(rawFormat);\n+\t\tif (sensorConfig->outputSize != rawSize ||\n+\t\t sensorConfig->bitDepth != info.bitsPerPixel)\n+\t\t\treturn Invalid;\n+\t}\n+\n \tstd::vector mbusCodes;\n+\tSize sz = maxSize;\n \n \tif (rawFormat.isValid()) {\n \t\tmbusCodes = { rawFormats.at(rawFormat) };\n+\t\tsz = rawSize;\n+\t} else if (sensorConfig) {\n+\t\t/* sensorConfig has been validated by RkISP1Path::validate(). */\n+\t\tfor (const auto &value : rawFormats) {\n+\t\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(value.first);\n+\t\t\tif (info.bitsPerPixel == sensorConfig->bitDepth)\n+\t\t\t\tmbusCodes.push_back(value.second);\n+\t\t}\n+\n+\t\tsz = sensorConfig->outputSize;\n \t} else {\n \t\tstd::transform(rawFormats.begin(), rawFormats.end(),\n \t\t\t std::back_inserter(mbusCodes),\n \t\t\t [](const auto &value) { return value.second; });\n \t}\n \n-\tsensorFormat_ = sensor->getFormat(mbusCodes, maxSize);\n+\tsensorFormat_ = sensor->getFormat(mbusCodes, sz);\n \n \tif (sensorFormat_.size.isNull())\n \t\tsensorFormat_.size = sensor->resolution();\n@@ -721,7 +764,14 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c)\n \tV4L2SubdeviceFormat format = config->sensorFormat();\n \tLOG(RkISP1, Debug) << \"Configuring sensor with \" << format;\n \n-\tret = sensor->setFormat(&format, config->combinedTransform());\n+\tif (config->sensorConfig) {\n+\t\tret = sensor->applyConfiguration(*config->sensorConfig,\n+\t\t\t\t\t\t config->combinedTransform(),\n+\t\t\t\t\t\t &format);\n+\t} else {\n+\t\tret = sensor->setFormat(&format, config->combinedTransform());\n+\t}\n+\n \tif (ret < 0)\n \t\treturn ret;\n \ndiff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\nindex e7f1f12a..93c7c7dc 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.cpp\n@@ -250,8 +250,10 @@ RkISP1Path::generateConfiguration(const CameraSensor *sensor, const Size &size,\n \treturn cfg;\n }\n \n-CameraConfiguration::Status RkISP1Path::validate(const CameraSensor *sensor,\n-\t\t\t\t\t\t StreamConfiguration *cfg)\n+CameraConfiguration::Status\n+RkISP1Path::validate(const CameraSensor *sensor,\n+\t\t std::optional &sensorConfig,\n+\t\t StreamConfiguration *cfg)\n {\n \tconst std::vector &mbusCodes = sensor->mbusCodes();\n \n@@ -278,11 +280,18 @@ CameraConfiguration::Status RkISP1Path::validate(const CameraSensor *sensor,\n \t\t\t\tcontinue;\n \n \t\t\t/*\n-\t\t\t * Store the raw format with the highest bits per pixel\n-\t\t\t * for later usage.\n+\t\t\t * If the bits per pixel is supplied from the sensor\n+\t\t\t * configuration, choose a raw format that complies with\n+\t\t\t * it. Otherwise, store the raw format with the highest\n+\t\t\t * bits per pixel for later usage.\n \t\t\t */\n \t\t\tconst PixelFormatInfo &info = PixelFormatInfo::info(format);\n-\t\t\tif (info.bitsPerPixel > rawBitsPerPixel) {\n+\n+\t\t\tif (sensorConfig &&\n+\t\t\t info.bitsPerPixel == sensorConfig->bitDepth) {\n+\t\t\t\trawFormat = format;\n+\t\t\t\trawBitsPerPixel = info.bitsPerPixel;\n+\t\t\t} else if (info.bitsPerPixel > rawBitsPerPixel) {\n \t\t\t\trawBitsPerPixel = info.bitsPerPixel;\n \t\t\t\trawFormat = format;\n \t\t\t}\n@@ -315,11 +324,35 @@ CameraConfiguration::Status RkISP1Path::validate(const CameraSensor *sensor,\n \t\t * size.\n \t\t */\n \t\tuint32_t mbusCode = formatToMediaBus.at(cfg->pixelFormat);\n+\t\tSize rawSize = sensorConfig ? sensorConfig->outputSize\n+\t\t\t\t\t : cfg->size;\n \t\tV4L2SubdeviceFormat sensorFormat =\n-\t\t\tsensor->getFormat({ mbusCode }, cfg->size);\n+\t\t\tsensor->getFormat({ mbusCode }, rawSize);\n+\n+\t\tif (sensorConfig &&\n+\t\t sensorFormat.size != sensorConfig->outputSize)\n+\t\t\treturn CameraConfiguration::Invalid;\n \n \t\tminResolution = sensorFormat.size;\n \t\tmaxResolution = sensorFormat.size;\n+\t} else if (sensorConfig) {\n+\t\t/*\n+\t\t * We have already ensured 'rawFormat' has the matching bit\n+\t\t * depth with sensorConfig.bitDepth hence, only validate the\n+\t\t * sensorConfig's output size here.\n+\t\t * */\n+\t\tuint32_t mbusCode = formatToMediaBus.at(rawFormat);\n+\t\tSize sensorSize = sensorConfig->outputSize;\n+\n+\t\tV4L2SubdeviceFormat sensorFormat =\n+\t\t\tsensor->getFormat({ mbusCode }, sensorSize);\n+\n+\t\tif (sensorFormat.size != sensorSize)\n+\t\t\treturn CameraConfiguration::Invalid;\n+\n+\t\tminResolution = minResolution_.expandedToAspectRatio(sensorSize);\n+\t\tmaxResolution = maxResolution_.boundedToAspectRatio(sensorSize)\n+\t\t\t\t\t .boundedTo(sensorSize);\n \t} else {\n \t\tSize resolution = filterSensorResolution(sensor);\n \ndiff --git a/src/libcamera/pipeline/rkisp1/rkisp1_path.h b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\nindex 777fcae7..ce9a5666 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1_path.h\n@@ -27,6 +27,7 @@ namespace libcamera {\n class CameraSensor;\n class MediaDevice;\n class V4L2Subdevice;\n+class SensorConfiguration;\n struct StreamConfiguration;\n struct V4L2SubdeviceFormat;\n \n@@ -44,6 +45,7 @@ public:\n \t\t\t\t\t\t const Size &resolution,\n \t\t\t\t\t\t StreamRole role);\n \tCameraConfiguration::Status validate(const CameraSensor *sensor,\n+\t\t\t\t\t std::optional &sensorConfig,\n \t\t\t\t\t StreamConfiguration *cfg);\n \n \tint configure(const StreamConfiguration &config,\n","prefixes":["v2","2/2"]}