From patchwork Tue Sep 30 12:26:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24523 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B9252C328C for ; Tue, 30 Sep 2025 13:24:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED8436B5F3; Tue, 30 Sep 2025 15:24:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="A69g1Y2X"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BE29562C35 for ; Tue, 30 Sep 2025 15:24:41 +0200 (CEST) Received: from ideasonboard.com (unknown [94.31.94.171]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 5960B226; Tue, 30 Sep 2025 15:23:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1759238593; bh=IGSYJPNbfIhqPySXe9VsD4deKqwMpIi5GaiOcVc4Qvs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A69g1Y2XQ0smoiu9eHcCue3ItqxwDX6UPSx/HlP+t7CEvhfso2dig5oCgb/ZMhovb +Kgea+RIMPA+Iap5GShcncjjt5QGvzXD6FIH6MR0I+cf0sDIRL2ddTspq74SEus4dC UUEumrdInfXPIBlgLAgNDSQTuLYMvYg6KV2jXk20= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 24/33] libcamera: rkisp1: Use vertex map to implement ScalerCrop Date: Tue, 30 Sep 2025 14:26:45 +0200 Message-ID: <20250930122726.1837524-25-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> References: <20250930122726.1837524-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The input crop implemented in the dw100 kernel driver is quite limited in that it doesn't allow arbitrary crop rectangles but only scale factors quantized to the underlying fixed point representation and only aspect ratio preserving crops. The vertex map based implementation allows for pixel perfect crops. The only downside is that ScalerCrop can no longer be set dynamically on older kernels. Warn if the user tries to set ScalerCrop on such a setup. As the vertex map is now set on start() it no longer needs to be updated for frame==0. Signed-off-by: Stefan Klug --- Changes in 0.9 - Code cleanup - Update vertex map before start() to partially support old kernels Changes in 0.7: - Removed double call to applyVertexMap() --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 118 +++++++++++------------ 1 file changed, 59 insertions(+), 59 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 740791ac9c02..e8902ce66b70 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -235,9 +235,6 @@ private: RkISP1SelfPath selfPath_; std::unique_ptr dewarper_; - Rectangle scalerMaxCrop_; - - std::optional activeCrop_; /* Internal buffers used when dewarper is being used */ std::vector> mainPathBuffers_; @@ -969,13 +966,22 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) return ret; /* - * Calculate the crop rectangle of the data - * flowing into the dewarper in sensor - * coordinates. + * Apply the actual sensor crop, for proper + * dewarp map calculation + */ + Rectangle sensorCrop = outputCrop.transformedBetween( + inputCrop, sensorInfo.analogCrop); + auto &vertexMap = dewarper_->vertexMap(cfg.stream()); + vertexMap.setSensorCrop(sensorCrop); + data->properties_.set(properties::ScalerCropMaximum, sensorCrop); + + /* + * Apply a default sensor crop that keeps the + * aspect ratio. */ - scalerMaxCrop_ = - outputCrop.transformedBetween(inputCrop, - sensorInfo.analogCrop); + Size crop = format.size.boundedToAspectRatio(cfg.size); + vertexMap.setScalerCrop(crop.centeredTo( + Rectangle(format.size).center())); } ret = mainPath_.configure(ispCfg, format); @@ -1191,6 +1197,18 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL actions += [&]() { stat_->streamOff(); }; if (data->usesDewarper_) { + /* + * Apply the vertex map before start to partially + * support ScalerCrop on kernels with a dw100 driver + * that has no dynamic vertex map/requests support. + */ + if (controls && controls->contains(controls::ScalerCrop.id())) { + const auto &crop = controls->get(controls::ScalerCrop); + auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + vertexMap.setScalerCrop(*crop); + } + dewarper_->applyVertexMap(&data->mainPathStream_); + ret = dewarper_->start(); if (ret) { LOG(RkISP1, Error) << "Failed to start dewarper"; @@ -1345,25 +1363,31 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) if (data->usesDewarper_) { std::pair cropLimits; - if (dewarper_->isConfigured(&data->mainPathStream_)) - cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); - else + if (dewarper_->isConfigured(&data->mainPathStream_)) { + auto &vertexMap = dewarper_->vertexMap(&data->mainPathStream_); + vertexMap.applyLimits(); + cropLimits = vertexMap.scalerCropBounds(); + controls[&controls::ScalerCrop] = ControlInfo(cropLimits.first, + cropLimits.second, + vertexMap.effectiveScalerCrop()); + } else { + /* This happens before configure() has run. Maybe we need a better solution.*/ + /* + * ScalerCrop is specified to be in Sensor coordinates. + * So we need to transform the limits to sensor coordinates. + * We can safely assume that the maximum crop limit contains the + * full fov of the dewarper. + */ cropLimits = dewarper_->inputCropBounds(); + Rectangle maxCrop = Rectangle(data->sensor_->resolution()); + Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, + maxCrop); - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform the limits to sensor coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, - scalerMaxCrop_); + controls[&controls::ScalerCrop] = ControlInfo(min, + maxCrop, + maxCrop); + } - controls[&controls::ScalerCrop] = ControlInfo(min, - scalerMaxCrop_, - scalerMaxCrop_); - data->properties_.set(properties::ScalerCropMaximum, scalerMaxCrop_); - activeCrop_ = scalerMaxCrop_; if (dewarper_->supportsRequests()) { controls[&controls::draft::Dw100Scale] = ControlInfo(0.2f, 8.0f, 1.0f); @@ -1415,8 +1439,6 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); - scalerMaxCrop_ = Rectangle(data->sensor_->resolution()); - const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, @@ -1675,42 +1697,20 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) update = true; } - if (update || info->frame == 0) { - dewarper_->applyVertexMap(&data->mainPathStream_, dewarpRequest); - } - /* Handle scaler crop control. */ const auto &crop = request->controls().get(controls::ScalerCrop); if (crop) { - Rectangle rect = crop.value(); - - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform it into dewarper coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - std::pair cropLimits = - dewarper_->inputCropBounds(&data->mainPathStream_); - - rect = rect.transformedBetween(scalerMaxCrop_, cropLimits.second); - int ret = dewarper_->setInputCrop(&data->mainPathStream_, - &rect); - rect = rect.transformedBetween(cropLimits.second, scalerMaxCrop_); - if (!ret && rect != crop.value()) { - /* - * If the rectangle is changed by setInputCrop on the - * dewarper, log a debug message and cache the actual - * applied rectangle for metadata reporting. - */ - LOG(RkISP1, Debug) - << "Applied rectangle " << rect.toString() - << " differs from requested " << crop.value().toString(); - } - - activeCrop_ = rect; + if (!dewarper_->supportsRequests()) + LOG(RkISP1, Error) + << "Dynamically setting ScalerCrop requires a " + "dw100 driver with requests support"; + vertexMap.setScalerCrop(*crop); + update = true; } + if (update) + dewarper_->applyVertexMap(&data->mainPathStream_, dewarpRequest); + /* * Queue input and output buffers to the dewarper. The output * buffers for the dewarper are the buffers of the request, supplied @@ -1746,7 +1746,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) meta.set(controls::draft::Dw100Scale, vertexMap.effectiveScale()); meta.set(controls::draft::Dw100Rotation, vertexMap.rotation()); meta.set(controls::draft::Dw100Offset, vertexMap.effectiveOffset()); - meta.set(controls::ScalerCrop, activeCrop_.value()); + meta.set(controls::ScalerCrop, vertexMap.effectiveScalerCrop()); } void PipelineHandlerRkISP1::dewarpRequestReady(V4L2Request *request)