From patchwork Tue Nov 25 16:28:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 25196 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 545B5C333D for ; Tue, 25 Nov 2025 16:29:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E4C3460AB1; Tue, 25 Nov 2025 17:29:46 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MJGSHdeM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D891960AA4 for ; Tue, 25 Nov 2025 17:29:44 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E5C7D6AF; Tue, 25 Nov 2025 17:27:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764088056; bh=Mk3k5f5BpwZwyQZfSvYKSz5AqfgWwA8+1FTcYbeX7D8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MJGSHdeMUQz5E+K8kUcoklpRRnlYnMPkTax0tOIuElcJp02Owbkfb+DcNy8zguJKa 9Bb3xBdM0wpZSK/c5wdR+UcomWWEhHgNOWYEY4/DAdSnAXsayw8fN5TrnOfAfGsJbJ Xuxi5XlraKoUxfj8ZT47chS6PWShv+HIoQzmkVKw= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 17/29] libcamera: rkisp1: Use the dw100 converter module instead of the generic v4l2 converter Date: Tue, 25 Nov 2025 17:28:29 +0100 Message-ID: <20251125162851.2301793-18-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> References: <20251125162851.2301793-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The dewarper integration into the rkisp1 pipeline is quite complicated. Simplify that by switching to the now available ConverterDW100Module. As there is no other known converter in combination with th rkisp1 ISP this is a safe step to do. This change also paves the way to implement dw100 specific features later. The input crop implemented in the dw100 kernel driver is quite limited in that it doesn't allow arbitrary crop rectangles but only scale factors quantized to the underlying fixed point representation and only aspect ratio preserving crops. The vertex map based implementation allows for pixel perfect crops. The only downside is that ScalerCrop can no longer be set dynamically on older kernels. A corresponding warning is already implemented in the converter module. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham --- Changes in v3: - Merge usage of the converter module and ScalerCrop handling into one patch as it is difficult to keep them separate with the new module concept Changes in 0.9: - Code cleanup - Update vertex map before start() to partially support old kernels Changes in 0.7: - Removed double call to applyVertexMap() --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 122 ++++++----------------- 1 file changed, 33 insertions(+), 89 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 138e1d5bf06b..6256a67bc31e 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -36,7 +36,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/camera_sensor_properties.h" -#include "libcamera/internal/converter/converter_v4l2_m2m.h" +#include "libcamera/internal/converter/converter_dw100.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/framebuffer.h" @@ -232,10 +232,7 @@ private: RkISP1MainPath mainPath_; RkISP1SelfPath selfPath_; - std::unique_ptr dewarper_; - Rectangle scalerMaxCrop_; - - std::optional activeCrop_; + std::unique_ptr dewarper_; /* Internal buffers used when dewarper is being used */ std::vector> mainPathBuffers_; @@ -940,6 +937,16 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) if (ret) return ret; + /* + * Apply the actual sensor crop, for proper + * dewarp map calculation + */ + Rectangle sensorCrop = outputCrop.transformedBetween( + inputCrop, sensorInfo.analogCrop); + if (data->usesDewarper_) + dewarper_->setSensorCrop(sensorCrop); + data->properties_.set(properties::ScalerCropMaximum, sensorCrop); + std::map streamConfig; std::vector> outputCfgs; @@ -965,13 +972,15 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) return ret; /* - * Calculate the crop rectangle of the data - * flowing into the dewarper in sensor - * coordinates. + * Apply a default scaler crop that keeps the + * aspect ratio. */ - scalerMaxCrop_ = - outputCrop.transformedBetween(inputCrop, - sensorInfo.analogCrop); + Size size = cfg.size; + size = sensorCrop.size().boundedToAspectRatio(size); + + ControlList ctrls; + ctrls.set(controls::ScalerCrop, size.centeredTo(sensorCrop.center())); + dewarper_->setControls(cfg.stream(), ctrls); } ret = mainPath_.configure(ispCfg, format); @@ -1165,6 +1174,9 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL actions += [&]() { stat_->streamOff(); }; if (data->usesDewarper_) { + if (controls) + dewarper_->setControls(&data->mainPathStream_, *controls); + ret = dewarper_->start(); if (ret) { LOG(RkISP1, Error) << "Failed to start dewarper"; @@ -1315,28 +1327,8 @@ int PipelineHandlerRkISP1::updateControls(RkISP1CameraData *data) { ControlInfoMap::Map controls; - if (data->usesDewarper_) { - std::pair cropLimits; - if (dewarper_->isConfigured(&data->mainPathStream_)) - cropLimits = dewarper_->inputCropBounds(&data->mainPathStream_); - else - cropLimits = dewarper_->inputCropBounds(); - - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform the limits to sensor coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - Rectangle min = cropLimits.first.transformedBetween(cropLimits.second, - scalerMaxCrop_); - - controls[&controls::ScalerCrop] = ControlInfo(min, - scalerMaxCrop_, - scalerMaxCrop_); - data->properties_.set(properties::ScalerCropMaximum, scalerMaxCrop_); - activeCrop_ = scalerMaxCrop_; - } + if (data->usesDewarper_) + dewarper_->updateControlInfos(&data->mainPathStream_, controls); /* Add the IPA registered controls to list of camera controls. */ for (const auto &ipaControl : data->ipaControls_) @@ -1376,8 +1368,6 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); - scalerMaxCrop_ = Rectangle(data->sensor_->resolution()); - const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, @@ -1472,27 +1462,11 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) stat_->bufferReady.connect(this, &PipelineHandlerRkISP1::statBufferReady); param_->bufferReady.connect(this, &PipelineHandlerRkISP1::paramBufferReady); - /* If dewarper is present, create its instance. */ - DeviceMatch dwp("dw100"); - dwp.add("dw100-source"); - dwp.add("dw100-sink"); - - std::shared_ptr dwpMediaDevice = enumerator->search(dwp); - if (dwpMediaDevice) { - dewarper_ = std::make_unique(dwpMediaDevice); - if (dewarper_->isValid()) { - dewarper_->outputBufferReady.connect( - this, &PipelineHandlerRkISP1::dewarpBufferReady); - - LOG(RkISP1, Info) - << "Found DW100 dewarper " << dewarper_->deviceNode(); - } else { - LOG(RkISP1, Warning) - << "Found DW100 dewarper " << dewarper_->deviceNode() - << " but invalid"; - - dewarper_.reset(); - } + dewarper_ = ConverterDW100Module::createModule(enumerator); + if (dewarper_) { + dewarper_->outputBufferReady.connect( + this, &PipelineHandlerRkISP1::dewarpBufferReady); + LOG(RkISP1, Debug) << "Found DW100 dewarper"; } /* @@ -1603,37 +1577,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) return; } - /* Handle scaler crop control. */ - const auto &crop = request->controls().get(controls::ScalerCrop); - if (crop) { - Rectangle rect = crop.value(); - - /* - * ScalerCrop is specified to be in Sensor coordinates. - * So we need to transform it into dewarper coordinates. - * We can safely assume that the maximum crop limit contains the - * full fov of the dewarper. - */ - std::pair cropLimits = - dewarper_->inputCropBounds(&data->mainPathStream_); - - rect = rect.transformedBetween(scalerMaxCrop_, cropLimits.second); - int ret = dewarper_->setInputCrop(&data->mainPathStream_, - &rect); - rect = rect.transformedBetween(cropLimits.second, scalerMaxCrop_); - if (!ret && rect != crop.value()) { - /* - * If the rectangle is changed by setInputCrop on the - * dewarper, log a debug message and cache the actual - * applied rectangle for metadata reporting. - */ - LOG(RkISP1, Debug) - << "Applied rectangle " << rect.toString() - << " differs from requested " << crop.value().toString(); - } - - activeCrop_ = rect; - } + dewarper_->setControls(&data->mainPathStream_, request->controls()); /* * Queue input and output buffers to the dewarper. The output @@ -1642,7 +1586,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) */ int ret = dewarper_->queueBuffers(buffer, request->buffers()); if (ret < 0) { - LOG(RkISP1, Error) << "Cannot queue buffers to dewarper: " + LOG(RkISP1, Error) << "Failed to queue buffers to dewarper: -" << strerror(-ret); cancelDewarpRequest(info); @@ -1650,7 +1594,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) return; } - request->metadata().set(controls::ScalerCrop, activeCrop_.value()); + dewarper_->populateMetadata(&data->mainPathStream_, request->metadata()); } void PipelineHandlerRkISP1::dewarpBufferReady(FrameBuffer *buffer)