From patchwork Thu Oct 29 13:26:30 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ray Strode X-Patchwork-Id: 28700 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6ABBCC3261 for ; Thu, 29 Oct 2026 13:27:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1CAD3690BB; Thu, 29 Oct 2026 14:27:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="L2MeTblX"; dkim-atps=neutral Received: from mail-ua1-x92c.google.com (mail-ua1-x92c.google.com [IPv6:2607:f8b0:4864:20::92c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C1DD5690B2 for ; Thu, 29 Oct 2026 14:27:26 +0100 (CET) Received: by mail-ua1-x92c.google.com with SMTP id a1e0cc1a2514c-9828fee1b4bso745218241.3 for ; Thu, 29 Oct 2026 06:27:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1793280445; x=1793885245; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=fpp5D11xjW5GknMdc9JoAnpOrym1MSWo25gamzd6rHg=; b=L2MeTblXddGL6xoph1HGf1e3Q7sDEeN3L3+rSMzNOGlX4Xq+16hmznjFAWmZK+2jlQ /GBLhd7vhryBS/vvbzrVfZl4Nt8y4bpSh1V5UKpJxIENTTC/3OmZUv8UGoOap0yPeZQf +rMAydE0g2cruRs9+ICCobLhOit+CXiUlLV08xWotb7xnBDI4sVIE9dZIpWSTJy2AUx0 NijlIlhKQrM55dcVLMl8JnHkBKRKQyI6+hqVCS1p3GsFP/cEE9vhiogSLPoGUkufneis SGOl7ashrZKssOiCsVyzWCNOmkw3ckeIQ4+vaqonytVGhJWuLkt270MHkUZWHZe//5QE XTEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1793280445; x=1793885245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=fpp5D11xjW5GknMdc9JoAnpOrym1MSWo25gamzd6rHg=; b=LZH0dgujp7K9DuRtq5KtcU6k03jihUtjWZLy4WF3dpBNG6/xeJ/K29YlyeW2FEGOhX eCTrxsHDK1ugWo2OW8NnrdeLt+JSKTQnrdBtBCX1moslNOZhEbOcVwRm0gAhgWIThBKo cDDgpPSud6rKxE9Nv5TuqJ1ELFI963ViHB9nEtHEbe0D7RQgupK2YkM+BSXp3+8yRVoR cqSgHc380yryaTmmqZ/SI2Sd+K5vPhmGbClgUqSQKu9jTK9886Mg4XvicuDdqyXNuC5p pxB8CU9TZkIEIlnB0QwtY/esmCfdymkFSnwANNsMOW/O+tbWrOQuGRoRSASFAtby5A3k kwkw== X-Gm-Message-State: AFq9FYKjoHq4A2UHpqlIUe4PqBTycGLB570+2KVpd8jjMCLrBVPfxJS0 A9Up0CNogjxMBmWFVptHTx4vgUgT4KMSVFTMhnpoeXDUmjMr4nEIIJqV1WZCldsPBYQ= X-Gm-Gg: AYBFou00azBNgKTfueN/uvppmCiWO3kqzkMgHmDg5MBYMtJrZyDVYwZM3sw4ggpaV3s kAgv+bkjaGEZtXZRxN60je4DS1AOYJV65O2/DGnwMZx58SCsm5tFj3lzhcXxfc0T/Id2TFW9sh1 CuOKMiqkgeTw+I7DRTisLcczuffVjg9aKwdtLRU5aS7iUVDftFoA1lkybFRD2AlJWx4DWvgPbeh LHOIfg321zpwvjZw9U9D1JqoX+gxYk4bvFML6ibW2aQMKCPNE01DYRZs3ZYa1ODHA9febFEbXwe zPmMOu3ktTivtja9u6vmFno7vG7uo80PyC2cAUZ2gwugBx0trhnKwiojm97uxsmu/bxUi/JO1y+ Jh12iVG/NEh7rTMDmw4XXIz+Q9CSSDgrgVfCg/9VMmwwXG6IZMCXOxLVX8o9rKgEaJVdmT3CV3n x3r+mTfARaXcBDkf6na3yVLUvpQbEGfLcDWF9WSsHa2aVJYzHOvRtEYC51jkr+pU2bwqPTaBANX 10lMjs8lhOO8mpQGh79XPYEK0ML93GhGCDGLodeelewJ6LnqSo7oxjnk1Ddumhbj+zGSc1RQlzY v6qUAIapHDFgDakPrR09S6cqZrJ1rsybO2yjo4mgJwG4Og3lVNPNCqGz X-Received: by 2002:a05:6102:605a:b0:7cc:32da:56e with SMTP id ada2fe7eead31-7d03a5a2346mr1967555137.19.1793280445510; Thu, 29 Oct 2026 06:27:25 -0700 (PDT) Received: from localhost ([144.121.52.163]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-99125931983sm9476046241.2.2026.10.29.06.27.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Oct 2026 06:27:24 -0700 (PDT) From: Ray Strode To: libcamera-devel@lists.libcamera.org Cc: kieran.bingham@ideasonboard.com, bod.linux@nxsw.ie, laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com, martincarvalho@gmail.com Subject: [PATCH v4 8/9] pipeline: simple: Report the software ISP's maximum crop Date: Thu, 29 Oct 2026 09:26:30 -0400 Message-ID: <20261029132631.370507-9-halfline@gmail.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261029132631.370507-1-halfline@gmail.com> References: <20261016202818.1405645-1-halfline@gmail.com> <20261029132631.370507-1-halfline@gmail.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Ray Strode This commit reports the software ISP's configured image region through the ScalerCropMaximum camera property. The simple pipeline currently exposes sensor properties without identifying the region retained by software processing. CPU debayering crops to the output dimensions, while EGL also scales the image, so the visible region can be smaller than the analog sensor crop. The reported rectangle uses physical pixel array coordinates in native sensor pixels. Require matching sensor, capture and intermediate pad sizes, and full-frame crop and compose selections where available. Unavailable selection stages preserve the preceding rectangle on supported pass-through paths. Query errors or reported intermediate cropping or scaling leave the mapping unknown and the property unreported. Update ScalerCropMaximum only after configuration succeeds, removing it when the new configuration cannot report a software ISP crop. Based on Kieran Bingham's ScalerCropMaximum reporting patch, extended to account for the software ISP's selected image region. Link: https://patchwork.libcamera.org/patch/26678/ Signed-off-by: Ray Strode --- src/libcamera/pipeline/simple/simple.cpp | 92 +++++++++++++++++++++++- 1 file changed, 89 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 38e94a15..3e6a3a50 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -3,74 +3,76 @@ * Copyright (C) 2020, Laurent Pinchart * Copyright (C) 2019, Martijn Braam * * Pipeline handler for simple pipelines */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include +#include #include #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_manager.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/camera_sensor_properties.h" #include "libcamera/internal/converter.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/formats.h" #include "libcamera/internal/global_configuration.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/request.h" +#include "libcamera/internal/software_isp/debayer_geometry.h" #include "libcamera/internal/software_isp/software_isp.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" namespace libcamera { LOG_DEFINE_CATEGORY(SimplePipeline) /* ----------------------------------------------------------------------------- * * Overview * -------- * * The SimplePipelineHandler relies on generic kernel APIs to control a camera * device, without any device-specific code and with limited device-specific * static data. * * To qualify for support by the simple pipeline handler, a device shall * * - be supported by V4L2 drivers, exposing the Media Controller API, the V4L2 * subdev APIs and the media bus format-based enumeration extension for the * VIDIOC_ENUM_FMT ioctl ; * - not expose any device-specific API from drivers to userspace ; * - include one or more camera sensor media entities and one or more video * capture devices ; * - have a capture pipeline with linear paths from the camera sensors to the * video capture devices ; and * - have an optional memory-to-memory device to perform format conversion * and/or scaling, exposed as a V4L2 M2M device. * @@ -265,60 +267,62 @@ static const SimplePipelineInfo supportedDevices[] = { { "mxc-isi", {}, false }, { "qcom-camss", {}, true }, { "sun6i-csi", {}, false }, }; bool isRaw(const StreamConfiguration &cfg) { return libcamera::PixelFormatInfo::info(cfg.pixelFormat).colourEncoding == libcamera::PixelFormatInfo::ColourEncodingRAW; } } /* namespace */ class SimpleCameraData : public Camera::Private { public: SimpleCameraData(SimplePipelineHandler *pipe, unsigned int numStreams, MediaEntity *sensor); bool isValid() const { return sensor_ != nullptr; } SimplePipelineHandler *pipe(); int init(); int setupLinks(); int setupFormats(V4L2SubdeviceFormat *format, V4L2Subdevice::Whence whence, Transform transform = Transform::Identity); void imageBufferReady(FrameBuffer *buffer); void clearIncompleteRequests(); + std::optional computeScalerCrop(const Size &inputSize, Transform transform); + void updateScalerCrop(std::optional crop); unsigned int streamIndex(const Stream *stream) const { return stream - &streams_.front(); } struct Entity { /* The media entity, always valid. */ MediaEntity *entity; /* * Whether or not the entity is a subdev that supports the * routing API. */ bool supportsRouting; /* * The local sink pad connected to the upstream entity, null for * the camera sensor at the beginning of the pipeline. */ const MediaPad *sink; /* * The local source pad connected to the downstream entity, null * for the video node at the end of the pipeline. */ const MediaPad *source; /* * The link on the source pad, to the downstream entity, null * for the video node at the end of the pipeline. */ MediaLink *sourceLink; }; @@ -333,60 +337,61 @@ public: }; std::vector streams_; Stream *rawStream_; /* * All entities in the pipeline, from the camera sensor to the video * node. */ std::list entities_; std::unique_ptr sensor_; V4L2VideoDevice *video_; V4L2Subdevice *frameStartEmitter_; std::vector configs_; std::map> formats_; std::unique_ptr delayedCtrls_; std::vector> conversionBuffers_; struct RequestOutputs { Request *request; std::map outputs; }; std::queue conversionQueue_; bool useConversion_; std::unique_ptr converter_; std::unique_ptr swIsp_; SimpleFrames frameInfo_; + std::optional scalerCrop_; private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); void tryCompleteRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); void ispStatsReady(uint32_t frame, uint32_t bufferId); void metadataReady(uint32_t frame, const ControlList &metadata); void setSensorControls(const ControlList &sensorControls); }; class SimpleCameraConfiguration : public CameraConfiguration { public: SimpleCameraConfiguration(Camera *camera, SimpleCameraData *data); Status validate() override; const SimpleCameraData::Configuration *pipeConfig() const { return pipeConfig_; } bool needConversion() const { return needConversion_; } const Transform &combinedTransform() const { return combinedTransform_; } private: @@ -956,60 +961,130 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) } if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front().outputs); else /* * request->sequence() cannot be retrieved from `buffer' inside * queueBuffers because unique_ptr's make buffer->request() invalid * already here. */ swIsp_->queueBuffers(request->sequence(), buffer, conversionQueue_.front().outputs); conversionQueue_.pop(); return; } /* Otherwise simply complete the request. */ pipe->completeBuffer(request, buffer); tryCompleteRequest(request); } void SimpleCameraData::clearIncompleteRequests() { while (!conversionQueue_.empty()) { pipe()->cancelRequest(conversionQueue_.front().request); conversionQueue_.pop(); } } +std::optional SimpleCameraData::computeScalerCrop(const Size &inputSize, + Transform transform) +{ + IPACameraSensorInfo sensorInfo; + if (sensor_->sensorInfo(&sensorInfo)) + return std::nullopt; + + /* + * The buffer-to-sensor mapping requires a capture path that preserves + * the sensor output. Intermediate cropping or scaling needs a separate + * mapping; do not report the analog crop as if it included that work. + * Unavailable selection stages preserve the preceding rectangle on the + * supported pass-through paths, provided the pad formats match. + */ + if (sensorInfo.outputSize != inputSize) + return std::nullopt; + + for (const Entity &entity : entities_) { + if (!entity.sink || !entity.source) + continue; + + V4L2Subdevice *subdev = pipe()->subdev(entity.entity); + for (const MediaPad *pad : { entity.sink, entity.source }) { + V4L2SubdeviceFormat format{}; + if (subdev->getFormat(pad->index(), &format) || + format.size != inputSize) + return std::nullopt; + + for (unsigned int target : { V4L2_SEL_TGT_CROP, V4L2_SEL_TGT_COMPOSE }) { + /* Composition is applied on the sink pad. */ + if (pad == entity.source && target == V4L2_SEL_TGT_COMPOSE) + continue; + + std::optional selection; + int ret = subdev->getSelection(pad->index(), target, &selection); + if (ret) + return std::nullopt; + if (!selection) + continue; + + if (*selection != Rectangle(inputSize)) + return std::nullopt; + } + } + } + + return debayerCropToSensor(swIsp_->inputCrop(), inputSize, + sensorInfo.analogCrop, transform); +} + +void SimpleCameraData::updateScalerCrop(std::optional crop) +{ + scalerCrop_ = crop; + if (crop) { + properties_.set(properties::ScalerCropMaximum, *crop); + return; + } + + if (!properties_.contains(properties::ScalerCropMaximum.id())) + return; + + ControlList updatedProperties = properties_; + updatedProperties.clear(); + for (const auto &[id, value] : properties_) { + if (id != properties::ScalerCropMaximum.id()) + updatedProperties.set(id, value); + } + properties_ = std::move(updatedProperties); +} + void SimpleCameraData::tryCompleteRequest(Request *request) { if (request->hasPendingBuffers()) return; SimpleFrameInfo *info = frameInfo_.find(request->sequence()); if (!info) { /* Something is really wrong, let's return. */ return; } if (info->metadataRequired && !info->metadataProcessed) return; frameInfo_.destroy(info->frame); pipe()->completeRequest(request); } void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { if (rawStream_) { /* Complete the input buffer as with raw-only processing. */ Request *request = buffer->request(); if (pipe()->completeBuffer(request, buffer)) tryCompleteRequest(request); } else { /* Queue the input buffer back for capture. */ video_->queueBuffer(buffer); } } @@ -1576,76 +1651,87 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) << pipeConfig->captureSize << "-" << videoFormat << " (got " << captureFormat << ")"; return -EINVAL; } if (requested_bpl && captureFormat.planes[0].bpl != requested_bpl) { LOG(SimplePipeline, Info) << "Input buffer stride ignored by the driver. " << "Requested " << requested_bpl << ", got " << captureFormat.planes[0].bpl; } /* Configure the converter if needed. */ std::vector> outputCfgs; data->useConversion_ = config->needConversion(); data->rawStream_ = nullptr; for (unsigned int i = 0; i < config->size(); ++i) { StreamConfiguration &cfg = config->at(i); bool rawStream = isRaw(cfg); cfg.setStream(&data->streams_[i]); if (data->useConversion_ && !rawStream) outputCfgs.push_back(cfg); if (rawStream) data->rawStream_ = &data->streams_[i]; } - if (outputCfgs.empty()) + if (outputCfgs.empty()) { + data->updateScalerCrop(std::nullopt); return 0; + } StreamConfiguration inputCfg; inputCfg.pixelFormat = videoFormat.toPixelFormat(); inputCfg.size = pipeConfig->captureSize; inputCfg.stride = captureFormat.planes[0].bpl; inputCfg.bufferCount = kNumInternalBuffers; if (data->converter_) { - return data->converter_->configure(inputCfg, outputCfgs); + ret = data->converter_->configure(inputCfg, outputCfgs); } else { ipa::softisp::IPAConfigInfo configInfo; configInfo.sensorControls = data->sensor_->controls(); - return data->swIsp_->configure(inputCfg, outputCfgs, configInfo, &data->controlInfo_); + ret = data->swIsp_->configure(inputCfg, outputCfgs, configInfo, &data->controlInfo_); } + if (ret) + return ret; + + std::optional crop; + if (!data->converter_) + crop = data->computeScalerCrop(inputCfg.size, config->combinedTransform()); + data->updateScalerCrop(crop); + + return 0; } int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) { SimpleCameraData *data = cameraData(camera); unsigned int count = stream->configuration().bufferCount; /* * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ if (data->useConversion_ && stream != data->rawStream_) return data->converter_ ? data->converter_->exportBuffers(stream, count, buffers) : data->swIsp_->exportBuffers(stream, count, buffers); else return data->video_->exportBuffers(count, buffers); } int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlList *controls) { SimpleCameraData *data = cameraData(camera); V4L2VideoDevice *video = data->video_; V4L2Subdevice *frameStartEmitter = data->frameStartEmitter_; int ret; const MediaPad *pad = acquirePipeline(data); if (pad) { LOG(SimplePipeline, Info)