From patchwork Thu Oct 29 15:28:25 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ray Strode X-Patchwork-Id: 28713 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EE6EBC324C for ; Thu, 29 Oct 2026 15:28:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AD7E9690BF; Thu, 29 Oct 2026 16:28:44 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="yeKMouho"; dkim-atps=neutral Received: from mail-ua1-x931.google.com (mail-ua1-x931.google.com [IPv6:2607:f8b0:4864:20::931]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 12511690A0 for ; Thu, 29 Oct 2026 16:28:43 +0100 (CET) Received: by mail-ua1-x931.google.com with SMTP id a1e0cc1a2514c-9908346255fso467868241.1 for ; Thu, 29 Oct 2026 08:28:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1793287722; x=1793892522; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Hx/FivPDLWAe81HIfDI1aU06X5jwYbGdLCMlToRwdJY=; b=yeKMouholhqIcvNiAw0djtWvt9sKTQsK96qY+ZLHyOJyd5Yn6GaWdJ7nn+daW+L3CN 2CiaIrQxRYWVwqSKyOJyTBDtBVttHIGcsYEUiGugfZwkuIYz1rBIjYcp/kqfWSbPdGCv oKOtWFSylmWTPOP8/Okcp2LmKhnnnenQERx0XpI+EclVVol8BPmVf2Mxp9D/9yNr27S3 ebjVFgbKVFkOgBJeNgjlsEu1EeZ509TkyrJZHE0SbfSKKQrZF3DQBIeR/GfDdzKdKFjH //JyqFuaAFfunuh9TUtxKI9JXif9ERwbG9x+TEDswt+b+4TQLRn/f2+NP0Xxvt5rOPBD 1lFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1793287722; x=1793892522; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Hx/FivPDLWAe81HIfDI1aU06X5jwYbGdLCMlToRwdJY=; b=HlCDIAG2aJoNwYj9zGclvFhngjDEqx+ia6PLBh/zWbP5Xetxc+Uquw4CCbhEq5ZZc2 B39Axl8bLKm/xtUVfexxgtmfswyWe3sjpsfqSHbBi6O7I6aLWtzB2KOpMFzZtZfIX0PI +vtDyEGaZ1NYn6VDLvM3Jg4DXHWwDcqCuOB8Q9iyRIOjr4M9Twb0zsagz9W1/sh7Yh6M AwNYRqLgT0G772eIQVRqXWG2hr3RnAGbHT+dNMmJ92riUx/ljso39jMHfLYDIldpDix6 4xhMLbzhqJEUM0guZWuTQT4f9W5OsXY9Pw4ybkSTdd7aPTjJ/V8FmAQrvfscn+tpfbwg E7nA== X-Gm-Message-State: AFq9FYI5EGs3QaIjtzQHR3n507BNYxPRJ8XkYHn08qcl2U6vwsgnASpJ ddMAJwUUZVfd73Bx8xWri599TNKdEd6NEPh4G56C8RbAyJcABPvrNwKecK47wmlPW0g= X-Gm-Gg: AYBFou02Uo2Z1Mxp87TLfAKVxARgjxo2EmN74ILFOfXp1aoav8a3KRNKLvTrMQlxQ6E p4GpLosKW/p/oBy7E9xPIQTYO3yc/+2RsUW8hZ5qUWE43Yi7CVXmAbyw9WOZZe/vzEtP3WgEVkL HXZL1m8KlBIUg4/1nW7DONplo7DhIXESIKjG+Ix7M/4S/aZNhqoKoUEXq22paEF96W+MrSzq7iO LnKqBwFex6EYP0ONPCZ0k6P9A7MvpvgHiPONa9xY4Lo9WQuiq6afyYzO1TGqjGpAiWCAT4rDPw3 jn+5vWxfKUkYMzaF8clHomI7L3tx1p70LimG5fMYnvrzW75F8k9iaWNmuBrMWQGR9euqAB0j+ke lr2COnH5WoI+LrCok4e16QwtscwdNuqcat4AVKMUjYRHxdjTmcE6oC1IfdM/OdhuHWEGK8Q5JUw szaCPDALSchd/jQZ+fJ0SOu43CvEJvsXxSSQVhMEG9Lp+G5HQkaDIC2l+fmHRNyLfZDGtz7p6/Q QU4neIsQPhlNjqW/BNDSqxmwU8Ea2IXG80A7i1rITPY8WhMpQwRG73IZNZK9LRHWZgq0XaWs2du /ObL9EFQjwdGJJceYum5OkBByRXyKOhAT0j2L0300tXCngJWxEYVHr5KrQ== X-Received: by 2002:a05:6102:e0b:b0:7cc:a4b:b0e3 with SMTP id ada2fe7eead31-7d0541814a2mr949960137.13.1793287720594; Thu, 29 Oct 2026 08:28:40 -0700 (PDT) Received: from localhost ([144.121.52.163]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-991f76aea18sm182670241.11.2026.10.29.08.28.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Oct 2026 08:28:39 -0700 (PDT) From: Ray Strode To: libcamera-devel@lists.libcamera.org Cc: kieran.bingham@ideasonboard.com, bod.linux@nxsw.ie, laurent.pinchart@ideasonboard.com, stefan.klug@ideasonboard.com, martincarvalho@gmail.com Subject: [PATCH v5 10/11] pipeline: simple: Report the software ISP's maximum crop Date: Thu, 29 Oct 2026 11:28:25 -0400 Message-ID: <20261029152058.896209-11-halfline@gmail.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261029152058.896209-1-halfline@gmail.com> References: <20261029152058.896209-1-halfline@gmail.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Ray Strode This commit reports the software ISP's configured image region through the ScalerCropMaximum camera property. The simple pipeline currently exposes sensor properties without identifying the region retained by software processing. CPU debayering crops to the output dimensions, while EGL also scales the image, so the visible region can be smaller than the analog sensor crop. The reported rectangle uses physical pixel array coordinates in native sensor pixels. The mapping requires matching sensor, capture and intermediate pad sizes, and full-frame crop and compose selections where available. Unavailable selection stages preserve the preceding rectangle on supported pass-through paths. Query errors or reported intermediate cropping or scaling leave the mapping unknown and the property unreported. ScalerCropMaximum is updated only after configuration succeeds and is removed when the new configuration cannot report a software ISP crop. Based on Kieran Bingham's ScalerCropMaximum reporting patch, extended to account for the software ISP's selected image region. Link: https://patchwork.libcamera.org/patch/26678/ Signed-off-by: Ray Strode --- src/libcamera/pipeline/simple/simple.cpp | 92 +++++++++++++++++++++++- 1 file changed, 89 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 38e94a15..3e6a3a50 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -3,74 +3,76 @@ * Copyright (C) 2020, Laurent Pinchart * Copyright (C) 2019, Martijn Braam * * Pipeline handler for simple pipelines */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include +#include #include #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_manager.h" #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/camera_sensor_properties.h" #include "libcamera/internal/converter.h" #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/formats.h" #include "libcamera/internal/global_configuration.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/request.h" +#include "libcamera/internal/software_isp/debayer_geometry.h" #include "libcamera/internal/software_isp/software_isp.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" namespace libcamera { LOG_DEFINE_CATEGORY(SimplePipeline) /* ----------------------------------------------------------------------------- * * Overview * -------- * * The SimplePipelineHandler relies on generic kernel APIs to control a camera * device, without any device-specific code and with limited device-specific * static data. * * To qualify for support by the simple pipeline handler, a device shall * * - be supported by V4L2 drivers, exposing the Media Controller API, the V4L2 * subdev APIs and the media bus format-based enumeration extension for the * VIDIOC_ENUM_FMT ioctl ; * - not expose any device-specific API from drivers to userspace ; * - include one or more camera sensor media entities and one or more video * capture devices ; * - have a capture pipeline with linear paths from the camera sensors to the * video capture devices ; and * - have an optional memory-to-memory device to perform format conversion * and/or scaling, exposed as a V4L2 M2M device. * @@ -265,60 +267,62 @@ static const SimplePipelineInfo supportedDevices[] = { { "mxc-isi", {}, false }, { "qcom-camss", {}, true }, { "sun6i-csi", {}, false }, }; bool isRaw(const StreamConfiguration &cfg) { return libcamera::PixelFormatInfo::info(cfg.pixelFormat).colourEncoding == libcamera::PixelFormatInfo::ColourEncodingRAW; } } /* namespace */ class SimpleCameraData : public Camera::Private { public: SimpleCameraData(SimplePipelineHandler *pipe, unsigned int numStreams, MediaEntity *sensor); bool isValid() const { return sensor_ != nullptr; } SimplePipelineHandler *pipe(); int init(); int setupLinks(); int setupFormats(V4L2SubdeviceFormat *format, V4L2Subdevice::Whence whence, Transform transform = Transform::Identity); void imageBufferReady(FrameBuffer *buffer); void clearIncompleteRequests(); + std::optional computeScalerCrop(const Size &inputSize, Transform transform); + void updateScalerCrop(std::optional crop); unsigned int streamIndex(const Stream *stream) const { return stream - &streams_.front(); } struct Entity { /* The media entity, always valid. */ MediaEntity *entity; /* * Whether or not the entity is a subdev that supports the * routing API. */ bool supportsRouting; /* * The local sink pad connected to the upstream entity, null for * the camera sensor at the beginning of the pipeline. */ const MediaPad *sink; /* * The local source pad connected to the downstream entity, null * for the video node at the end of the pipeline. */ const MediaPad *source; /* * The link on the source pad, to the downstream entity, null * for the video node at the end of the pipeline. */ MediaLink *sourceLink; }; @@ -333,60 +337,61 @@ public: }; std::vector streams_; Stream *rawStream_; /* * All entities in the pipeline, from the camera sensor to the video * node. */ std::list entities_; std::unique_ptr sensor_; V4L2VideoDevice *video_; V4L2Subdevice *frameStartEmitter_; std::vector configs_; std::map> formats_; std::unique_ptr delayedCtrls_; std::vector> conversionBuffers_; struct RequestOutputs { Request *request; std::map outputs; }; std::queue conversionQueue_; bool useConversion_; std::unique_ptr converter_; std::unique_ptr swIsp_; SimpleFrames frameInfo_; + std::optional scalerCrop_; private: void tryPipeline(unsigned int code, const Size &size); static std::vector routedSourcePads(MediaPad *sink); void tryCompleteRequest(Request *request); void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); void ispStatsReady(uint32_t frame, uint32_t bufferId); void metadataReady(uint32_t frame, const ControlList &metadata); void setSensorControls(const ControlList &sensorControls); }; class SimpleCameraConfiguration : public CameraConfiguration { public: SimpleCameraConfiguration(Camera *camera, SimpleCameraData *data); Status validate() override; const SimpleCameraData::Configuration *pipeConfig() const { return pipeConfig_; } bool needConversion() const { return needConversion_; } const Transform &combinedTransform() const { return combinedTransform_; } private: @@ -956,60 +961,130 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) } if (converter_) converter_->queueBuffers(buffer, conversionQueue_.front().outputs); else /* * request->sequence() cannot be retrieved from `buffer' inside * queueBuffers because unique_ptr's make buffer->request() invalid * already here. */ swIsp_->queueBuffers(request->sequence(), buffer, conversionQueue_.front().outputs); conversionQueue_.pop(); return; } /* Otherwise simply complete the request. */ pipe->completeBuffer(request, buffer); tryCompleteRequest(request); } void SimpleCameraData::clearIncompleteRequests() { while (!conversionQueue_.empty()) { pipe()->cancelRequest(conversionQueue_.front().request); conversionQueue_.pop(); } } +std::optional SimpleCameraData::computeScalerCrop(const Size &inputSize, + Transform transform) +{ + IPACameraSensorInfo sensorInfo; + if (sensor_->sensorInfo(&sensorInfo)) + return std::nullopt; + + /* + * The buffer-to-sensor mapping requires a capture path that preserves + * the sensor output. Intermediate cropping or scaling needs a separate + * mapping; do not report the analog crop as if it included that work. + * Unavailable selection stages preserve the preceding rectangle on the + * supported pass-through paths, provided the pad formats match. + */ + if (sensorInfo.outputSize != inputSize) + return std::nullopt; + + for (const Entity &entity : entities_) { + if (!entity.sink || !entity.source) + continue; + + V4L2Subdevice *subdev = pipe()->subdev(entity.entity); + for (const MediaPad *pad : { entity.sink, entity.source }) { + V4L2SubdeviceFormat format{}; + if (subdev->getFormat(pad->index(), &format) || + format.size != inputSize) + return std::nullopt; + + for (unsigned int target : { V4L2_SEL_TGT_CROP, V4L2_SEL_TGT_COMPOSE }) { + /* Composition is applied on the sink pad. */ + if (pad == entity.source && target == V4L2_SEL_TGT_COMPOSE) + continue; + + std::optional selection; + int ret = subdev->getSelection(pad->index(), target, &selection); + if (ret) + return std::nullopt; + if (!selection) + continue; + + if (*selection != Rectangle(inputSize)) + return std::nullopt; + } + } + } + + return debayerCropToSensor(swIsp_->inputCrop(), inputSize, + sensorInfo.analogCrop, transform); +} + +void SimpleCameraData::updateScalerCrop(std::optional crop) +{ + scalerCrop_ = crop; + if (crop) { + properties_.set(properties::ScalerCropMaximum, *crop); + return; + } + + if (!properties_.contains(properties::ScalerCropMaximum.id())) + return; + + ControlList updatedProperties = properties_; + updatedProperties.clear(); + for (const auto &[id, value] : properties_) { + if (id != properties::ScalerCropMaximum.id()) + updatedProperties.set(id, value); + } + properties_ = std::move(updatedProperties); +} + void SimpleCameraData::tryCompleteRequest(Request *request) { if (request->hasPendingBuffers()) return; SimpleFrameInfo *info = frameInfo_.find(request->sequence()); if (!info) { /* Something is really wrong, let's return. */ return; } if (info->metadataRequired && !info->metadataProcessed) return; frameInfo_.destroy(info->frame); pipe()->completeRequest(request); } void SimpleCameraData::conversionInputDone(FrameBuffer *buffer) { if (rawStream_) { /* Complete the input buffer as with raw-only processing. */ Request *request = buffer->request(); if (pipe()->completeBuffer(request, buffer)) tryCompleteRequest(request); } else { /* Queue the input buffer back for capture. */ video_->queueBuffer(buffer); } } @@ -1576,76 +1651,87 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) << pipeConfig->captureSize << "-" << videoFormat << " (got " << captureFormat << ")"; return -EINVAL; } if (requested_bpl && captureFormat.planes[0].bpl != requested_bpl) { LOG(SimplePipeline, Info) << "Input buffer stride ignored by the driver. " << "Requested " << requested_bpl << ", got " << captureFormat.planes[0].bpl; } /* Configure the converter if needed. */ std::vector> outputCfgs; data->useConversion_ = config->needConversion(); data->rawStream_ = nullptr; for (unsigned int i = 0; i < config->size(); ++i) { StreamConfiguration &cfg = config->at(i); bool rawStream = isRaw(cfg); cfg.setStream(&data->streams_[i]); if (data->useConversion_ && !rawStream) outputCfgs.push_back(cfg); if (rawStream) data->rawStream_ = &data->streams_[i]; } - if (outputCfgs.empty()) + if (outputCfgs.empty()) { + data->updateScalerCrop(std::nullopt); return 0; + } StreamConfiguration inputCfg; inputCfg.pixelFormat = videoFormat.toPixelFormat(); inputCfg.size = pipeConfig->captureSize; inputCfg.stride = captureFormat.planes[0].bpl; inputCfg.bufferCount = kNumInternalBuffers; if (data->converter_) { - return data->converter_->configure(inputCfg, outputCfgs); + ret = data->converter_->configure(inputCfg, outputCfgs); } else { ipa::softisp::IPAConfigInfo configInfo; configInfo.sensorControls = data->sensor_->controls(); - return data->swIsp_->configure(inputCfg, outputCfgs, configInfo, &data->controlInfo_); + ret = data->swIsp_->configure(inputCfg, outputCfgs, configInfo, &data->controlInfo_); } + if (ret) + return ret; + + std::optional crop; + if (!data->converter_) + crop = data->computeScalerCrop(inputCfg.size, config->combinedTransform()); + data->updateScalerCrop(crop); + + return 0; } int SimplePipelineHandler::exportFrameBuffers(Camera *camera, Stream *stream, std::vector> *buffers) { SimpleCameraData *data = cameraData(camera); unsigned int count = stream->configuration().bufferCount; /* * Export buffers on the converter or capture video node, depending on * whether the converter is used or not. */ if (data->useConversion_ && stream != data->rawStream_) return data->converter_ ? data->converter_->exportBuffers(stream, count, buffers) : data->swIsp_->exportBuffers(stream, count, buffers); else return data->video_->exportBuffers(count, buffers); } int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlList *controls) { SimpleCameraData *data = cameraData(camera); V4L2VideoDevice *video = data->video_; V4L2Subdevice *frameStartEmitter = data->frameStartEmitter_; int ret; const MediaPad *pad = acquirePipeline(data); if (pad) { LOG(SimplePipeline, Info)