From patchwork Fri Oct 22 11:55:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14253 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0685DBF415 for ; Fri, 22 Oct 2021 11:55:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7836668F59; Fri, 22 Oct 2021 13:55:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="dddLCyk8"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2C4746012A for ; Fri, 22 Oct 2021 13:55:42 +0200 (CEST) Received: by mail-wr1-x434.google.com with SMTP id i12so1875964wrb.7 for ; Fri, 22 Oct 2021 04:55:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1+yQGT8m0ouNkcWR6TMH7jsBxo1G5lAbrwkO3omOMbI=; b=dddLCyk8xzdbd3MIStqiyTHKBC29VqqPKykYVPK9Vg/sM2T7/pa0kUVC8B32JrbiHC G2vUguzbo8vmjNQHAnfqx42H7tIABzxAS7L7Rj4vx4Pb7WSYgkIj/HrbewYwZ13eTnLk bLRAxwdWBirxUJcE/6Ch6/PnQE6U5tZvM7y1o2baAHYymiiDEbZzceXk900qxxRsJ+S9 G1+M/7uUoIvPyRtfb8sqwy8cbcf1Me7/4KMatvmvfkyIzfqeCdkCZmHFDHD0wKTMfUNg 4lLkjBCj1MDts45oG3HFRsSV5ewibM63ZWbzhi/z4XMV9wZDWfzo8Q/krQ3nIPT+9BCz rCZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1+yQGT8m0ouNkcWR6TMH7jsBxo1G5lAbrwkO3omOMbI=; b=wDGyvX+07LQ+wzbqDen0aQM9ITt4tUv2V15J+hSAlLeXCGmHphmHEnhYwYtPqdYx6m O8HBI/ZIJwG1yBc21E4gPZzzX+CY1asjwgWEvp0kC6hoiDkEPxjiTNN3XEYK0V7at6n2 TpmlSOFCmT64oQO14c8edsBqt7dA8SCDXC/+eiaZITgbFlA5+qoNQOREKZZx1NyJ7rw+ iEcCHPXHkPjRBJSpy+T7mgLIA7/XGAPUjIRBFFRtc1bhARohnAmhCwhjnJ3dHlzpv1tP M9zH3Ts4YxSj6bOFH37hxc5BVSUGgrbONxK/zcSEiJz85uAu1czw/AXYQ7s2xrf4pcIV HFPQ== X-Gm-Message-State: AOAM532/BdWlx8WsidHQJlUsOINRbgobua3CgESvFsC9fZxIQLVjV5D3 6l0juxyWgkIKvA4MMnhr5h1wHn3zwxp7zA== X-Google-Smtp-Source: ABdhPJyTdNFsNsJdJ9U84JtNBld5TMfhAHuyi+RQOYU9nXC6gw4AX2iP0tX0OC4s/QmVGZ0hsjZlyw== X-Received: by 2002:adf:fe45:: with SMTP id m5mr15114630wrs.197.1634903741586; Fri, 22 Oct 2021 04:55:41 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id l5sm7853364wru.24.2021.10.22.04.55.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 04:55:41 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 12:55:32 +0100 Message-Id: <20211022115537.2964533-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022115537.2964533-1-naush@raspberrypi.com> References: <20211022115537.2964533-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/6] libcamera: bayer_format: Add conversion helpers to the BayerFormat class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a BayerFormat::toMbusCode() member function to convert a BayerFormat to an mbus code defined by the V4L2 specification. Add a BayerFormat::toPixelFormat() member function to convert a BayerFormat to a PixelFormat type. This conversion uses the existing bayerToV4l2 conversion table. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- include/libcamera/internal/bayer_format.h | 4 +++ src/libcamera/bayer_format.cpp | 31 +++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/include/libcamera/internal/bayer_format.h b/include/libcamera/internal/bayer_format.h index 723382d4168d..217da54e90a2 100644 --- a/include/libcamera/internal/bayer_format.h +++ b/include/libcamera/internal/bayer_format.h @@ -10,6 +10,8 @@ #include #include +#include + #include "libcamera/internal/v4l2_pixelformat.h" namespace libcamera { @@ -43,6 +45,7 @@ public: { } + unsigned int toMbusCode() const; static const BayerFormat &fromMbusCode(unsigned int mbusCode); bool isValid() const { return bitDepth != 0; } @@ -50,6 +53,7 @@ public: V4L2PixelFormat toV4L2PixelFormat() const; static BayerFormat fromV4L2PixelFormat(V4L2PixelFormat v4l2Format); + PixelFormat toPixelFormat() const; BayerFormat transform(Transform t) const; Order order; diff --git a/src/libcamera/bayer_format.cpp b/src/libcamera/bayer_format.cpp index 11355f144f66..c138e7a09597 100644 --- a/src/libcamera/bayer_format.cpp +++ b/src/libcamera/bayer_format.cpp @@ -169,6 +169,24 @@ const std::unordered_map mbusCodeToBayer{ * \param[in] p The type of packing applied to the pixel values */ +/** + * \brief Retrieve the media bus code associated with a BayerFormat + * + * The media bus code numeric identifiers are defined by the V4L2 specification. + */ +unsigned int BayerFormat::toMbusCode() const +{ + const auto it = std::find_if(mbusCodeToBayer.begin(), mbusCodeToBayer.end(), + [this](const auto &p) { + return p.second == *this; + }); + + if (it == mbusCodeToBayer.end()) + return it->first; + + return 0; +} + /** * \brief Retrieve the BayerFormat associated with a media bus code * \param[in] mbusCode The media bus code to convert into a BayerFormat @@ -269,6 +287,19 @@ BayerFormat BayerFormat::fromV4L2PixelFormat(V4L2PixelFormat v4l2Format) return BayerFormat(); } +/** + * \brief Convert a BayerFormat into the corresponding PixelFormat + * \return The PixelFormat corresponding to this BayerFormat + */ +PixelFormat BayerFormat::toPixelFormat() const +{ + const auto it = bayerToV4l2.find(*this); + if (it != bayerToV4l2.end()) + return PixelFormat(it->second); + + return PixelFormat(); +} + /** * \brief Apply a transform to this BayerFormat * \param[in] t The transform to apply From patchwork Fri Oct 22 11:55:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14254 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7E5F6BF415 for ; Fri, 22 Oct 2021 11:55:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E000768F63; Fri, 22 Oct 2021 13:55:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="KlxdnJxV"; dkim-atps=neutral Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 862446012A for ; Fri, 22 Oct 2021 13:55:43 +0200 (CEST) Received: by mail-wr1-x430.google.com with SMTP id v17so294200wrv.9 for ; Fri, 22 Oct 2021 04:55:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sCKKLF8TUAwbk+gOcEEPUCf0R0HWOcxv1bN+xA23Zrs=; b=KlxdnJxVePVYLGthIvBR02mdQVFJUaDyVDODk8nNGGOLiU4d0T6NZThNkU++wzqLIE Pa+i9WioOQYSyoqD+XfuUFKs8EHtZbPW82xZMnn2GH1HRec6HPCCMvVqNAg/EQ8jHqVL ftDPvbKwK1nj4pO/0aYlA6lQRbIBAi0BF3G1YhrK6ZNvdbIWUOU1k+202A6UzE26gQN3 Ax9rgbd535Nw+EooI5SDK/5ASbybiJEsX3KsUFbPb0ZVQEGpYf9kOTOZSwzlpFRLDpW+ SSyGIpUs8p4joKiXRwVyS7X+CxxluQFOjIWxxRczuyaA2IZA7qq2j6kkdTAtB0VDoE7e 5EqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sCKKLF8TUAwbk+gOcEEPUCf0R0HWOcxv1bN+xA23Zrs=; b=4S2gniyOp+R7rlCO10khW1w8/4Val8a9+6PWpYu8pNjx3yB7Oh421aHwoBdWRPzY0Q hWH60jaJA2Y7r7z2RmKktvuoqoX2xCiK880lviQCqLS8pza8oGaJC12K6RNPvBJAcsAh KEIcc6SuwF2BaqnuXt41Zt6MQuGIHzcAD3LljkMxCr7r2VnTwJsGKvvKcSpqlCUUwLaM l9SjmjvAeHuTJjgZSH0KryNJ+ko4Yk7UVg/YI8I3mKV/eKq7R93T7il9g/2lmaa0B19W rvJObMvQzx2NNyXpI9RPeollFpOOkhCeZRDX7CqffBeNoC30xhxzKUGZCiryXU2c+jDk lxRg== X-Gm-Message-State: AOAM530BpLQMwavto+Q1bL3PItJMyQb0ZOkRYwvDEqNmexG1DiNaiibJ nJXPIwQ3SJeq9yeXad8TvWSv9SFSjCUlDUaJ X-Google-Smtp-Source: ABdhPJzAFj8zNp0q/ZjVmIye1ebKA7WuRvshaDXhhrqK0nVZVCD/QNIApI86/d+l+6Gd2aHzONkSpA== X-Received: by 2002:a05:6000:86:: with SMTP id m6mr7978169wrx.129.1634903742813; Fri, 22 Oct 2021 04:55:42 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id l5sm7853364wru.24.2021.10.22.04.55.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 04:55:41 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 12:55:33 +0100 Message-Id: <20211022115537.2964533-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022115537.2964533-1-naush@raspberrypi.com> References: <20211022115537.2964533-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/6] pipeline: raspberrypi: Convert the pipeline handler to use media controller X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Switch the pipeline handler to use the new Unicam media controller based driver. With this change, we directly talk to the sensor device driver to set controls and set/get formats in the pipeline handler. This change requires the accompanying Raspberry Pi linux kernel change at https://github.com/raspberrypi/linux/pull/4645. If this kernel change is not present, the pipeline handler will fail to run with an error message informing the user to update the kernel build. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 113 +++++++++++------- 1 file changed, 67 insertions(+), 46 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 1634ca98f481..730f1575095c 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -48,6 +48,19 @@ LOG_DEFINE_CATEGORY(RPI) namespace { +/* Map of mbus codes to supported sizes reported by the sensor. */ +using SensorFormats = std::map>; + +SensorFormats populateSensorFormats(std::unique_ptr &sensor) +{ + SensorFormats formats; + + for (auto const mbusCode : sensor->mbusCodes()) + formats.emplace(mbusCode, sensor->sizes(mbusCode)); + + return formats; +} + bool isRaw(PixelFormat &pixFmt) { /* @@ -74,8 +87,7 @@ double scoreFormat(double desired, double actual) return score; } -V4L2DeviceFormat findBestMode(V4L2VideoDevice::Formats &formatsMap, - const Size &req) +V4L2DeviceFormat findBestMode(const SensorFormats &formatsMap, const Size &req) { double bestScore = std::numeric_limits::max(), score; V4L2DeviceFormat bestMode; @@ -88,18 +100,17 @@ V4L2DeviceFormat findBestMode(V4L2VideoDevice::Formats &formatsMap, /* Calculate the closest/best mode from the user requested size. */ for (const auto &iter : formatsMap) { - V4L2PixelFormat v4l2Format = iter.first; + const unsigned int mbus_code = iter.first; + const V4L2PixelFormat v4l2Format = BayerFormat::fromMbusCode(mbus_code).toV4L2PixelFormat(); const PixelFormatInfo &info = PixelFormatInfo::info(v4l2Format); - for (const SizeRange &sz : iter.second) { - double modeWidth = sz.contains(req) ? req.width : sz.max.width; - double modeHeight = sz.contains(req) ? req.height : sz.max.height; + for (const Size &sz : iter.second) { double reqAr = static_cast(req.width) / req.height; - double modeAr = modeWidth / modeHeight; + double modeAr = sz.width / sz.height; /* Score the dimensions for closeness. */ - score = scoreFormat(req.width, modeWidth); - score += scoreFormat(req.height, modeHeight); + score = scoreFormat(req.width, sz.width); + score += scoreFormat(req.height, sz.height); score += PENALTY_AR * scoreFormat(reqAr, modeAr); /* Add any penalties... this is not an exact science! */ @@ -116,10 +127,10 @@ V4L2DeviceFormat findBestMode(V4L2VideoDevice::Formats &formatsMap, if (score <= bestScore) { bestScore = score; bestMode.fourcc = v4l2Format; - bestMode.size = Size(modeWidth, modeHeight); + bestMode.size = sz; } - LOG(RPI, Info) << "Mode: " << modeWidth << "x" << modeHeight + LOG(RPI, Info) << "Mode: " << sz.width << "x" << sz.height << " fmt " << v4l2Format.toString() << " Score: " << score << " (best " << bestScore << ")"; @@ -170,6 +181,7 @@ public: std::unique_ptr ipa_; std::unique_ptr sensor_; + SensorFormats sensorFormats_; /* Array of Unicam and ISP device streams and associated buffers/streams. */ RPi::Device unicam_; RPi::Device isp_; @@ -352,8 +364,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() * Calculate the best sensor mode we can use based on * the user request. */ - V4L2VideoDevice::Formats fmts = data_->unicam_[Unicam::Image].dev()->formats(); - V4L2DeviceFormat sensorFormat = findBestMode(fmts, cfg.size); + V4L2DeviceFormat sensorFormat = findBestMode(data_->sensorFormats_, cfg.size); int ret = data_->unicam_[Unicam::Image].dev()->tryFormat(&sensorFormat); if (ret) return Invalid; @@ -487,8 +498,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, switch (role) { case StreamRole::Raw: size = data->sensor_->resolution(); - fmts = data->unicam_[Unicam::Image].dev()->formats(); - sensorFormat = findBestMode(fmts, size); + sensorFormat = findBestMode(data->sensorFormats_, size); pixelFormat = sensorFormat.fourcc.toPixelFormat(); ASSERT(pixelFormat.isValid()); bufferCount = 2; @@ -599,32 +609,32 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) } /* First calculate the best sensor mode we can use based on the user request. */ - V4L2VideoDevice::Formats fmts = data->unicam_[Unicam::Image].dev()->formats(); - V4L2DeviceFormat sensorFormat = findBestMode(fmts, rawStream ? sensorSize : maxSize); + V4L2DeviceFormat unicamFormat = findBestMode(data->sensorFormats_, rawStream ? sensorSize : maxSize); + + unsigned int mbus_code = BayerFormat::fromV4L2PixelFormat(unicamFormat.fourcc).toMbusCode(); + V4L2SubdeviceFormat sensorFormat { .mbus_code = mbus_code, .size = unicamFormat.size }; + + ret = data->sensor_->setFormat(&sensorFormat); + if (ret) + return ret; /* * Unicam image output format. The ISP input format gets set at start, * just in case we have swapped bayer orders due to flips. */ - ret = data->unicam_[Unicam::Image].dev()->setFormat(&sensorFormat); + ret = data->unicam_[Unicam::Image].dev()->setFormat(&unicamFormat); if (ret) return ret; - /* - * The control ranges associated with the sensor may need updating - * after a format change. - * \todo Use the CameraSensor::setFormat API instead. - */ - data->sensor_->updateControlInfo(); - LOG(RPI, Info) << "Sensor: " << camera->id() - << " - Selected mode: " << sensorFormat.toString(); + << " - Selected sensor mode: " << sensorFormat.toString() + << " - Selected unicam mode: " << unicamFormat.toString(); /* * This format may be reset on start() if the bayer order has changed * because of flips in the sensor. */ - ret = data->isp_[Isp::Input].dev()->setFormat(&sensorFormat); + ret = data->isp_[Isp::Input].dev()->setFormat(&unicamFormat); if (ret) return ret; @@ -746,8 +756,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) data->ispMinCropSize_ = testCrop.size(); /* Adjust aspect ratio by providing crops on the input image. */ - Size size = sensorFormat.size.boundedToAspectRatio(maxSize); - Rectangle crop = size.centeredTo(Rectangle(sensorFormat.size).center()); + Size size = unicamFormat.size.boundedToAspectRatio(maxSize); + Rectangle crop = size.centeredTo(Rectangle(unicamFormat.size).center()); data->ispCrop_ = crop; data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); @@ -761,10 +771,11 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) * supports it. */ if (data->sensorMetadata_) { - format = {}; - format.fourcc = V4L2PixelFormat(V4L2_META_FMT_SENSOR_DATA); + V4L2SubdeviceFormat embeddedFormat; - LOG(RPI, Debug) << "Setting embedded data format."; + data->sensor_->device()->getFormat(1, &embeddedFormat); + format.fourcc = V4L2PixelFormat(V4L2_META_FMT_SENSOR_DATA); + format.planes[0].size = embeddedFormat.size.width * embeddedFormat.size.height; ret = data->unicam_[Unicam::Embedded].dev()->setFormat(&format); if (ret) { LOG(RPI, Error) << "Failed to set format on Unicam embedded: " @@ -847,9 +858,14 @@ int PipelineHandlerRPi::start(Camera *camera, const ControlList *controls) * IPA configure may have changed the sensor flips - hence the bayer * order. Get the sensor format and set the ISP input now. */ - V4L2DeviceFormat sensorFormat; - data->unicam_[Unicam::Image].dev()->getFormat(&sensorFormat); - ret = data->isp_[Isp::Input].dev()->setFormat(&sensorFormat); + V4L2SubdeviceFormat sensorFormat; + data->sensor_->device()->getFormat(0, &sensorFormat); + + V4L2DeviceFormat ispFormat; + ispFormat.fourcc = BayerFormat::fromMbusCode(sensorFormat.mbus_code).toV4L2PixelFormat(); + ispFormat.size = sensorFormat.size; + + ret = data->isp_[Isp::Input].dev()->setFormat(&ispFormat); if (ret) { stop(camera); return ret; @@ -1004,6 +1020,8 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) if (data->sensor_->init()) return false; + data->sensorFormats_ = populateSensorFormats(data->sensor_); + ipa::RPi::SensorConfig sensorConfig; if (data->loadIPA(&sensorConfig)) { LOG(RPI, Error) << "Failed to load a suitable IPA library"; @@ -1030,6 +1048,11 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) return false; } + if (!(data->unicam_[Unicam::Image].dev()->caps().device_caps() & V4L2_CAP_IO_MC)) { + LOG(RPI, Error) << "Unicam driver did not advertise V4L2_CAP_IO_MC, please update your kernel!"; + return false; + } + /* * Setup our delayed control writer with the sensor default * gain and exposure delays. Mark VBLANK for priority write. @@ -1039,7 +1062,7 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) { V4L2_CID_EXPOSURE, { sensorConfig.exposureDelay, false } }, { V4L2_CID_VBLANK, { sensorConfig.vblankDelay, true } } }; - data->delayedCtrls_ = std::make_unique(data->unicam_[Unicam::Image].dev(), params); + data->delayedCtrls_ = std::make_unique(data->sensor_->device(), params); data->sensorMetadata_ = sensorConfig.sensorMetadata; /* Register the controls that the Raspberry Pi IPA can handle. */ @@ -1066,15 +1089,14 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) * As part of answering the final question, we reset the camera to * no transform at all. */ - - V4L2VideoDevice *dev = data->unicam_[Unicam::Image].dev(); - const struct v4l2_query_ext_ctrl *hflipCtrl = dev->controlInfo(V4L2_CID_HFLIP); + const V4L2Subdevice *sensor = data->sensor_->device(); + const struct v4l2_query_ext_ctrl *hflipCtrl = sensor->controlInfo(V4L2_CID_HFLIP); if (hflipCtrl) { /* We assume it will support vflips too... */ data->supportsFlips_ = true; data->flipsAlterBayerOrder_ = hflipCtrl->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT; - ControlList ctrls(dev->controls()); + ControlList ctrls(data->sensor_->controls()); ctrls.set(V4L2_CID_HFLIP, 0); ctrls.set(V4L2_CID_VFLIP, 0); data->setSensorControls(ctrls); @@ -1082,9 +1104,8 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) /* Look for a valid Bayer format. */ BayerFormat bayerFormat; - for (const auto &iter : dev->formats()) { - V4L2PixelFormat v4l2Format = iter.first; - bayerFormat = BayerFormat::fromV4L2PixelFormat(v4l2Format); + for (const auto &iter : data->sensorFormats_) { + bayerFormat = BayerFormat::fromMbusCode(iter.first); if (bayerFormat.isValid()) break; } @@ -1271,7 +1292,7 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) } } - entityControls.emplace(0, unicam_[Unicam::Image].dev()->controls()); + entityControls.emplace(0, sensor_->controls()); entityControls.emplace(1, isp_[Isp::Input].dev()->controls()); /* Always send the user transform to the IPA. */ @@ -1406,10 +1427,10 @@ void RPiCameraData::setSensorControls(ControlList &controls) ControlList vblank_ctrl; vblank_ctrl.set(V4L2_CID_VBLANK, controls.get(V4L2_CID_VBLANK)); - unicam_[Unicam::Image].dev()->setControls(&vblank_ctrl); + sensor_->setControls(&vblank_ctrl); } - unicam_[Unicam::Image].dev()->setControls(&controls); + sensor_->setControls(&controls); } void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) From patchwork Fri Oct 22 11:55:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14255 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EF323BF415 for ; Fri, 22 Oct 2021 11:55:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AE88068F68; Fri, 22 Oct 2021 13:55:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="sK2flQGo"; dkim-atps=neutral Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ACFBE68F5E for ; Fri, 22 Oct 2021 13:55:44 +0200 (CEST) Received: by mail-wr1-x42e.google.com with SMTP id e12so3577351wra.4 for ; Fri, 22 Oct 2021 04:55:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZtQR4caqAxW5pY0otRC3Bfy56XbPIGt7rkbYnd54QuY=; b=sK2flQGokXoH/h69QXg7khQWxelLq+evFy/zNGXLDZk0jXtxanq630O9d/S3Dp3gNx JOg6AaWBNmtGP/DFroWsokL5lTyRUsgz5UGeM/Wtn5gNtV/Iu/t2w3OCn2Tsp6t4U6Ei uV0+CVSEROl6lyhzJRDYji8ZUsYNwhR5kif1E6iHn/kACcLqMu3JhTOOd4GwVz1BXUpq Iyp9Cwobkr6AIbi2CZIds1UKNPT2x+/eOLbs4j7/EUF6KbfAAEXMx/bZTsqWyHlqNIqu aASPgnRKRIW4Z+HFI5scmg7o6sdj/+jM/AkT891++5oxp4/wgTZ7Q8i8OlopoxPNCptD OIWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZtQR4caqAxW5pY0otRC3Bfy56XbPIGt7rkbYnd54QuY=; b=aaS9Vrd3oLkCOIADFy0XtcVIiWnIuQmPvWx/YBCQpHo4VXDryFNfCUARCBhMVRvIW/ xl2M8Nda9YT/P1Un5g3j2T3FstQr2IAb3irXObF3h30X4g+Dk39IZwg78VDIxItXvG1b 5urY4GFqnIwV0Y5DgB8H95vt4/AgArsBCXwBQCl19u6SZZwq77x1kV4UIDhPg8praBNh l8lYZf4iAok584xxi832CNTr0e6covaECeoi8miCuU3KQ9l1Ycz8ewPXDN9Yq10TUcXP LRy4gWt0iVyF10PgP5zfFOD6gslrcFoG66XwQ8CQGkktVwc6OpBMymGI0VTr5Hl7u9bZ qPaQ== X-Gm-Message-State: AOAM5322WJcGl1IpmXKer5LicIgJNTYNj3jayB/+0HWl6LTaBQvXiGCD egMJRmdpr5rLpFJueWqdqvQbwTO/lm0DFSeC X-Google-Smtp-Source: ABdhPJxcPJ2GK5LGE3g41fdjDWz3gZ7ecMnIRUqqh2rmsGTvy35gUTQUVMZSkp5MQ4vp12b9uD9kvw== X-Received: by 2002:adf:d1eb:: with SMTP id g11mr14926382wrd.16.1634903744017; Fri, 22 Oct 2021 04:55:44 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id l5sm7853364wru.24.2021.10.22.04.55.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 04:55:43 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 12:55:34 +0100 Message-Id: <20211022115537.2964533-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022115537.2964533-1-naush@raspberrypi.com> References: <20211022115537.2964533-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/6] pipeline: raspberrypi: Return a PixelFormat from findBestMode() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" It is more convenient to return a PixelFormat from findBestMode(), as the conversions from PixelFormat to V4L2SubdeviceFormat and V4L2DeviceFormat are simpler. Add some internal helpers to perform these conversions. Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 73 +++++++++++++------ 1 file changed, 52 insertions(+), 21 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 730f1575095c..0f13127a7748 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -50,6 +50,7 @@ namespace { /* Map of mbus codes to supported sizes reported by the sensor. */ using SensorFormats = std::map>; +using SensorMode = std::pair; SensorFormats populateSensorFormats(std::unique_ptr &sensor) { @@ -61,6 +62,34 @@ SensorFormats populateSensorFormats(std::unique_ptr &sensor) return formats; } +inline V4L2DeviceFormat toV4L2DeviceFormat(SensorMode &mode) +{ + V4L2DeviceFormat deviceFormat; + + deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(mode.first); + deviceFormat.size = mode.second; + return deviceFormat; +} + +inline V4L2DeviceFormat toV4L2DeviceFormat(V4L2SubdeviceFormat &format) +{ + V4L2DeviceFormat deviceFormat; + + deviceFormat.fourcc = BayerFormat::fromMbusCode(format.mbus_code).toV4L2PixelFormat(); + deviceFormat.size = format.size; + return deviceFormat; +} + +inline V4L2SubdeviceFormat toV4L2SubdeviceFormat(SensorMode &mode) +{ + V4L2SubdeviceFormat subdeviceFormat; + V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(mode.first); + + subdeviceFormat.mbus_code = BayerFormat::fromV4L2PixelFormat(fourcc).toMbusCode(); + subdeviceFormat.size = mode.second; + return subdeviceFormat; +} + bool isRaw(PixelFormat &pixFmt) { /* @@ -87,10 +116,10 @@ double scoreFormat(double desired, double actual) return score; } -V4L2DeviceFormat findBestMode(const SensorFormats &formatsMap, const Size &req) +SensorMode findBestMode(const SensorFormats &formatsMap, const Size &req) { double bestScore = std::numeric_limits::max(), score; - V4L2DeviceFormat bestMode; + SensorMode bestMode; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -101,8 +130,8 @@ V4L2DeviceFormat findBestMode(const SensorFormats &formatsMap, const Size &req) /* Calculate the closest/best mode from the user requested size. */ for (const auto &iter : formatsMap) { const unsigned int mbus_code = iter.first; - const V4L2PixelFormat v4l2Format = BayerFormat::fromMbusCode(mbus_code).toV4L2PixelFormat(); - const PixelFormatInfo &info = PixelFormatInfo::info(v4l2Format); + const PixelFormat format = BayerFormat::fromMbusCode(mbus_code).toPixelFormat(); + const PixelFormatInfo &info = PixelFormatInfo::info(format); for (const Size &sz : iter.second) { double reqAr = static_cast(req.width) / req.height; @@ -126,12 +155,12 @@ V4L2DeviceFormat findBestMode(const SensorFormats &formatsMap, const Size &req) if (score <= bestScore) { bestScore = score; - bestMode.fourcc = v4l2Format; - bestMode.size = sz; + bestMode.first = format; + bestMode.second = sz; } LOG(RPI, Info) << "Mode: " << sz.width << "x" << sz.height - << " fmt " << v4l2Format.toString() + << " fmt " << format.toString() << " Score: " << score << " (best " << bestScore << ")"; } @@ -364,8 +393,9 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() * Calculate the best sensor mode we can use based on * the user request. */ - V4L2DeviceFormat sensorFormat = findBestMode(data_->sensorFormats_, cfg.size); - int ret = data_->unicam_[Unicam::Image].dev()->tryFormat(&sensorFormat); + SensorMode sensorMode = findBestMode(data_->sensorFormats_, cfg.size); + V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorMode); + int ret = data_->unicam_[Unicam::Image].dev()->tryFormat(&unicamFormat); if (ret) return Invalid; @@ -377,7 +407,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() * fetch the "native" (i.e. untransformed) Bayer order, * because the sensor may currently be flipped! */ - V4L2PixelFormat fourcc = sensorFormat.fourcc; + V4L2PixelFormat fourcc = unicamFormat.fourcc; if (data_->flipsAlterBayerOrder_) { BayerFormat bayer = BayerFormat::fromV4L2PixelFormat(fourcc); bayer.order = data_->nativeBayerOrder_; @@ -386,15 +416,15 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() } PixelFormat sensorPixFormat = fourcc.toPixelFormat(); - if (cfg.size != sensorFormat.size || + if (cfg.size != unicamFormat.size || cfg.pixelFormat != sensorPixFormat) { - cfg.size = sensorFormat.size; + cfg.size = unicamFormat.size; cfg.pixelFormat = sensorPixFormat; status = Adjusted; } - cfg.stride = sensorFormat.planes[0].bpl; - cfg.frameSize = sensorFormat.planes[0].size; + cfg.stride = unicamFormat.planes[0].bpl; + cfg.frameSize = unicamFormat.planes[0].size; rawCount++; } else { @@ -483,7 +513,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, { RPiCameraData *data = cameraData(camera); CameraConfiguration *config = new RPiCameraConfiguration(data); - V4L2DeviceFormat sensorFormat; + V4L2DeviceFormat unicamFormat; + SensorMode sensorMode; unsigned int bufferCount; PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; @@ -498,8 +529,9 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, switch (role) { case StreamRole::Raw: size = data->sensor_->resolution(); - sensorFormat = findBestMode(data->sensorFormats_, size); - pixelFormat = sensorFormat.fourcc.toPixelFormat(); + sensorMode = findBestMode(data->sensorFormats_, size); + unicamFormat = toV4L2DeviceFormat(sensorMode); + pixelFormat = sensorMode.first; ASSERT(pixelFormat.isValid()); bufferCount = 2; rawCount++; @@ -609,10 +641,9 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) } /* First calculate the best sensor mode we can use based on the user request. */ - V4L2DeviceFormat unicamFormat = findBestMode(data->sensorFormats_, rawStream ? sensorSize : maxSize); - - unsigned int mbus_code = BayerFormat::fromV4L2PixelFormat(unicamFormat.fourcc).toMbusCode(); - V4L2SubdeviceFormat sensorFormat { .mbus_code = mbus_code, .size = unicamFormat.size }; + SensorMode sensorMode = findBestMode(data->sensorFormats_, rawStream ? sensorSize : maxSize); + V4L2SubdeviceFormat sensorFormat = toV4L2SubdeviceFormat(sensorMode); + V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorMode); ret = data->sensor_->setFormat(&sensorFormat); if (ret) From patchwork Fri Oct 22 11:55:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14256 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 821C7BF415 for ; Fri, 22 Oct 2021 11:55:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 186D168F57; Fri, 22 Oct 2021 13:55:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="OPHQN6od"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 66BAE6012A for ; Fri, 22 Oct 2021 13:55:45 +0200 (CEST) Received: by mail-wr1-x436.google.com with SMTP id d13so4337360wrf.11 for ; Fri, 22 Oct 2021 04:55:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PDQBlbl4qInGaGsRRpYvBAqua2Eo/HUT+WJSofI6L8c=; b=OPHQN6oddmyjbEQMIoUE/Lzf6TG0icVOlzxnMOYQoqH16z0pO6FSnxGcAdmWL4N5dY wiES6vLY2jN21/CjrtNY0qSPEnktoRzv7m5xM9IxKrEdomUI0pBjxt6cLY87pzvlo99z HBCwIggkH3MzbrnXrerN4UvBBgd7Q8V2ZFLXkVOiRoEiwMduswXpK7CilW4VlGXCiPDV 3d/uxiSZfUOeO56n+Kdp6FB0nyA64HapGp3gOYpvXoj9O+ipaRznfQdDW98VkeDwYL5v 5Nn3KH/GiYfsh9dIcQ1Bzb0rv84PIxlvoZiIrfVzjRpPLvizSsLXabPmRYZdL9GUsh// DtzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PDQBlbl4qInGaGsRRpYvBAqua2Eo/HUT+WJSofI6L8c=; b=h/V9i8+HxUemPqe+TDQWfKubTDDYHcTBlWG9x3rozCZFamqnKZ88WzGofw6SCkOTbv wfBxGo5QefuVR7xm4CkV+kq0MAPRtBCXL8U2l32y4XcvCSCQwW36negK0Pu92YDDTw+x SKa4z0t23zVKE6Hs3bpcoBwsXKeg8RYcsgOnkYutyPXrxTtA2JfPa+LQqUG2YW949HWj hLBOFa8JBobTV22qqZQJ+dsQn4ucZPXIOm/eidbsqCSlSp6ugC0jdL9/0jCxHEc5Nf48 /a7P4fp86u+9/AJYY5yy5LfT83D62Q4JDu9BmOKMeoYI7UVibm6F+Rwaq0teAy67hfqB BAsA== X-Gm-Message-State: AOAM533BszkThF8mjlngWB4jtQCJ07zauoCJ9fEiQPmgO03skHo0Ni15 rFEFu8WoNKIl25bRcn24E2uur5Ut1ZdYtNIJ X-Google-Smtp-Source: ABdhPJx0Zq3y8ryWrwZy+Tz3UHK33wGcMvEaV3dN7gZySF0KU1RJg4IBXsDFnuWXcRqm8KNM4Jj89A== X-Received: by 2002:a5d:59aa:: with SMTP id p10mr15040842wrr.45.1634903744912; Fri, 22 Oct 2021 04:55:44 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id l5sm7853364wru.24.2021.10.22.04.55.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 04:55:44 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 12:55:35 +0100 Message-Id: <20211022115537.2964533-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022115537.2964533-1-naush@raspberrypi.com> References: <20211022115537.2964533-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/6] pipeline: raspberrypi: Remove "unpacked" format penalty in mode selection X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the recent change to switch to programming the sensor device directly, the notion of packed vs unpacked modes are not relevent, since that is a Unicam format construct. Remove any scoring based on packed/unpacked modes. Signed-off-by: Naushir Patuck --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 4 ---- 1 file changed, 4 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 0f13127a7748..fc190e39732a 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -125,7 +125,6 @@ SensorMode findBestMode(const SensorFormats &formatsMap, const Size &req) #define PENALTY_8BIT 2000.0 #define PENALTY_10BIT 1000.0 #define PENALTY_12BIT 0.0 -#define PENALTY_UNPACKED 500.0 /* Calculate the closest/best mode from the user requested size. */ for (const auto &iter : formatsMap) { @@ -143,9 +142,6 @@ SensorMode findBestMode(const SensorFormats &formatsMap, const Size &req) score += PENALTY_AR * scoreFormat(reqAr, modeAr); /* Add any penalties... this is not an exact science! */ - if (!info.packed) - score += PENALTY_UNPACKED; - if (info.bitsPerPixel == 12) score += PENALTY_12BIT; else if (info.bitsPerPixel == 10) From patchwork Fri Oct 22 11:55:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14257 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EBBC3C324E for ; Fri, 22 Oct 2021 11:55:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 74ECC68F62; Fri, 22 Oct 2021 13:55:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="AqcSBPs3"; dkim-atps=neutral Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1C95E68F65 for ; Fri, 22 Oct 2021 13:55:46 +0200 (CEST) Received: by mail-wr1-x42f.google.com with SMTP id s19so125497wra.2 for ; Fri, 22 Oct 2021 04:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iRRAspqPBslKSGeN5NDTBveEOZheEHwOQC+2ZoH4+7E=; b=AqcSBPs36tN5sSMhDlH/tJw04bEQ0GxlvZfo30oAP8xLlRWF5gOkJifgieQqWrwG8T iv8vwNGOn+4sMMJE/I/cZznAXK+NJHucvfMFh9pZeJp+GZNOUZ9zpfGb9mZAiN4QkrA4 fDrBSn/vlDqMVwzq0w3vXe8kkb6wo1Z9KQi9EP31FJSV37jMiuYmrQVC9bTwVgnINzMx a20/04zrafudsxP4vbQ25nFwdxIsqciefLd7V6aSb4AW/XmDqjCa5um4wdHDYH0yy5j1 MMh6WMTKsWZerXkBY3SOU07DdN/Z9MAOs11vkG4rHGWfOWPqPXfFHinPRagsFBoRADYJ Psxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iRRAspqPBslKSGeN5NDTBveEOZheEHwOQC+2ZoH4+7E=; b=J4g7IihlL5gU9ktcoqX1cb+KILAL1scKg94P5wh0sPeCv+Gagbjtu6bVFviXt04QN3 QtQR/qtPdkLPnax8Gc33447XfO5To3HnxbAvwi4k+PWS6Dneiwvvqb2717GgdQs0+VtG 6K2rBhABMTXiX51jtC2jrTRRxPw+nmrD/XpwjGX3ENgE8TetRvtkYQp58K5OAolkQCI1 fSsKZDbr8Hd215Evl3DNa6tegE5+cso8XBfIEtrwFE9H6Yi+JYGjokuqKjgO0VQ+XQvX T2yjw5FYtPDuFP741GKgUaXTVXYEmiWITDQgbuGp1Wem+9lrWAdoL+kr8xE4m3CcnG0C DXFQ== X-Gm-Message-State: AOAM530HXPfEltQYhyYfICcikCZfcP2/aJKByyHX7PFQPl2CdQJJFhvg Mr++bG/z1m9SHp2BYLzVSGoNigBv26iREgux X-Google-Smtp-Source: ABdhPJyz3qLyQhi27q82+hd1kLIefBIivusjVAnyBtbjBgDrs49SvdRCuRPKYyarDDT6/Y3Njb3Ehg== X-Received: by 2002:a5d:4a4d:: with SMTP id v13mr12399543wrs.353.1634903745615; Fri, 22 Oct 2021 04:55:45 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id l5sm7853364wru.24.2021.10.22.04.55.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 04:55:45 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 12:55:36 +0100 Message-Id: <20211022115537.2964533-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022115537.2964533-1-naush@raspberrypi.com> References: <20211022115537.2964533-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/6] pipeline: raspberrypi: Account for a missing Unicam embedded data node X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The unicam driver no longer regesters an embedded data node if the sensor does not provide this stream. Account for this in the pipeline handler match routine by not assuming it is always present. Add a warning if Unicam and the CamHelper do not agree on the presense of sensor embedded data, and disable its useage in these cases. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index fc190e39732a..5aaf24436f27 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -994,7 +994,6 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) DeviceMatch unicam("unicam"); DeviceMatch isp("bcm2835-isp"); - unicam.add("unicam-embedded"); unicam.add("unicam-image"); isp.add("bcm2835-isp0-output0"); /* Input */ @@ -1015,9 +1014,16 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) return false; /* Locate and open the unicam video streams. */ - data->unicam_[Unicam::Embedded] = RPi::Stream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); data->unicam_[Unicam::Image] = RPi::Stream("Unicam Image", unicam_->getEntityByName("unicam-image")); + /* An embedded data node will not be present if the sensor does not support it. */ + MediaEntity *embeddedEntity = unicam_->getEntityByName("unicam-embedded"); + if (embeddedEntity) { + data->unicam_[Unicam::Embedded] = RPi::Stream("Unicam Embedded", embeddedEntity); + data->unicam_[Unicam::Embedded].dev()->bufferReady.connect(data.get(), + &RPiCameraData::unicamBufferDequeue); + } + /* Tag the ISP input stream as an import stream. */ data->isp_[Isp::Input] = RPi::Stream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); data->isp_[Isp::Output0] = RPi::Stream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); @@ -1027,7 +1033,6 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) /* Wire up all the buffer connections. */ data->unicam_[Unicam::Image].dev()->frameStart.connect(data.get(), &RPiCameraData::frameStarted); data->unicam_[Unicam::Image].dev()->bufferReady.connect(data.get(), &RPiCameraData::unicamBufferDequeue); - data->unicam_[Unicam::Embedded].dev()->bufferReady.connect(data.get(), &RPiCameraData::unicamBufferDequeue); data->isp_[Isp::Input].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispInputDequeue); data->isp_[Isp::Output0].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispOutputDequeue); data->isp_[Isp::Output1].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispOutputDequeue); @@ -1055,6 +1060,11 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) return false; } + if (sensorConfig.sensorMetadata ^ !!embeddedEntity) { + LOG(RPI, Warning) << "Mismatch between Unicam and CamHelper for embedded data usage!"; + sensorConfig.sensorMetadata = false; + } + /* * Open all Unicam and ISP streams. The exception is the embedded data * stream, which only gets opened below if the IPA reports that the sensor From patchwork Fri Oct 22 11:55:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14258 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 69BF3BF415 for ; Fri, 22 Oct 2021 11:55:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EB84868F5F; Fri, 22 Oct 2021 13:55:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Q/hoYl4c"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D952568F5A for ; Fri, 22 Oct 2021 13:55:46 +0200 (CEST) Received: by mail-wr1-x436.google.com with SMTP id e12so3577518wra.4 for ; Fri, 22 Oct 2021 04:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=86xqRCs6+mA3Dlk2ANmJcOkjcIa6VXvLIhE5udLMFg4=; b=Q/hoYl4ck1jHOmE8ESc3qlpcKF3Z/zMwgEni3PXHMmVXoVNyJ/qAeAIZGVzAlac3vu xDoaX42jjRl+XrSYkBpOXrA06Djj5Mt3MsTrroCEtlQ6yOBlZgT2oIugVyfBWKtUzxX4 9lfKPNiYn/Y//Jk8nsmoAWBiE6/wDxVRMOuxrmQXksw6WV9kasu2RG4VhjKjvoLqqHUn UuLErgwOUU+iBoBrz6/JVqIhPPs37amucBvbGgqHvR56jZiVz069JvfQqBcqYkaEJ2X9 CP5uVBVgMBfjjUspA19rarHEBth4VzvwP1oLpWsuLg4iZSiC154DdrlAwLRHv0OCUnT+ TwAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=86xqRCs6+mA3Dlk2ANmJcOkjcIa6VXvLIhE5udLMFg4=; b=OQC7ua6sbbuwI8wVZitcNjzgZImFywinbm/ItlZJO0vgNabjvTcg87eEG4oae6mshg 0RQUJ3RxMDKn7oc8OUrWUCDtrgS7NjJDrWMabkYKcYfwrZ81Ha+C9wWxKEay5QNa4amh VJvbQPnpjIg9DnGCkGthy3xhMK4lcr+kO0HXIfHXzzXuLjKYD7A7NmTFOA4DmJNR8XC/ 7Xvgx8rSw00W13rNUKCTueGys66yCKT0u2Ff5icvgsS8ann1tB7u8yvU6ZBJa5Jkkuo2 8SFuwHboe2rvWwJDR02DE/sVXWt1ZakSiBJxhYtiN+ueXI64ZZvh2JmBqqnHjMHwTVf1 pgoQ== X-Gm-Message-State: AOAM530MKexfCrRzlU9fiqrys3H4dmAUf+BHC38NUT7Hd//E0rr0ZFqL 0IyDbbXoo1HIb+sL42/WF1oriHsHQ97c9/uv X-Google-Smtp-Source: ABdhPJzOHGgJxtRB2Qymi3nZyAzpFA4HBDv1EbsQG9i3ldSd4oq2Ce9m5mjIYbyo98ClJQoeP8HZ9g== X-Received: by 2002:adf:fa8b:: with SMTP id h11mr15270659wrr.74.1634903746367; Fri, 22 Oct 2021 04:55:46 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id l5sm7853364wru.24.2021.10.22.04.55.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 04:55:45 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 12:55:37 +0100 Message-Id: <20211022115537.2964533-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022115537.2964533-1-naush@raspberrypi.com> References: <20211022115537.2964533-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 6/6] pipeline: raspberrypi: Apply sensor flips at the start of configure() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Sensor flips might change the Bayer order of the requested format. The existing code would set a sensor format along with the appropriate Unicam and ISP input formats, but reset the latter two on start() once the flips had been requested. We can now set the sensor flips just after we set the sensor mode in configure(), thereby not needing the second pair of format sets in start(). Signed-off-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 61 +++++++------------ 1 file changed, 22 insertions(+), 39 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5aaf24436f27..8fb6197ec283 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -639,16 +639,35 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) /* First calculate the best sensor mode we can use based on the user request. */ SensorMode sensorMode = findBestMode(data->sensorFormats_, rawStream ? sensorSize : maxSize); V4L2SubdeviceFormat sensorFormat = toV4L2SubdeviceFormat(sensorMode); - V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorMode); ret = data->sensor_->setFormat(&sensorFormat); if (ret) return ret; /* - * Unicam image output format. The ISP input format gets set at start, - * just in case we have swapped bayer orders due to flips. + * Configure the H/V flip controls based on the combination of + * the sensor and user transform. */ + if (data->supportsFlips_) { + const RPiCameraConfiguration *rpiConfig = + static_cast(config); + ControlList controls; + + controls.set(V4L2_CID_HFLIP, + static_cast(!!(rpiConfig->combinedTransform_ & Transform::HFlip))); + controls.set(V4L2_CID_VFLIP, + static_cast(!!(rpiConfig->combinedTransform_ & Transform::VFlip))); + data->setSensorControls(controls); + + /* + * IPA configure may have changed the sensor flips - hence the bayer + * order. So update the sensor format now. + */ + data->sensor_->device()->getFormat(0, &sensorFormat); + sensorMode.first = BayerFormat::fromMbusCode(sensorFormat.mbus_code).toPixelFormat(); + } + + V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorMode); ret = data->unicam_[Unicam::Image].dev()->setFormat(&unicamFormat); if (ret) return ret; @@ -657,10 +676,6 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) << " - Selected sensor mode: " << sensorFormat.toString() << " - Selected unicam mode: " << unicamFormat.toString(); - /* - * This format may be reset on start() if the bayer order has changed - * because of flips in the sensor. - */ ret = data->isp_[Isp::Input].dev()->setFormat(&unicamFormat); if (ret) return ret; @@ -881,23 +896,6 @@ int PipelineHandlerRPi::start(Camera *camera, const ControlList *controls) return ret; } - /* - * IPA configure may have changed the sensor flips - hence the bayer - * order. Get the sensor format and set the ISP input now. - */ - V4L2SubdeviceFormat sensorFormat; - data->sensor_->device()->getFormat(0, &sensorFormat); - - V4L2DeviceFormat ispFormat; - ispFormat.fourcc = BayerFormat::fromMbusCode(sensorFormat.mbus_code).toV4L2PixelFormat(); - ispFormat.size = sensorFormat.size; - - ret = data->isp_[Isp::Input].dev()->setFormat(&ispFormat); - if (ret) { - stop(camera); - return ret; - } - /* Enable SOF event generation. */ data->unicam_[Unicam::Image].dev()->setFrameStartEnabled(true); @@ -1311,10 +1309,6 @@ int RPiCameraData::loadIPA(ipa::RPi::SensorConfig *sensorConfig) int RPiCameraData::configureIPA(const CameraConfiguration *config) { - /* We know config must be an RPiCameraConfiguration. */ - const RPiCameraConfiguration *rpiConfig = - static_cast(config); - std::map streamConfig; std::map entityControls; ipa::RPi::IPAConfig ipaConfig; @@ -1365,17 +1359,6 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) return -EPIPE; } - /* - * Configure the H/V flip controls based on the combination of - * the sensor and user transform. - */ - if (supportsFlips_) { - controls.set(V4L2_CID_HFLIP, - static_cast(!!(rpiConfig->combinedTransform_ & Transform::HFlip))); - controls.set(V4L2_CID_VFLIP, - static_cast(!!(rpiConfig->combinedTransform_ & Transform::VFlip))); - } - if (!controls.empty()) setSensorControls(controls);