From patchwork Fri Oct 22 14:39:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14260 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E71BBBDB1C for ; Fri, 22 Oct 2021 14:40:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 89E7668F5F; Fri, 22 Oct 2021 16:40:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="jET7vTux"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D10346023B for ; Fri, 22 Oct 2021 16:40:13 +0200 (CEST) Received: by mail-wm1-x32f.google.com with SMTP id z11-20020a1c7e0b000000b0030db7b70b6bso3665019wmc.1 for ; Fri, 22 Oct 2021 07:40:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mi1BmveZO0MDVL5rAfXrj96e0ld5nDzmh+AbmduPjhM=; b=jET7vTuxFjUYaUUm/WFUJNSVo+SB7CcF4uLj/nZdhepNpmiKs8DurCm9Pdk/V/xGfH bgfwKCesbcx0b3JlXUdRoI1DG5T2RyBNIYYGZXDXvqjyaimiVmDa/tXQE1tB5vS4hFpz w+qCQPd0y1xNjUPcbEotBMIAt1ph694e5dPxELpkQPyFiUnvRaMhREQB5f/jnVbnma8C HK9gkvGByCVOPnNS9qq5RbTcmAYgn1cqElm4fir43FLxDj5nmdLUKdb/qIUvTEyoNNCn VKdgGD4C5kmS4+vml8EBHBAa56Ji8Lu2gCaoA9N4yi2/2goVqslZjLIVKJC13/A+a0cr RD3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mi1BmveZO0MDVL5rAfXrj96e0ld5nDzmh+AbmduPjhM=; b=CxqMXas//3DiU5wNTGSbWIIUHqetcedK3QdjyNsVp/s5snYbhTrjvtiHnZCK5XZxIv xjtIbp1Z13dL/CU3PM0Vn4XTBOzx56KzL6U/t3Yix4coX+3EceORI0rzdi57ofjlKO9c +wSKrKJjSp1OepGv/jk/5i5MTEO49ebphcKJSwkHExbH7j7QhLfyt4WKvi8+o6vjuCos F4wNzR3EFyo9kLU7XyACIRBQ+Uzk6e9ZRNiY0hWUiQdONF0VCO1T3+dLT0m5e99/ouo9 TQt71PE5sym5Eo8AnIYEssW0B/pMi1F0NuhLuWch6H+X2Cgvtf6jAfIooCkA8zCy2g/d 7PtA== X-Gm-Message-State: AOAM532HAKocxZsjx8ML4JTHPtO0SXbWfJRS0hfgvBZwf6nH3AUYFopd DNzwXkQ/fWg/PhZZ+La2QGH38OmvoY4oEUI7 X-Google-Smtp-Source: ABdhPJzj1e0J7sS9TmES0mGZ8M8uEpC9UAJcB6MFnydQuK4efryMW6v9j2r1ShnEXhCvXe10mdP5Qw== X-Received: by 2002:a05:600c:3b88:: with SMTP id n8mr119428wms.93.1634913613400; Fri, 22 Oct 2021 07:40:13 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id r205sm8577878wma.3.2021.10.22.07.40.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 07:40:12 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 15:39:02 +0100 Message-Id: <20211022143907.3089419-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022143907.3089419-1-naush@raspberrypi.com> References: <20211022143907.3089419-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/6] libcamera: bayer_format: Add a toPixelFormat() helpers to the BayerFormat class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a BayerFormat::toPixelFormat() member function to convert a BayerFormat to a PixelFormat type. This conversion uses the existing bayerToV4l2 conversion table. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- include/libcamera/internal/bayer_format.h | 3 +++ src/libcamera/bayer_format.cpp | 13 +++++++++++++ 2 files changed, 16 insertions(+) diff --git a/include/libcamera/internal/bayer_format.h b/include/libcamera/internal/bayer_format.h index 723382d4168d..247a60cf0e36 100644 --- a/include/libcamera/internal/bayer_format.h +++ b/include/libcamera/internal/bayer_format.h @@ -10,6 +10,8 @@ #include #include +#include + #include "libcamera/internal/v4l2_pixelformat.h" namespace libcamera { @@ -50,6 +52,7 @@ public: V4L2PixelFormat toV4L2PixelFormat() const; static BayerFormat fromV4L2PixelFormat(V4L2PixelFormat v4l2Format); + PixelFormat toPixelFormat() const; BayerFormat transform(Transform t) const; Order order; diff --git a/src/libcamera/bayer_format.cpp b/src/libcamera/bayer_format.cpp index 11355f144f66..c662ba36604c 100644 --- a/src/libcamera/bayer_format.cpp +++ b/src/libcamera/bayer_format.cpp @@ -269,6 +269,19 @@ BayerFormat BayerFormat::fromV4L2PixelFormat(V4L2PixelFormat v4l2Format) return BayerFormat(); } +/** + * \brief Convert a BayerFormat into the corresponding PixelFormat + * \return The PixelFormat corresponding to this BayerFormat + */ +PixelFormat BayerFormat::toPixelFormat() const +{ + const auto it = bayerToV4l2.find(*this); + if (it != bayerToV4l2.end()) + return PixelFormat(it->second); + + return PixelFormat(); +} + /** * \brief Apply a transform to this BayerFormat * \param[in] t The transform to apply From patchwork Fri Oct 22 14:39:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14261 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 41A9FBDB1C for ; Fri, 22 Oct 2021 14:40:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0461268F61; Fri, 22 Oct 2021 16:40:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GF8jST3s"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 33D7A6012A for ; Fri, 22 Oct 2021 16:40:15 +0200 (CEST) Received: by mail-wm1-x329.google.com with SMTP id j205so2443434wmj.3 for ; Fri, 22 Oct 2021 07:40:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wJpyur6FGxUGBGkl1qE65WWe1yICqyvTwggHyrkXtb8=; b=GF8jST3stXA8+VnkDBriDE5zE7JNjtPjDkzojOOyf99L7i4yTNHvgup8EP38vcXUNv soSNPQ2Ey7U6KF25Gt4RuyYmD5E/iTXXNIw5eD1tAMXcowtpYDzd3rXwBVMNHdRoa1qq 27uQBZbqQVnvw2pjgzHN/6qNSRSi+Ct/myak8sfCA3K1B7Q4pH5w/T3IOeTRbCFlS6zO MAclMvcQlI1ffAGy5DdHyZt2jdOo07G/86kV+u4Usn9B44Hk11NrZ1vy8GvppPV40jeC /FIM74Zcbo6ZM6yx2K7sFt4AGUqGZKp4VdvMoLIRKyZfLH4UXoIKQOXaH5oXyPFgWvt+ BUeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wJpyur6FGxUGBGkl1qE65WWe1yICqyvTwggHyrkXtb8=; b=K9oScpjsu6yDoPlAHi2E6ZsdBzL8OktNFkkFzDJnKw4FrYOzWXxho80ARv+GhqiV6C PYzX2O+QAGIxc3wgLai3/1ZkI24bkokFztY9JaxWkTc3rrYtYm3J/SG+aYH2YWd8rCJS tPaE4uL7Wib7r2AaATQ3OmSux5Dib0sIXIQFhtrESGlDvTiXp0DvOWtS79NlwSQRFmxe uR4RAyvxfLz3zP2bQ3sVZaUoVGZnTRF6PpSboN+zct4/3b61MMrV7fSTRGuoyuzz73Ok 5iYkiwBzCo2AOMH6wUOfNYOu3oTybzYRTRu9avNJAyhPBV4oPQVpv2mwl5v/qlRGAhAl mhGg== X-Gm-Message-State: AOAM53196zbBMGwCmFitcjlSBVqiXJ1/U2n5+/QKf5wFviItGGinBrRv W+UiwlY0EtyIM4FN/00KSJU4WaoT8428fBUE X-Google-Smtp-Source: ABdhPJzb+TpjEkb7PRbNgfB7Hn+2hsEK2DkWetEEm+shaQofaFxN4moIDRGodx+N9yuHMtJRRPPdPQ== X-Received: by 2002:a1c:3b44:: with SMTP id i65mr107502wma.161.1634913614133; Fri, 22 Oct 2021 07:40:14 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id r205sm8577878wma.3.2021.10.22.07.40.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 07:40:13 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 15:39:03 +0100 Message-Id: <20211022143907.3089419-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022143907.3089419-1-naush@raspberrypi.com> References: <20211022143907.3089419-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/6] pipeline: raspberrypi: Convert the pipeline handler to use media controller X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Switch the pipeline handler to use the new Unicam media controller based driver. With this change, we directly talk to the sensor device driver to set controls and set/get formats in the pipeline handler. This change requires the accompanying Raspberry Pi linux kernel change at https://github.com/raspberrypi/linux/pull/4645. If this kernel change is not present, the pipeline handler will fail to run with an error message informing the user to update the kernel build. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 146 +++++++++++------- 1 file changed, 90 insertions(+), 56 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 1634ca98f481..a31b0f81eba7 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -48,6 +48,29 @@ LOG_DEFINE_CATEGORY(RPI) namespace { +/* Map of mbus codes to supported sizes reported by the sensor. */ +using SensorFormats = std::map>; + +SensorFormats populateSensorFormats(std::unique_ptr &sensor) +{ + SensorFormats formats; + + for (auto const mbusCode : sensor->mbusCodes()) + formats.emplace(mbusCode, sensor->sizes(mbusCode)); + + return formats; +} + +inline V4L2DeviceFormat toV4L2DeviceFormat(V4L2SubdeviceFormat &mode) +{ + V4L2DeviceFormat deviceFormat; + BayerFormat bayer = BayerFormat::fromMbusCode(mode.mbus_code); + + deviceFormat.fourcc = bayer.toV4L2PixelFormat(); + deviceFormat.size = mode.size; + return deviceFormat; +} + bool isRaw(PixelFormat &pixFmt) { /* @@ -74,11 +97,10 @@ double scoreFormat(double desired, double actual) return score; } -V4L2DeviceFormat findBestMode(V4L2VideoDevice::Formats &formatsMap, - const Size &req) +V4L2SubdeviceFormat findBestMode(const SensorFormats &formatsMap, const Size &req) { double bestScore = std::numeric_limits::max(), score; - V4L2DeviceFormat bestMode; + V4L2SubdeviceFormat bestMode; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -88,18 +110,17 @@ V4L2DeviceFormat findBestMode(V4L2VideoDevice::Formats &formatsMap, /* Calculate the closest/best mode from the user requested size. */ for (const auto &iter : formatsMap) { - V4L2PixelFormat v4l2Format = iter.first; - const PixelFormatInfo &info = PixelFormatInfo::info(v4l2Format); + const unsigned int mbus_code = iter.first; + const PixelFormat format = BayerFormat::fromMbusCode(mbus_code).toPixelFormat(); + const PixelFormatInfo &info = PixelFormatInfo::info(format); - for (const SizeRange &sz : iter.second) { - double modeWidth = sz.contains(req) ? req.width : sz.max.width; - double modeHeight = sz.contains(req) ? req.height : sz.max.height; + for (const Size &sz : iter.second) { double reqAr = static_cast(req.width) / req.height; - double modeAr = modeWidth / modeHeight; + double modeAr = sz.width / sz.height; /* Score the dimensions for closeness. */ - score = scoreFormat(req.width, modeWidth); - score += scoreFormat(req.height, modeHeight); + score = scoreFormat(req.width, sz.width); + score += scoreFormat(req.height, sz.height); score += PENALTY_AR * scoreFormat(reqAr, modeAr); /* Add any penalties... this is not an exact science! */ @@ -115,12 +136,12 @@ V4L2DeviceFormat findBestMode(V4L2VideoDevice::Formats &formatsMap, if (score <= bestScore) { bestScore = score; - bestMode.fourcc = v4l2Format; - bestMode.size = Size(modeWidth, modeHeight); + bestMode.mbus_code = mbus_code; + bestMode.size = sz; } - LOG(RPI, Info) << "Mode: " << modeWidth << "x" << modeHeight - << " fmt " << v4l2Format.toString() + LOG(RPI, Info) << "Mode: " << sz.width << "x" << sz.height + << " fmt " << format.toString() << " Score: " << score << " (best " << bestScore << ")"; } @@ -170,6 +191,7 @@ public: std::unique_ptr ipa_; std::unique_ptr sensor_; + SensorFormats sensorFormats_; /* Array of Unicam and ISP device streams and associated buffers/streams. */ RPi::Device unicam_; RPi::Device isp_; @@ -352,9 +374,9 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() * Calculate the best sensor mode we can use based on * the user request. */ - V4L2VideoDevice::Formats fmts = data_->unicam_[Unicam::Image].dev()->formats(); - V4L2DeviceFormat sensorFormat = findBestMode(fmts, cfg.size); - int ret = data_->unicam_[Unicam::Image].dev()->tryFormat(&sensorFormat); + V4L2SubdeviceFormat sensorFormat = findBestMode(data_->sensorFormats_, cfg.size); + V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorFormat); + int ret = data_->unicam_[Unicam::Image].dev()->tryFormat(&unicamFormat); if (ret) return Invalid; @@ -366,7 +388,7 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() * fetch the "native" (i.e. untransformed) Bayer order, * because the sensor may currently be flipped! */ - V4L2PixelFormat fourcc = sensorFormat.fourcc; + V4L2PixelFormat fourcc = unicamFormat.fourcc; if (data_->flipsAlterBayerOrder_) { BayerFormat bayer = BayerFormat::fromV4L2PixelFormat(fourcc); bayer.order = data_->nativeBayerOrder_; @@ -375,15 +397,15 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() } PixelFormat sensorPixFormat = fourcc.toPixelFormat(); - if (cfg.size != sensorFormat.size || + if (cfg.size != unicamFormat.size || cfg.pixelFormat != sensorPixFormat) { - cfg.size = sensorFormat.size; + cfg.size = unicamFormat.size; cfg.pixelFormat = sensorPixFormat; status = Adjusted; } - cfg.stride = sensorFormat.planes[0].bpl; - cfg.frameSize = sensorFormat.planes[0].size; + cfg.stride = unicamFormat.planes[0].bpl; + cfg.frameSize = unicamFormat.planes[0].size; rawCount++; } else { @@ -472,7 +494,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, { RPiCameraData *data = cameraData(camera); CameraConfiguration *config = new RPiCameraConfiguration(data); - V4L2DeviceFormat sensorFormat; + V4L2SubdeviceFormat sensorFormat; + V4L2DeviceFormat unicamFormat; unsigned int bufferCount; PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; @@ -487,9 +510,9 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, switch (role) { case StreamRole::Raw: size = data->sensor_->resolution(); - fmts = data->unicam_[Unicam::Image].dev()->formats(); - sensorFormat = findBestMode(fmts, size); - pixelFormat = sensorFormat.fourcc.toPixelFormat(); + sensorFormat = findBestMode(data->sensorFormats_, size); + unicamFormat = toV4L2DeviceFormat(sensorFormat); + pixelFormat = BayerFormat::fromMbusCode(sensorFormat.mbus_code).toPixelFormat(); ASSERT(pixelFormat.isValid()); bufferCount = 2; rawCount++; @@ -599,32 +622,30 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) } /* First calculate the best sensor mode we can use based on the user request. */ - V4L2VideoDevice::Formats fmts = data->unicam_[Unicam::Image].dev()->formats(); - V4L2DeviceFormat sensorFormat = findBestMode(fmts, rawStream ? sensorSize : maxSize); + V4L2SubdeviceFormat sensorFormat = findBestMode(data->sensorFormats_, rawStream ? sensorSize : maxSize); + V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorFormat); + + ret = data->sensor_->setFormat(&sensorFormat); + if (ret) + return ret; /* * Unicam image output format. The ISP input format gets set at start, * just in case we have swapped bayer orders due to flips. */ - ret = data->unicam_[Unicam::Image].dev()->setFormat(&sensorFormat); + ret = data->unicam_[Unicam::Image].dev()->setFormat(&unicamFormat); if (ret) return ret; - /* - * The control ranges associated with the sensor may need updating - * after a format change. - * \todo Use the CameraSensor::setFormat API instead. - */ - data->sensor_->updateControlInfo(); - LOG(RPI, Info) << "Sensor: " << camera->id() - << " - Selected mode: " << sensorFormat.toString(); + << " - Selected sensor mode: " << sensorFormat.toString() + << " - Selected unicam mode: " << unicamFormat.toString(); /* * This format may be reset on start() if the bayer order has changed * because of flips in the sensor. */ - ret = data->isp_[Isp::Input].dev()->setFormat(&sensorFormat); + ret = data->isp_[Isp::Input].dev()->setFormat(&unicamFormat); if (ret) return ret; @@ -746,8 +767,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) data->ispMinCropSize_ = testCrop.size(); /* Adjust aspect ratio by providing crops on the input image. */ - Size size = sensorFormat.size.boundedToAspectRatio(maxSize); - Rectangle crop = size.centeredTo(Rectangle(sensorFormat.size).center()); + Size size = unicamFormat.size.boundedToAspectRatio(maxSize); + Rectangle crop = size.centeredTo(Rectangle(unicamFormat.size).center()); data->ispCrop_ = crop; data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); @@ -761,8 +782,11 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) * supports it. */ if (data->sensorMetadata_) { - format = {}; + V4L2SubdeviceFormat embeddedFormat; + + data->sensor_->device()->getFormat(1, &embeddedFormat); format.fourcc = V4L2PixelFormat(V4L2_META_FMT_SENSOR_DATA); + format.planes[0].size = embeddedFormat.size.width * embeddedFormat.size.height; LOG(RPI, Debug) << "Setting embedded data format."; ret = data->unicam_[Unicam::Embedded].dev()->setFormat(&format); @@ -847,9 +871,14 @@ int PipelineHandlerRPi::start(Camera *camera, const ControlList *controls) * IPA configure may have changed the sensor flips - hence the bayer * order. Get the sensor format and set the ISP input now. */ - V4L2DeviceFormat sensorFormat; - data->unicam_[Unicam::Image].dev()->getFormat(&sensorFormat); - ret = data->isp_[Isp::Input].dev()->setFormat(&sensorFormat); + V4L2SubdeviceFormat sensorFormat; + data->sensor_->device()->getFormat(0, &sensorFormat); + + V4L2DeviceFormat ispFormat; + ispFormat.fourcc = BayerFormat::fromMbusCode(sensorFormat.mbus_code).toV4L2PixelFormat(); + ispFormat.size = sensorFormat.size; + + ret = data->isp_[Isp::Input].dev()->setFormat(&ispFormat); if (ret) { stop(camera); return ret; @@ -1004,6 +1033,8 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) if (data->sensor_->init()) return false; + data->sensorFormats_ = populateSensorFormats(data->sensor_); + ipa::RPi::SensorConfig sensorConfig; if (data->loadIPA(&sensorConfig)) { LOG(RPI, Error) << "Failed to load a suitable IPA library"; @@ -1030,6 +1061,11 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) return false; } + if (!(data->unicam_[Unicam::Image].dev()->caps().device_caps() & V4L2_CAP_IO_MC)) { + LOG(RPI, Error) << "Unicam driver did not advertise V4L2_CAP_IO_MC, please update your kernel!"; + return false; + } + /* * Setup our delayed control writer with the sensor default * gain and exposure delays. Mark VBLANK for priority write. @@ -1039,7 +1075,7 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) { V4L2_CID_EXPOSURE, { sensorConfig.exposureDelay, false } }, { V4L2_CID_VBLANK, { sensorConfig.vblankDelay, true } } }; - data->delayedCtrls_ = std::make_unique(data->unicam_[Unicam::Image].dev(), params); + data->delayedCtrls_ = std::make_unique(data->sensor_->device(), params); data->sensorMetadata_ = sensorConfig.sensorMetadata; /* Register the controls that the Raspberry Pi IPA can handle. */ @@ -1066,15 +1102,14 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) * As part of answering the final question, we reset the camera to * no transform at all. */ - - V4L2VideoDevice *dev = data->unicam_[Unicam::Image].dev(); - const struct v4l2_query_ext_ctrl *hflipCtrl = dev->controlInfo(V4L2_CID_HFLIP); + const V4L2Subdevice *sensor = data->sensor_->device(); + const struct v4l2_query_ext_ctrl *hflipCtrl = sensor->controlInfo(V4L2_CID_HFLIP); if (hflipCtrl) { /* We assume it will support vflips too... */ data->supportsFlips_ = true; data->flipsAlterBayerOrder_ = hflipCtrl->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT; - ControlList ctrls(dev->controls()); + ControlList ctrls(data->sensor_->controls()); ctrls.set(V4L2_CID_HFLIP, 0); ctrls.set(V4L2_CID_VFLIP, 0); data->setSensorControls(ctrls); @@ -1082,9 +1117,8 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) /* Look for a valid Bayer format. */ BayerFormat bayerFormat; - for (const auto &iter : dev->formats()) { - V4L2PixelFormat v4l2Format = iter.first; - bayerFormat = BayerFormat::fromV4L2PixelFormat(v4l2Format); + for (const auto &iter : data->sensorFormats_) { + bayerFormat = BayerFormat::fromMbusCode(iter.first); if (bayerFormat.isValid()) break; } @@ -1271,7 +1305,7 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) } } - entityControls.emplace(0, unicam_[Unicam::Image].dev()->controls()); + entityControls.emplace(0, sensor_->controls()); entityControls.emplace(1, isp_[Isp::Input].dev()->controls()); /* Always send the user transform to the IPA. */ @@ -1406,10 +1440,10 @@ void RPiCameraData::setSensorControls(ControlList &controls) ControlList vblank_ctrl; vblank_ctrl.set(V4L2_CID_VBLANK, controls.get(V4L2_CID_VBLANK)); - unicam_[Unicam::Image].dev()->setControls(&vblank_ctrl); + sensor_->setControls(&vblank_ctrl); } - unicam_[Unicam::Image].dev()->setControls(&controls); + sensor_->setControls(&controls); } void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) From patchwork Fri Oct 22 14:39:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14262 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0334FBDB1C for ; Fri, 22 Oct 2021 14:40:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6507C68F69; Fri, 22 Oct 2021 16:40:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="pi6IUNDn"; dkim-atps=neutral Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 551C86023B for ; Fri, 22 Oct 2021 16:40:15 +0200 (CEST) Received: by mail-wr1-x42a.google.com with SMTP id k7so1927579wrd.13 for ; Fri, 22 Oct 2021 07:40:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dlKh+cB15Cw4lo2b5aY9K5NmMsiPItPmPHaK4JaBV0U=; b=pi6IUNDnL2smEDHm6G0NkaEjSjRXG2v+1O7rAACWRoBzStAGYgoeUtzsFCukbiRGAp PYtw/ae12c4MBxXTMwCeJf4peLSMJQJl0IYsv41YWOfus6/bSWkelxGhgbyphKEOZHHw 8RgQWzPaaIFH3V53utqVh6VnnngpqKcJ5BR5al3Ow/0kS9UEwQcju+ynfS64QWFCK0Rz 2/YGbpAQQ4FOG01ZKXA2bXZHcQLpJsnm7rcMAB6SU4OERjEhZGb6/JakMxuqZZcAYGOL sv112pPHKmCdf1GigvM4+PW0l5dzPQrPi71KCousSw4bVM/YJmB1XwRuF1SHx2h/kFq4 q60A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dlKh+cB15Cw4lo2b5aY9K5NmMsiPItPmPHaK4JaBV0U=; b=u7Ohuu2zb1f+uaby3R3dP4+WnZWlsSbn2m8SMx0nIUEE/TxVEElN1X81IWLFkZC3pT XQDSsi0Ay4Ey/SOG6kbNI12qgJZ1jr2AtwaqgHFdfubfTqajQfU8bdrLioAOf4lwtEKL hqq33Ae0+UMoB175KVEENQDYP38TKO3CfRwiVj31lBmD/llC942/ABTKMXBhERLeFYEb Y+XL0iDkv89QAXse0uKG8xMSjSPZ725jFhakPiUgWMEx+B9caQu2v3TJnrpWXXCCgGNp 0/9uT7KLX2PjCcZxcYTYNWgooMh/IrhemsrR/mheJagQUf1lqCqo4uzUAC2ivNGgSdsP Trgg== X-Gm-Message-State: AOAM530DWh1TSdzN4WSMJ0k+VdL0gX17qqLRlTqNaepk2Va9vZMi9fGS eBt/c9sbGiRYVDvLLLXUEkR4Wa7IMnpYbSY+ X-Google-Smtp-Source: ABdhPJwjwf00xnQZ1nBt0tfR0mhXym4ld5rkDebwCYguipeg/Npc3NkW9hKS9bVMdZO2wo3hatQAuw== X-Received: by 2002:a05:6000:1376:: with SMTP id q22mr272095wrz.359.1634913614727; Fri, 22 Oct 2021 07:40:14 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id r205sm8577878wma.3.2021.10.22.07.40.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 07:40:14 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 15:39:04 +0100 Message-Id: <20211022143907.3089419-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022143907.3089419-1-naush@raspberrypi.com> References: <20211022143907.3089419-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/6] pipeline: raspberrypi: Set packing formats for the Unicam image node X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Default to using CSI2 packed formats when setting up the Unicam image format, but use an unpacked format if the user requests one through StreamConfiguration. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index a31b0f81eba7..45725527d66e 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -61,11 +61,14 @@ SensorFormats populateSensorFormats(std::unique_ptr &sensor) return formats; } -inline V4L2DeviceFormat toV4L2DeviceFormat(V4L2SubdeviceFormat &mode) +inline V4L2DeviceFormat toV4L2DeviceFormat(V4L2SubdeviceFormat &mode, bool unpacked = false) { V4L2DeviceFormat deviceFormat; BayerFormat bayer = BayerFormat::fromMbusCode(mode.mbus_code); + bayer.packing = unpacked ? BayerFormat::Packing::None + : BayerFormat::Packing::CSI2Packed; + deviceFormat.fourcc = bayer.toV4L2PixelFormat(); deviceFormat.size = mode.size; return deviceFormat; @@ -598,6 +601,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) Size maxSize, sensorSize; unsigned int maxIndex = 0; bool rawStream = false; + bool unpacked = false; /* * Look for the RAW stream (if given) size as well as the largest @@ -613,6 +617,10 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) */ sensorSize = cfg.size; rawStream = true; + /* Check if the user has explicitly set an unpacked format. */ + V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); + BayerFormat bayer = BayerFormat::fromV4L2PixelFormat(fourcc); + unpacked = bayer.packing == BayerFormat::Packing::None; } else { if (cfg.size > maxSize) { maxSize = config->at(i).size; @@ -623,7 +631,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) /* First calculate the best sensor mode we can use based on the user request. */ V4L2SubdeviceFormat sensorFormat = findBestMode(data->sensorFormats_, rawStream ? sensorSize : maxSize); - V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorFormat); + V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorFormat, unpacked); ret = data->sensor_->setFormat(&sensorFormat); if (ret) From patchwork Fri Oct 22 14:39:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14263 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A8DE7BDB1C for ; Fri, 22 Oct 2021 14:40:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B6A6468F6B; Fri, 22 Oct 2021 16:40:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="CF7R+4n9"; dkim-atps=neutral Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4F5FC6012A for ; Fri, 22 Oct 2021 16:40:16 +0200 (CEST) Received: by mail-wm1-x335.google.com with SMTP id y22-20020a1c7d16000000b003231ea3d705so3623206wmc.4 for ; Fri, 22 Oct 2021 07:40:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LPz2Ck+cam+tyTq1UgTzhzey7O0P6Ctm4/xvxMVlKak=; b=CF7R+4n9LPN9R0s6yF8WHf9Aj+aFI1s+2/upYLGakqLx3aseZtb/FqHFEHU+L/hWd4 43EdDotsMEzrMZGFQfMgSLJnlWGkxYy35LU2ena1AGmiYzq+q797Adue8g4JL3IY2TcE P9rLV+MZbH8jK17y1QyIhUiH3LbwhdrqLZ1681l78xlkfnYOY2YWn+S/QdIwPjHaJYeV XKyYnlDw2+ph44kOJ84j5jhxdJLb+Kx6SQo+7wVBTGVmj0+dUGpTvcp7SoTDs34hF5Sf ZZuf30fvdO8b/Yk/dmMSMMpJG7epealDbbxkLHt/Z2W+/Y50zYmGnckugIf0nZFvoe4T t7sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LPz2Ck+cam+tyTq1UgTzhzey7O0P6Ctm4/xvxMVlKak=; b=sUp7J7q747+LwOLnF4hfDa7bE/IrysShaUluNKmdUyugs8pVCwlDEA0haSGiQLHEek jVxhTS/Q9Mf7pj+k4Pe1lr8X1jrZkVzvTcglKU4Bohz6hpUJQr00iwuOTNhZGyaxpjWd DrmXZirvUW+1pFgXH7xmYs9OzxWqHtW6++cEQghiJk2AUUy57sgpixYdK6hfZg5UwOWa 3dfjROT9AbL0S5WlPFllbkqpVekCs5MTP1sX7KUoBvn/EUJ2kzwquVczPjgmR4TC9o3u LkV85m7GmsRG/tVOQMCXobyrgvoFseL326/COsqTkC0sKGyomADzFtrrxJ4+4Eil0lLx FLxg== X-Gm-Message-State: AOAM533y0hD4/7awrJMjICYnh66lYDeTLlGpaUWQlEsSEp4+Aze0tNNA zOdrKW/EgN/9DwWQazJINFAxHIevkkQNWDQ/ X-Google-Smtp-Source: ABdhPJztNC+0vmIcLRFyyo/xK34xRnkkxf+vswD15Ju9KH72gfUZaNuZno0cV2fzeoWgKvUpbrCsbQ== X-Received: by 2002:a7b:c088:: with SMTP id r8mr29495440wmh.120.1634913615823; Fri, 22 Oct 2021 07:40:15 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id r205sm8577878wma.3.2021.10.22.07.40.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 07:40:15 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 15:39:05 +0100 Message-Id: <20211022143907.3089419-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022143907.3089419-1-naush@raspberrypi.com> References: <20211022143907.3089419-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/6] pipeline: raspberrypi: Remove "unpacked" format penalty in mode selection X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the recent change to switch to programming the sensor device directly, the notion of packed vs unpacked modes are not relevent, since that is a Unicam format construct. Remove any scoring based on packed/unpacked modes. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 4 ---- 1 file changed, 4 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 45725527d66e..2b70b877e70a 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -109,7 +109,6 @@ V4L2SubdeviceFormat findBestMode(const SensorFormats &formatsMap, const Size &re #define PENALTY_8BIT 2000.0 #define PENALTY_10BIT 1000.0 #define PENALTY_12BIT 0.0 -#define PENALTY_UNPACKED 500.0 /* Calculate the closest/best mode from the user requested size. */ for (const auto &iter : formatsMap) { @@ -127,9 +126,6 @@ V4L2SubdeviceFormat findBestMode(const SensorFormats &formatsMap, const Size &re score += PENALTY_AR * scoreFormat(reqAr, modeAr); /* Add any penalties... this is not an exact science! */ - if (!info.packed) - score += PENALTY_UNPACKED; - if (info.bitsPerPixel == 12) score += PENALTY_12BIT; else if (info.bitsPerPixel == 10) From patchwork Fri Oct 22 14:39:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14264 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 08617C324E for ; Fri, 22 Oct 2021 14:40:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3211F68F6D; Fri, 22 Oct 2021 16:40:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="q7YqgHEM"; dkim-atps=neutral Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D5CB68F5D for ; Fri, 22 Oct 2021 16:40:17 +0200 (CEST) Received: by mail-wr1-x432.google.com with SMTP id s19so906097wra.2 for ; Fri, 22 Oct 2021 07:40:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oT0OVsIP0543VwfoaGee+9LOY2F+eiYz5IwkYiOUns4=; b=q7YqgHEMj/J3BS236OdPF/tJ60SGyIjPvxcKveLXkdNWo0/ch48uiBf9z2i9YLdB6G QgZIvQb+Xm390yL99GX0F88QWWalOI1r6l70IJ3fdf3kAd663z4qy9ljUSeJ5YuEWl1O Wpq/pfMSjc9ffecDz29skYvF8rRGjbqYTOSE8k9TlJqyhCJMjD7Js9fPVDDN5KPfSxrs geYIeF97k8PWG3TMsmXMUhGu4uvFX6blw2L903b0H90gHwwAYvC1QHRZzpcHHidcJTx8 PI2fw6Um9mVBfWL2PPbatxcIT3SBLCubvtyMKTRT2/3oAn7sVaL4MzFj4RPsIgw+KTqu Wk/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oT0OVsIP0543VwfoaGee+9LOY2F+eiYz5IwkYiOUns4=; b=zXly+czC72UICxSnpquKxdZLzxqX++rmOJcr7iyn17l5Qzykwva+TQyXnL5xhkYv0k +ztSitJb8RvmF/encf3dKLHjQJA0Hvv/+OwHdDfYhZxFyb41a9PZqem3iynS1n1Pwnib 5NbpCn7YNnxR0lXg3JjOyxbNKalTa0P333Os8eL+QV/1Y2id/PtE0hiYD4Wdr/gkTRp6 Z5sGf4noveZ2/lY2ZHJfPg7GT0e/GzqNSy/L8J+jYUcyH+L0OvQkLPOBHp/n0PdbDOSE DF7XM+rtrS2CA15OLlWaG/5BY4hgx+dyySCK6pyD5FZKYlx/hiS07mUbbjgQwveOz2+V S8Iw== X-Gm-Message-State: AOAM530zpm7D+jqG7q7QWaROE3yeO4Yf0SDoLxgCBBVaXm3qOsLHcbyn RdW/zDg1/ILzVp23tddMaBxVtQEU2A2jt6J+ X-Google-Smtp-Source: ABdhPJxBWHWBAws4MWXrxpRETB9RmJWtoQzUOHOyht9pcxoSEqi/hzOOZ4jAOZoP6rZlNOt/qaQc6A== X-Received: by 2002:a05:6000:1289:: with SMTP id f9mr355000wrx.192.1634913616642; Fri, 22 Oct 2021 07:40:16 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id r205sm8577878wma.3.2021.10.22.07.40.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 07:40:16 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 15:39:06 +0100 Message-Id: <20211022143907.3089419-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022143907.3089419-1-naush@raspberrypi.com> References: <20211022143907.3089419-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/6] pipeline: raspberrypi: Account for a missing Unicam embedded data node X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The unicam driver no longer regesters an embedded data node if the sensor does not provide this stream. Account for this in the pipeline handler match routine by not assuming it is always present. Add a warning if Unicam and the CamHelper do not agree on the presense of sensor embedded data, and disable its useage in these cases. Signed-off-by: Naushir Patuck Review-by: David Plowman Reviewed-by: Laurent Pinchart --- .../pipeline/raspberrypi/raspberrypi.cpp | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 2b70b877e70a..c5e9607c7d95 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -984,7 +984,6 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) DeviceMatch unicam("unicam"); DeviceMatch isp("bcm2835-isp"); - unicam.add("unicam-embedded"); unicam.add("unicam-image"); isp.add("bcm2835-isp0-output0"); /* Input */ @@ -1005,9 +1004,16 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) return false; /* Locate and open the unicam video streams. */ - data->unicam_[Unicam::Embedded] = RPi::Stream("Unicam Embedded", unicam_->getEntityByName("unicam-embedded")); data->unicam_[Unicam::Image] = RPi::Stream("Unicam Image", unicam_->getEntityByName("unicam-image")); + /* An embedded data node will not be present if the sensor does not support it. */ + MediaEntity *embeddedEntity = unicam_->getEntityByName("unicam-embedded"); + if (embeddedEntity) { + data->unicam_[Unicam::Embedded] = RPi::Stream("Unicam Embedded", embeddedEntity); + data->unicam_[Unicam::Embedded].dev()->bufferReady.connect(data.get(), + &RPiCameraData::unicamBufferDequeue); + } + /* Tag the ISP input stream as an import stream. */ data->isp_[Isp::Input] = RPi::Stream("ISP Input", isp_->getEntityByName("bcm2835-isp0-output0"), true); data->isp_[Isp::Output0] = RPi::Stream("ISP Output0", isp_->getEntityByName("bcm2835-isp0-capture1")); @@ -1017,7 +1023,6 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) /* Wire up all the buffer connections. */ data->unicam_[Unicam::Image].dev()->frameStart.connect(data.get(), &RPiCameraData::frameStarted); data->unicam_[Unicam::Image].dev()->bufferReady.connect(data.get(), &RPiCameraData::unicamBufferDequeue); - data->unicam_[Unicam::Embedded].dev()->bufferReady.connect(data.get(), &RPiCameraData::unicamBufferDequeue); data->isp_[Isp::Input].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispInputDequeue); data->isp_[Isp::Output0].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispOutputDequeue); data->isp_[Isp::Output1].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispOutputDequeue); @@ -1045,6 +1050,11 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) return false; } + if (sensorConfig.sensorMetadata ^ !!embeddedEntity) { + LOG(RPI, Warning) << "Mismatch between Unicam and CamHelper for embedded data usage!"; + sensorConfig.sensorMetadata = false; + } + /* * Open all Unicam and ISP streams. The exception is the embedded data * stream, which only gets opened below if the IPA reports that the sensor From patchwork Fri Oct 22 14:39:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14265 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 44476BDB1C for ; Fri, 22 Oct 2021 14:40:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 07AF268F5C; Fri, 22 Oct 2021 16:40:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ol88NpK6"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3EFD968F5C for ; Fri, 22 Oct 2021 16:40:18 +0200 (CEST) Received: by mail-wr1-x435.google.com with SMTP id d13so628087wrf.11 for ; Fri, 22 Oct 2021 07:40:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rS8souYoYk4gsvn/YKRYkOCcLKCa7xP51raLarZqbo4=; b=ol88NpK6Ey5/Okp8QARlt78mRmmJMXUgV+amxGlHTmLYOFo2+mGvln4OG+2bV9De/0 T8ELAwmOPKvijfKvNGAkQNoQMOEDApPLP9HWo96AkpeVH/Twe8ngiRBrnav6dwwF3EBM tkcZ1VEYFGr21wZ+UEJloPzKn97b0i2ZsdxawcvNwr+wzAhWs7zhGlvqiTmwFmats6/s HHsu0cHImga/nJreJwp/vQXEXZk1tfAR7+yCBbvmtZmPb0LNnDC1RqJfWCD3byDFr2pE mEBUgyW75CZqxcl9OVYs74g66w0AXbFPXhk3dTFh8xtX5q0Wstzef/R456BW2wnIXBen JVxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rS8souYoYk4gsvn/YKRYkOCcLKCa7xP51raLarZqbo4=; b=Rgj+3QHTKQqF0S7KEsiUVAKqfyvsV3Jmn2HpyU6e2g/XpYrvsACb7hVrov2Atd/5Hp lafq7TlaleE1y5DFPMEg/BcTnefD4pSxlUvI33BJHm/eBcTayP61sOu5Jc3I9F8SUAaU TlwpsrVAQb88b7/2ZGb3pax/bUyoCAHFCQ5Dd9fsp+0IFmjWVQfFsTNifp8Pi2FNn9fc 5sz9XhbAK5pudysmUspLtbrfjHf+EGQuq/TntsOAHtYiqQAX8/Kh988xDhz9aLctnH3Y thDVyMJK3Q0KRQtyg8s7/BoHKCxpM2/CVDv6c206nsWb2Us/JnUPXnHlRl0OYGm5edDW qxwg== X-Gm-Message-State: AOAM533Fi7ltb8p414V0po3vVJd8eXStrZZCN0Cj+ydPsiH9YPz1tXfC AxLmKKP3XJTzn2KROX41z6/UPl2GtwnXnfec X-Google-Smtp-Source: ABdhPJzl6sbD06rcbkqF5xcv33M7elMJNUXvLDXpz9008h8GPX4+9lZFpSm8JyArskhIK0XQYxsipw== X-Received: by 2002:adf:e689:: with SMTP id r9mr320484wrm.426.1634913617698; Fri, 22 Oct 2021 07:40:17 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:f00d:ddf0:61b0:a4cd]) by smtp.gmail.com with ESMTPSA id r205sm8577878wma.3.2021.10.22.07.40.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 07:40:16 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Fri, 22 Oct 2021 15:39:07 +0100 Message-Id: <20211022143907.3089419-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211022143907.3089419-1-naush@raspberrypi.com> References: <20211022143907.3089419-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 6/6] pipeline: raspberrypi: Apply sensor flips at the start of configure() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Sensor flips might change the Bayer order of the requested format. The existing code would set a sensor format along with the appropriate Unicam and ISP input formats, but reset the latter two on start() once the flips had been requested. We can now set the sensor flips just after we set the sensor mode in configure(), thereby not needing the second pair of format sets in start(). Signed-off-by: Naushir Patuck Review-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 60 +++++++------------ 1 file changed, 21 insertions(+), 39 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index c5e9607c7d95..ad6f1231bbf6 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -627,16 +627,34 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) /* First calculate the best sensor mode we can use based on the user request. */ V4L2SubdeviceFormat sensorFormat = findBestMode(data->sensorFormats_, rawStream ? sensorSize : maxSize); - V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorFormat, unpacked); ret = data->sensor_->setFormat(&sensorFormat); if (ret) return ret; /* - * Unicam image output format. The ISP input format gets set at start, - * just in case we have swapped bayer orders due to flips. + * Configure the H/V flip controls based on the combination of + * the sensor and user transform. */ + if (data->supportsFlips_) { + const RPiCameraConfiguration *rpiConfig = + static_cast(config); + ControlList controls; + + controls.set(V4L2_CID_HFLIP, + static_cast(!!(rpiConfig->combinedTransform_ & Transform::HFlip))); + controls.set(V4L2_CID_VFLIP, + static_cast(!!(rpiConfig->combinedTransform_ & Transform::VFlip))); + data->setSensorControls(controls); + + /* + * IPA configure may have changed the sensor flips - hence the bayer + * order. So update the sensor format now. + */ + data->sensor_->device()->getFormat(0, &sensorFormat); + } + + V4L2DeviceFormat unicamFormat = toV4L2DeviceFormat(sensorFormat, unpacked); ret = data->unicam_[Unicam::Image].dev()->setFormat(&unicamFormat); if (ret) return ret; @@ -645,10 +663,6 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) << " - Selected sensor mode: " << sensorFormat.toString() << " - Selected unicam mode: " << unicamFormat.toString(); - /* - * This format may be reset on start() if the bayer order has changed - * because of flips in the sensor. - */ ret = data->isp_[Isp::Input].dev()->setFormat(&unicamFormat); if (ret) return ret; @@ -871,23 +885,6 @@ int PipelineHandlerRPi::start(Camera *camera, const ControlList *controls) return ret; } - /* - * IPA configure may have changed the sensor flips - hence the bayer - * order. Get the sensor format and set the ISP input now. - */ - V4L2SubdeviceFormat sensorFormat; - data->sensor_->device()->getFormat(0, &sensorFormat); - - V4L2DeviceFormat ispFormat; - ispFormat.fourcc = BayerFormat::fromMbusCode(sensorFormat.mbus_code).toV4L2PixelFormat(); - ispFormat.size = sensorFormat.size; - - ret = data->isp_[Isp::Input].dev()->setFormat(&ispFormat); - if (ret) { - stop(camera); - return ret; - } - /* Enable SOF event generation. */ data->unicam_[Unicam::Image].dev()->setFrameStartEnabled(true); @@ -1301,10 +1298,6 @@ int RPiCameraData::loadIPA(ipa::RPi::SensorConfig *sensorConfig) int RPiCameraData::configureIPA(const CameraConfiguration *config) { - /* We know config must be an RPiCameraConfiguration. */ - const RPiCameraConfiguration *rpiConfig = - static_cast(config); - std::map streamConfig; std::map entityControls; ipa::RPi::IPAConfig ipaConfig; @@ -1355,17 +1348,6 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) return -EPIPE; } - /* - * Configure the H/V flip controls based on the combination of - * the sensor and user transform. - */ - if (supportsFlips_) { - controls.set(V4L2_CID_HFLIP, - static_cast(!!(rpiConfig->combinedTransform_ & Transform::HFlip))); - controls.set(V4L2_CID_VFLIP, - static_cast(!!(rpiConfig->combinedTransform_ & Transform::VFlip))); - } - if (!controls.empty()) setSensorControls(controls);