From patchwork Sat Aug 29 11:54:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 9432 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2CE7CBF019 for ; Sat, 29 Aug 2020 11:54:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EDFC76293F; Sat, 29 Aug 2020 13:54:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="l4pIK0dQ"; dkim-atps=neutral Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BD93C62920 for ; Sat, 29 Aug 2020 13:54:39 +0200 (CEST) Received: by mail-wr1-x42b.google.com with SMTP id b18so1552013wrs.7 for ; Sat, 29 Aug 2020 04:54:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zDr2YMtYWCg4u86sRPc68xnTPi105UzxUzv42mZx7uw=; b=l4pIK0dQ5OOYP6lifyWPIpOQCAr3S1cVqBxOvzhArCLB43ROPF1upMRR1uo0g8BY5N eQisYiExZpB0YlbNIezjxIDn6CK+4ZclNktb0YSSq9XsFooNjUDFTqlFBKEN9gkFVqTY cndBcfs0eJB3MVXGvhDgnxhbSpQ7vwUNsmeyJ1qPf6Zg34wnv2mj3esGxvoWPNIf04Yy B6GfkCK2m0c49o9VLGUNB5SAnZD0KdaNh0tuHdfKsHRyJrTG74sgwtJzgluQobGa41MW iM0SjO2Lt2Q9RgcMJGuL61+xAxFmRBpN9b2qxu2zNUptA2Fr1mxKlWOlN4JxDieB4Sz5 x39A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zDr2YMtYWCg4u86sRPc68xnTPi105UzxUzv42mZx7uw=; b=cp0cJyJDpo9iXXTGKwMkGqwFcf4Z+6MG10g4gUTjSrXEicXA7dvNevXvsoFiqF7ySl quncB3bpDyJFqwRzuJuFHa3JFfOPrt2cZxu7CTztFtJN9UsN+7wvuHzHluZLSnBvQvut wC8/TFGezBxFsAKYWqW9xVImcFVL2sRRssmubRn90Ki6R1TItZyA+ZRNirugYixZ6cKF yTWpGpRfp8hYywklzM6X09Y4qSkml3rxmcK6PtlTFkZP2OEAs0yMy6voixwQMIXqj0DD KEDvuYsXGSf05GwwScrxLM8R23D/snOjNoqk5gQeuJozfCEBZlgSvswgRqMzlk8jRgBr a7kg== X-Gm-Message-State: AOAM530OrenagWmg7XfEIxdvOtb5Pk92HSV3hymzMibtBmVS8VrM0q1c Sdhwfido32GTFu5TE4n7Q6rrPUgZ1zfk/g== X-Google-Smtp-Source: ABdhPJweY8ywZbVOribAyDsMtQj5rjI/4HeHkM5MREYB3GHV1SaQ26jaEHOc8ugrLicVItcpo1EuVA== X-Received: by 2002:adf:ef0a:: with SMTP id e10mr1414775wro.362.1598702079089; Sat, 29 Aug 2020 04:54:39 -0700 (PDT) Received: from pi4-davidp.lan (plowpeople3.plus.com. [80.229.223.72]) by smtp.gmail.com with ESMTPSA id v16sm3071523wmj.14.2020.08.29.04.54.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 29 Aug 2020 04:54:38 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Date: Sat, 29 Aug 2020 12:54:27 +0100 Message-Id: <20200829115429.30010-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200829115429.30010-1-david.plowman@raspberrypi.com> References: <20200829115429.30010-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 6/8] libcamera: raspberrypi: Set camera flips correctly from user transform X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler allows all transforms except those involving a transpose. The user transform is combined with any inherent rotation of the camera, and the camera's H and V flip bits are set accordingly. Note that the validate() method has to work out what the final Bayer order of any raw streams will be, before configure() actually applies the transform to the sensor. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 82 ++++++++++++++++--- 1 file changed, 71 insertions(+), 11 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index c554532..333aa94 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -294,7 +294,7 @@ public: void frameStarted(uint32_t sequence); int loadIPA(); - int configureIPA(); + int configureIPA(CameraConfiguration *config); void queueFrameAction(unsigned int frame, const IPAOperationData &action); @@ -400,8 +400,34 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; - if (transform != Transform::Identity) { - transform = Transform::Identity; + /* + * What if the platform has a non-90 degree rotation? We can't even + * "adjust" the configuration and carry on. Alternatively, raising an + * error means the platform can never run. Let's just print a warning + * and continue regardless; the rotation is effectively set to zero. + */ + int32_t rotation = data_->sensor_->properties().get(properties::Rotation); + bool success; + Transform combined = transform * transformFromRotation(rotation, &success); + if (!success) + LOG(RPI, Warning) << "Invalid rotation of " << rotation + << " degrees - ignoring"; + + /* + * We combine the platform and user transform, but must "adjust away" + * any combined result that includes a transform, as we can't do those. + * In this case, flipping only the transpose bit is helpful to + * applications - they either get the transform they requested, or have + * to do a simple transpose themselves (they don't have to worry about + * the other possible cases). + */ + if (!!(combined & Transform::Transpose)) { + /* + * Flipping the transpose bit in "transform" flips it in + * combined result too (as it's the last thing that happens). + */ + transform ^= Transform::Transpose; + combined ^= Transform::Transpose; status = Adjusted; } @@ -414,13 +440,42 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() * Calculate the best sensor mode we can use based on * the user request. */ - V4L2VideoDevice::Formats fmts = data_->unicam_[Unicam::Image].dev()->formats(); + V4L2VideoDevice *dev = data_->unicam_[Unicam::Image].dev(); + V4L2VideoDevice::Formats fmts = dev->formats(); V4L2DeviceFormat sensorFormat = findBestMode(fmts, cfg.size); - int ret = data_->unicam_[Unicam::Image].dev()->tryFormat(&sensorFormat); + int ret = dev->tryFormat(&sensorFormat); if (ret) return Invalid; - PixelFormat sensorPixFormat = sensorFormat.fourcc.toPixelFormat(); + /* + * Some sensors change their Bayer order when they are + * h-flipped or v-flipped, according to the transform. + * If the controls own up to "modifying the layout" we + * will assume that's what is going on and advertise + * the transformed Bayer order in the stream. + */ + V4L2PixelFormat fourcc = sensorFormat.fourcc; + + /* + * The camera may already be transformed, so we must + * only transform the fourcc if the new transform is + * different. + */ + ControlList ctrls = dev->getControls({ V4L2_CID_HFLIP, V4L2_CID_VFLIP }); + + const struct v4l2_query_ext_ctrl *hflipCtrl = dev->queryControl(V4L2_CID_HFLIP); + if (hflipCtrl && + (hflipCtrl->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT) && + ctrls.get(V4L2_CID_HFLIP).get() != !!(combined & Transform::HFlip)) + fourcc = fourcc.transform(Transform::HFlip); + + const struct v4l2_query_ext_ctrl *vflipCtrl = dev->queryControl(V4L2_CID_VFLIP); + if (vflipCtrl && + (vflipCtrl->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT) && + ctrls.get(V4L2_CID_VFLIP).get() != !!(combined & Transform::VFlip)) + fourcc = fourcc.transform(Transform::VFlip); + + PixelFormat sensorPixFormat = fourcc.toPixelFormat(); if (cfg.size != sensorFormat.size || cfg.pixelFormat != sensorPixFormat) { cfg.size = sensorFormat.size; @@ -756,7 +811,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) crop.y = (sensorFormat.size.height - crop.height) >> 1; data->isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &crop); - ret = data->configureIPA(); + ret = data->configureIPA(config); if (ret) LOG(RPI, Error) << "Failed to configure the IPA: " << ret; @@ -1112,7 +1167,7 @@ int RPiCameraData::loadIPA() return ipa_->init(settings); } -int RPiCameraData::configureIPA() +int RPiCameraData::configureIPA(CameraConfiguration *config) { std::map streamConfig; std::map entityControls; @@ -1170,11 +1225,16 @@ int RPiCameraData::configureIPA() sensorMetadata_ = result.data[2]; } - /* Configure the H/V flip controls based on the sensor rotation. */ + /* + * Configure the H/V flip controls based on the sensor rotation + * and user transform. + */ ControlList ctrls(unicam_[Unicam::Image].dev()->controls()); int32_t rotation = sensor_->properties().get(properties::Rotation); - ctrls.set(V4L2_CID_HFLIP, static_cast(!!rotation)); - ctrls.set(V4L2_CID_VFLIP, static_cast(!!rotation)); + /* The rotation angle was already checked in validate(). */ + Transform combined = config->transform * transformFromRotation(rotation); + ctrls.set(V4L2_CID_HFLIP, static_cast(!!(combined & Transform::HFlip))); + ctrls.set(V4L2_CID_VFLIP, static_cast(!!(combined & Transform::VFlip))); unicam_[Unicam::Image].dev()->setControls(&ctrls); }