{"id":19733,"url":"https://patchwork.libcamera.org/api/patches/19733/?format=json","web_url":"https://patchwork.libcamera.org/patch/19733/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20240315001613.2033-15-laurent.pinchart@ideasonboard.com>","date":"2024-03-15T00:16:13","name":"[v2,14/14] libcamera: camera_sensor: Expose the Bayer order","commit_ref":"284919ef2e509d8109b84eedcc6d5c2f78cddbef","pull_url":null,"state":"accepted","archived":false,"hash":"ccf49c5bf933190dafafb95cd805ecf7da445eff","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/19733/mbox/","series":[{"id":4226,"url":"https://patchwork.libcamera.org/api/series/4226/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4226","date":"2024-03-15T00:15:59","name":"libcamera: Prepare for new camera sensor class","version":2,"mbox":"https://patchwork.libcamera.org/series/4226/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/19733/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/19733/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1663FC3273\n\tfor ;\n\tFri, 15 Mar 2024 00:16:48 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 41C5162D2B;\n\tFri, 15 Mar 2024 01:16:47 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5642062CAB\n\tfor ;\n\tFri, 15 Mar 2024 01:16:36 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A05C3667\n\tfor ;\n\tFri, 15 Mar 2024 01:16:12 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"G+cwvex5\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1710461772;\n\tbh=WMHCsHL+yRdCG0onYI/V4JB1bXqEme74Az2ttZ7EQbQ=;\n\th=From:To:Subject:Date:In-Reply-To:References:From;\n\tb=G+cwvex5Tpdr0vRjHICNxNUwFM1jIjxBWvzG1X1ApDfKtIsdRDwMH0DlF5lH4Bnw9\n\tr/jbZ1cyybwGbcvhzvZWh4kBRXkJteL+j6/kGCyeAz5AYF0vYn8xc3rI5GUd7WoXVp\n\tqre99qvsk9/v+RUVmFmCJS6SwKs2nBFrvYoQUW1Y=","From":"Laurent Pinchart ","To":"libcamera-devel@lists.libcamera.org","Subject":"[PATCH v2 14/14] libcamera: camera_sensor: Expose the Bayer order","Date":"Fri, 15 Mar 2024 02:16:13 +0200","Message-ID":"<20240315001613.2033-15-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.43.2","In-Reply-To":"<20240315001613.2033-1-laurent.pinchart@ideasonboard.com>","References":"<20240315001613.2033-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Pipeline handlers may need to know the Bayer order produced by the\nsensor when a Transform is applied (horizontal or vertical flip). This\nis currently implemented manually in the Raspberry Pi pipeline handler.\nMove the implementation to the CameraSensor class to make it usable in\nother pipeline handlers.\n\nSigned-off-by: Laurent Pinchart \nReviewed-by: Jacopo Mondi \nReviewed-by: Daniel Scally \n---\n include/libcamera/internal/camera_sensor.h | 4 +-\n .../pipeline/rpi/common/pipeline_base.cpp | 54 +++----------------\n .../pipeline/rpi/common/pipeline_base.h | 6 +--\n src/libcamera/sensor/camera_sensor.cpp | 37 ++++++++++++-\n 4 files changed, 45 insertions(+), 56 deletions(-)","diff":"diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h\nindex 750d6d729cac..d05f48ebeebe 100644\n--- a/include/libcamera/internal/camera_sensor.h\n+++ b/include/libcamera/internal/camera_sensor.h\n@@ -22,12 +22,12 @@\n \n #include \n \n+#include \"libcamera/internal/bayer_format.h\"\n #include \"libcamera/internal/formats.h\"\n #include \"libcamera/internal/v4l2_subdevice.h\"\n \n namespace libcamera {\n \n-class BayerFormat;\n class CameraLens;\n class MediaEntity;\n class SensorConfiguration;\n@@ -69,6 +69,7 @@ public:\n \tconst ControlList &properties() const { return properties_; }\n \tint sensorInfo(IPACameraSensorInfo *info) const;\n \tTransform computeTransform(Orientation *orientation) const;\n+\tBayerFormat::Order bayerOrder(Transform t) const;\n \n \tconst ControlInfoMap &controls() const;\n \tControlList getControls(const std::vector &ids);\n@@ -114,6 +115,7 @@ private:\n \tRectangle activeArea_;\n \tconst BayerFormat *bayerFormat_;\n \tbool supportFlips_;\n+\tbool flipsAlterBayerOrder_;\n \tOrientation mountingOrientation_;\n \n \tControlList properties_;\ndiff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\nindex 9449c3dc458c..7e420b3f90a4 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n@@ -235,24 +235,16 @@ CameraConfiguration::Status RPiCameraConfiguration::validate()\n \tfor (auto &raw : rawStreams_) {\n \t\tStreamConfiguration *rawStream = raw.cfg;\n \n-\t\t/* Adjust the RAW stream to match the computed sensor format. */\n-\t\tBayerFormat sensorBayer = BayerFormat::fromMbusCode(sensorFormat_.code);\n-\n \t\t/*\n-\t\t * Some sensors change their Bayer order when they are h-flipped\n-\t\t * or v-flipped, according to the transform. If this one does, we\n-\t\t * must advertise the transformed Bayer order in the raw stream.\n-\t\t * Note how we must fetch the \"native\" (i.e. untransformed) Bayer\n-\t\t * order, because the sensor may currently be flipped!\n+\t\t * Some sensors change their Bayer order when they are\n+\t\t * h-flipped or v-flipped, according to the transform. Adjust\n+\t\t * the RAW stream to match the computed sensor format by\n+\t\t * applying the sensor Bayer order resulting from the transform\n+\t\t * to the user request.\n \t\t */\n-\t\tif (data_->flipsAlterBayerOrder_) {\n-\t\t\tsensorBayer.order = data_->nativeBayerOrder_;\n-\t\t\tsensorBayer = sensorBayer.transform(combinedTransform_);\n-\t\t}\n \n-\t\t/* Apply the sensor adjusted Bayer order to the user request. */\n \t\tBayerFormat cfgBayer = BayerFormat::fromPixelFormat(rawStream->pixelFormat);\n-\t\tcfgBayer.order = sensorBayer.order;\n+\t\tcfgBayer.order = data_->sensor_->bayerOrder(combinedTransform_);\n \n \t\tif (rawStream->pixelFormat != cfgBayer.toPixelFormat()) {\n \t\t\trawStream->pixelFormat = cfgBayer.toPixelFormat();\n@@ -840,40 +832,6 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr &camera\n \t */\n \tdata->properties_.set(properties::ScalerCropMaximum, Rectangle{});\n \n-\t/*\n-\t * We cache two things about the sensor in relation to transforms\n-\t * (meaning horizontal and vertical flips): if they affect the Bayer\n-\t * ordering, and what the \"native\" Bayer order is, when no transforms\n-\t * are applied.\n-\t *\n-\t * If flips are supported verify if they affect the Bayer ordering\n-\t * and what the \"native\" Bayer order is, when no transforms are\n-\t * applied.\n-\t *\n-\t * We note that the sensor's cached list of supported formats is\n-\t * already in the \"native\" order, with any flips having been undone.\n-\t */\n-\tconst V4L2Subdevice *sensor = data->sensor_->device();\n-\tconst struct v4l2_query_ext_ctrl *hflipCtrl = sensor->controlInfo(V4L2_CID_HFLIP);\n-\tif (hflipCtrl) {\n-\t\t/* We assume it will support vflips too... */\n-\t\tdata->flipsAlterBayerOrder_ = hflipCtrl->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT;\n-\t}\n-\n-\t/* Look for a valid Bayer format. */\n-\tBayerFormat bayerFormat;\n-\tfor (const auto &iter : data->sensorFormats_) {\n-\t\tbayerFormat = BayerFormat::fromMbusCode(iter.first);\n-\t\tif (bayerFormat.isValid())\n-\t\t\tbreak;\n-\t}\n-\n-\tif (!bayerFormat.isValid()) {\n-\t\tLOG(RPI, Error) << \"No Bayer format found\";\n-\t\treturn -EINVAL;\n-\t}\n-\tdata->nativeBayerOrder_ = bayerFormat.order;\n-\n \tret = platformRegister(cameraData, frontend, backend);\n \tif (ret)\n \t\treturn ret;\ndiff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h\nindex 267eef1102f1..0608bbe5f0c7 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.h\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h\n@@ -48,7 +48,7 @@ class CameraData : public Camera::Private\n public:\n \tCameraData(PipelineHandler *pipe)\n \t\t: Camera::Private(pipe), state_(State::Stopped),\n-\t\t flipsAlterBayerOrder_(false), dropFrameCount_(0), buffersAllocated_(false),\n+\t\t dropFrameCount_(0), buffersAllocated_(false),\n \t\t ispOutputCount_(0), ispOutputTotal_(0)\n \t{\n \t}\n@@ -131,10 +131,6 @@ public:\n \n \tstd::queue requestQueue_;\n \n-\t/* Store the \"native\" Bayer order (that is, with no transforms applied). */\n-\tbool flipsAlterBayerOrder_;\n-\tBayerFormat::Order nativeBayerOrder_;\n-\n \t/* For handling digital zoom. */\n \tIPACameraSensorInfo sensorInfo_;\n \tRectangle ispCrop_; /* crop in ISP (camera mode) pixels */\ndiff --git a/src/libcamera/sensor/camera_sensor.cpp b/src/libcamera/sensor/camera_sensor.cpp\nindex 402025566544..5c4f35324055 100644\n--- a/src/libcamera/sensor/camera_sensor.cpp\n+++ b/src/libcamera/sensor/camera_sensor.cpp\n@@ -58,7 +58,7 @@ LOG_DEFINE_CATEGORY(CameraSensor)\n CameraSensor::CameraSensor(const MediaEntity *entity)\n \t: entity_(entity), pad_(UINT_MAX), staticProps_(nullptr),\n \t bayerFormat_(nullptr), supportFlips_(false),\n-\t properties_(properties::properties)\n+\t flipsAlterBayerOrder_(false), properties_(properties::properties)\n {\n }\n \n@@ -271,9 +271,14 @@ int CameraSensor::validateSensorDriver()\n \tconst struct v4l2_query_ext_ctrl *hflipInfo = subdev_->controlInfo(V4L2_CID_HFLIP);\n \tconst struct v4l2_query_ext_ctrl *vflipInfo = subdev_->controlInfo(V4L2_CID_VFLIP);\n \tif (hflipInfo && !(hflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY) &&\n-\t vflipInfo && !(vflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY))\n+\t vflipInfo && !(vflipInfo->flags & V4L2_CTRL_FLAG_READ_ONLY)) {\n \t\tsupportFlips_ = true;\n \n+\t\tif (hflipInfo->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT ||\n+\t\t vflipInfo->flags & V4L2_CTRL_FLAG_MODIFY_LAYOUT)\n+\t\t\tflipsAlterBayerOrder_ = true;\n+\t}\n+\n \tif (!supportFlips_)\n \t\tLOG(CameraSensor, Debug)\n \t\t\t<< \"Camera sensor does not support horizontal/vertical flip\";\n@@ -1041,6 +1046,34 @@ Transform CameraSensor::computeTransform(Orientation *orientation) const\n \treturn transform;\n }\n \n+/**\n+ * \\brief Compute the Bayer order that results from the given Transform\n+ * \\param[in] t The Transform to apply to the sensor\n+ *\n+ * Some sensors change their Bayer order when they are h-flipped or v-flipped.\n+ * This function computes and returns the Bayer order that would result from the\n+ * given transform applied to the sensor.\n+ *\n+ * This function is valid only when the sensor produces raw Bayer formats.\n+ *\n+ * \\return The Bayer order produced by the sensor when the Transform is applied\n+ */\n+BayerFormat::Order CameraSensor::bayerOrder(Transform t) const\n+{\n+\t/* Return a defined by meaningless value for non-Bayer sensors. */\n+\tif (!bayerFormat_)\n+\t\treturn BayerFormat::Order::BGGR;\n+\n+\tif (!flipsAlterBayerOrder_)\n+\t\treturn bayerFormat_->order;\n+\n+\t/*\n+\t * Apply the transform to the native (i.e. untransformed) Bayer order,\n+\t * using the rest of the Bayer format supplied by the caller.\n+\t */\n+\treturn bayerFormat_->transform(t).order;\n+}\n+\n /**\n * \\brief Retrieve the supported V4L2 controls and their information\n *\n","prefixes":["v2","14/14"]}