{"id":15078,"url":"https://patchwork.libcamera.org/api/1.1/patches/15078/?format=json","web_url":"https://patchwork.libcamera.org/patch/15078/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"","date":"2021-12-08T12:35:23","name":"[libcamera-devel] libcamera: pipeline: simple: Add support for controls","commit_ref":null,"pull_url":null,"state":"rejected","archived":false,"hash":"db3402114efd5ce7dfe9aab29425e781d54f87ca","submitter":{"id":108,"url":"https://patchwork.libcamera.org/api/1.1/people/108/?format=json","name":"Benjamin Schaaf","email":"ben.schaaf@gmail.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/15078/mbox/","series":[{"id":2826,"url":"https://patchwork.libcamera.org/api/1.1/series/2826/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2826","date":"2021-12-08T12:35:23","name":"[libcamera-devel] libcamera: pipeline: simple: Add support for controls","version":1,"mbox":"https://patchwork.libcamera.org/series/2826/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/15078/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/15078/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 712F2BDB13\n\tfor ;\n\tWed, 8 Dec 2021 12:35:39 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AB2C460822;\n\tWed, 8 Dec 2021 13:35:38 +0100 (CET)","from mail-yb1-xb2a.google.com (mail-yb1-xb2a.google.com\n\t[IPv6:2607:f8b0:4864:20::b2a])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6226560225\n\tfor ;\n\tWed, 8 Dec 2021 13:35:37 +0100 (CET)","by mail-yb1-xb2a.google.com with SMTP id y68so5698713ybe.1\n\tfor ;\n\tWed, 08 Dec 2021 04:35:37 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"qHGv2Juy\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;\n\th=mime-version:from:date:message-id:subject:to;\n\tbh=a1aJq3pkhIbDTHBvNy9eMXicBodX1KPs2MqBUQOXTe4=;\n\tb=qHGv2JuyyhYdHunchhG6Qqm8qE9pV8AsjPWyR09T4w5vOUxeS8UgXObBVCG/g0uk8j\n\tx8GnuTnkUxgHxnRPmycrOXV//edESQjG+CyNth6+efMBDpac1ShaiDE5W5qdDFza3p1N\n\t4gHbMWIOdVB8HzTR4fwYQDbbFcWD4+TbSUFlgElNntYYKwRlRP+c8KmXK+mbjKLgPsvS\n\t7lsxVWSIcAm4ay182FfnTBYBOf/oK62CHI744eLvJD7Ov+/oQgjfjdnJ+iLqQ9MvmMOc\n\tXn6nNoVJ6X3B178x/jHojT3TCn6a17y9+tR7t9d+2bzPdg5nsYgii8EiLTZjRyLyVnl0\n\ti1BA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:mime-version:from:date:message-id:subject:to;\n\tbh=a1aJq3pkhIbDTHBvNy9eMXicBodX1KPs2MqBUQOXTe4=;\n\tb=eIiGH0/HSkFvnNkcxgLJ6uDQv3WRYWi5Rhm6be7pn0wvac0hvuRQn/gCW3wNN6Kh8M\n\tK+/hdt6aXHoP2IeW5yOWEW1j0zO8ppo62zDrikwHEDetD1IUqKxTQmZU4yzvx5ReEgLq\n\tu2Ykte7QL8lRCh4tCcR8Ysu/Rh94FyPYGFW2booBdHBVMT8OZW/mFHCiSmeVh0S/XKAx\n\t66cR7yiL3FsnBF0KQxAZmOSaOERS1Nl/GMhRc/ZuNb0zTbEe3LGYwKHHPCrJYkchMvsJ\n\t7X60ocfejv3p1q7RsztGXklzqeQNHUvGT2iGXa3KcFmGZcJQ1uGqSBiY/0Pqb8BSUl1s\n\tnKcQ==","X-Gm-Message-State":"AOAM533KJFebUvzlF1MlasFRq6Ztc5z9YqdOymU3MejL52n9ciyZ61cX\n\t2+Tmay3/5p/koa8bAaaOHW+Fl6q23xEAMl1sNDt6p4MflRg=","X-Google-Smtp-Source":"ABdhPJzVEgkl+kf5DcTSyfiIeKj+e9U2GTmzGgkBNLZ0zZhi9VSn1+0DhhLgvMcDxodQTiQZ1t6BgI0gJ0J86o1WYys=","X-Received":"by 2002:a25:9bc9:: with SMTP id\n\tw9mr58032128ybo.398.1638966935031; \n\tWed, 08 Dec 2021 04:35:35 -0800 (PST)","MIME-Version":"1.0","From":"Benjamin Schaaf ","Date":"Wed, 8 Dec 2021 23:35:23 +1100","Message-ID":"","To":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"[libcamera-devel] [PATCH] libcamera: pipeline: simple: Add support\n\tfor controls","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Controls and control info are translated between libcamera and V4L2\ninside the simple pipeline. Request controls are applied when a request\nis next to be completed.\n\nThis also adds some additional draft controls needed for the PinePhone.\n\nSigned-off-by: Benjamin Schaaf \n---\n src/libcamera/control_ids.yaml | 24 +++\n src/libcamera/controls.cpp | 6 -\n src/libcamera/pipeline/simple/controls.cpp | 230 +++++++++++++++++++++\n src/libcamera/pipeline/simple/controls.h | 26 +++\n src/libcamera/pipeline/simple/meson.build | 1 +\n src/libcamera/pipeline/simple/simple.cpp | 30 ++-\n 6 files changed, 310 insertions(+), 7 deletions(-)\n create mode 100644 src/libcamera/pipeline/simple/controls.cpp\n create mode 100644 src/libcamera/pipeline/simple/controls.h\n\n SimplePipelineHandler *pipe = SimpleCameraData::pipe();\n@@ -666,6 +686,10 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer)\n return;\n }\n\n+ // Set the controls for the next queued request\n+ if (!queuedRequests_.empty())\n+ setRequestControls(queuedRequests_.front());\n+\n /*\n * Record the sensor's timestamp in the request metadata. The request\n * needs to be obtained from the user-facing buffer, as internal\n@@ -1033,6 +1057,10 @@ int SimplePipelineHandler::start(Camera\n*camera, [[maybe_unused]] const ControlL\n return ret;\n }\n\n+ // Apply controls from first request\n+ if (!data->queuedRequests_.empty())\n+ data->setRequestControls(data->queuedRequests_.front());\n+\n if (data->useConverter_) {\n ret = data->converter_->start();\n if (ret < 0) {","diff":"diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\nindex 9d4638ae..2af230c3 100644\n--- a/src/libcamera/control_ids.yaml\n+++ b/src/libcamera/control_ids.yaml\n@@ -406,6 +406,30 @@ controls:\n The camera will cancel any active or completed metering sequence.\n The AE algorithm is reset to its initial state.\n\n+ - AutoGain:\n+ type: bool\n+ draft: true\n+ description: |\n+ Control for Automatic Gain. Currently identical to V4L2_CID_AUTOGAIN.\n+\n+ - AfEnabled:\n+ type: bool\n+ draft: true\n+ description: |\n+ Control for AF. Currently identical to V4L2_CID_FOCUS_AUTO.\n+\n+ - AfStart:\n+ type: void\n+ draft: true\n+ description: |\n+ Control for AF. Currently identical to V4L2_CID_AUTO_FOCUS_START.\n+\n+ - AfStop:\n+ type: void\n+ draft: true\n+ description: |\n+ Control for AF. Currently identical to V4L2_CID_AUTO_FOCUS_END.\n+\n - AfTrigger:\n type: int32_t\n draft: true\ndiff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp\nindex 0d8c0a5c..1f65fc73 100644\n--- a/src/libcamera/controls.cpp\n+++ b/src/libcamera/controls.cpp\n@@ -1052,9 +1052,6 @@ const ControlValue *ControlList::find(unsigned\nint id) const\n {\n const auto iter = controls_.find(id);\n if (iter == controls_.end()) {\n- LOG(Controls, Error)\n- << \"Control \" << utils::hex(id) << \" not found\";\n-\n return nullptr;\n }\n\n@@ -1064,9 +1061,6 @@ const ControlValue *ControlList::find(unsigned\nint id) const\n ControlValue *ControlList::find(unsigned int id)\n {\n if (validator_ && !validator_->validate(id)) {\n- LOG(Controls, Error)\n- << \"Control \" << utils::hex(id)\n- << \" is not valid for \" << validator_->name();\n return nullptr;\n }\n\ndiff --git a/src/libcamera/pipeline/simple/controls.cpp\nb/src/libcamera/pipeline/simple/controls.cpp\nnew file mode 100644\nindex 00000000..32695749\n--- /dev/null\n+++ b/src/libcamera/pipeline/simple/controls.cpp\n@@ -0,0 +1,230 @@\n+#include \"controls.h\"\n+\n+#include \n+\n+#include \n+\n+#include \n+\n+namespace libcamera {\n+\n+LOG_DECLARE_CATEGORY(SimplePipeline)\n+\n+/*\n+ * These controls can be directly mapped between libcamera and V4L2 without\n+ * doing any conversion to the ControlValue.\n+ */\n+static std::unordered_map controlsToV4L2 = {\n+ { controls::AUTO_GAIN, V4L2_CID_AUTOGAIN },\n+ { controls::AF_ENABLED, V4L2_CID_FOCUS_AUTO },\n+ { controls::AF_START, V4L2_CID_AUTO_FOCUS_START },\n+ { controls::AF_STOP, V4L2_CID_AUTO_FOCUS_STOP },\n+ { controls::AE_ENABLE, V4L2_CID_EXPOSURE_AUTO },\n+ { controls::EXPOSURE_VALUE, V4L2_CID_EXPOSURE },\n+ { controls::DIGITAL_GAIN, V4L2_CID_GAIN },\n+ { controls::ANALOGUE_GAIN, V4L2_CID_ANALOGUE_GAIN },\n+ { controls::AF_STATE, V4L2_CID_AUTO_FOCUS_STATUS },\n+};\n+\n+/*\n+ * Convert from a libcamera control to a V4L2 control, optionally\nalso convert a\n+ * set of ControlValues.\n+ */\n+bool simpleControlToV4L2(unsigned int control,\n+ unsigned int *v4l2_control,\n+ const ControlValue *control_values,\n+ ControlValue *v4l2_values,\n+ size_t num_values)\n+{\n+ // Convert controls\n+ if (v4l2_control) {\n+ auto it = controlsToV4L2.find(control);\n+ if (it == controlsToV4L2.end())\n+ return false;\n+\n+ *v4l2_control = it->second;\n+ }\n+\n+ // Convert values\n+ if (num_values == 0)\n+ return true;\n+\n+ switch (control) {\n+ case controls::AE_ENABLE:\n+ for (size_t i = 0; i < num_values; ++i)\n+ v4l2_values[i] =\nControlValue((int32_t)(control_values[i].get() ?\nV4L2_EXPOSURE_AUTO : V4L2_EXPOSURE_MANUAL));\n+ return true;\n+ case controls::EXPOSURE_VALUE:\n+ case controls::DIGITAL_GAIN:\n+ case controls::ANALOGUE_GAIN:\n+ for (size_t i = 0; i < num_values; ++i)\n+ v4l2_values[i] =\nControlValue((int32_t)control_values[i].get());\n+ return true;\n+ // Read only\n+ case controls::AF_STATE:\n+ return false;\n+ default:\n+ for (size_t i = 0; i < num_values; ++i)\n+ v4l2_values[i] = control_values[i];\n+ return true;\n+ }\n+\n+}\n+\n+static std::unordered_map controlsFromV4L2;\n+\n+/*\n+ * Convert from a V4L2 control to a libcamera control, optionally\nalso convert a\n+ * set of ControlValues.\n+ */\n+bool simpleControlFromV4L2(unsigned int v4l2_control,\n+ unsigned int *control,\n+ const ControlValue *v4l2_values,\n+ ControlValue *control_values,\n+ size_t num_values)\n+{\n+ // Initialize the inverse of controlsToV4L2\n+ if (controlsFromV4L2.empty()) {\n+ for (const auto &v : controlsToV4L2) {\n+ controlsFromV4L2[v.second] = v.first;\n+ }\n+ }\n+\n+ // Convert control\n+ if (control) {\n+ auto it = controlsFromV4L2.find(v4l2_control);\n+ if (it == controlsFromV4L2.end())\n+ return false;\n+\n+ *control = it->second;\n+ }\n+\n+ // Convert values\n+ if (num_values == 0)\n+ return true;\n+\n+ switch (v4l2_control) {\n+ case V4L2_CID_EXPOSURE_AUTO:\n+ for (size_t i = 0; i < num_values; ++i)\n+ control_values[i] =\nControlValue(v4l2_values[i].get() == V4L2_EXPOSURE_AUTO);\n+ return true;\n+ case V4L2_CID_EXPOSURE:\n+ case V4L2_CID_GAIN:\n+ case V4L2_CID_ANALOGUE_GAIN:\n+ for (size_t i = 0; i < num_values; ++i)\n+ control_values[i] =\nControlValue((float)v4l2_values[i].get());\n+ return true;\n+ case V4L2_CID_AUTO_FOCUS_STATUS:\n+ for (size_t i = 0; i < num_values; ++i) {\n+ switch (v4l2_values[i].get()) {\n+ case V4L2_AUTO_FOCUS_STATUS_IDLE:\n+ control_values[i] =\nControlValue((int32_t)controls::draft::AfStateInactive);\n+ break;\n+ case V4L2_AUTO_FOCUS_STATUS_BUSY:\n+ control_values[i] =\nControlValue((int32_t)controls::draft::AfStateActiveScan);\n+ break;\n+ case V4L2_AUTO_FOCUS_STATUS_REACHED:\n+ control_values[i] =\nControlValue((int32_t)controls::draft::AfStatePassiveFocused);\n+ break;\n+ case V4L2_AUTO_FOCUS_STATUS_FAILED:\n+ control_values[i] =\nControlValue((int32_t)controls::draft::AfStatePassiveUnfocused);\n+ break;\n+ default:\n+ LOG(SimplePipeline, Error)\n+ << \"AUTO_FOCUS_STATUS has invalid value: \"\n+ << utils::hex(v4l2_values[i].get());\n+ /*TODO: Log Error*/\n+ return false;\n+ }\n+ }\n+ return true;\n+ default:\n+ for (size_t i = 0; i < num_values; ++i)\n+ control_values[i] = v4l2_values[i];\n+ return true;\n+ }\n+}\n+\n+/*\n+ * Convert a ControlInfoMap from V4L2 to libcamera. Converts both the control\n+ * identifiers as well as all values.\n+ */\n+ControlInfoMap simpleControlInfoFromV4L2(const ControlInfoMap &v4l2_info_map)\n+{\n+ ControlInfoMap::Map info_map;\n+\n+ for (const auto &pair : v4l2_info_map) {\n+ unsigned int v4l2_control = pair.first->id();\n+ const ControlInfo &v4l2_info = pair.second;\n+\n+ unsigned int control;\n+ ControlValue def;\n+ if (!simpleControlFromV4L2(v4l2_control, &control,\n&v4l2_info.def(), &def, 1))\n+ continue;\n+\n+ const ControlId *control_id = controls::controls.at(control);\n+\n+ // ControlInfo has either a list of values or a minimum and\n+ // maximum. This includes controls that have no values or are\n+ // booleans.\n+ ControlInfo info;\n+ if (v4l2_info.values().empty()) {\n+ ControlValue min, max;\n+ simpleControlFromV4L2(v4l2_control, nullptr,\n&v4l2_info.min(), &min, 1);\n+ simpleControlFromV4L2(v4l2_control, nullptr,\n&v4l2_info.max(), &max, 1);\n+ info = ControlInfo(std::move(min), std::move(max), std::move(def));\n+ } else {\n+ std::vector values;\n+ values.resize(v4l2_info.values().size());\n+ simpleControlFromV4L2(v4l2_control, nullptr,\nv4l2_info.values().data(), values.data(), values.size());\n+ info = ControlInfo(std::move(values), std::move(def));\n+ }\n+ info_map.emplace(control_id, std::move(info));\n+ }\n+\n+ return ControlInfoMap(std::move(info_map), controls::controls);\n+}\n+\n+/*\n+ * Convert a control list from libcamera to V4L2.\n+ */\n+ControlList simpleControlListToV4L2(const ControlList &controls)\n+{\n+ ControlList v4l2_controls;\n+ for (const auto &pair : controls) {\n+ unsigned int control = pair.first;\n+ const ControlValue &value = pair.second;\n+\n+ unsigned int v4l2_control;\n+ ControlValue v4l2_value;\n+ if (!simpleControlToV4L2(control, &v4l2_control, &value,\n&v4l2_value, 1)) {\n+ LOG(SimplePipeline, Warning)\n+ << \"Control \" << utils::hex(control)\n+ << \" does not have a V4L2 equivalent\";\n+ continue;\n+ }\n+\n+ v4l2_controls.set(v4l2_control, v4l2_value);\n+ }\n+ return v4l2_controls;\n+}\n+\n+/*\n+ * Convert a control list from V4L2 to libcamera.\n+ */\n+ControlList simpleControlListFromV4L2(const ControlList &v4l2_controls)\n+{\n+ ControlList controls;\n+ for (const auto &pair : v4l2_controls) {\n+ unsigned int v4l2_control = pair.first;\n+ const ControlValue &v4l2_value = pair.second;\n+\n+ unsigned int control;\n+ ControlValue value;\n+ if (simpleControlFromV4L2(v4l2_control, &control,\n&v4l2_value, &value, 1))\n+ controls.set(control, value);\n+ }\n+ return controls;\n+}\n+\n+} /* namespace libcamera */\ndiff --git a/src/libcamera/pipeline/simple/controls.h\nb/src/libcamera/pipeline/simple/controls.h\nnew file mode 100644\nindex 00000000..114c5fc2\n--- /dev/null\n+++ b/src/libcamera/pipeline/simple/controls.h\n@@ -0,0 +1,26 @@\n+#ifndef __LIBCAMERA_PIPELINE_SIMPLE_CONTROLS_H__\n+#define __LIBCAMERA_PIPELINE_SIMPLE_CONTROLS_H__\n+\n+#include \n+\n+namespace libcamera {\n+\n+bool simpleControlToV4L2(unsigned int control,\n+ unsigned int *v4l2_control,\n+ const ControlValue *control_values,\n+ ControlValue *v4l2_values,\n+ size_t num_values);\n+bool simpleControlFromV4L2(unsigned int v4l2_control,\n+ unsigned int *control,\n+ const ControlValue *v4l2_values,\n+ ControlValue *control_values,\n+ size_t num_values);\n+\n+ControlInfoMap simpleControlInfoFromV4L2(const ControlInfoMap &v4l2_info);\n+\n+ControlList simpleControlListToV4L2(const ControlList &controls);\n+ControlList simpleControlListFromV4L2(const ControlList &controls);\n+\n+} /* namespace libcamera */\n+\n+#endif /* __LIBCAMERA_PIPELINE_SIMPLE_CONTROLS_H__ */\ndiff --git a/src/libcamera/pipeline/simple/meson.build\nb/src/libcamera/pipeline/simple/meson.build\nindex 9c99b32f..0c60d65a 100644\n--- a/src/libcamera/pipeline/simple/meson.build\n+++ b/src/libcamera/pipeline/simple/meson.build\n@@ -1,6 +1,7 @@\n # SPDX-License-Identifier: CC0-1.0\n\n libcamera_sources += files([\n+ 'controls.cpp',\n 'converter.cpp',\n 'simple.cpp',\n ])\ndiff --git a/src/libcamera/pipeline/simple/simple.cpp\nb/src/libcamera/pipeline/simple/simple.cpp\nindex a597e27f..b0d4a62a 100644\n--- a/src/libcamera/pipeline/simple/simple.cpp\n+++ b/src/libcamera/pipeline/simple/simple.cpp\n@@ -37,6 +37,7 @@\n #include \"libcamera/internal/v4l2_videodevice.h\"\n\n #include \"converter.h\"\n+#include \"controls.h\"\n\n namespace libcamera {\n\n@@ -181,6 +182,7 @@ public:\n int setupLinks();\n int setupFormats(V4L2SubdeviceFormat *format,\n V4L2Subdevice::Whence whence);\n+ int setRequestControls(Request *request);\n void bufferReady(FrameBuffer *buffer);\n\n unsigned int streamIndex(const Stream *stream) const\n@@ -519,7 +521,8 @@ int SimpleCameraData::init()\n formats_[fmt] = &config;\n }\n\n- properties_ = sensor_->properties();\n+ properties_ = simpleControlListFromV4L2(sensor_->properties());\n+ controlInfo_ = simpleControlInfoFromV4L2(sensor_->controls());\n\n return 0;\n }\n@@ -624,6 +627,23 @@ int\nSimpleCameraData::setupFormats(V4L2SubdeviceFormat *format,\n return 0;\n }\n\n+int SimpleCameraData::setRequestControls(Request *request)\n+{\n+ SimplePipelineHandler *pipe = SimpleCameraData::pipe();\n+\n+ // Apply controls only to one entity. If there's a subdevice use that.\n+ V4L2Device *control_device = video_;\n+ for (const SimpleCameraData::Entity &e : entities_) {\n+ V4L2Subdevice *subdev = pipe->subdev(e.entity);\n+ if (subdev) {\n+ control_device = subdev;\n+ }\n+ }\n+\n+ ControlList controls = simpleControlListToV4L2(request->controls());\n+ return control_device->setControls(&controls);\n+}\n+\n void SimpleCameraData::bufferReady(FrameBuffer *buffer)\n {\n","prefixes":["libcamera-devel"]}