From patchwork Wed Jul 23 14:07:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Giacomo Cappellini X-Patchwork-Id: 23902 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7FF97C3237 for ; Wed, 23 Jul 2025 14:08:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 380BC69057; Wed, 23 Jul 2025 16:08:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="lOmIVvxu"; dkim-atps=neutral Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CB0AE6904F for ; Wed, 23 Jul 2025 16:08:11 +0200 (CEST) Received: by mail-ed1-x529.google.com with SMTP id 4fb4d7f45d1cf-60c93c23b08so12763968a12.3 for ; Wed, 23 Jul 2025 07:08:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1753279691; x=1753884491; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=ouJizimE/r8DhMQJqObV/l0/z8FA1ewKVVko2ukHIdk=; b=lOmIVvxuiYSv6Jmag05qeFKMtgPHVJ14lPpvNKAPPkfc55mwDNCB3jdBFN+G8IeKYr POTq/NhKNpkLUFFIMJSVDH6t5HqlzGM4ETUVcfoDbGz27PPrVnfXA6lvMX6uF9DOBnz6 EGljC8YbCUwNnvO4rSA6RQ8SOtRh8hUUCXs2hpXsNscHvxD0nyn7Xz81OWRt0/HzMCVw PuWRZzuSfxoOb6/GlgQUpiNY+wtHEVmndA3yzX8nSl7jVmdgNqE06gUDHKymeorzsM8C pagyYPvuQ0y0XopeQmW8m2SftJCJdyV0YW2pdHpXxgixs8KVwew8Tb73mRn2tNfcuLlf CTgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753279691; x=1753884491; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ouJizimE/r8DhMQJqObV/l0/z8FA1ewKVVko2ukHIdk=; b=FYAAdNwp+6iQmIfuav95466qCWg5E/3gJFNe/W7aS9yzSU/cwgKfCC7Dnhy/GM/wWN 6WUOqjuLaVJwm0TeTSnJu6ZZaNlIhrWCRurQ43T5k7oYPl27YlobNQ7bMijuiwlz5nnn Mz/Oc3kbaqepyVAZFEwE4eQaEP55ZDfbo01sdnnCZJJvCl86DSoscXfZaYtGuvAYYup6 BFPPmI/dl+lByfviecScwPmBawOJvwjA4Lp6asw6cEbbXm3wAvWqh+Bm9LHNbpBDNWQt zcJdW3abS0nwYs3gek3Si1i8B4jhh8q3dTl695q2GnCyIxQxzw5f7MyjH3VE9M5MMrcy 2B2Q== X-Gm-Message-State: AOJu0YyCoTlwRw+Wu5pzSZE7h3e9XavRvUeJT5HaSje7qD9lnq8i1tNz xXnwfgk3DE8bQ05BgoT/2LxlTvqttT7/9MuJBhVPg1HVp/u/hB2zFS2i2elEZB0O X-Gm-Gg: ASbGncupL7xvoWY1WYNZwOpguYWjMTnmE3YuQi2bOtWTptzKDpfuW934p+39E3FLjnq 34tC3xdui10BduraacyU5ZaI1DHWIKsMLeqaTtSfrwuTRyC6NfkLVgMHlYV0FMKZ35RkIkvCJRc 4gQ5XG6NazcuLRujUSjD28gY+dlgJ9KE8kPYbjowMFwdUzy70vX6XovmGVTQYDLAGanh9hphXn5 Uca9hDi0Gx4J8fXmS1Mj2NuxLJ3tCg+ESR4BbT3INXKvUd4TcvKJlyYZiEiG2jo3V3fNJOduEF/ WAn8i1XEv40iTa23GNyVTomn2sc67z/DHZBvfyuB7qx5vU2NzezfgptVYLQZXO2YVbjwB4X0JHY RzxXBdRXZzbeleyIs+E0qfq4AOFhXJV4FGj99sHejtnL4u5ohRX46migyBfzSrlvnUMr88mKqp4 oM179pEy0NkQmrCFeOgC/WoQ== X-Google-Smtp-Source: AGHT+IHV1rjZUQr39BVje9lb9VbZ46DrOFMGd6fcorl0Bsmff3aZ1N2SUKK/B/tpsq71DDKwv9LrEg== X-Received: by 2002:a05:6402:270d:b0:614:9a3d:2e4c with SMTP id 4fb4d7f45d1cf-6149b5a9e15mr2887573a12.30.1753279690951; Wed, 23 Jul 2025 07:08:10 -0700 (PDT) Received: from jasus.ad.servtec.it (host-95-251-230-143.retail.telecomitalia.it. [95.251.230.143]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-612c903fbd3sm8523859a12.36.2025.07.23.07.08.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Jul 2025 07:08:10 -0700 (PDT) From: Giacomo Cappellini To: libcamera-devel@lists.libcamera.org Cc: Giacomo Cappellini Subject: [PATCH v3] gstreamer: Add support for Orientation Date: Wed, 23 Jul 2025 16:07:55 +0200 Message-ID: <20250723140801.648652-1-giacomo.cappellini.87@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" libcamera allows to control the images orientation through the CameraConfiguration::orientation field, expose a GST_PARAM_MUTABLE_READY parameter of type GstVideoOrientationMethod in GstLibcameraSrc to control it. Parameter is mapped internally to libcamera::Orientation via new gstlibcamera-utils functions: - gst_video_orientation_to_libcamera_orientation - libcamera_orientation_to_gst_video_orientation Update CameraConfiguration::Adjusted case in negotiation to warn about changes in StreamConfiguration and SensorConfiguration, as well as the new Orientation parameter. Signed-off-by: Giacomo Cappellini --- src/gstreamer/gstlibcamera-utils.cpp | 39 ++++++++ src/gstreamer/gstlibcamera-utils.h | 4 + src/gstreamer/gstlibcamerasrc.cpp | 132 +++++++++++++++++++++++++-- 3 files changed, 166 insertions(+), 9 deletions(-) diff --git a/src/gstreamer/gstlibcamera-utils.cpp b/src/gstreamer/gstlibcamera-utils.cpp index a548b0c1..95d3813e 100644 --- a/src/gstreamer/gstlibcamera-utils.cpp +++ b/src/gstreamer/gstlibcamera-utils.cpp @@ -10,6 +10,9 @@ #include #include +#include + +#include using namespace libcamera; @@ -659,3 +662,39 @@ gst_libcamera_get_camera_manager(int &ret) return cm; } + +static const struct { + Orientation orientation; + GstVideoOrientationMethod method; +} orientation_map[]{ + { Orientation::Rotate0, GST_VIDEO_ORIENTATION_IDENTITY }, + { Orientation::Rotate90, GST_VIDEO_ORIENTATION_90R }, + { Orientation::Rotate180, GST_VIDEO_ORIENTATION_180 }, + { Orientation::Rotate270, GST_VIDEO_ORIENTATION_90L }, + { Orientation::Rotate0Mirror, GST_VIDEO_ORIENTATION_HORIZ }, + { Orientation::Rotate180Mirror, GST_VIDEO_ORIENTATION_VERT }, + { Orientation::Rotate90Mirror, GST_VIDEO_ORIENTATION_UL_LR }, + { Orientation::Rotate270Mirror, GST_VIDEO_ORIENTATION_UR_LL }, +}; + +Orientation +gst_video_orientation_to_libcamera_orientation(GstVideoOrientationMethod method) +{ + for (auto &b : orientation_map) { + if (b.method == method) + return b.orientation; + } + + return Orientation::Rotate0; +} + +GstVideoOrientationMethod +libcamera_orientation_to_gst_video_orientation(Orientation orientation) +{ + for (auto &a : orientation_map) { + if (a.orientation == orientation) + return a.method; + } + + return GST_VIDEO_ORIENTATION_IDENTITY; +} diff --git a/src/gstreamer/gstlibcamera-utils.h b/src/gstreamer/gstlibcamera-utils.h index 5f4e8a0f..bbbd33db 100644 --- a/src/gstreamer/gstlibcamera-utils.h +++ b/src/gstreamer/gstlibcamera-utils.h @@ -10,6 +10,7 @@ #include #include +#include #include #include @@ -92,3 +93,6 @@ public: private: GRecMutex *mutex_; }; + +libcamera::Orientation gst_video_orientation_to_libcamera_orientation(GstVideoOrientationMethod method); +GstVideoOrientationMethod libcamera_orientation_to_gst_video_orientation(libcamera::Orientation orientation); diff --git a/src/gstreamer/gstlibcamerasrc.cpp b/src/gstreamer/gstlibcamerasrc.cpp index 3aca4eed..5f483701 100644 --- a/src/gstreamer/gstlibcamerasrc.cpp +++ b/src/gstreamer/gstlibcamerasrc.cpp @@ -29,6 +29,7 @@ #include #include +#include #include #include #include @@ -38,6 +39,7 @@ #include #include +#include #include "gstlibcamera-controls.h" #include "gstlibcamera-utils.h" @@ -146,6 +148,7 @@ struct _GstLibcameraSrc { GstTask *task; gchar *camera_name; + GstVideoOrientationMethod orientation; std::atomic pending_eos; @@ -157,6 +160,7 @@ struct _GstLibcameraSrc { enum { PROP_0, PROP_CAMERA_NAME, + PROP_ORIENTATION, PROP_LAST }; @@ -166,8 +170,8 @@ static void gst_libcamera_src_child_proxy_init(gpointer g_iface, G_DEFINE_TYPE_WITH_CODE(GstLibcameraSrc, gst_libcamera_src, GST_TYPE_ELEMENT, G_IMPLEMENT_INTERFACE(GST_TYPE_CHILD_PROXY, gst_libcamera_src_child_proxy_init) - GST_DEBUG_CATEGORY_INIT(source_debug, "libcamerasrc", 0, - "libcamera Source")) + GST_DEBUG_CATEGORY_INIT(source_debug, "libcamerasrc", 0, + "libcamera Source")) #define TEMPLATE_CAPS GST_STATIC_CAPS("video/x-raw; image/jpeg; video/x-bayer") @@ -225,8 +229,7 @@ int GstLibcameraSrcState::queueRequest() return 0; } -void -GstLibcameraSrcState::requestCompleted(Request *request) +void GstLibcameraSrcState::requestCompleted(Request *request) { GST_DEBUG_OBJECT(src_, "buffers are ready"); @@ -616,9 +619,109 @@ gst_libcamera_src_negotiate(GstLibcameraSrc *self) gst_libcamera_get_framerate_from_caps(caps, element_caps); } + /* Set orientation control. */ + state->config_->orientation = gst_video_orientation_to_libcamera_orientation(self->orientation); + + /* Save original configuration for comparison after validation */ + std::vector orig_stream_cfgs; + for (gsize i = 0; i < state->config_->size(); i++) + orig_stream_cfgs.push_back(state->config_->at(i)); + std::optional orig_sensor_cfg = state->config_->sensorConfig; + Orientation orig_orientation = state->config_->orientation; + /* Validate the configuration. */ - if (state->config_->validate() == CameraConfiguration::Invalid) + switch (state->config_->validate()) { + case CameraConfiguration::Valid: + GST_DEBUG_OBJECT(self, "Camera configuration is valid"); + break; + case CameraConfiguration::Adjusted: { + bool warned = false; + // Warn if number of StreamConfigurations changed + if (orig_stream_cfgs.size() != state->config_->size()) { + GST_WARNING_OBJECT(self, "Number of StreamConfiguration elements changed: requested=%zu, actual=%zu", + orig_stream_cfgs.size(), state->config_->size()); + warned = true; + } + // Warn about changes in each StreamConfiguration + // TODO implement diffing in StreamConfiguration + for (gsize i = 0; i < std::min(orig_stream_cfgs.size(), state->config_->size()); i++) { + if (orig_stream_cfgs[i].toString() != state->config_->at(i).toString()) { + GST_WARNING_OBJECT(self, "StreamConfiguration %zu changed: %s -> %s", + i, orig_stream_cfgs[i].toString().c_str(), + state->config_->at(i).toString().c_str()); + warned = true; + } + } + // Warn about SensorConfiguration changes + // TODO implement diffing in SensorConfiguration + if (orig_sensor_cfg.has_value() || state->config_->sensorConfig.has_value()) { + const SensorConfiguration *orig = orig_sensor_cfg.has_value() ? &orig_sensor_cfg.value() : nullptr; + const SensorConfiguration *curr = state->config_->sensorConfig.has_value() ? &state->config_->sensorConfig.value() : nullptr; + bool sensor_changed = false; + std::ostringstream diff; + if ((orig == nullptr) != (curr == nullptr)) { + diff << "SensorConfiguration presence changed: " + << (orig ? "was present" : "was absent") + << " -> " + << (curr ? "present" : "absent"); + sensor_changed = true; + } else if (orig && curr) { + if (orig->bitDepth != curr->bitDepth) { + diff << "bitDepth: " << orig->bitDepth << " -> " << curr->bitDepth << "; "; + sensor_changed = true; + } + if (orig->analogCrop != curr->analogCrop) { + diff << "analogCrop: " << orig->analogCrop.toString() << " -> " << curr->analogCrop.toString() << "; "; + sensor_changed = true; + } + if (orig->binning.binX != curr->binning.binX || + orig->binning.binY != curr->binning.binY) { + diff << "binning: (" << orig->binning.binX << "," << orig->binning.binY << ") -> (" + << curr->binning.binX << "," << curr->binning.binY << "); "; + sensor_changed = true; + } + if (orig->skipping.xOddInc != curr->skipping.xOddInc || + orig->skipping.xEvenInc != curr->skipping.xEvenInc || + orig->skipping.yOddInc != curr->skipping.yOddInc || + orig->skipping.yEvenInc != curr->skipping.yEvenInc) { + diff << "skipping: (" + << orig->skipping.xOddInc << "," << orig->skipping.xEvenInc << "," + << orig->skipping.yOddInc << "," << orig->skipping.yEvenInc << ") -> (" + << curr->skipping.xOddInc << "," << curr->skipping.xEvenInc << "," + << curr->skipping.yOddInc << "," << curr->skipping.yEvenInc << "); "; + sensor_changed = true; + } + if (orig->outputSize != curr->outputSize) { + diff << "outputSize: " << orig->outputSize.toString() << " -> " << curr->outputSize.toString() << "; "; + sensor_changed = true; + } + } + if (sensor_changed) { + GST_WARNING_OBJECT(self, "SensorConfiguration changed: %s", diff.str().c_str()); + warned = true; + } + } + // Warn about orientation change + if (orig_orientation != state->config_->orientation) { + GEnumClass *enum_class = (GEnumClass *)g_type_class_ref(GST_TYPE_VIDEO_ORIENTATION_METHOD); + const char *orig_orientation_str = g_enum_get_value(enum_class, libcamera_orientation_to_gst_video_orientation(orig_orientation))->value_nick; + const char *new_orientation_str = g_enum_get_value(enum_class, libcamera_orientation_to_gst_video_orientation(state->config_->orientation))->value_nick; + GST_WARNING_OBJECT(self, "Orientation changed: %s -> %s", orig_orientation_str, new_orientation_str); + warned = true; + } + if (!warned) { + GST_DEBUG_OBJECT(self, "Camera configuration adjusted, but no significant changes detected."); + } + // Update Gst orientation property to match adjusted config + self->orientation = libcamera_orientation_to_gst_video_orientation(state->config_->orientation); + break; + } + case CameraConfiguration::Invalid: + GST_ELEMENT_ERROR(self, RESOURCE, SETTINGS, + ("Camera configuration is not supported"), + ("CameraConfiguration::validate() returned Invalid")); return false; + } int ret = state->cam_->configure(state->config_.get()); if (ret) { @@ -926,6 +1029,9 @@ gst_libcamera_src_set_property(GObject *object, guint prop_id, g_free(self->camera_name); self->camera_name = g_value_dup_string(value); break; + case PROP_ORIENTATION: + self->orientation = (GstVideoOrientationMethod)g_value_get_enum(value); + break; default: if (!state->controls_.setProperty(prop_id - PROP_LAST, value, pspec)) G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); @@ -945,6 +1051,9 @@ gst_libcamera_src_get_property(GObject *object, guint prop_id, GValue *value, case PROP_CAMERA_NAME: g_value_set_string(value, self->camera_name); break; + case PROP_ORIENTATION: + g_value_set_enum(value, (gint)self->orientation); + break; default: if (!state->controls_.getProperty(prop_id - PROP_LAST, value, pspec)) G_OBJECT_WARN_INVALID_PROPERTY_ID(object, prop_id, pspec); @@ -1148,12 +1257,17 @@ gst_libcamera_src_class_init(GstLibcameraSrcClass *klass) GParamSpec *spec = g_param_spec_string("camera-name", "Camera Name", "Select by name which camera to use.", nullptr, - (GParamFlags)(GST_PARAM_MUTABLE_READY - | G_PARAM_CONSTRUCT - | G_PARAM_READWRITE - | G_PARAM_STATIC_STRINGS)); + (GParamFlags)(GST_PARAM_MUTABLE_READY | G_PARAM_CONSTRUCT | G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)); g_object_class_install_property(object_class, PROP_CAMERA_NAME, spec); + /* Register the orientation enum type. */ + spec = g_param_spec_enum("orientation", "Orientation", + "Select the orientation of the camera.", + GST_TYPE_VIDEO_ORIENTATION_METHOD, + GST_VIDEO_ORIENTATION_IDENTITY, + (GParamFlags)(GST_PARAM_MUTABLE_READY | G_PARAM_CONSTRUCT | G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)); + g_object_class_install_property(object_class, PROP_ORIENTATION, spec); + GstCameraControls::installProperties(object_class, PROP_LAST); }