From patchwork Thu Mar 24 13:58:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Curtin X-Patchwork-Id: 15535 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 86E7FBD80A for ; Thu, 24 Mar 2022 13:58:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AE912604D5; Thu, 24 Mar 2022 14:58:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1648130291; bh=SG8MnUOMj3tVtciFc4EC8/3agrNf6VmtIpovFHaAAyk=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=dvC7CTOPPGnF5NDj75ld4dM7UDg93sC8MyqDuTbFnRBtam/Xt8/Hwe3niycMphEwR kQhVI7IXhvqZ5DyoweItXfaU5PU1VobtL0uS2rhAyEDcF9SPRh9++mXb9KhKpA3Z8A oiUexR03nZ71KS6gVxqdzscw5F9r5NcAJLBhg3O2J/jYRtJSbblZcKKLPuAsQ9hfNf 3/hF+GoKEt1mSWCKlrpHbzE0G0FUwDJpHCj4qW9gDujlI/DxKseE/ozYQZnSPRYvsd eQvgpP9uRieoP6COmiUh9u8ag1nR6Mm94SodcjOS32Na3dFzGWvfQ8l/+vlyhahyAg Y5cZI7dPXRKwQ== Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2289860397 for ; Thu, 24 Mar 2022 14:58:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="cFR7dcsn"; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648130288; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=BLmxiauL1NtN884DxqKPUud+ma+fTdy75GL58X/VFf8=; b=cFR7dcsnFvLxUT941H6Ea2XjJyNQL+7LwWdO7gkyA2PM77jVkL/O/lHuBUISoZHVqu2NW0 1Vh+X6EUG04MI7B30cb2q1bZMPaXPz6X7FofzUVmdX4fsPS2h8Ye8RWEKQWrywDGClSYPC G40nwFEL3cukgTB3lu1qnQQBYng4qHc= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-220-TRr2rHoLMseSviHgUjY1EQ-1; Thu, 24 Mar 2022 09:58:07 -0400 X-MC-Unique: TRr2rHoLMseSviHgUjY1EQ-1 Received: by mail-wm1-f72.google.com with SMTP id v67-20020a1cac46000000b00383e71bb26fso1596291wme.1 for ; Thu, 24 Mar 2022 06:58:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=BLmxiauL1NtN884DxqKPUud+ma+fTdy75GL58X/VFf8=; b=bePMyBjDv/YYi7HqUdm5jFBaYQbMO2kIcZ8GDDcX3HZ3MftXiia4YfgN39VAxFgmm3 Nh7jsBGhf0dvjEIE4txyfzuuYTNaV3L901iz+/vvp1/CLw2CFzRHJGkT2+gO1dvGJifK czMHk9p/qGcyIx6i+/jdX5z5E+9Ud19082J0EygEN3lywSy03/aF3tmUW2Htc2bLPKh4 h8331yClhZzAbsKpEGhslST8XLv+K9rdMNkTsFBr6RwzxRo0OYayoVtQzlpRKWMR4WIb bA/WnzMrvaSgaYcb3HdyABuIujCqmROHPM9Xh2jHX5I6UWNPKzoenExFmgKZHVZLbgfG 5lsA== X-Gm-Message-State: AOAM533y4y3CB4r6h0HYJMeyoQgb296DLwNneeuvzSCI11NdjopsJFLD kwBRhS3n70Cvv8C8sl39auYAMZhg62tmYWjE4T5G4kc4SeFxijax92lFnLN+CGCrfBnqrMtgqAv rmsfp9qH92DJyEzq2ugt7RTjOMGC188hJeTnj5k6vVTnryh2DBHTUA85sDninX5E0ZMFa5mxwS5 Z+xl6t0nsP X-Received: by 2002:a05:600c:1e02:b0:38c:a5c2:243 with SMTP id ay2-20020a05600c1e0200b0038ca5c20243mr14471917wmb.150.1648130286090; Thu, 24 Mar 2022 06:58:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw4mHDDYsu4Mp9vl3EDA12tABGqe9s96wO1FRtSuejhNftjGOJQbvV7WMq/i/G0VOCT4KKBqw== X-Received: by 2002:a05:600c:1e02:b0:38c:a5c2:243 with SMTP id ay2-20020a05600c1e0200b0038ca5c20243mr14471897wmb.150.1648130285743; Thu, 24 Mar 2022 06:58:05 -0700 (PDT) Received: from p1.Home ([2001:8a0:6724:4500:a69c:e66f:828e:b340]) by smtp.gmail.com with ESMTPSA id o11-20020adf9d4b000000b001f0077ea337sm2764054wre.22.2022.03.24.06.58.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Mar 2022 06:58:05 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Mar 2022 13:58:02 +0000 Message-Id: <20220324135802.35230-1-ecurtin@redhat.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=ecurtin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [libcamera-devel] [PATCH v3] cam: sdl_sink: Add SDL sink X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Eric Curtin via libcamera-devel From: Eric Curtin Reply-To: Eric Curtin Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds more portability to existing cam sinks. You can pass a YUYV camera buffer for example and SDL will handle the pixel buffer conversion, although SDL does not support decompression for pixelformats like MJPEG. This allows cam reference implementation to display images on VMs, Mac M1, Raspberry Pi, etc. This also enables cam reference implementation, to run as a desktop application in wayland or x11. SDL also has support for Android and ChromeOS which I have not tested. Also tested on simpledrm raspberry pi 4 framebuffer successfully where existing kms sink did not work. Can also be used as kmsdrm sink. Signed-off-by: Eric Curtin --- Changes in v2: - Remove hardcoded pixel format from SDL_CreateTexture call Changes in v3: - Drop blank line - The contents of the if..endif are indented - Split configure function into start and configure - Add SDL_DestroyRenderer - Remove assign and test in the same statement --- src/cam/camera_session.cpp | 8 +++ src/cam/main.cpp | 5 ++ src/cam/main.h | 1 + src/cam/meson.build | 10 +++ src/cam/sdl_sink.cpp | 133 +++++++++++++++++++++++++++++++++++++ src/cam/sdl_sink.h | 41 ++++++++++++ 6 files changed, 198 insertions(+) create mode 100644 src/cam/sdl_sink.cpp create mode 100644 src/cam/sdl_sink.h diff --git a/src/cam/camera_session.cpp b/src/cam/camera_session.cpp index 0428b538..30162dbd 100644 --- a/src/cam/camera_session.cpp +++ b/src/cam/camera_session.cpp @@ -19,6 +19,9 @@ #ifdef HAVE_KMS #include "kms_sink.h" #endif +#ifdef HAVE_SDL +#include "sdl_sink.h" +#endif #include "main.h" #include "stream_options.h" @@ -187,6 +190,11 @@ int CameraSession::start() sink_ = std::make_unique(options_[OptDisplay].toString()); #endif +#ifdef HAVE_SDL + if (options_.isSet(OptSDL)) + sink_ = std::make_unique(); +#endif + if (options_.isSet(OptFile)) { if (!options_[OptFile].toString().empty()) sink_ = std::make_unique(streamNames_, diff --git a/src/cam/main.cpp b/src/cam/main.cpp index c7f664b9..1d62a64a 100644 --- a/src/cam/main.cpp +++ b/src/cam/main.cpp @@ -137,6 +137,11 @@ int CamApp::parseOptions(int argc, char *argv[]) "Display viewfinder through DRM/KMS on specified connector", "display", ArgumentOptional, "connector", false, OptCamera); +#endif +#ifdef HAVE_SDL + parser.addOption(OptSDL, OptionNone, + "Display viewfinder through SDL", + "sdl", ArgumentNone, "", false, OptCamera); #endif parser.addOption(OptFile, OptionString, "Write captured frames to disk\n" diff --git a/src/cam/main.h b/src/cam/main.h index 62f7bbc9..a64f95a0 100644 --- a/src/cam/main.h +++ b/src/cam/main.h @@ -11,6 +11,7 @@ enum { OptCamera = 'c', OptCapture = 'C', OptDisplay = 'D', + OptSDL = 'S', OptFile = 'F', OptHelp = 'h', OptInfo = 'I', diff --git a/src/cam/meson.build b/src/cam/meson.build index e8e2ae57..bd536c5b 100644 --- a/src/cam/meson.build +++ b/src/cam/meson.build @@ -32,11 +32,21 @@ cam_sources += files([ ]) endif +libsdl2 = dependency('SDL2', required : false) + +if libsdl2.found() + cam_cpp_args += [ '-DHAVE_SDL' ] + cam_sources += files([ + 'sdl_sink.cpp' + ]) +endif + cam = executable('cam', cam_sources, dependencies : [ libatomic, libcamera_public, libdrm, + libsdl2, libevent, ], cpp_args : cam_cpp_args, diff --git a/src/cam/sdl_sink.cpp b/src/cam/sdl_sink.cpp new file mode 100644 index 00000000..97f601b1 --- /dev/null +++ b/src/cam/sdl_sink.cpp @@ -0,0 +1,133 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * sdl_sink.cpp - SDL Sink + */ + +#include "sdl_sink.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "image.h" + +using namespace libcamera; + +SDLSink::SDLSink() + : sdlRenderer_(0) +{ + memset(&sdlRect_, 0, sizeof(sdlRect_)); +} + +SDLSink::~SDLSink() +{ + if (sdlRenderer_) + SDL_DestroyRenderer(sdlRenderer_); + SDL_Quit(); +} + +int SDLSink::configure(const libcamera::CameraConfiguration &cfg) +{ + int ret = FrameSink::configure(cfg); + if (ret < 0) + return ret; + + const libcamera::StreamConfiguration &sCfg = cfg.at(0); + pf = (SDL_PixelFormatEnum)sCfg.pixelFormat.fourcc(); + if (pf == SDL_DEFINE_PIXELFOURCC('Y', 'U', 'Y', 'V')) { + pf = SDL_PIXELFORMAT_YUY2; + } else if (int ne = strcmp(SDL_GetPixelFormatName(pf), "SDL_PIXELFORMAT_UNKNOWN"); !ne) { + std::cerr << "SDL_GetPixelFormatName error - exiting: SDL_PIXELFORMAT_UNKNOWN, no " << sCfg.pixelFormat.toString() << " support\n"; + return -EINVAL; + } + + sdlRect_.w = sCfg.size.width; + sdlRect_.h = sCfg.size.height; + + return 0; +} + +int SDLSink::start() +{ + int ret = SDL_Init(SDL_INIT_VIDEO); + if (ret) { + std::cerr << "SDL_Init error - exiting: " << SDL_GetError() << std::endl; + return ret; + } + + sdlScreen_ = SDL_CreateWindow("", SDL_WINDOWPOS_UNDEFINED, + SDL_WINDOWPOS_UNDEFINED, sdlRect_.w, + sdlRect_.h, SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE); + if (!sdlScreen_) { + std::cerr << "SDL_CreateWindow error - exiting: " << SDL_GetError() << std::endl; + return -EINVAL; + } + + sdlRenderer_ = SDL_CreateRenderer( + sdlScreen_, -1, 0); + if (!sdlRenderer_) { + std::cerr << "SDL_CreateRenderer error - exiting: " << SDL_GetError() << std::endl; + return -EINVAL; + } + + SDL_RenderSetLogicalSize(sdlRenderer_, sdlRect_.w, + sdlRect_.h); + + sdlTexture_ = + SDL_CreateTexture(sdlRenderer_, pf, + SDL_TEXTUREACCESS_STREAMING, sdlRect_.w, + sdlRect_.h); + + return 0; +} + +void SDLSink::mapBuffer(FrameBuffer *buffer) +{ + std::unique_ptr image = + Image::fromFrameBuffer(buffer, Image::MapMode::ReadOnly); + assert(image != nullptr); + + mappedBuffers_[buffer] = std::move(image); +} + +bool SDLSink::processRequest(Request *request) +{ + for (auto [stream, buffer] : request->buffers()) + writeBuffer(buffer); + + for (SDL_Event e; SDL_PollEvent(&e);) { + if (e.type == SDL_QUIT) { // click close icon then quit + kill(getpid(), SIGINT); + } + } + + return true; +} + +void SDLSink::writeBuffer(FrameBuffer *buffer) +{ + Image *image = mappedBuffers_[buffer].get(); + + for (unsigned int i = 0; i < buffer->planes().size(); ++i) { + const FrameMetadata::Plane &meta = buffer->metadata().planes()[i]; + + Span data = image->data(i); + if (meta.bytesused > data.size()) + std::cerr << "payload size " << meta.bytesused + << " larger than plane size " << data.size() + << std::endl; + + SDL_UpdateTexture(sdlTexture_, &sdlRect_, data.data(), sdlRect_.w * 2); + SDL_RenderClear(sdlRenderer_); + SDL_RenderCopy(sdlRenderer_, sdlTexture_, NULL, NULL); + SDL_RenderPresent(sdlRenderer_); + } +} diff --git a/src/cam/sdl_sink.h b/src/cam/sdl_sink.h new file mode 100644 index 00000000..f5f8982a --- /dev/null +++ b/src/cam/sdl_sink.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * sdl_sink.h - SDL Sink + */ + +#pragma once + +#include +#include +#include + +#include + +#include + +#include "frame_sink.h" + +class Image; + +class SDLSink : public FrameSink +{ +public: + SDLSink(); + ~SDLSink(); + + int configure(const libcamera::CameraConfiguration &cfg) override; + int start() override; + void mapBuffer(libcamera::FrameBuffer *buffer) override; + + bool processRequest(libcamera::Request *request) override; + +private: + void writeBuffer(libcamera::FrameBuffer *buffer); + + std::map> mappedBuffers_; + SDL_Window *sdlScreen_; + SDL_Renderer *sdlRenderer_; + SDL_Texture *sdlTexture_; + SDL_Rect sdlRect_; + SDL_PixelFormatEnum pf; +};