From patchwork Tue Mar 29 11:17:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Curtin X-Patchwork-Id: 15580 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BBE64C0F1B for ; Tue, 29 Mar 2022 11:17:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0077465632; Tue, 29 Mar 2022 13:17:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1648552656; bh=ylE6KRzyGbxBVszbgevvSzwM0kPna1RogbZ/qxptmdo=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=Jfnp8S5MrXKcnpnDkD3taH/ydZLyc5mvgKfglxhZSvCnVqkMzJ7pj+g1R1W1NDylT 7Mu2UdyNCNjSWNFha7n8KVbhxv3voAR0WKAUBoVnpexnCBSh7/kuK7FkuXEIhke9Ik hyYVQls2Om/gzSefgo0eWCZMhlbhnl3+XUdlIFWK0X5H4504ikZagW/G6z+i/nqoxq 6Q0UURInTJ5/6T8NE9xnF8LIUqzVlzbkqeBs6jUHoxSueKl2etiei6XgFvszregudP 9SDVWAHkC+UL8M2W4kXtr79upqkPiGwBEIe0gvi0PzMSW/Uuwsar4d/YgRNZ+qGmFJ 1+h+gBkEcLlFg== Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 32A6F60135 for ; Tue, 29 Mar 2022 13:17:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="hdB7Gsmv"; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648552652; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=2xxcOCFy8kVVunHVssTOocKJpmwLLDNxnHbPyEFQAVA=; b=hdB7GsmvAdHmlFmB26jG9anIjw/EwOscmaGqzL4YfzRJKHTmLMjzscfuvdG135BApBvx2C ZRf4ccmppTSXyUdwFs09SJSJZjVVtbQdXHnNKFyQRd/NPCxQ4k04aPMmdTkC4dCPfjlv0v +twowyubGh5Eg6ZJQeiD+1j5Wf3XEwU= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-589-RrgTNtytM8C1IZ-l9rJGFQ-1; Tue, 29 Mar 2022 07:17:31 -0400 X-MC-Unique: RrgTNtytM8C1IZ-l9rJGFQ-1 Received: by mail-wr1-f70.google.com with SMTP id h11-20020a5d430b000000b001f01a35a86fso4924367wrq.4 for ; Tue, 29 Mar 2022 04:17:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=2xxcOCFy8kVVunHVssTOocKJpmwLLDNxnHbPyEFQAVA=; b=yQxLiQBw6SGWusNPwEpVUl0eXyRrQS2FXCeqhvWRVzLuNNnUhEIEWZihPWWX2WnZiE 7DQLey9shF2yz4L7Mtt38K4csC7PkC0/ACp0uWACxG78CB6mXYBrUpfz3ZBtT5YGy5MC uUMoudo5ZQ+M39ZU2MmWswvgjf8BJq/fL2BZacvmyoSObbyVDgk5ONTjOnvbkM1gwato gJqflkKmXPJDKD/HLTNVqplaXX+bzuFgP+rMaJXMQlGwI1ks+UECgGw/Qhsj1FT8Ep1F 5xzMC70aAadVg9dXSDuGKteqwva58jFKZa1mz0MrniAkwItgelVFfwxGBEYZYUU/uVBX 256Q== X-Gm-Message-State: AOAM531ZuRYzbsAdUV6sNctV0hzQ3fNYdhISuvLjRxgkvrerp/Yufh8z aGpHg6+NdV6Dbj1KBC7BhIG69fcxIL614n5Sp1iuhnzUdnFeJEmJPjy43VlHbwblVIzkBdxt20n 8VZOiRWXT/facFdW22STxZzjh0wfzebhoujpWbeDFIQ46fe8La/reRBch4YNRCSMgEK5bfGwCfy LgZKsumT77 X-Received: by 2002:a05:600c:3504:b0:38c:bcbc:f95e with SMTP id h4-20020a05600c350400b0038cbcbcf95emr6032895wmq.170.1648552649655; Tue, 29 Mar 2022 04:17:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxcYCHfu3fHJaDrrcDw/AgZtRVAM602Q2HUOpzeg8+HmyhcjZ1TNNnZE2phYsFyTc0b5y7FSw== X-Received: by 2002:a05:600c:3504:b0:38c:bcbc:f95e with SMTP id h4-20020a05600c350400b0038cbcbcf95emr6032860wmq.170.1648552649339; Tue, 29 Mar 2022 04:17:29 -0700 (PDT) Received: from p1.Home ([2001:8a0:6724:4500:a69c:e66f:828e:b340]) by smtp.gmail.com with ESMTPSA id q6-20020adffec6000000b00205b60faeeesm8107591wrs.24.2022.03.29.04.17.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Mar 2022 04:17:28 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Tue, 29 Mar 2022 12:17:25 +0100 Message-Id: <20220329111725.24565-1-ecurtin@redhat.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=ecurtin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [libcamera-devel] [PATCH v4] cam: sdl_sink: Add SDL sink X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Eric Curtin via libcamera-devel From: Eric Curtin Reply-To: Eric Curtin Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds more portability to existing cam sinks. You can pass a YUYV camera buffer for example and SDL will handle the pixel buffer conversion, although SDL does not support decompression for pixelformats like MJPEG. This allows cam reference implementation to display images on VMs, Mac M1, Raspberry Pi, etc. This also enables cam reference implementation, to run as a desktop application in wayland or x11. SDL also has support for Android and ChromeOS which I have not tested. Also tested on simpledrm raspberry pi 4 framebuffer successfully where existing kms sink did not work. Can also be used as kmsdrm sink. Signed-off-by: Eric Curtin --- Changes in v2: - Remove hardcoded pixel format from SDL_CreateTexture call Changes in v3: - Drop blank line - The contents of the if..endif are indented - Split configure function into start and configure - Add SDL_DestroyRenderer - Remove assign and test in the same statement Changes in v4: - Add processSDLEvents as a timed event in the evloop --- src/cam/camera_session.cpp | 8 +++ src/cam/event_loop.cpp | 10 ++- src/cam/event_loop.h | 1 + src/cam/main.cpp | 5 ++ src/cam/main.h | 1 + src/cam/meson.build | 10 +++ src/cam/sdl_sink.cpp | 142 +++++++++++++++++++++++++++++++++++++ src/cam/sdl_sink.h | 42 +++++++++++ 8 files changed, 218 insertions(+), 1 deletion(-) create mode 100644 src/cam/sdl_sink.cpp create mode 100644 src/cam/sdl_sink.h diff --git a/src/cam/camera_session.cpp b/src/cam/camera_session.cpp index 0428b538..30162dbd 100644 --- a/src/cam/camera_session.cpp +++ b/src/cam/camera_session.cpp @@ -19,6 +19,9 @@ #ifdef HAVE_KMS #include "kms_sink.h" #endif +#ifdef HAVE_SDL +#include "sdl_sink.h" +#endif #include "main.h" #include "stream_options.h" @@ -187,6 +190,11 @@ int CameraSession::start() sink_ = std::make_unique(options_[OptDisplay].toString()); #endif +#ifdef HAVE_SDL + if (options_.isSet(OptSDL)) + sink_ = std::make_unique(); +#endif + if (options_.isSet(OptFile)) { if (!options_[OptFile].toString().empty()) sink_ = std::make_unique(streamNames_, diff --git a/src/cam/event_loop.cpp b/src/cam/event_loop.cpp index e25784c0..41339d4e 100644 --- a/src/cam/event_loop.cpp +++ b/src/cam/event_loop.cpp @@ -75,7 +75,15 @@ void EventLoop::addEvent(int fd, EventType type, return; } - int ret = event_add(event->event_, nullptr); + struct timeval *tp = nullptr; + struct timeval tv; + if (events == EV_PERSIST) { + tp = &tv; + tv.tv_sec = 0; + tv.tv_usec = 10000; /* every 10 ms */ + } + + int ret = event_add(event->event_, tp); if (ret < 0) { std::cerr << "Failed to add event for fd " << fd << std::endl; return; diff --git a/src/cam/event_loop.h b/src/cam/event_loop.h index a4613eb2..8fd9bd20 100644 --- a/src/cam/event_loop.h +++ b/src/cam/event_loop.h @@ -20,6 +20,7 @@ class EventLoop { public: enum EventType { + Default = 0, /* the event can be triggered only by a timeout or by manual activation */ Read = 1, Write = 2, }; diff --git a/src/cam/main.cpp b/src/cam/main.cpp index c7f664b9..1d62a64a 100644 --- a/src/cam/main.cpp +++ b/src/cam/main.cpp @@ -137,6 +137,11 @@ int CamApp::parseOptions(int argc, char *argv[]) "Display viewfinder through DRM/KMS on specified connector", "display", ArgumentOptional, "connector", false, OptCamera); +#endif +#ifdef HAVE_SDL + parser.addOption(OptSDL, OptionNone, + "Display viewfinder through SDL", + "sdl", ArgumentNone, "", false, OptCamera); #endif parser.addOption(OptFile, OptionString, "Write captured frames to disk\n" diff --git a/src/cam/main.h b/src/cam/main.h index 62f7bbc9..a64f95a0 100644 --- a/src/cam/main.h +++ b/src/cam/main.h @@ -11,6 +11,7 @@ enum { OptCamera = 'c', OptCapture = 'C', OptDisplay = 'D', + OptSDL = 'S', OptFile = 'F', OptHelp = 'h', OptInfo = 'I', diff --git a/src/cam/meson.build b/src/cam/meson.build index e8e2ae57..bd536c5b 100644 --- a/src/cam/meson.build +++ b/src/cam/meson.build @@ -32,11 +32,21 @@ cam_sources += files([ ]) endif +libsdl2 = dependency('SDL2', required : false) + +if libsdl2.found() + cam_cpp_args += [ '-DHAVE_SDL' ] + cam_sources += files([ + 'sdl_sink.cpp' + ]) +endif + cam = executable('cam', cam_sources, dependencies : [ libatomic, libcamera_public, libdrm, + libsdl2, libevent, ], cpp_args : cam_cpp_args, diff --git a/src/cam/sdl_sink.cpp b/src/cam/sdl_sink.cpp new file mode 100644 index 00000000..6c6e37d0 --- /dev/null +++ b/src/cam/sdl_sink.cpp @@ -0,0 +1,142 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * sdl_sink.cpp - SDL Sink + */ + +#include "sdl_sink.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "event_loop.h" +#include "image.h" + +using namespace libcamera; + +SDLSink::SDLSink() + : sdlRenderer_(0) +{ + memset(&sdlRect_, 0, sizeof(sdlRect_)); +} + +SDLSink::~SDLSink() +{ + if (sdlRenderer_) + SDL_DestroyRenderer(sdlRenderer_); + SDL_Quit(); +} + +int SDLSink::configure(const libcamera::CameraConfiguration &cfg) +{ + int ret = FrameSink::configure(cfg); + if (ret < 0) + return ret; + + const libcamera::StreamConfiguration &sCfg = cfg.at(0); + pf = (SDL_PixelFormatEnum)sCfg.pixelFormat.fourcc(); + if (pf == SDL_DEFINE_PIXELFOURCC('Y', 'U', 'Y', 'V')) { + pf = SDL_PIXELFORMAT_YUY2; + } else if (int ne = strcmp(SDL_GetPixelFormatName(pf), "SDL_PIXELFORMAT_UNKNOWN"); !ne) { + std::cerr << "SDL_GetPixelFormatName error - exiting: SDL_PIXELFORMAT_UNKNOWN, no " << sCfg.pixelFormat.toString() << " support\n"; + return -EINVAL; + } + + sdlRect_.w = sCfg.size.width; + sdlRect_.h = sCfg.size.height; + + return 0; +} + +int SDLSink::start() +{ + int ret = SDL_Init(SDL_INIT_VIDEO); + if (ret) { + std::cerr << "SDL_Init error - exiting: " << SDL_GetError() << std::endl; + return ret; + } + + sdlScreen_ = SDL_CreateWindow("", SDL_WINDOWPOS_UNDEFINED, + SDL_WINDOWPOS_UNDEFINED, sdlRect_.w, + sdlRect_.h, SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE); + if (!sdlScreen_) { + std::cerr << "SDL_CreateWindow error - exiting: " << SDL_GetError() << std::endl; + return -EINVAL; + } + + sdlRenderer_ = SDL_CreateRenderer( + sdlScreen_, -1, 0); + if (!sdlRenderer_) { + std::cerr << "SDL_CreateRenderer error - exiting: " << SDL_GetError() << std::endl; + return -EINVAL; + } + + SDL_RenderSetLogicalSize(sdlRenderer_, sdlRect_.w, + sdlRect_.h); + + sdlTexture_ = + SDL_CreateTexture(sdlRenderer_, pf, + SDL_TEXTUREACCESS_STREAMING, sdlRect_.w, + sdlRect_.h); + EventLoop::instance()->addEvent(-1, EventLoop::Default, + std::bind(&SDLSink::processSDLEvents, this)); + + return 0; +} + +void SDLSink::mapBuffer(FrameBuffer *buffer) +{ + std::unique_ptr image = + Image::fromFrameBuffer(buffer, Image::MapMode::ReadOnly); + assert(image != nullptr); + + mappedBuffers_[buffer] = std::move(image); +} + +bool SDLSink::processRequest(Request *request) +{ + for (auto [stream, buffer] : request->buffers()) + writeBuffer(buffer); + + return true; +} + +/* + * Process SDL events, required for things like window resize and quit button + */ +void SDLSink::processSDLEvents() +{ + for (SDL_Event e; SDL_PollEvent(&e);) { + if (e.type == SDL_QUIT) { /* click close icon then quit */ + kill(getpid(), SIGINT); + } + } +} + +void SDLSink::writeBuffer(FrameBuffer *buffer) +{ + Image *image = mappedBuffers_[buffer].get(); + + for (unsigned int i = 0; i < buffer->planes().size(); ++i) { + const FrameMetadata::Plane &meta = buffer->metadata().planes()[i]; + + Span data = image->data(i); + if (meta.bytesused > data.size()) + std::cerr << "payload size " << meta.bytesused + << " larger than plane size " << data.size() + << std::endl; + + SDL_UpdateTexture(sdlTexture_, &sdlRect_, data.data(), sdlRect_.w * 2); + SDL_RenderClear(sdlRenderer_); + SDL_RenderCopy(sdlRenderer_, sdlTexture_, NULL, NULL); + SDL_RenderPresent(sdlRenderer_); + } +} diff --git a/src/cam/sdl_sink.h b/src/cam/sdl_sink.h new file mode 100644 index 00000000..6f20e2d6 --- /dev/null +++ b/src/cam/sdl_sink.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * sdl_sink.h - SDL Sink + */ + +#pragma once + +#include +#include +#include + +#include + +#include + +#include "frame_sink.h" + +class Image; + +class SDLSink : public FrameSink +{ +public: + SDLSink(); + ~SDLSink(); + + int configure(const libcamera::CameraConfiguration &cfg) override; + int start() override; + void mapBuffer(libcamera::FrameBuffer *buffer) override; + + bool processRequest(libcamera::Request *request) override; + +private: + void writeBuffer(libcamera::FrameBuffer *buffer); + void processSDLEvents(); + + std::map> mappedBuffers_; + SDL_Window *sdlScreen_; + SDL_Renderer *sdlRenderer_; + SDL_Texture *sdlTexture_; + SDL_Rect sdlRect_; + SDL_PixelFormatEnum pf; +};