From patchwork Thu Apr 7 13:21:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Curtin X-Patchwork-Id: 15651 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 62E1AC0F1B for ; Thu, 7 Apr 2022 13:21:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 14F3B65645; Thu, 7 Apr 2022 15:21:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1649337681; bh=NzczoHGeAt1nNyHa1vD/1JmZA8esztlaJyvkDnUYSPk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=rGrxGHXnuc4HB6gPBX5mr2TJ5bSCt9EioIvauZ6Ic2JIJ7xzzTZByHQkl/KHRZx6d BB8wuWKdb9W8GPtR4RyM3M2CDSENV57zLTnC7s1e9mfW/QgoZRpCbbBpRw8eVj9J0V S878j7HnvutE5VL47DsEfxF4Rw6tBBXwz3YlkkL+qGbS8mwE6qeOq8VSIdWhfdVz6z eR7ebKn6ApQNx62fobDMrfA0YlmTOr7R/0ZHrp0EglTZccfrcog96qfqAquyU24PJ+ wp4SORVgMdeN1vKKYA+sL9RJUJlevSUzXO/iq2EGeT55vp5eznjyn1Rzl5J+aqwx1P 4ADpUy57+25vg== Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 72CF16563F for ; Thu, 7 Apr 2022 15:21:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="JR9/Q+z6"; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649337678; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+xjFK55a0gDKzq6744CEy92BTWAtPSBocnNHraT6Cng=; b=JR9/Q+z6/mmB7eQNaUtParMxtQbSVZSCoTetMJ7RH19Xd2TXYSmmzqGmX8IXwqHAPD4IkM Z0PqPB77rz4bD9YZY6atRrmt5vP3fhNgce0PTLYkn3y81AT0hn+0RRZPDrqa1hdMW5etv9 3OJRIrehLiWzkjFZpGCiCZYB7SyXUa0= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-395-DiWyOzM1MjuLvVz7g2udjw-1; Thu, 07 Apr 2022 09:21:17 -0400 X-MC-Unique: DiWyOzM1MjuLvVz7g2udjw-1 Received: by mail-wm1-f69.google.com with SMTP id r64-20020a1c2b43000000b0038b59eb1940so3347220wmr.0 for ; Thu, 07 Apr 2022 06:21:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+xjFK55a0gDKzq6744CEy92BTWAtPSBocnNHraT6Cng=; b=WT+e5XM+GR++w3MDdjN0Z2EdKhorpuyPCxxRb1v3Ymp53BMOzER4qQwJFB2pX2WJtY Ld4L6N7DfK4ILM7iIfxd8PnWMQYPJNNxQ8vEx4uAjUY/OZgUD1WEuTHYXv7Oix2evC/6 6GKwYHKhceYiVE8bkRBvP0/pkOkLa1MzFeI3rVgzEkcp7erv8ieDiGsEGNFSAXgktnCp OfpvCRkP89Tr1WGqKElV8skpJCLo3ZuVGoigNSuTjtHJYFIGw3nzxGkwzoAfBvNCeiE8 D/z9T21ek+GqqbsWBHPFRWklHvEiVHubOvNQuwv8RpVfKfoP9v30onyvuoAUsRgIukq7 Pn2w== X-Gm-Message-State: AOAM5302ff7SkT/6jpt1tpdzHng+qRhD4M1uquyN1zKKqV7KDFo7/3dm FLZgcoTFPTrIOyobjuvnCXBEK86LWcHPFyyAmIhh8Nzj9257fXvghhJ7I558HqLmPRZfYvTYSve ptfhD74n2oBD9YdCgdiDEtFEFb+dC+tAFZR7q6OGTdIParEa18JbIyTiU2zKdMWQHVjElB3SmbL YnOOx7FreW X-Received: by 2002:adf:f88d:0:b0:206:1102:60c5 with SMTP id u13-20020adff88d000000b00206110260c5mr10380506wrp.600.1649337675672; Thu, 07 Apr 2022 06:21:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxsThzgkKBV4H2u6Q3C2NuhK0kVJDwjXzjII4gTLvHAZHZ50QFIs238mvhdPaYG2pUsiYMXyA== X-Received: by 2002:adf:f88d:0:b0:206:1102:60c5 with SMTP id u13-20020adff88d000000b00206110260c5mr10380466wrp.600.1649337675006; Thu, 07 Apr 2022 06:21:15 -0700 (PDT) Received: from p1.Home ([2001:8a0:6724:4500:a69c:e66f:828e:b340]) by smtp.gmail.com with ESMTPSA id m18-20020a05600c4f5200b0038e8f9d7b57sm1789575wmq.42.2022.04.07.06.21.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Apr 2022 06:21:14 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Thu, 7 Apr 2022 14:21:01 +0100 Message-Id: <20220407132101.33140-2-ecurtin@redhat.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220407132101.33140-1-ecurtin@redhat.com> References: <20220407132101.33140-1-ecurtin@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=ecurtin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [libcamera-devel] [PATCH v6 2/2] cam: sdl_sink: Add SDL sink X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Eric Curtin via libcamera-devel From: Eric Curtin Reply-To: Eric Curtin Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds more portability to existing cam sinks. You can pass a YUYV camera buffer for example and SDL will handle the pixel buffer conversion, although SDL does not support decompression for pixelformats like MJPEG. This allows cam reference implementation to display images on VMs, Mac M1, Raspberry Pi, etc. This also enables cam reference implementation, to run as a desktop application in wayland or x11. SDL also has support for Android and ChromeOS which has not been tested. Also tested on simpledrm raspberry pi 4 framebuffer successfully where existing kms sink did not work. Can also be used as kmsdrm sink. Only supports one camera stream at present. --- src/cam/camera_session.cpp | 8 ++ src/cam/main.cpp | 5 + src/cam/main.h | 1 + src/cam/meson.build | 10 ++ src/cam/sdl_sink.cpp | 198 +++++++++++++++++++++++++++++++++++++ src/cam/sdl_sink.h | 46 +++++++++ 6 files changed, 268 insertions(+) create mode 100644 src/cam/sdl_sink.cpp create mode 100644 src/cam/sdl_sink.h diff --git a/src/cam/camera_session.cpp b/src/cam/camera_session.cpp index 0428b538..30162dbd 100644 --- a/src/cam/camera_session.cpp +++ b/src/cam/camera_session.cpp @@ -19,6 +19,9 @@ #ifdef HAVE_KMS #include "kms_sink.h" #endif +#ifdef HAVE_SDL +#include "sdl_sink.h" +#endif #include "main.h" #include "stream_options.h" @@ -187,6 +190,11 @@ int CameraSession::start() sink_ = std::make_unique(options_[OptDisplay].toString()); #endif +#ifdef HAVE_SDL + if (options_.isSet(OptSDL)) + sink_ = std::make_unique(); +#endif + if (options_.isSet(OptFile)) { if (!options_[OptFile].toString().empty()) sink_ = std::make_unique(streamNames_, diff --git a/src/cam/main.cpp b/src/cam/main.cpp index c7f664b9..1d62a64a 100644 --- a/src/cam/main.cpp +++ b/src/cam/main.cpp @@ -137,6 +137,11 @@ int CamApp::parseOptions(int argc, char *argv[]) "Display viewfinder through DRM/KMS on specified connector", "display", ArgumentOptional, "connector", false, OptCamera); +#endif +#ifdef HAVE_SDL + parser.addOption(OptSDL, OptionNone, + "Display viewfinder through SDL", + "sdl", ArgumentNone, "", false, OptCamera); #endif parser.addOption(OptFile, OptionString, "Write captured frames to disk\n" diff --git a/src/cam/main.h b/src/cam/main.h index 62f7bbc9..2b285808 100644 --- a/src/cam/main.h +++ b/src/cam/main.h @@ -18,6 +18,7 @@ enum { OptListProperties = 'p', OptMonitor = 'm', OptStream = 's', + OptSDL = 'S', OptListControls = 256, OptStrictFormats = 257, OptMetadata = 258, diff --git a/src/cam/meson.build b/src/cam/meson.build index 5bab8c9e..59787741 100644 --- a/src/cam/meson.build +++ b/src/cam/meson.build @@ -32,11 +32,21 @@ if libdrm.found() ]) endif +libsdl2 = dependency('SDL2', required : false) + +if libsdl2.found() + cam_cpp_args += ['-DHAVE_SDL'] + cam_sources += files([ + 'sdl_sink.cpp' + ]) +endif + cam = executable('cam', cam_sources, dependencies : [ libatomic, libcamera_public, libdrm, + libsdl2, libevent, ], cpp_args : cam_cpp_args, diff --git a/src/cam/sdl_sink.cpp b/src/cam/sdl_sink.cpp new file mode 100644 index 00000000..03511974 --- /dev/null +++ b/src/cam/sdl_sink.cpp @@ -0,0 +1,198 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * sdl_sink.cpp - SDL Sink + */ + +#include "sdl_sink.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "event_loop.h" +#include "image.h" + +using namespace libcamera; + +SDLSink::SDLSink() + : sdlWindow_(nullptr), sdlRenderer_(nullptr), sdlTexture_(nullptr), sdlRect_({}), sdlInit_(false) +{ +} + +SDLSink::~SDLSink() +{ + stop(); +} + +int SDLSink::configure(const libcamera::CameraConfiguration &cfg) +{ + int ret = FrameSink::configure(cfg); + if (ret < 0) + return ret; + + if (cfg.size() > 1) { + std::cerr << "SDL sink only supports one camera stream at present, streaming first camera stream" + << std::endl; + } else if (cfg.empty()) { + std::cerr << "Require at least one camera stream to process" << std::endl; + return -EINVAL; + } + + const libcamera::StreamConfiguration &sCfg = cfg.at(0); + sdlRect_.w = sCfg.size.width; + sdlRect_.h = sCfg.size.height; + switch (sCfg.pixelFormat) { + case libcamera::formats::YUYV: + pixelFormat_ = SDL_PIXELFORMAT_YUY2; + pitch_ = 4 * ((sdlRect_.w + 1) / 2); + break; + + /* From here down the fourcc values are identical between SDL, drm, libcamera */ + case libcamera::formats::NV21: + pixelFormat_ = SDL_PIXELFORMAT_NV21; + pitch_ = sdlRect_.w; + break; + case libcamera::formats::NV12: + pixelFormat_ = SDL_PIXELFORMAT_NV12; + pitch_ = sdlRect_.w; + break; + case libcamera::formats::YVU420: + pixelFormat_ = SDL_PIXELFORMAT_YV12; + pitch_ = sdlRect_.w; + break; + case libcamera::formats::YVYU: + pixelFormat_ = SDL_PIXELFORMAT_YVYU; + pitch_ = 4 * ((sdlRect_.w + 1) / 2); + break; + case libcamera::formats::UYVY: + pixelFormat_ = SDL_PIXELFORMAT_UYVY; + pitch_ = 4 * ((sdlRect_.w + 1) / 2); + break; + default: + std::cerr << sCfg.pixelFormat.toString() << " not present in libcamera <-> SDL map" + << std::endl; + return -EINVAL; + }; + + return 0; +} + +int SDLSink::start() +{ + int ret = SDL_Init(SDL_INIT_VIDEO); + if (ret) { + std::cerr << "Failed to initialize SDL: " << SDL_GetError() << std::endl; + return ret; + } + + sdlInit_ = true; + sdlWindow_ = SDL_CreateWindow("", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, sdlRect_.w, sdlRect_.h, SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE); + if (!sdlWindow_) { + std::cerr << "Failed to create SDL window: " << SDL_GetError() << std::endl; + return -EINVAL; + } + + sdlRenderer_ = SDL_CreateRenderer(sdlWindow_, -1, 0); + if (!sdlRenderer_) { + std::cerr << "Failed to create SDL renderer: " << SDL_GetError() << std::endl; + return -EINVAL; + } + + ret = SDL_RenderSetLogicalSize(sdlRenderer_, sdlRect_.w, sdlRect_.h); + if (ret) { /* Not critical to set, don't return in this case, set for scaling purposes */ + std::cerr << "Failed to set SDL render logical size: " << SDL_GetError() << std::endl; + } + + sdlTexture_ = SDL_CreateTexture(sdlRenderer_, pixelFormat_, SDL_TEXTUREACCESS_STREAMING, sdlRect_.w, sdlRect_.h); + if (!sdlTexture_) { + std::cerr << "Failed to create SDL texture: " << SDL_GetError() << std::endl; + return -EINVAL; + } + + EventLoop::instance()->addTimerEvent(10000, std::bind(&SDLSink::processSDLEvents, this)); + + return 0; +} + +int SDLSink::stop() +{ + if (sdlTexture_) { + SDL_DestroyTexture(sdlTexture_); + sdlTexture_ = nullptr; + } + + if (sdlRenderer_) { + SDL_DestroyRenderer(sdlRenderer_); + sdlRenderer_ = nullptr; + } + + if (sdlWindow_) { + SDL_DestroyWindow(sdlWindow_); + sdlWindow_ = nullptr; + } + + if (sdlInit_) { + SDL_Quit(); + sdlInit_ = false; + } + + return FrameSink::stop(); +} + +void SDLSink::mapBuffer(FrameBuffer *buffer) +{ + std::unique_ptr image = Image::fromFrameBuffer(buffer, Image::MapMode::ReadOnly); + assert(image != nullptr); + + mappedBuffers_[buffer] = std::move(image); +} + +bool SDLSink::processRequest(Request *request) +{ + for (auto [stream, buffer] : request->buffers()) { + renderBuffer(buffer); + break; /* to be expanded to launch SDL window per buffer */ + } + + return true; +} + +/* + * Process SDL events, required for things like window resize and quit button + */ +void SDLSink::processSDLEvents() +{ + for (SDL_Event e; SDL_PollEvent(&e);) { + if (e.type == SDL_QUIT) { /* click close icon then quit */ + EventLoop::instance()->exit(0); + } + } +} + +void SDLSink::renderBuffer(FrameBuffer *buffer) +{ + Image *image = mappedBuffers_[buffer].get(); + + for (unsigned int i = 0; i < buffer->planes().size(); ++i) { + const FrameMetadata::Plane &meta = buffer->metadata().planes()[i]; + + Span data = image->data(i); + if (meta.bytesused > data.size()) + std::cerr << "payload size " << meta.bytesused + << " larger than plane size " << data.size() + << std::endl; + + SDL_UpdateTexture(sdlTexture_, &sdlRect_, data.data(), pitch_); + SDL_RenderClear(sdlRenderer_); + SDL_RenderCopy(sdlRenderer_, sdlTexture_, nullptr, nullptr); + SDL_RenderPresent(sdlRenderer_); + } +} diff --git a/src/cam/sdl_sink.h b/src/cam/sdl_sink.h new file mode 100644 index 00000000..c9b0ab8e --- /dev/null +++ b/src/cam/sdl_sink.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * sdl_sink.h - SDL Sink + */ + +#pragma once + +#include +#include +#include + +#include + +#include + +#include "frame_sink.h" + +class Image; + +class SDLSink : public FrameSink +{ +public: + SDLSink(); + ~SDLSink(); + + int configure(const libcamera::CameraConfiguration &cfg) override; + int start() override; + int stop() override; + void mapBuffer(libcamera::FrameBuffer *buffer) override; + + bool processRequest(libcamera::Request *request) override; + +private: + void renderBuffer(libcamera::FrameBuffer *buffer); + void processSDLEvents(); + + std::map> mappedBuffers_; + + SDL_Window *sdlWindow_; + SDL_Renderer *sdlRenderer_; + SDL_Texture *sdlTexture_; + SDL_Rect sdlRect_; + SDL_PixelFormatEnum pixelFormat_; + bool sdlInit_; + int pitch_; +};