From patchwork Tue Mar 22 19:09:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Curtin X-Patchwork-Id: 15511 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BDB12BD80A for ; Tue, 22 Mar 2022 19:09:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0E56E604DB; Tue, 22 Mar 2022 20:09:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1647976186; bh=fLh7zaT/up2tSEjy4C405D9nz9VvNaWDxee7I9RGaa0=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=gL0KbIlE6hwLJBIRfCpjRKqQRHOKpuRyeorfcTjHYcjzNSgz/DOlDlTaQCD6ZqD1L YEF+1MWalbrrGrc9R+nVRQUw04FG030lLiL2izMrYmYrfNIWZmW0qiefGLteHMoZud eo1c3yor8UiLtjrQqGi4nEevjmy7JVwTzXE4PId1AquyZWx9NeqwpWjNhBt3ugNzwZ L8MeJU0OcudxRRQhQ/CAdEYX0B7i7s+6ODQtS6reFzHpbzxtmFu0pvcTijCJkvvEVJ 1pgZcy9YMAMTeVFqIFp9pgbaAiZZdUMM3JPhgOXEXg59elmiOt7MYfZdCW8yB1lXKx wf95MPqZHhyYg== Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 382FF604C5 for ; Tue, 22 Mar 2022 20:09:44 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Li/d3BPB"; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647976183; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=34Z95dkjOR83XONsplmy94S4L3sm0x1AHzE0gAaxhP8=; b=Li/d3BPBwEsOFPUg8Y9ijFUR2Aie344XlPpEBTvIyfn5NYi4fDvigDHL67t+JJK9o2WD8m lf/jz26x3u/Cw66ZC3WvQ7vhPSwzShi8iJUeXC0F4ahEDVEOS8F96bnejJqHf4NNR+npW8 vrrQVcD+gJbqpOC2mc20u4dYg/qQd+4= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-593-e1QTKaWjO92-pP34Li94-Q-1; Tue, 22 Mar 2022 15:09:41 -0400 X-MC-Unique: e1QTKaWjO92-pP34Li94-Q-1 Received: by mail-wm1-f71.google.com with SMTP id 12-20020a05600c24cc00b0038c6caa95f7so1300659wmu.4 for ; Tue, 22 Mar 2022 12:09:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=34Z95dkjOR83XONsplmy94S4L3sm0x1AHzE0gAaxhP8=; b=b/QNnvMn5wn9mlsjJNjcvP8hhxCe80mNvu1RCLDHkTzVJRsHZ8yQmkEHMZJ68NObVU u4oAWZVe6sYNZj8wBr3PV8LW+sK52DmrXfJh9fZ8Z4XT2C9EDZCiIqhUGIai72d6+uOM FOHy1ZsU4/t/cgCeO7fK9sD4t1PXRAsYlWp1wJAXGe9cA0CjJwcg/oXTxYfv+PmP0k/u BbeMtZ/DApfUdcdtZha3sKPudQCV0ZlN/G7NMxmHb8OU51NyuHvCAXO6qge0M4kZmts0 acWVKyZHozWgoM7H9mKB7f/EtHoCd2JOIwOiTjNYgoP/O0jYeEbAIXgbadMyhjw002ku zj5g== X-Gm-Message-State: AOAM533PKv2AsJHHrUN1hzI4lc6PDRHk5y3Ggfhr0CZkBxrzPM88X4un Xjawn/UJloFfkiK2hotZdi67QvcBL4UgMxpsMYLVkgObroC7bBHtodvG0/1sacX+tM4fp6xzLni VwtRWlTIFCHXIVRtSO9ITfcDWSJ163wo4YVJosv+Wxtwtn3SmslKJ5CzDuDdtFRdcZJoSuhoHH3 df8XYWw6g9 X-Received: by 2002:a05:600c:1990:b0:38c:c0a2:c0ab with SMTP id t16-20020a05600c199000b0038cc0a2c0abmr884740wmq.72.1647976179978; Tue, 22 Mar 2022 12:09:39 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyVjnvwDPRds6kcttCkvTM83WFKsj/nO3S8j4u3NR77XfrDZArHay9R0eRlvF+3kOxX/U3B8A== X-Received: by 2002:a05:600c:1990:b0:38c:c0a2:c0ab with SMTP id t16-20020a05600c199000b0038cc0a2c0abmr884716wmq.72.1647976179681; Tue, 22 Mar 2022 12:09:39 -0700 (PDT) Received: from p1.Home ([2001:8a0:6724:4500:a69c:e66f:828e:b340]) by smtp.gmail.com with ESMTPSA id p8-20020a5d4e08000000b002054b5437f2sm1884100wrt.115.2022.03.22.12.09.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Mar 2022 12:09:39 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Tue, 22 Mar 2022 19:09:36 +0000 Message-Id: <20220322190936.69281-1-ecurtin@redhat.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=ecurtin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [libcamera-devel] [PATCH v2] cam: sdl_sink: Add SDL sink X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Eric Curtin via libcamera-devel From: Eric Curtin Reply-To: Eric Curtin Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This adds more portability to existing cam sinks. You can pass a YUYV camera buffer for example and SDL will handle the pixel buffer conversion, although SDL does not support decompression for pixelformats like MJPEG. This allows cam reference implementation to display images on VMs, Mac M1, Raspberry Pi, etc. This also enables cam reference implementation, to run as a desktop application in wayland or x11. SDL also has support for Android and ChromeOS which I have not tested. Also tested on simpledrm raspberry pi 4 framebuffer successfully where existing kms sink did not work. Signed-off-by: Eric Curtin --- Changes in v2: - Remove hardcoded pixel format from SDL_CreateTexture call --- src/cam/camera_session.cpp | 8 +++ src/cam/main.cpp | 6 ++ src/cam/main.h | 1 + src/cam/meson.build | 10 +++ src/cam/sdl_sink.cpp | 125 +++++++++++++++++++++++++++++++++++++ src/cam/sdl_sink.h | 40 ++++++++++++ 6 files changed, 190 insertions(+) create mode 100644 src/cam/sdl_sink.cpp create mode 100644 src/cam/sdl_sink.h diff --git a/src/cam/camera_session.cpp b/src/cam/camera_session.cpp index 0428b538..30162dbd 100644 --- a/src/cam/camera_session.cpp +++ b/src/cam/camera_session.cpp @@ -19,6 +19,9 @@ #ifdef HAVE_KMS #include "kms_sink.h" #endif +#ifdef HAVE_SDL +#include "sdl_sink.h" +#endif #include "main.h" #include "stream_options.h" @@ -187,6 +190,11 @@ int CameraSession::start() sink_ = std::make_unique(options_[OptDisplay].toString()); #endif +#ifdef HAVE_SDL + if (options_.isSet(OptSDL)) + sink_ = std::make_unique(); +#endif + if (options_.isSet(OptFile)) { if (!options_[OptFile].toString().empty()) sink_ = std::make_unique(streamNames_, diff --git a/src/cam/main.cpp b/src/cam/main.cpp index c7f664b9..406d61bc 100644 --- a/src/cam/main.cpp +++ b/src/cam/main.cpp @@ -138,6 +138,12 @@ int CamApp::parseOptions(int argc, char *argv[]) "display", ArgumentOptional, "connector", false, OptCamera); #endif + +#ifdef HAVE_SDL + parser.addOption(OptSDL, OptionNone, + "Display viewfinder through SDL", + "sdl", ArgumentNone, "", false, OptCamera); +#endif parser.addOption(OptFile, OptionString, "Write captured frames to disk\n" "If the file name ends with a '/', it sets the directory in which\n" diff --git a/src/cam/main.h b/src/cam/main.h index 62f7bbc9..a64f95a0 100644 --- a/src/cam/main.h +++ b/src/cam/main.h @@ -11,6 +11,7 @@ enum { OptCamera = 'c', OptCapture = 'C', OptDisplay = 'D', + OptSDL = 'S', OptFile = 'F', OptHelp = 'h', OptInfo = 'I', diff --git a/src/cam/meson.build b/src/cam/meson.build index e8e2ae57..44202ef0 100644 --- a/src/cam/meson.build +++ b/src/cam/meson.build @@ -32,11 +32,21 @@ cam_sources += files([ ]) endif +libsdl2 = dependency('SDL2', required : false) + +if libsdl2.found() +cam_cpp_args += [ '-DHAVE_SDL' ] +cam_sources += files([ + 'sdl_sink.cpp' +]) +endif + cam = executable('cam', cam_sources, dependencies : [ libatomic, libcamera_public, libdrm, + libsdl2, libevent, ], cpp_args : cam_cpp_args, diff --git a/src/cam/sdl_sink.cpp b/src/cam/sdl_sink.cpp new file mode 100644 index 00000000..480df1b4 --- /dev/null +++ b/src/cam/sdl_sink.cpp @@ -0,0 +1,125 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * sdl_sink.cpp - SDL Sink + */ + +#include "sdl_sink.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "image.h" + +using namespace libcamera; + +SDLSink::SDLSink() +{ + memset(&sdlRect_, 0, sizeof(sdlRect_)); +} + +SDLSink::~SDLSink() +{ + SDL_Quit(); +} + +int SDLSink::configure(const libcamera::CameraConfiguration &cfg) +{ + int ret = FrameSink::configure(cfg); + if (ret < 0) + return ret; + + if ((ret = SDL_Init(SDL_INIT_VIDEO))) { + std::cout << "Could not initialize SDL - " << SDL_GetError() << "\n"; + return ret; + } + + const libcamera::StreamConfiguration &sCfg = cfg.at(0); + sdlScreen_ = SDL_CreateWindow("", SDL_WINDOWPOS_UNDEFINED, + SDL_WINDOWPOS_UNDEFINED, sCfg.size.width, + sCfg.size.height, SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE); + if (!sdlScreen_) { + std::cerr << "SDL: could not create window - exiting: " << SDL_GetError() << "\n"; + return -1; + } + + sdlRenderer_ = SDL_CreateRenderer( + sdlScreen_, -1, 0); + if (!sdlRenderer_) { + std::cerr << "SDL_CreateRenderer Error\n"; + return -2; + } + + SDL_RenderSetLogicalSize(sdlRenderer_, sCfg.size.width, + sCfg.size.height); + + SDL_PixelFormatEnum pf = (SDL_PixelFormatEnum)sCfg.pixelFormat.fourcc(); + if (pf == SDL_DEFINE_PIXELFOURCC('Y', 'U', 'Y', 'V')) { + pf = SDL_PIXELFORMAT_YUY2; + } + + if (!(ret = strcmp(SDL_GetPixelFormatName(pf), "SDL_PIXELFORMAT_UNKNOWN"))) { + std::cerr << "error: SDL_PIXELFORMAT_UNKNOWN, no " << sCfg.pixelFormat.toString() << " support\n"; + return -3; + } + + sdlTexture_ = + SDL_CreateTexture(sdlRenderer_, pf, + SDL_TEXTUREACCESS_STREAMING, sCfg.size.width, + sCfg.size.height); + sdlRect_.w = sCfg.size.width; + sdlRect_.h = sCfg.size.height; + + return 0; +} + +void SDLSink::mapBuffer(FrameBuffer *buffer) +{ + std::unique_ptr image = + Image::fromFrameBuffer(buffer, Image::MapMode::ReadOnly); + assert(image != nullptr); + + mappedBuffers_[buffer] = std::move(image); +} + +bool SDLSink::processRequest(Request *request) +{ + for (auto [stream, buffer] : request->buffers()) + writeBuffer(buffer); + + return true; +} + +void SDLSink::writeBuffer(FrameBuffer *buffer) +{ + Image *image = mappedBuffers_[buffer].get(); + + for (unsigned int i = 0; i < buffer->planes().size(); ++i) { + const FrameMetadata::Plane &meta = buffer->metadata().planes()[i]; + + Span data = image->data(i); + if (meta.bytesused > data.size()) + std::cerr << "payload size " << meta.bytesused + << " larger than plane size " << data.size() + << std::endl; + + SDL_UpdateTexture(sdlTexture_, &sdlRect_, data.data(), sdlRect_.w * 2); + SDL_RenderClear(sdlRenderer_); + SDL_RenderCopy(sdlRenderer_, sdlTexture_, NULL, NULL); + SDL_RenderPresent(sdlRenderer_); + SDL_Event e; + while (SDL_PollEvent(&e)) { + if (e.type == SDL_QUIT) { // click close icon then quit + kill(getpid(), SIGINT); + } + }; + } +} diff --git a/src/cam/sdl_sink.h b/src/cam/sdl_sink.h new file mode 100644 index 00000000..f4f843fa --- /dev/null +++ b/src/cam/sdl_sink.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * sdl_sink.h - SDL Sink + */ + +#pragma once + +#include +#include +#include + +#include + +#include + +#include "frame_sink.h" + +class Image; + +class SDLSink : public FrameSink +{ +public: + SDLSink(); + ~SDLSink(); + + int configure(const libcamera::CameraConfiguration &cfg) override; + + void mapBuffer(libcamera::FrameBuffer *buffer) override; + + bool processRequest(libcamera::Request *request) override; + +private: + void writeBuffer(libcamera::FrameBuffer *buffer); + + std::map> mappedBuffers_; + SDL_Window *sdlScreen_; + SDL_Renderer *sdlRenderer_; + SDL_Texture *sdlTexture_; + SDL_Rect sdlRect_; +};