From patchwork Mon Oct 19 22:49:02 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Robert Mader X-Patchwork-Id: 28530 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2FE11C324C for ; Mon, 19 Oct 2026 22:49:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8FC1368EE9; Tue, 20 Oct 2026 00:49:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=collabora.com header.i=robert.mader@collabora.com header.b="JgEKFqC+"; dkim-atps=neutral Received: from sender5-op-o11.zoho.com (sender5-op-o11.zoho.com [165.173.182.11]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6FAE468EC6 for ; Tue, 20 Oct 2026 00:49:19 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; t=1792450155; cv=none; d=zohomail.com; s=zohoarc; b=Tj/p95dwNvEqatFmgDXLOf5oMizNf15YdNRBtQ4hz5IQh1QFYhoV+zOf3Aj/qBh41TNnQTL81E+gQuZyGOK96L51KjPRY6EAZIEPSDaVlEzABq1jA0EdazFj2NblBjZlCnxq8ZuVKCeCh28mFwXC8iZgB6TeoTvUKfeyecm01Jk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1792450155; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To; bh=Vdeb51ZcJiTWQFI4niKv525S31T9C/gGsGwhvEDUVYw=; b=MW9u+EKTUUV8pLtZA/geX5oi12amqXosU89Tg0Sg3Y85ZikxRODs0AEuIzM16nWB7pwBs3aBhWb6TIr75iFarTMUA5/QFd8s5a4FZEzffv6LRJYgQ8OEm+l4M9jU01h3ky1DTH72wqqOGtplqpQ8xceedyOV+t5/5vodWWGIBlQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=collabora.com; spf=pass smtp.mailfrom=robert.mader@collabora.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1792450155; s=zohomail; d=collabora.com; i=robert.mader@collabora.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=Vdeb51ZcJiTWQFI4niKv525S31T9C/gGsGwhvEDUVYw=; b=JgEKFqC+ij+GhZzeT9wN0iTxFP91FUWsUjz3dJohC2+tl6ZX/xw68FMwJSUai3i7 CkCr2W50sAzQAZtcdutM+MO+mvGQXXTv0z3+i1zxR1fTmmxS53uUJ/E5sY4TDQOncMD z7niD4o0+L2hKhobw7YdTYfZ8zbXh7z6+U67y++I= Received: by smtp.zohomail.com with SMTPS id 1792450154468167.38656109269346; Mon, 19 Oct 2026 15:49:14 -0700 (PDT) From: Robert Mader To: libcamera-devel@lists.libcamera.org Cc: Robert Mader , Kieran Bingham , =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= Subject: [PATCH v5 3/3] virtual: Support more common pixel formats Date: Tue, 20 Oct 2026 00:49:02 +0200 Message-ID: <20261019224902.177382-4-robert.mader@collabora.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261019224902.177382-1-robert.mader@collabora.com> References: <20261019224902.177382-1-robert.mader@collabora.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" While NV12 is the most common native format for hardware video encoders, YUV420 is the equivalent for most software video encoders. Thus supporting both is great for testing. Similarly having some RGB formats with and without alpha can be very useful, especially as the swISP is currently limited to such formats. Signed-off-by: Robert Mader Reviewed-by: Kieran Bingham Tested-by: Kieran Bingham Reviewed-by: Barnabás Pőcze Reviewed-by: Laurent Pinchart --- .../pipeline/virtual/frame_generator.h | 3 +- .../virtual/image_frame_generator.cpp | 61 ++++++++++++++++--- .../pipeline/virtual/image_frame_generator.h | 2 +- .../virtual/test_pattern_generator.cpp | 52 +++++++++++++--- .../pipeline/virtual/test_pattern_generator.h | 2 +- src/libcamera/pipeline/virtual/virtual.cpp | 45 +++++++++++--- 6 files changed, 135 insertions(+), 30 deletions(-) diff --git a/src/libcamera/pipeline/virtual/frame_generator.h b/src/libcamera/pipeline/virtual/frame_generator.h index a0658c45cd82..fd34a91a5e50 100644 --- a/src/libcamera/pipeline/virtual/frame_generator.h +++ b/src/libcamera/pipeline/virtual/frame_generator.h @@ -9,6 +9,7 @@ #include #include +#include namespace libcamera { @@ -19,7 +20,7 @@ public: virtual void configure(const Size &size) = 0; - virtual int generateFrame(const Size &size, + virtual int generateFrame(const StreamConfiguration &cfg, const FrameBuffer *buffer) = 0; protected: diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.cpp b/src/libcamera/pipeline/virtual/image_frame_generator.cpp index 5caad8b72763..7a9bc9ac1ebd 100644 --- a/src/libcamera/pipeline/virtual/image_frame_generator.cpp +++ b/src/libcamera/pipeline/virtual/image_frame_generator.cpp @@ -126,7 +126,7 @@ void ImageFrameGenerator::configure(const Size &size) } } -int ImageFrameGenerator::generateFrame(const Size &size, const FrameBuffer *buffer) +int ImageFrameGenerator::generateFrame(const StreamConfiguration &cfg, const FrameBuffer *buffer) { ASSERT(!scaledFrameDatas_.empty()); @@ -143,17 +143,62 @@ int ImageFrameGenerator::generateFrame(const Size &size, const FrameBuffer *buff MappedFrameBuffer::MapFlag::Write); const auto &planes = mappedFrameBuffer.planes(); - const PixelFormatInfo &info = PixelFormatInfo::info(formats::NV12); + const Size size = cfg.size; + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); /* Loop only around the number of images available */ frameIndex_ %= imageFrameDatas_.size(); + int ret; + auto Y = scaledFrameDatas_[frameIndex_].Y.get(); + auto UV = scaledFrameDatas_[frameIndex_].UV.get(); + /* Write the scaledY and scaledUV to the mapped frame buffer */ - libyuv::NV12Copy(scaledFrameDatas_[frameIndex_].Y.get(), size.width, - scaledFrameDatas_[frameIndex_].UV.get(), size.width, - planes[0].data(), info.stride(size.width, 0), - planes[1].data(), info.stride(size.width, 1), - size.width, size.height); + switch (cfg.pixelFormat) { + case formats::NV12: + ret = libyuv::NV12Copy(Y, + size.width, + UV, + size.width, + planes[0].data(), + info.stride(size.width, 0), + planes[1].data(), + info.stride(size.width, 1), + size.width, size.height); + break; + + case formats::YUV420: + ret = libyuv::NV12ToI420(Y, + size.width, + UV, + size.width, + planes[0].data(), + info.stride(size.width, 0), + planes[1].data(), + info.stride(size.width, 1), + planes[2].data(), + info.stride(size.width, 2), + size.width, size.height); + break; + + case formats::ABGR8888: + case formats::XBGR8888: + ret = libyuv::NV12ToABGR(Y, + size.width, + UV, + size.width, + planes[0].data(), + info.stride(size.width, 0), + size.width, size.height); + break; + + default: + LOG(Virtual, Error) << "Unsupported format"; + return -1; + } + + if (ret != 0) + LOG(Virtual, Error) << "Conversation failed with " << ret; /* Proceed to the next image every 4 frames */ /* \todo Consider setting the frameRepeat in the config file */ @@ -161,7 +206,7 @@ int ImageFrameGenerator::generateFrame(const Size &size, const FrameBuffer *buff if (parameter_ % frameRepeat == 0) frameIndex_++; - return 0; + return ret; } /* diff --git a/src/libcamera/pipeline/virtual/image_frame_generator.h b/src/libcamera/pipeline/virtual/image_frame_generator.h index 42a077ba803c..19b6fe38654e 100644 --- a/src/libcamera/pipeline/virtual/image_frame_generator.h +++ b/src/libcamera/pipeline/virtual/image_frame_generator.h @@ -37,7 +37,7 @@ private: }; void configure(const Size &size) override; - int generateFrame(const Size &size, const FrameBuffer *buffer) override; + int generateFrame(const StreamConfiguration &cfg, const FrameBuffer *buffer) override; std::vector imageFrameDatas_; std::vector scaledFrameDatas_; diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.cpp b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp index 571be4fbf65d..e214048d288a 100644 --- a/src/libcamera/pipeline/virtual/test_pattern_generator.cpp +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.cpp @@ -44,7 +44,7 @@ LOG_DECLARE_CATEGORY(Virtual) static const unsigned int kARGBSize = 4; -int TestPatternGenerator::generateFrame(const Size &size, +int TestPatternGenerator::generateFrame(const StreamConfiguration &cfg, const FrameBuffer *buffer) { /* @@ -60,19 +60,53 @@ int TestPatternGenerator::generateFrame(const Size &size, MappedFrameBuffer::MapFlag::Write); const auto &planes = mappedFrameBuffer.planes(); - const PixelFormatInfo &info = PixelFormatInfo::info(formats::NV12); + const Size size = cfg.size; + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); rotateLeft1Column(size, template_.get()); + int ret; + /* Convert the template_ to the frame buffer */ - int ret = libyuv::ARGBToNV12(template_.get(), size.width * kARGBSize, - planes[0].data(), - info.stride(size.width, 0), - planes[1].data(), - info.stride(size.width, 1), - size.width, size.height); + switch (cfg.pixelFormat) { + case formats::NV12: + ret = libyuv::ARGBToNV12(template_.get(), + size.width * kARGBSize, + planes[0].data(), + info.stride(size.width, 0), + planes[1].data(), + info.stride(size.width, 1), + size.width, size.height); + break; + + case formats::YUV420: + ret = libyuv::ARGBToI420(template_.get(), + size.width * kARGBSize, + planes[0].data(), + info.stride(size.width, 0), + planes[1].data(), + info.stride(size.width, 1), + planes[2].data(), + info.stride(size.width, 2), + size.width, size.height); + break; + + case formats::ABGR8888: + case formats::XBGR8888: + ret = libyuv::ARGBCopy(template_.get(), + size.width * kARGBSize, + planes[0].data(), + info.stride(size.width, 0), + size.width, size.height); + break; + + default: + LOG(Virtual, Error) << "Unsupported format"; + return -1; + } + if (ret != 0) - LOG(Virtual, Error) << "ARGBToNV12() failed with " << ret; + LOG(Virtual, Error) << "Conversation failed with " << ret; return ret; } diff --git a/src/libcamera/pipeline/virtual/test_pattern_generator.h b/src/libcamera/pipeline/virtual/test_pattern_generator.h index 2a51bd31abdf..9f72d652ab2c 100644 --- a/src/libcamera/pipeline/virtual/test_pattern_generator.h +++ b/src/libcamera/pipeline/virtual/test_pattern_generator.h @@ -24,7 +24,7 @@ enum class TestPattern : char { class TestPatternGenerator : public FrameGenerator { public: - int generateFrame(const Size &size, const FrameBuffer *buffer) override; + int generateFrame(const StreamConfiguration &cfg, const FrameBuffer *buffer) override; protected: /* Buffer of test pattern template */ diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index e25dd85a7391..f35af97b103d 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -47,6 +47,13 @@ LOG_DEFINE_CATEGORY(Virtual) namespace { +constexpr PixelFormat outputFormats[] = { + formats::NV12, + formats::YUV420, + formats::ABGR8888, + formats::XBGR8888, +}; + uint64_t currentTimestamp() { const auto now = std::chrono::steady_clock::now(); @@ -148,7 +155,7 @@ void VirtualCameraData::processRequest(Request *request) found = true; if (streamConfig.frameGenerator->generateFrame( - stream->configuration().size, buffer)) + stream->configuration(), buffer)) fmd.status = FrameMetadata::Status::FrameError; bufferCompleted.emit(buffer); @@ -202,18 +209,37 @@ CameraConfiguration::Status VirtualCameraConfiguration::validate() adjusted = true; } - if (cfg.pixelFormat != formats::NV12) { - cfg.pixelFormat = formats::NV12; + if (std::find(std::begin(outputFormats), + std::end(outputFormats), + cfg.pixelFormat) == std::end(outputFormats)) { + cfg.pixelFormat = outputFormats[0]; status = Adjusted; adjusted = true; } - if (cfg.colorSpace != ColorSpace::Smpte170m) { - cfg.colorSpace = ColorSpace::Smpte170m; + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + std::optional colorSpace; + + switch (info.colourEncoding) { + case PixelFormatInfo::ColourEncodingRGB: + colorSpace = ColorSpace::Srgb; + break; + + case PixelFormatInfo::ColourEncodingYUV: + colorSpace = ColorSpace::Smpte170m; + break; + + default: + ASSERT(false); + } + + if (cfg.colorSpace && cfg.colorSpace != colorSpace) { status = Adjusted; adjusted = true; } + cfg.colorSpace = colorSpace; + if (validateColorSpaces() == Adjusted) { status = Adjusted; adjusted = true; @@ -223,7 +249,6 @@ CameraConfiguration::Status VirtualCameraConfiguration::validate() LOG(Virtual, Info) << "Stream configuration adjusted to " << cfg.toString(); - const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); cfg.stride = info.stride(cfg.size.width, 0, 1); cfg.frameSize = info.frameSize(cfg.size, 1); @@ -275,12 +300,12 @@ PipelineHandlerVirtual::generateConfiguration(Camera *camera, } std::map> streamFormats; - PixelFormat pixelFormat = formats::NV12; - streamFormats[pixelFormat] = { { data->config_.minResolutionSize, - data->config_.maxResolutionSize } }; + for (auto pixelFormat : outputFormats) + streamFormats[pixelFormat] = { { data->config_.minResolutionSize, + data->config_.maxResolutionSize } }; StreamFormats formats(streamFormats); StreamConfiguration cfg(formats); - cfg.pixelFormat = pixelFormat; + cfg.pixelFormat = outputFormats[0]; cfg.size = data->config_.maxResolutionSize; cfg.bufferCount = VirtualCameraConfiguration::kBufferCount; cfg.colorSpace = ColorSpace::Smpte170m;