From patchwork Fri Sep 15 07:57:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattijs Korpershoek X-Patchwork-Id: 19012 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 199D9C32B1 for ; Fri, 15 Sep 2023 07:57:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C643562922; Fri, 15 Sep 2023 09:57:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694764656; bh=IBL60Z2Vz20HmwGlky/i1f7Jcbli+IO2LFw0N0ieF8k=; h=Date:References:In-Reply-To:To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=tGEMjS5divFR3Adetvs4JyvQAnEvP8lmDxjdaY55gGNYybSlJSEGC9tCryAjqgRDw wH48RuCMX2MW3UYxUbig19jodA5c+yyXjShkSbmpLX+cqXm35JEu5XDyuj5EWljYsr Gs/AG9nN3ywhPFotZAH+afFL3+KBfRQUsy643lDry2QMpgzHbmn+mGzV7QV5Z7FR1d /QOdluB7GfbjjJ35Ywhqb9A4J9K1aqv9rX6tRSJcVv/SRNvyhaSzXpkLbmOkA0+cnV xXsh10zLp23jDyqLVjptpfJVi8/+dtMWQAXgygmNhBLToQuP8buKSgSu6Ud96pZIcC WuNhkBBnldlZA== Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E29616291E for ; Fri, 15 Sep 2023 09:57:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.b="lkKojsLj"; dkim-atps=neutral Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-31dd10c2b8bso1649808f8f.3 for ; Fri, 15 Sep 2023 00:57:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1694764651; x=1695369451; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=xovdNtMrWbtfBJY3joA+lnakbzSinm4xINrzUSJhrX8=; b=lkKojsLjb5hyBeopBpzA6oqAyb/3h6ES6TUUxsaGZ8Aac+NXSWr5xP0Geu4dC1GBb2 KyDMuEgI4RX1yu5nhcvMojLGl4lbSUaZicjnbuzCtdl4iStdWyWMjcq9xb6+0zev1df6 ZAPgb94bgH4iGhGCwIWhhzbxMRn9UgAfu4VcIpF3B0XSQOGQAPd2reLtSjTQT8dPOSeF KvSgxuiJCOERGhHeoQmNyxtqmcrODTyaaKa7LRfYPE2EAcD7Sgp6+gGwAeALAhbQ+bEr yQUO/aKk4dym4aBf+S/d4d4i3ZIFUscjam1+t5O9oswO4TKPlgB3IRgVI6KKuMiCtnFu 4rPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764651; x=1695369451; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xovdNtMrWbtfBJY3joA+lnakbzSinm4xINrzUSJhrX8=; b=aPOcGSePnJps5ECJVMv2FH/kI7SGJMM/vjeJ4Matw/sbivVMoYNZtFr5HyYztg7UWf +L32YoQjPxzDPoIk12vLD3Eodv/eJsPsS3C8Q6XWwZZQG52wxldvH4P7Uk6lj+tThVxC qJNihZuOEg2M5+Hp7aBM/FvfXEp6YmlpFl8WyryAYFTxh2HhYXJFtvTD7JQGIAWdOfKv ILfY1dAXKzilYMXYTrZsKvy6mRZ7CRA/xIK63II7GziAh/s+KH1Oq0ub2bOSrxfXCvWZ WUfi307NLdA+AZZAicDnRBdP4mvTnjEdUoRqnqP1vRJgqQ7/zhbCm3LoqA8HBecJbpLS lWPA== X-Gm-Message-State: AOJu0YwFGtg4jL/80bFZbH/BE14scfIRkC5hMFVwqQjKuK851MplT6JT Id/gOLskM92r/tPx35SCOHl+o/3oEj97f77/4fw= X-Google-Smtp-Source: AGHT+IE7DLoHjaO1Dpz99bNJFqTaqJW3nSxT40SOP8pdS7dcLdsr+utje8ItLsiajfLsuRrD4mEzvg== X-Received: by 2002:adf:fc8c:0:b0:319:7fdc:d15f with SMTP id g12-20020adffc8c000000b003197fdcd15fmr800723wrr.26.1694764651355; Fri, 15 Sep 2023 00:57:31 -0700 (PDT) Received: from [192.168.1.20] ([2a01:cb19:8704:be00:4f55:bd9d:611a:6c8e]) by smtp.gmail.com with ESMTPSA id m2-20020a056000174200b0031fe9a47a87sm2506942wrf.112.2023.09.15.00.57.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 00:57:31 -0700 (PDT) Date: Fri, 15 Sep 2023 09:57:30 +0200 MIME-Version: 1.0 Message-Id: <20230915-libyuv-convert-v1-6-1e5bcf68adac@baylibre.com> References: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> In-Reply-To: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> To: libcamera-devel@lists.libcamera.org X-Mailer: b4 0.12.4-dev-6aa5d Subject: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV -> NV12 conversion X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mattijs Korpershoek via libcamera-devel From: Mattijs Korpershoek Reply-To: Mattijs Korpershoek Cc: Guillaume La Roque Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" On am62x platforms, the receiver driver (j721e-csi2rx) only supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY. The receiver and the sensor (ov5640) hardware are both capable of YUV420, however: * we are not aware of OV5640 being tested with YUV420 formats on any vendor tree. * NV12 has different line lines (even lines are twice as long as odd lines) Different line-sized DMA transfers have not been tested on the TI CSI-RX SHIM IP. On the other hand, the graphics allocator (gralloc) cannot allocate YUV422 buffers directly. It mainly allocated NV12 buffers when userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED. Because of the above, we need a pixel conversion from YUYV to NV12, which is handled in the yuv processor via libyuv. Signed-off-by: Mattijs Korpershoek --- src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++--------- 1 file changed, 68 insertions(+), 23 deletions(-) diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index 734bb85b7351..b680b833dafb 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -7,6 +7,9 @@ #include "post_processor_yuv.h" +#include + +#include #include #include @@ -22,14 +25,42 @@ using namespace libcamera; LOG_DEFINE_CATEGORY(YUV) +namespace { + +/** + * \var supportedConversions + * \brief list of supported output pixel formats for an input pixel format + */ +const std::map> supportedConversions = { + { formats::YUYV, { formats::NV12 } }, +}; + +} /* namespace */ + int PostProcessorYuv::configure(const StreamConfiguration &inCfg, const StreamConfiguration &outCfg) { if (inCfg.pixelFormat != outCfg.pixelFormat) { - LOG(YUV, Error) << "Pixel format conversion is not supported" - << " (from " << inCfg.pixelFormat - << " to " << outCfg.pixelFormat << ")"; - return -EINVAL; + const auto it = supportedConversions.find(inCfg.pixelFormat); + if (it == supportedConversions.end()) { + LOG(YUV, Error) << "Unsupported source format " << inCfg.pixelFormat; + return -EINVAL; + } + + std::vector outFormats = it->second; + const auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat); + if (match == outFormats.end()) { + LOG(YUV, Error) << "Requested pixel format conversion is not supported" + << " (from " << inCfg.pixelFormat + << " to " << outCfg.pixelFormat << ")"; + return -EINVAL; + } + } else { + if (inCfg.pixelFormat != formats::NV12) { + LOG(YUV, Error) << "Unsupported format " << inCfg.pixelFormat + << " (only NV12 is supported for scaling)"; + return -EINVAL; + } } if (inCfg.size < outCfg.size) { @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg, return -EINVAL; } - if (inCfg.pixelFormat != formats::NV12) { - LOG(YUV, Error) << "Unsupported format " << inCfg.pixelFormat - << " (only NV12 is supported)"; - return -EINVAL; - } - calculateLengths(inCfg, outCfg); return 0; } @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf return; } - int ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(), - sourceStride_[0], - sourceMapped.planes()[1].data(), - sourceStride_[1], - sourceSize_.width, sourceSize_.height, - destination->plane(0).data(), - destinationStride_[0], - destination->plane(1).data(), - destinationStride_[1], - destinationSize_.width, - destinationSize_.height, - libyuv::FilterMode::kFilterBilinear); + int ret = 0; + switch (sourceFormat_) { + case formats::NV12: + ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(), + sourceStride_[0], + sourceMapped.planes()[1].data(), + sourceStride_[1], + sourceSize_.width, sourceSize_.height, + destination->plane(0).data(), + destinationStride_[0], + destination->plane(1).data(), + destinationStride_[1], + destinationSize_.width, + destinationSize_.height, + libyuv::FilterMode::kFilterBilinear); + break; + case formats::YUYV: + ret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(), + sourceStride_[0], + destination->plane(0).data(), + destinationStride_[0], + destination->plane(1).data(), + destinationStride_[1], + destinationSize_.width, + destinationSize_.height); + break; + default: + LOG(YUV, Error) << "Unsupported source format " << sourceFormat_; + processComplete.emit(streamBuffer, PostProcessor::Status::Error); + break; + } + if (ret) { - LOG(YUV, Error) << "Failed NV12 scaling: " << ret; + LOG(YUV, Error) << "Libyuv operation failure: " << ret; processComplete.emit(streamBuffer, PostProcessor::Status::Error); return; }