From patchwork Fri Sep 15 07:57:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattijs Korpershoek X-Patchwork-Id: 19007 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8D7AEBD160 for ; Fri, 15 Sep 2023 07:57:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6F72462922; Fri, 15 Sep 2023 09:57:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694764651; bh=BlimaKRl0xyLPQ0r4LM+wfkESh4Lofh1vu5Ia6W5Lns=; h=Date:References:In-Reply-To:To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=zqBrziG973kVpkXnlw6dhR96AM2xGB1S5etxutBLVc81UFTLjMPvOQ+dNo0YtIHcE 6FdnVWRwEM8IN2u4nrfLxBwG76tiAMMV0VhQ7YZEZ+OtWZ0P5Ng5UAl47H/InrTxKD FBtscG4PWT2u/UQsoyJv40S8WTAdjgCvnkJTieZ48Vv9X8zFArgZT7IY5PEiSonf3V hg9M21fVrNtoI3CAk3m7gLGScE+otZj89mlV1Q5YQVVkMPn8Rj7QA0AlTqpsGFGJ+R Qg5el2ixSaVNbmKYSOJt8ZnridnJkXnAI3MHeGGqB1rawxYDrDRouyERFeDwSeQqwF D3r8BNIkbab/w== Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CC74262911 for ; Fri, 15 Sep 2023 09:57:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.b="NFJqjrLI"; dkim-atps=neutral Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-401b393ddd2so21128535e9.0 for ; Fri, 15 Sep 2023 00:57:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1694764648; x=1695369448; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=FR44mURV+pW3PtQs626ezxxVyqhFayqZPrHLiDOr1BA=; b=NFJqjrLIX290GGyJ8EUEM+zHhjN60okgBgvoJJp/zkhNnBV7fC9f65Je53A/kQoI+P nhQc6p873JMr+7T3iC69cTmDmoWrOvpte6BrNqR6VIcHXTj0QEeFTEjqlfzUVbLAwAek a0y/dQIiT5SPcnArEP1y1En64aXkCUSBHZoNj2gn+ZR4pV01Szh2FG5U8QoT30w76ixP 8A4VFngT1lC0wOM4eNxh2+RFRE+/mWuRWhN90flitrfGQfl13lMrWK++ZxnVt80TQF8z UeFKRlHBP9BU+MWop5FzWVTqm+Y1pfWWJ/t2a0Q72uk/FK6W1KsCXoXTQn22TjrchOVM yeLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764648; x=1695369448; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FR44mURV+pW3PtQs626ezxxVyqhFayqZPrHLiDOr1BA=; b=hy7bBWX1nrPwHc+qgYgeF2B7/0vGeT5iM+EIntuUjJyF/xPpiwc9vIXzDsUlKiD6PG EVFfZBuriUMXulFHILY8S/wHTLl7rCIbc/UdQkcWTFpOXhEOqRX8NSCPELE6NJpozCeY xDeRLcZzvvzbVCFEIxZhyaCp62qDvaOMndXrru+yrnY8KIdBQ5rVNIfOPlJGkXqCTl1S NfDEjSoNWoNw9ei6UYh7vUMh7Ahqvt6VwvrsPVTtoWYXsE0ssqNu+5pos8ppaaISv5nZ SkWwnkl/+pttMW4V97qEuCpmnVLK5gA5oMyZT354PTK06qCb1nPQU+6jebNY7zeOSY5d DS+g== X-Gm-Message-State: AOJu0Yzg8HFxG79cC1QxsDWtFk8kywQBZJ47yyPMnSdT+jJMxxxrrkQ/ NaGhvQd0u480L8lmPv6pWm1y0zzW1th15t/f3rY= X-Google-Smtp-Source: AGHT+IG/o48nMAFDqtTdhmW98ZQ9QbtRFVthl1zYYqzjnCnJafEHyqeZp5w7BIvqTIikacsctL5ALA== X-Received: by 2002:adf:b604:0:b0:31f:f662:8ec5 with SMTP id f4-20020adfb604000000b0031ff6628ec5mr543006wre.55.1694764648249; Fri, 15 Sep 2023 00:57:28 -0700 (PDT) Received: from [192.168.1.20] ([2a01:cb19:8704:be00:4f55:bd9d:611a:6c8e]) by smtp.gmail.com with ESMTPSA id m2-20020a056000174200b0031fe9a47a87sm2506942wrf.112.2023.09.15.00.57.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 00:57:27 -0700 (PDT) Date: Fri, 15 Sep 2023 09:57:25 +0200 MIME-Version: 1.0 Message-Id: <20230915-libyuv-convert-v1-1-1e5bcf68adac@baylibre.com> References: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> In-Reply-To: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> To: libcamera-devel@lists.libcamera.org X-Mailer: b4 0.12.4-dev-6aa5d Subject: [libcamera-devel] [PATCH RFC 1/7] android: yuv: separate source destination in length check X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mattijs Korpershoek via libcamera-devel From: Mattijs Korpershoek Reply-To: Mattijs Korpershoek Cc: Guillaume La Roque Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Right now, the assumption of PostProcessorYuv is that both the source and the destination are in formats::NV12. This might change in the future, if we add other formats or pixel format conversion as supported in libyuv. Split out source and destination check to prepare for that. No functional change. Signed-off-by: Mattijs Korpershoek --- src/android/yuv/post_processor_yuv.cpp | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index ed44e6fe02da..9631c9617154 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -133,14 +133,17 @@ void PostProcessorYuv::calculateLengths(const StreamConfiguration &inCfg, sourceSize_ = inCfg.size; destinationSize_ = outCfg.size; - const PixelFormatInfo &nv12Info = PixelFormatInfo::info(formats::NV12); + const PixelFormatInfo &sourceInfo = PixelFormatInfo::info(formats::NV12); for (unsigned int i = 0; i < 2; i++) { sourceStride_[i] = inCfg.stride; - destinationStride_[i] = nv12Info.stride(destinationSize_.width, i, 1); + sourceLength_[i] = sourceInfo.planeSize(sourceSize_.height, i, + sourceStride_[i]); + } - sourceLength_[i] = nv12Info.planeSize(sourceSize_.height, i, - sourceStride_[i]); - destinationLength_[i] = nv12Info.planeSize(destinationSize_.height, i, - destinationStride_[i]); + const PixelFormatInfo &destinationInfo = PixelFormatInfo::info(formats::NV12); + for (unsigned int i = 0; i < 2; i++) { + destinationStride_[i] = sourceInfo.stride(destinationSize_.width, i, 1); + destinationLength_[i] = sourceInfo.planeSize(destinationSize_.height, i, + destinationStride_[i]); } } From patchwork Fri Sep 15 07:57:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattijs Korpershoek X-Patchwork-Id: 19008 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D3A78C3260 for ; Fri, 15 Sep 2023 07:57:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 456AC6291C; Fri, 15 Sep 2023 09:57:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694764652; bh=Gmyz+JFbdgJxh5sAYtlji1fX/uQaXkb+Fc2zUb8qSPM=; h=Date:References:In-Reply-To:To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=t4vGyKs8nN/RmYolPeyDjcHv4B5jkdcuVfAmSS7o/kz4Lk29ZYyL85vYu/ekoOqKw CxruRGOwZtASKV7fJc96kgelSKRTJRlgjzVvv+qRnqXYs3Qu1uJ+aUoyHZ9LQ2+6se Elz2Z3g9tbtrY6Os5XVNR3E1OvqnaX1gKDPylNHHAgmET2EG6jnqQGnstVn/KnaBgN +8HPASxpSgsZbo+mZPZmkBI5B+wGjcDMIssile/BlU8OgS9ZvVXoFScL/he6MiEyyt do+w3P6ao8GE8qnXHpOvbWYqpsO8OwiktDlZZR3ImeDR394e+7sBTU+WS36FcYEtAc d1NwGE8pywKRg== Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4578A62911 for ; Fri, 15 Sep 2023 09:57:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.b="npbt+OKi"; dkim-atps=neutral Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-404773f2501so10197855e9.0 for ; Fri, 15 Sep 2023 00:57:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1694764649; x=1695369449; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=vZTpit9hOwhOaXXsmkIT8ZLwPgPwB4u4Lnyf5QP21iw=; b=npbt+OKiRvslG9ezoG8pHHS2mjP6ED1hiHLCkuarfIy3Fa3zlouy+LqHSYuvCumSoy /7DbmuSt8q/CJ6OQ3dkmxNusc0e20M02QoRvowIMLA4Mi3KRpdipJlWOA0zRBGRjLo7L sEICo27dLWj/OvM2yEbZrqhTrBAsrwuwy1K9WfkVqrmb+82ZBfsLnVHfvraGOkxZbKa/ S7L3+9myUQLdGV+0BWnM2g74nWKUhRqk7Crq2wn0SdofaXqGNXL0LwvZmrdPO2Uqnr62 CGFBhPNZzkBeyBVo5vXGoswupXAERlksO3y5H3x8snDpCCVgouaZYRyJqOYqXQsx7XGQ 1DCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764649; x=1695369449; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vZTpit9hOwhOaXXsmkIT8ZLwPgPwB4u4Lnyf5QP21iw=; b=C4L3SfKq5SiPLsmIbrvsxUKTHoR07T8wDjpb1BAUu1gGKW9U1ZBhoDQVA1et/vty4h pSZ/ajExS9JoRyQtYm4FsW3MXIKG8W039a+Rol5bsJIqeZrlaT/RLkDhIrZVthC3BKCc koW18zkgjNBcl0nkgWE5Bo3msA2u9lF3V5chSSUvt/654MBQwxmugMerPf24EBJEqwv6 jW8knKE1yIu+izMNGIWH2Tq2IOZFVeJBTpo6d7w3WRc8fCfTNIW8bNEhzc+pcYXGAqde UKBGj2QkTMG9h9/XS72DkrWPCMhoUuHeF35fVZYMfM4L06JbnF5JcBVCAp3/QnrNdkun 7HiQ== X-Gm-Message-State: AOJu0YxEcN3mbeOvxIOsdK1wifK2rrQ3iWjqRnO0QE+DIKLy3ov2BKex PA0kueWxOdXNJiQ70JTU38peDQ== X-Google-Smtp-Source: AGHT+IEHF4VshMwkLrBmR5lEmGo/34nPonQBdXtP86jwoLsGjlpUIeoQIgi2rxvhPNAjAqCjWfkn6g== X-Received: by 2002:adf:a303:0:b0:31f:f84e:f63e with SMTP id c3-20020adfa303000000b0031ff84ef63emr305113wrb.54.1694764648864; Fri, 15 Sep 2023 00:57:28 -0700 (PDT) Received: from [192.168.1.20] ([2a01:cb19:8704:be00:4f55:bd9d:611a:6c8e]) by smtp.gmail.com with ESMTPSA id m2-20020a056000174200b0031fe9a47a87sm2506942wrf.112.2023.09.15.00.57.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 00:57:28 -0700 (PDT) Date: Fri, 15 Sep 2023 09:57:26 +0200 MIME-Version: 1.0 Message-Id: <20230915-libyuv-convert-v1-2-1e5bcf68adac@baylibre.com> References: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> In-Reply-To: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> To: libcamera-devel@lists.libcamera.org X-Mailer: b4 0.12.4-dev-6aa5d Subject: [libcamera-devel] [PATCH RFC 2/7] android: yuv: loop over each plane for size check X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mattijs Korpershoek via libcamera-devel From: Mattijs Korpershoek Reply-To: Mattijs Korpershoek Cc: Guillaume La Roque Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" PostProcessorYuv assumption is that only formats::NV12 is supported for source and destination pixelFormat. Because of this, we don't loop on each plane when size checking, but we always assume a 2 plane buffer. To prepare for adding more YUV formats such as YUYV, loop over the planes instead of using an if. No functional change, besides the logs only printing the first faulty plane length. Signed-off-by: Mattijs Korpershoek Reviewed-by: Jacopo Mondi --- src/android/yuv/post_processor_yuv.cpp | 40 +++++++++++++++++----------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index 9631c9617154..d58090db14ee 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -101,27 +101,27 @@ bool PostProcessorYuv::isValidBuffers(const FrameBuffer &source, return false; } - if (source.planes()[0].length < sourceLength_[0] || - source.planes()[1].length < sourceLength_[1]) { - LOG(YUV, Error) - << "The source planes lengths are too small, actual size: {" - << source.planes()[0].length << ", " - << source.planes()[1].length - << "}, expected size: {" - << sourceLength_[0] << ", " - << sourceLength_[1] << "}"; - return false; + for (unsigned int i = 0; i < 2; i++) { + if (source.planes()[i].length < sourceLength_[i]) { + LOG(YUV, Error) + << "The source planes lengths are too small, " + << "actual size[" << i << "]=" + << source.planes()[i].length + << ", expected size[" << i << "]=" + << sourceLength_[i]; + return false; + } } - if (destination.plane(0).size() < destinationLength_[0] || - destination.plane(1).size() < destinationLength_[1]) { - LOG(YUV, Error) - << "The destination planes lengths are too small, actual size: {" - << destination.plane(0).size() << ", " - << destination.plane(1).size() - << "}, expected size: {" - << sourceLength_[0] << ", " - << sourceLength_[1] << "}"; - return false; + for (unsigned int i = 0; i < 2; i++) { + if (destination.plane(i).size() < destinationLength_[i]) { + LOG(YUV, Error) + << "The destination planes lengths are too small, " + << "actual size[" << i << "]=" + << destination.plane(i).size() + << ", expected size[" << i << "]=" + << sourceLength_[i]; + return false; + } } return true; From patchwork Fri Sep 15 07:57:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattijs Korpershoek X-Patchwork-Id: 19009 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AF1BABD160 for ; Fri, 15 Sep 2023 07:57:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C01FC62927; Fri, 15 Sep 2023 09:57:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694764652; bh=0hHduLXnGmQRUinzMw3HA8Pdweuxi4OWV720ADh/2cI=; h=Date:References:In-Reply-To:To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=R5CzMKXxImhKZbpAYgZ9dcXkn/ytdf1NciRSuUqCYppLvC4Y7ZLsulZY98jk/+fWy fap9m/1qFFQIbSlJkyYfe5NBZwBnvPl6ImdCu4ohmESsyIJwXo09pFDL4Jb2JstbcN kCATv5K3TBW5CVRKAlGXmlUEsMg7rwoA349WbamsIEdKcsxDXFarfURSC0AyrWOWvY Mr09XCoaYZPx7S3R57kvlyL15lX2ilEploI8xhemSQLjF2WnTZH0zQFjD2IuSGEjqs Xk4G2OzUWzqPsNgWjqmy637IJ+5ZTp0tg3Y90kgf1og/n6Lm2Vmh4+ttjSf6fsIb// 8H1/iwMtI7b5Q== Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D1CF16291A for ; Fri, 15 Sep 2023 09:57:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.b="Pg6kyn7C"; dkim-atps=neutral Received: by mail-wr1-x434.google.com with SMTP id ffacd0b85a97d-313e742a787so1181775f8f.1 for ; Fri, 15 Sep 2023 00:57:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1694764649; x=1695369449; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=b/wOgWma/1S7FO6/zEXOhIIMNxsMj5cc2vXJjrkX00w=; b=Pg6kyn7Cu+8gSvX8VjPyGujWhHvyg+Lz6l3UG0OB7wWFA0jSl5cYewawwZmFsxWC5h 08OEPYl4uNeFxwxosNKbk0FQ1sjOhSR4szh2wzETbiqFw3QJ677IAfof2TvEbLRx3A3T CklEP0b+0zgjgRfCfBllWV0y384FTIA+FLhf2MwE18uhfRlk3KlrOKWIEfHnn8tsmZL/ Ddq1FuilEnJBNkaMtLks+IQq94ZcrNVdP3A1k3wzPSkfgYLDMijPlPbkWwEy9nk5TDiG hxA3noYonXXQM/uoWUSX55iic0G0au9TH+CmjQL/tz/GiAJ2XbgRW2QqUiGTPi/B6Qqm q1Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764649; x=1695369449; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b/wOgWma/1S7FO6/zEXOhIIMNxsMj5cc2vXJjrkX00w=; b=uYXqoyPtJgTdMGsnwl4QN73uaXdSAPWfuQA3UTVuWMxiK7sUIJ90cahbhq49Hp7G7Z qXINxzoIddvfNtFs5fFw5qavST65KYBhs4VmYEN3H+t7yPKCEM6jBJcmqf+9dD59vWNZ EVKHhjp9cTuJ+F/Byh2aIgsVg/FWetZvWUTuZCnscS9pnTUAVgWwex2zW12Irc+Mt5fZ QpludYYxdz2Q7/qiJhn9LyKCtwKOFX0l3sFSgIIklLr/hiDQ8Cxk6eYqlE33SVdJtHtp +uMjVJpL3Ja3gFnxKvoliM4YNRyZmTXAYZsrx/g4kcOi/V5mvX/btMTOPv7BnorAk+dw AMIQ== X-Gm-Message-State: AOJu0Ywy2uaumUakvp7MbVDiJoN5fOSGn1UVLvGBNTbxCpdDbiMbb9AN 3qtki7r4E3tpOSgpzZlZpBwczQ== X-Google-Smtp-Source: AGHT+IHyYHYC0JnSqUmB2Q5e6L82bCm/r40Bkes++y72PayEMp/FFHAFkQBpVTJ/83t2PsndqIeu2w== X-Received: by 2002:adf:fc42:0:b0:31f:d8be:a313 with SMTP id e2-20020adffc42000000b0031fd8bea313mr599172wrs.5.1694764649474; Fri, 15 Sep 2023 00:57:29 -0700 (PDT) Received: from [192.168.1.20] ([2a01:cb19:8704:be00:4f55:bd9d:611a:6c8e]) by smtp.gmail.com with ESMTPSA id m2-20020a056000174200b0031fe9a47a87sm2506942wrf.112.2023.09.15.00.57.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 00:57:29 -0700 (PDT) Date: Fri, 15 Sep 2023 09:57:27 +0200 MIME-Version: 1.0 Message-Id: <20230915-libyuv-convert-v1-3-1e5bcf68adac@baylibre.com> References: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> In-Reply-To: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> To: libcamera-devel@lists.libcamera.org X-Mailer: b4 0.12.4-dev-6aa5d Subject: [libcamera-devel] [PATCH RFC 3/7] android: yuv: prepare support for other pixel formats X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mattijs Korpershoek via libcamera-devel From: Mattijs Korpershoek Reply-To: Mattijs Korpershoek Cc: Guillaume La Roque Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" PostProcessorYuv assumes that both source and destination pixel formats will always be formats::NV12. This might change in the future, for example when doing pixel format conversion via libyuv. Add the necessary plumbing so that other formats can be added easily in the future. Also increase plane number to 3, since some YUV format [1] are fully planar (and thus have 3 planes). [1] https://docs.kernel.org/userspace-api/media/v4l/pixfmt-yuv-planar.html#fully-planar-yuv-formats Signed-off-by: Mattijs Korpershoek Reviewed-by: Jacopo Mondi --- src/android/yuv/post_processor_yuv.cpp | 26 +++++++++++++++----------- src/android/yuv/post_processor_yuv.h | 12 ++++++++---- 2 files changed, 23 insertions(+), 15 deletions(-) diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index d58090db14ee..734bb85b7351 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -90,18 +90,18 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf bool PostProcessorYuv::isValidBuffers(const FrameBuffer &source, const CameraBuffer &destination) const { - if (source.planes().size() != 2) { + if (source.planes().size() != sourceNumPlanes_) { LOG(YUV, Error) << "Invalid number of source planes: " << source.planes().size(); return false; } - if (destination.numPlanes() != 2) { + if (destination.numPlanes() != destinationNumPlanes_) { LOG(YUV, Error) << "Invalid number of destination planes: " << destination.numPlanes(); return false; } - for (unsigned int i = 0; i < 2; i++) { + for (unsigned int i = 0; i < sourceNumPlanes_; i++) { if (source.planes()[i].length < sourceLength_[i]) { LOG(YUV, Error) << "The source planes lengths are too small, " @@ -112,7 +112,7 @@ bool PostProcessorYuv::isValidBuffers(const FrameBuffer &source, return false; } } - for (unsigned int i = 0; i < 2; i++) { + for (unsigned int i = 0; i < destinationNumPlanes_; i++) { if (destination.plane(i).size() < destinationLength_[i]) { LOG(YUV, Error) << "The destination planes lengths are too small, " @@ -132,18 +132,22 @@ void PostProcessorYuv::calculateLengths(const StreamConfiguration &inCfg, { sourceSize_ = inCfg.size; destinationSize_ = outCfg.size; + sourceFormat_ = inCfg.pixelFormat; + destinationFormat_ = outCfg.pixelFormat; - const PixelFormatInfo &sourceInfo = PixelFormatInfo::info(formats::NV12); - for (unsigned int i = 0; i < 2; i++) { + const PixelFormatInfo &sourceInfo = PixelFormatInfo::info(sourceFormat_); + sourceNumPlanes_ = sourceInfo.numPlanes(); + for (unsigned int i = 0; i < sourceInfo.numPlanes(); i++) { sourceStride_[i] = inCfg.stride; sourceLength_[i] = sourceInfo.planeSize(sourceSize_.height, i, sourceStride_[i]); } - const PixelFormatInfo &destinationInfo = PixelFormatInfo::info(formats::NV12); - for (unsigned int i = 0; i < 2; i++) { - destinationStride_[i] = sourceInfo.stride(destinationSize_.width, i, 1); - destinationLength_[i] = sourceInfo.planeSize(destinationSize_.height, i, - destinationStride_[i]); + const PixelFormatInfo &destinationInfo = PixelFormatInfo::info(destinationFormat_); + destinationNumPlanes_ = destinationInfo.numPlanes(); + for (unsigned int i = 0; i < destinationInfo.numPlanes(); i++) { + destinationStride_[i] = destinationInfo.stride(destinationSize_.width, i, 1); + destinationLength_[i] = destinationInfo.planeSize(destinationSize_.height, i, + destinationStride_[i]); } } diff --git a/src/android/yuv/post_processor_yuv.h b/src/android/yuv/post_processor_yuv.h index a7ac17c564b6..bfe35f46c6dc 100644 --- a/src/android/yuv/post_processor_yuv.h +++ b/src/android/yuv/post_processor_yuv.h @@ -28,8 +28,12 @@ private: libcamera::Size sourceSize_; libcamera::Size destinationSize_; - unsigned int sourceLength_[2] = {}; - unsigned int destinationLength_[2] = {}; - unsigned int sourceStride_[2] = {}; - unsigned int destinationStride_[2] = {}; + libcamera::PixelFormat sourceFormat_; + libcamera::PixelFormat destinationFormat_; + unsigned int sourceLength_[3] = {}; + unsigned int destinationLength_[3] = {}; + unsigned int sourceStride_[3] = {}; + unsigned int destinationStride_[3] = {}; + unsigned int sourceNumPlanes_; + unsigned int destinationNumPlanes_; }; From patchwork Fri Sep 15 07:57:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattijs Korpershoek X-Patchwork-Id: 19010 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3A111C32B0 for ; Fri, 15 Sep 2023 07:57:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6C57D6291D; Fri, 15 Sep 2023 09:57:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694764653; bh=4ps3su9sccE3KcyunR2qOZ3yin2JQeACKjKB208QI34=; h=Date:References:In-Reply-To:To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=JlDE/qfRL5o7hNr9tKXJZIGgHWljhAqwp4hyYXBmweREhdF8GNnCech20T11Hpi5o Y62sIUPk5Szso39sFXx4ZsghNuGip2UpXCaCalxDEJcBSJFC1LRR9ELxy5f1Ogu7gG HTvBKgqA3WE0VLMCa5WbsefDQ32G1ChO14BlCHA4loUzKsW1+sj4ybK4glzsc58QbX 7Ou6GGRRRUps8OyUD3ZDvVFfMIGmwmaq9a3kn9szdudw0aMfDR/q6BqZIxj3o7GZkO JEMV40yZWJ8PAzynL8VLw6AECvPie/SIXDXad5Qhj3f/dZM5MwPNp21l0RkLmOd2RJ UJZAZNej5rrTQ== Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7200262911 for ; Fri, 15 Sep 2023 09:57:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.b="CFwB92r3"; dkim-atps=neutral Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-307d58b3efbso1673981f8f.0 for ; Fri, 15 Sep 2023 00:57:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1694764650; x=1695369450; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=q8b8Y5ZxAxJoxZYo1iZ+1EV95/CTGwBGZwZrmCm9cTk=; b=CFwB92r3B0RJ5ySxaKY5kCtE8Z+gv3AuaY0jq4T2ySHhBNvi1hVezkULPCTU3bX2AD 75nADo/Wv5gfOLxBJpEHtk+D+njeuA/KZmkR2/M15vXoXTdHvRs3JzAWnD9h24wIxju0 i7uk3NaxjATA++CcQ4C1KnEbb5NGx4CkY34IBnowjSWvAgnKEJky9AaA+mbjqWfFCRM/ QD52+wkM4m/UVUNEAPex0nLZf/yZxAnEwa1gysU8nxFdNOjGjcWHcnzOAjHYoApaFI03 pB7cQ9CXMsOl5wewFiJijynogighXmDEgONtroqOf50zvyPu3BrbQEnQKk556IquFJ4X Ag9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764650; x=1695369450; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q8b8Y5ZxAxJoxZYo1iZ+1EV95/CTGwBGZwZrmCm9cTk=; b=bB1ZaddDfmQGENdWWsE6GVhOfl3f4gp/NrjxqzL26WsRLxPyaShMoeUszUvlXzRRLW yyA4xCIUYeWtcMOrMgEU09RwDqY7aQTIs9aniH3FhqmZBkgjzl4r6o9R1VsLHifbuIBv Nyf0P04psaHlWeMjUcIVRNPhzPrd1zZtwBaKRM/VSF5BMnTQfQ4KObJq/ElDeKhFLc/g rSDtmAgPu871q6MXgS64xF8PNV+XqnbVq/pkWD47UV7m9sy00WX3tW3b0MFSH+kTm78T 59KpnG3DmJD99kZ+VEK36lvx4tj9GzB24WImbXy4Pc2fNHlTGUKRO3xN+x85+dJLUWpN +a9g== X-Gm-Message-State: AOJu0Yy7jFnixZAY/sU4VlLm+iJdsgPpfjaY2V00wCusK4Fc78xYt7B1 VyvZ2c848qnoT1J7EKROATrBJGffzVatv98vASk= X-Google-Smtp-Source: AGHT+IEnFidxUslLAIz8lykDzfVgh4ciiXc6ftDajrKB5KwNJ3dXqWnAWJ8gydLMoEZ1ztxJ6IGWcw== X-Received: by 2002:a5d:534c:0:b0:314:1f1e:3a85 with SMTP id t12-20020a5d534c000000b003141f1e3a85mr638608wrv.61.1694764650073; Fri, 15 Sep 2023 00:57:30 -0700 (PDT) Received: from [192.168.1.20] ([2a01:cb19:8704:be00:4f55:bd9d:611a:6c8e]) by smtp.gmail.com with ESMTPSA id m2-20020a056000174200b0031fe9a47a87sm2506942wrf.112.2023.09.15.00.57.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 00:57:29 -0700 (PDT) Date: Fri, 15 Sep 2023 09:57:28 +0200 MIME-Version: 1.0 Message-Id: <20230915-libyuv-convert-v1-4-1e5bcf68adac@baylibre.com> References: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> In-Reply-To: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> To: libcamera-devel@lists.libcamera.org X-Mailer: b4 0.12.4-dev-6aa5d Subject: [libcamera-devel] [PATCH RFC 4/7] android: camera_device: support when no Direct stream is found X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mattijs Korpershoek via libcamera-devel From: Mattijs Korpershoek Reply-To: Mattijs Korpershoek Cc: Guillaume La Roque Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As the comment states, it's possible that the first element in streamConfig.streams is a Internal stream. In that case sourceStream stays a nullptr and all the Mapped streams are mapped towards a nullptr. Make sure that the Mapped streams are mapped to an Internal stream in case that's the first element. Signed-off-by: Mattijs Korpershoek Reviewed-by: Laurent Pinchart --- src/android/camera_device.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 1f7ce4402b63..d34bae715a47 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -724,7 +724,8 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) * be used when constructing the subsequent mapped * streams. */ - if (stream.type == CameraStream::Type::Direct) + if (stream.type == CameraStream::Type::Direct || + stream.type == CameraStream::Type::Internal) sourceStream = &streams_.back(); } } From patchwork Fri Sep 15 07:57:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattijs Korpershoek X-Patchwork-Id: 19011 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 97091C3260 for ; Fri, 15 Sep 2023 07:57:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 265EE62925; Fri, 15 Sep 2023 09:57:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694764655; bh=jVKLL5QEF5z5C0TNFbo0tgqqRfWikpDWJ129oNMsrl0=; h=Date:References:In-Reply-To:To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Yk1bCfA91vEmPWOSkYqt/ARouZOmxndzhsFS5dOp5NATGmZ8lZQJKcsG3VUJLjybs Ss7TCNwoWrRaF3V5YcyPUe035ovtvOjMDfUvXiiK+Grag+gR/k0sHbLv7gS3N6mziv Q0Lvm1vCDEPIvsvzRxDrAtXzfUqs37aJJcFESwhvgwY/QzAtHL/r8BqrQhFEo4wyEo LcFTJnx6afNRzaYXD35Cgl2j8ZwEBwN9uH03gFV949qdjwdfm76scIx2hIqTw/08eF CHvIIm4M2kasiM53rQ7t7q/2H1QavXTpF9jypujt1DqKTXbLhPt56/LaK0GBHInPjx k6sMjPPHH4SBg== Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4771F62921 for ; Fri, 15 Sep 2023 09:57:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.b="JlmqQ/hU"; dkim-atps=neutral Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-402cc6b8bedso21187035e9.1 for ; Fri, 15 Sep 2023 00:57:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1694764650; x=1695369450; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=7Yd76vAKqd9nCx1+b0DVl9wG9TBhJs7jAHDcebruqiE=; b=JlmqQ/hU5CxrdP7dFcuKk3cvEqHMxjvbLlghZ85mdx0KX2gB2ScDHb7T5sI7bszUuI e7DI2ZkLY/PZ2tHF6LtOuxbggzf5iP7BTuzOLHSu6ASSQzGK3ZFBUPxOPo7JqdvonNQc RvhY99CGvgPdbpHqLHNj0/5WvcmFz9xPshHrTzNb/yAyLWr4xgtZBocftelanMus0r1Q 1cE3ut1v/jJp962G8gY5d0hvpfI2xgFN/UU/jYHhXsPh1EWdIiAMeBt67cG80NAr3J5U VsHYEK03YfOfzGpjRQ/XamGIRQhC/fJ4kbwbucLyQ7bnjCHyyoDzawQdGTq5aR2Wt3js 28Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764650; x=1695369450; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7Yd76vAKqd9nCx1+b0DVl9wG9TBhJs7jAHDcebruqiE=; b=Nv7UMmFBCsZLJdesOrgYutftK6/a5oYE/4+/djDm/l1kTjzDmaxBjuvorC+npzCBFu 27kr6G6/n7qIiYscNymYzqSUAh2shbBy1z9TDtsKlZy0ArY6cSLcP3Qt2LwV9wv+3jv9 5N/bEw/LOHITeokyl2Z5odirft27US5/rQYCRB0ExhF5zzFuqJbaUxhds5fBnhe6/him tqJUM/QBoO12Uo0+Ghwjn32+HlCNTT8tkdVVnOnLXpSdsQeGuEd8kTmIPZYj5EsQisOH X1ujBHdAe029JYrXRZ1ze8fz/ONPOqwwTh1QumEVUC30zgGYgaD86qDmXq6GuPOhwrX4 4CvQ== X-Gm-Message-State: AOJu0YyR71EdZIe1v7W/GPMPQhWxCx/43LfhPIMw9tT+AFdz+a+DSWh6 7P3CuOCVXahoB0AF/bjQF3wslg== X-Google-Smtp-Source: AGHT+IHP8tAsP3EL6+g/m2wKXhxeM8R24gXxBLlhhy0By/5lmTstw9JxaZuOhknj7d2gzDFgpJZ8uw== X-Received: by 2002:adf:e845:0:b0:31a:d6cb:7f94 with SMTP id d5-20020adfe845000000b0031ad6cb7f94mr692758wrn.23.1694764650741; Fri, 15 Sep 2023 00:57:30 -0700 (PDT) Received: from [192.168.1.20] ([2a01:cb19:8704:be00:4f55:bd9d:611a:6c8e]) by smtp.gmail.com with ESMTPSA id m2-20020a056000174200b0031fe9a47a87sm2506942wrf.112.2023.09.15.00.57.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 00:57:30 -0700 (PDT) Date: Fri, 15 Sep 2023 09:57:29 +0200 MIME-Version: 1.0 Message-Id: <20230915-libyuv-convert-v1-5-1e5bcf68adac@baylibre.com> References: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> In-Reply-To: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> To: libcamera-devel@lists.libcamera.org X-Mailer: b4 0.12.4-dev-6aa5d Subject: [libcamera-devel] [PATCH RFC 5/7] android: camera_stream: add add explicit input configuration X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mattijs Korpershoek via libcamera-devel From: Mattijs Korpershoek Reply-To: Mattijs Korpershoek Cc: Guillaume La Roque Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" PostProcessors are configured using and input StreamConfigurations. Right now, we use configuration() for the input StreamConfiguration. Use an intermediate variable instead, to prepare for software conversion using libyuv. Signed-off-by: Mattijs Korpershoek Reviewed-by: Jacopo Mondi --- src/android/camera_stream.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 045e60061a20..4fd05dda5ed3 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -87,6 +87,7 @@ int CameraStream::configure() if (type_ == Type::Internal || type_ == Type::Mapped) { const PixelFormat outFormat = cameraDevice_->capabilities()->toPixelFormat(camera3Stream_->format); + StreamConfiguration input = configuration(); StreamConfiguration output = configuration(); output.pixelFormat = outFormat; output.size.width = camera3Stream_->width; @@ -106,7 +107,7 @@ int CameraStream::configure() return -EINVAL; } - int ret = postProcessor_->configure(configuration(), output); + int ret = postProcessor_->configure(input, output); if (ret) return ret; From patchwork Fri Sep 15 07:57:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattijs Korpershoek X-Patchwork-Id: 19012 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 199D9C32B1 for ; Fri, 15 Sep 2023 07:57:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C643562922; Fri, 15 Sep 2023 09:57:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694764656; bh=IBL60Z2Vz20HmwGlky/i1f7Jcbli+IO2LFw0N0ieF8k=; h=Date:References:In-Reply-To:To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=tGEMjS5divFR3Adetvs4JyvQAnEvP8lmDxjdaY55gGNYybSlJSEGC9tCryAjqgRDw wH48RuCMX2MW3UYxUbig19jodA5c+yyXjShkSbmpLX+cqXm35JEu5XDyuj5EWljYsr Gs/AG9nN3ywhPFotZAH+afFL3+KBfRQUsy643lDry2QMpgzHbmn+mGzV7QV5Z7FR1d /QOdluB7GfbjjJ35Ywhqb9A4J9K1aqv9rX6tRSJcVv/SRNvyhaSzXpkLbmOkA0+cnV xXsh10zLp23jDyqLVjptpfJVi8/+dtMWQAXgygmNhBLToQuP8buKSgSu6Ud96pZIcC WuNhkBBnldlZA== Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E29616291E for ; Fri, 15 Sep 2023 09:57:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.b="lkKojsLj"; dkim-atps=neutral Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-31dd10c2b8bso1649808f8f.3 for ; Fri, 15 Sep 2023 00:57:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1694764651; x=1695369451; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=xovdNtMrWbtfBJY3joA+lnakbzSinm4xINrzUSJhrX8=; b=lkKojsLjb5hyBeopBpzA6oqAyb/3h6ES6TUUxsaGZ8Aac+NXSWr5xP0Geu4dC1GBb2 KyDMuEgI4RX1yu5nhcvMojLGl4lbSUaZicjnbuzCtdl4iStdWyWMjcq9xb6+0zev1df6 ZAPgb94bgH4iGhGCwIWhhzbxMRn9UgAfu4VcIpF3B0XSQOGQAPd2reLtSjTQT8dPOSeF KvSgxuiJCOERGhHeoQmNyxtqmcrODTyaaKa7LRfYPE2EAcD7Sgp6+gGwAeALAhbQ+bEr yQUO/aKk4dym4aBf+S/d4d4i3ZIFUscjam1+t5O9oswO4TKPlgB3IRgVI6KKuMiCtnFu 4rPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764651; x=1695369451; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xovdNtMrWbtfBJY3joA+lnakbzSinm4xINrzUSJhrX8=; b=aPOcGSePnJps5ECJVMv2FH/kI7SGJMM/vjeJ4Matw/sbivVMoYNZtFr5HyYztg7UWf +L32YoQjPxzDPoIk12vLD3Eodv/eJsPsS3C8Q6XWwZZQG52wxldvH4P7Uk6lj+tThVxC qJNihZuOEg2M5+Hp7aBM/FvfXEp6YmlpFl8WyryAYFTxh2HhYXJFtvTD7JQGIAWdOfKv ILfY1dAXKzilYMXYTrZsKvy6mRZ7CRA/xIK63II7GziAh/s+KH1Oq0ub2bOSrxfXCvWZ WUfi307NLdA+AZZAicDnRBdP4mvTnjEdUoRqnqP1vRJgqQ7/zhbCm3LoqA8HBecJbpLS lWPA== X-Gm-Message-State: AOJu0YwFGtg4jL/80bFZbH/BE14scfIRkC5hMFVwqQjKuK851MplT6JT Id/gOLskM92r/tPx35SCOHl+o/3oEj97f77/4fw= X-Google-Smtp-Source: AGHT+IE7DLoHjaO1Dpz99bNJFqTaqJW3nSxT40SOP8pdS7dcLdsr+utje8ItLsiajfLsuRrD4mEzvg== X-Received: by 2002:adf:fc8c:0:b0:319:7fdc:d15f with SMTP id g12-20020adffc8c000000b003197fdcd15fmr800723wrr.26.1694764651355; Fri, 15 Sep 2023 00:57:31 -0700 (PDT) Received: from [192.168.1.20] ([2a01:cb19:8704:be00:4f55:bd9d:611a:6c8e]) by smtp.gmail.com with ESMTPSA id m2-20020a056000174200b0031fe9a47a87sm2506942wrf.112.2023.09.15.00.57.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 00:57:31 -0700 (PDT) Date: Fri, 15 Sep 2023 09:57:30 +0200 MIME-Version: 1.0 Message-Id: <20230915-libyuv-convert-v1-6-1e5bcf68adac@baylibre.com> References: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> In-Reply-To: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> To: libcamera-devel@lists.libcamera.org X-Mailer: b4 0.12.4-dev-6aa5d Subject: [libcamera-devel] [PATCH RFC 6/7] android: yuv: add YUYV -> NV12 conversion X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mattijs Korpershoek via libcamera-devel From: Mattijs Korpershoek Reply-To: Mattijs Korpershoek Cc: Guillaume La Roque Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" On am62x platforms, the receiver driver (j721e-csi2rx) only supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY. The receiver and the sensor (ov5640) hardware are both capable of YUV420, however: * we are not aware of OV5640 being tested with YUV420 formats on any vendor tree. * NV12 has different line lines (even lines are twice as long as odd lines) Different line-sized DMA transfers have not been tested on the TI CSI-RX SHIM IP. On the other hand, the graphics allocator (gralloc) cannot allocate YUV422 buffers directly. It mainly allocated NV12 buffers when userspace requests HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED. Because of the above, we need a pixel conversion from YUYV to NV12, which is handled in the yuv processor via libyuv. Signed-off-by: Mattijs Korpershoek --- src/android/yuv/post_processor_yuv.cpp | 91 +++++++++++++++++++++++++--------- 1 file changed, 68 insertions(+), 23 deletions(-) diff --git a/src/android/yuv/post_processor_yuv.cpp b/src/android/yuv/post_processor_yuv.cpp index 734bb85b7351..b680b833dafb 100644 --- a/src/android/yuv/post_processor_yuv.cpp +++ b/src/android/yuv/post_processor_yuv.cpp @@ -7,6 +7,9 @@ #include "post_processor_yuv.h" +#include + +#include #include #include @@ -22,14 +25,42 @@ using namespace libcamera; LOG_DEFINE_CATEGORY(YUV) +namespace { + +/** + * \var supportedConversions + * \brief list of supported output pixel formats for an input pixel format + */ +const std::map> supportedConversions = { + { formats::YUYV, { formats::NV12 } }, +}; + +} /* namespace */ + int PostProcessorYuv::configure(const StreamConfiguration &inCfg, const StreamConfiguration &outCfg) { if (inCfg.pixelFormat != outCfg.pixelFormat) { - LOG(YUV, Error) << "Pixel format conversion is not supported" - << " (from " << inCfg.pixelFormat - << " to " << outCfg.pixelFormat << ")"; - return -EINVAL; + const auto it = supportedConversions.find(inCfg.pixelFormat); + if (it == supportedConversions.end()) { + LOG(YUV, Error) << "Unsupported source format " << inCfg.pixelFormat; + return -EINVAL; + } + + std::vector outFormats = it->second; + const auto &match = std::find(outFormats.begin(), outFormats.end(), outCfg.pixelFormat); + if (match == outFormats.end()) { + LOG(YUV, Error) << "Requested pixel format conversion is not supported" + << " (from " << inCfg.pixelFormat + << " to " << outCfg.pixelFormat << ")"; + return -EINVAL; + } + } else { + if (inCfg.pixelFormat != formats::NV12) { + LOG(YUV, Error) << "Unsupported format " << inCfg.pixelFormat + << " (only NV12 is supported for scaling)"; + return -EINVAL; + } } if (inCfg.size < outCfg.size) { @@ -39,12 +70,6 @@ int PostProcessorYuv::configure(const StreamConfiguration &inCfg, return -EINVAL; } - if (inCfg.pixelFormat != formats::NV12) { - LOG(YUV, Error) << "Unsupported format " << inCfg.pixelFormat - << " (only NV12 is supported)"; - return -EINVAL; - } - calculateLengths(inCfg, outCfg); return 0; } @@ -66,20 +91,40 @@ void PostProcessorYuv::process(Camera3RequestDescriptor::StreamBuffer *streamBuf return; } - int ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(), - sourceStride_[0], - sourceMapped.planes()[1].data(), - sourceStride_[1], - sourceSize_.width, sourceSize_.height, - destination->plane(0).data(), - destinationStride_[0], - destination->plane(1).data(), - destinationStride_[1], - destinationSize_.width, - destinationSize_.height, - libyuv::FilterMode::kFilterBilinear); + int ret = 0; + switch (sourceFormat_) { + case formats::NV12: + ret = libyuv::NV12Scale(sourceMapped.planes()[0].data(), + sourceStride_[0], + sourceMapped.planes()[1].data(), + sourceStride_[1], + sourceSize_.width, sourceSize_.height, + destination->plane(0).data(), + destinationStride_[0], + destination->plane(1).data(), + destinationStride_[1], + destinationSize_.width, + destinationSize_.height, + libyuv::FilterMode::kFilterBilinear); + break; + case formats::YUYV: + ret = libyuv::YUY2ToNV12(sourceMapped.planes()[0].data(), + sourceStride_[0], + destination->plane(0).data(), + destinationStride_[0], + destination->plane(1).data(), + destinationStride_[1], + destinationSize_.width, + destinationSize_.height); + break; + default: + LOG(YUV, Error) << "Unsupported source format " << sourceFormat_; + processComplete.emit(streamBuffer, PostProcessor::Status::Error); + break; + } + if (ret) { - LOG(YUV, Error) << "Failed NV12 scaling: " << ret; + LOG(YUV, Error) << "Libyuv operation failure: " << ret; processComplete.emit(streamBuffer, PostProcessor::Status::Error); return; } From patchwork Fri Sep 15 07:57:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattijs Korpershoek X-Patchwork-Id: 19013 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 161AEC32B2 for ; Fri, 15 Sep 2023 07:57:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 89BFF62921; Fri, 15 Sep 2023 09:57:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694764657; bh=YPidGSqcc2kQoHQvTM7e/QJD48dRt0kqNOJQpbXRc/k=; h=Date:References:In-Reply-To:To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Y7KDstVR+OqRHZof2Ot2U9a6wAh9TYjObg9caIQPiiyyrr+D1j00q220YRpTTIsHS HLS3+xgz9cmUoCBaiKrnsXociKy41qXwS4LypAIkz3Rzwg+tZLM25gjbFiu7mfDWpP uTIV+832jBZYjDLyZNTSMIbBNPvotd5f2azy0KfNOB2Do1H+qxXRv+XoGs4XZwf9cw u8JwQgDcBr/1K3d7Xb70lIj3Yj4ZwSq5944RYUB+mJCPCVBAQLJJ8vPz/0bwVw+O8o IGNG0WFi3a0Jmfi0tZ70Pogt6iheKIHlqSBXclYS7bDX3+l0P2UUyLhsV8ngmGOCjq +FGlyO1yBvP6Q== Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 75BF862925 for ; Fri, 15 Sep 2023 09:57:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.b="xIJ0+ULC"; dkim-atps=neutral Received: by mail-wr1-x433.google.com with SMTP id ffacd0b85a97d-31ad9155414so1650692f8f.3 for ; Fri, 15 Sep 2023 00:57:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1694764652; x=1695369452; darn=lists.libcamera.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=wDS1MohSUECROpdzIgju5oIQIUyzBwhqYjp/QXcKlUY=; b=xIJ0+ULC+Q7PajvxI7j2hnZo1RK3zmDHY2nicxgD/8LfynkLzIEbdRD+bBYgNICrrH QsZxzDP1UuEni51Y4ZitQdO5SzzdulQPrA6iwR78MkN1hSBSrEmZSiXUmjvGpTQajBUv fYpLYhu/D8j7I8Y3Jy524eaRh62guFw1xmUz+qBSKKlIbwtqmpexY7/UOe6+8EfaS8Sw q2Qyu078MMkmqxJid8cwtnrbwOVtieP83Xcrk8mUgRRhJrFc/uia+r93T48xKc4M40Zc AWZ2y3TvMvtKqMf9MWhNEm/cPymKxl8saYBYGV6XX1u3Elz765t+VeKXRLBbaGEh4Cp0 7EyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764652; x=1695369452; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wDS1MohSUECROpdzIgju5oIQIUyzBwhqYjp/QXcKlUY=; b=xNGjasd7eW35GD/0b1ua6swqisxp9fBbP37n+669davFwSHFup2R8jFG4CuiUUjGVS e+OIZLVxJCl5lKzX5mErI5fdYi92rJsg8+Es9uOuYmoq6j0dXL2UX25G86f5iQxtJUqs QOVYuc4aMkHxqitpKoHpEO8nJYiZlibXZ2z6VxzBhGMVrtGMqX69Zdgdzgw6ZQ75UDT1 s6P0Q+iKd/eaLymFNt/tf21h2pTMHfq0KWycGvt9xT7WzMHVz0LMbetrf2osLIR+jTRR TW/RCO2OGn8oRzT3WYLPI0gPqiaPh5w4sezZhHDutxh7B/z8GYcjLe8VmwMbpIoY0rd6 HJUg== X-Gm-Message-State: AOJu0YwRsyAyT+7jcnPEqHLlu32WuOxoFdaxKHjCg+hoedetW1oty/iN E/0tvW1AfU1CLG5ubY4/nFqbhAqqrKDMk43tTvo= X-Google-Smtp-Source: AGHT+IEn5r1+/swhrQCGIePp8ScDtHHUkkwAordKLV11fE6EDDzxnWDlL48LeypmU89FnNaLWR1hRQ== X-Received: by 2002:adf:d4c7:0:b0:31a:ed75:75d6 with SMTP id w7-20020adfd4c7000000b0031aed7575d6mr696501wrk.15.1694764651971; Fri, 15 Sep 2023 00:57:31 -0700 (PDT) Received: from [192.168.1.20] ([2a01:cb19:8704:be00:4f55:bd9d:611a:6c8e]) by smtp.gmail.com with ESMTPSA id m2-20020a056000174200b0031fe9a47a87sm2506942wrf.112.2023.09.15.00.57.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 00:57:31 -0700 (PDT) Date: Fri, 15 Sep 2023 09:57:31 +0200 MIME-Version: 1.0 Message-Id: <20230915-libyuv-convert-v1-7-1e5bcf68adac@baylibre.com> References: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> In-Reply-To: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> To: libcamera-devel@lists.libcamera.org X-Mailer: b4 0.12.4-dev-6aa5d Subject: [libcamera-devel] [PATCH RFC 7/7] WIP: android: add YUYV->NV12 format conversion via libyuv X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mattijs Korpershoek via libcamera-devel From: Mattijs Korpershoek Reply-To: Mattijs Korpershoek Cc: Guillaume La Roque Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For some platforms, it's possible that the gralloc implementation and the CSI receiver cannot agree on a pixel format. When that happens, there is usually a m2m converter in the pipeline which handles pixel format conversion. On platforms without pixel format converters, such as the AM62x, we need to do software conversion. The AM62x platform: * uses a CSI receiver (j721e-csi2rx), that only supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY. * Has a gralloc implementation that only supports of semi-planar YUV420 formats such as NV12. Implement YUYV->NV12 format conversion using libyuv. This is mainly done by transforming the first stream from Type::Direct into Type::Internal so that it goes through the post-processor loop. ``` The WIP: part is mainly around computeYUYVSize(): Since gralloc and j721e-csi2rx are incompatible, we need a way to get gralloc to allocate (NV12) the kernel-requested buffer length (YUYV). In other words, we should make sure that the first plane of the NV12 allocated buffer is long enough to fit a YUYV image. According to [1], NV12 has 8 bits (one byte) per component, and the first plane is the Y component. So a 1920x1080 image in NV12 has plane[0].length=1920*1080=2073600 According to [2], YUYV stores 2 pixels per container of 32 bits, which gives us 16 bits (2 bytes for one pixel). So a 1920x1080 image in YUYV has plane[0].length=1920*1080*2=4147200 So apply a *2 factor to make the kernel believe it's receiving a YUYV buffer. Note: this also means that we are wasting NV12's plane[1] buffer with each allocation. [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/pixfmt-yuv-planar.html [2] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/pixfmt-packed-yuv.html ``` Signed-off-by: Mattijs Korpershoek --- src/android/camera_capabilities.cpp | 90 ++++++++++++++++++++++++++++++++++++- src/android/camera_capabilities.h | 4 ++ src/android/camera_device.cpp | 6 ++- src/android/camera_stream.cpp | 54 +++++++++++++++++++++- src/android/camera_stream.h | 5 +++ 5 files changed, 154 insertions(+), 5 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 1bfeaea4b121..e2e0f7409e94 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -124,6 +124,16 @@ const std::map camera3FormatsMap = { }, }; +/** + * \var yuvConversions + * \brief list of supported pixel formats for an input pixel format + * + * \todo This should be retrieved statically from yuv/post_processor_yuv instead + */ +const std::map> yuvConversions = { + { formats::YUYV, { formats::NV12 } }, +}; + const std::map hwLevelStrings = { { ANDROID_INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED, "LIMITED" }, @@ -582,8 +592,10 @@ int CameraCapabilities::initializeStreamConfigurations() LOG(HAL, Debug) << "Testing " << pixelFormat; /* - * The stream configuration size can be adjusted, - * not the pixel format. + * The stream configuration size can be adjusted. + * The pixel format might be converted via libyuv. + * Conversion check is done in another loop after + * testing native supported formats. * * \todo This could be simplified once all pipeline * handlers will report the StreamFormats list of @@ -603,7 +615,46 @@ int CameraCapabilities::initializeStreamConfigurations() /* If the format is not mandatory, skip it. */ if (!camera3Format.mandatory) continue; + } + + /* + * Test if we can map the format via a software conversion. + * This means that the converter can produce an "output" that is + * compatible with the format defined in Android. + */ + bool needConversion = false; + for (const PixelFormat &pixelFormat : libcameraFormats) { + LOG(HAL, Debug) << "Testing " << pixelFormat << " using conversion"; + + /* \todo move this into a separate function */ + for (const auto &[inputFormat, outputFormats] : yuvConversions) { + /* check if the converter can produce pixelFormat */ + auto it = std::find(outputFormats.begin(), outputFormats.end(), pixelFormat); + if (it == outputFormats.end()) + continue; + + /* + * The converter can produce output pixelFormat, see if we can configure + * the camera with the associated input pixelFormat. + */ + cfg.pixelFormat = inputFormat; + CameraConfiguration::Status status = cameraConfig->validate(); + + if (status != CameraConfiguration::Invalid && cfg.pixelFormat == inputFormat) { + mappedFormat = inputFormat; + conversionMap_[androidFormat] = std::make_pair(inputFormat, *it); + needConversion = true; + break; + } + } + + /* We found a valid conversion format, so bail out */ + if (mappedFormat.isValid()) + break; + } + + if (!mappedFormat.isValid()) { LOG(HAL, Error) << "Failed to map mandatory Android format " << camera3Format.name << " (" @@ -619,6 +670,11 @@ int CameraCapabilities::initializeStreamConfigurations() LOG(HAL, Debug) << "Mapped Android format " << camera3Format.name << " to " << mappedFormat; + if (needConversion) { + LOG(HAL, Debug) << mappedFormat + << " will be converted into " + << conversionMap_[androidFormat].second; + } std::vector resolutions; const PixelFormatInfo &info = PixelFormatInfo::info(mappedFormat); @@ -1457,6 +1513,36 @@ PixelFormat CameraCapabilities::toPixelFormat(int format) const return it->second; } +/* + * Check if we need to do software conversion via a post-processor + * for an Android format code + */ +bool CameraCapabilities::needConversion(int format) const +{ + auto it = conversionMap_.find(format); + if (it == conversionMap_.end()) { + LOG(HAL, Error) << "Requested format " << utils::hex(format) + << " not supported for conversion"; + return false; + } + + return true; +} + +/* + * Returns a conversion (input,output) pair for a given Android format code + */ +std::pair CameraCapabilities::conversionFormats(int format) const +{ + auto it = conversionMap_.find(format); + if (it == conversionMap_.end()) { + LOG(HAL, Error) << "Requested format " << utils::hex(format) + << " not supported for conversion"; + } + + return it->second; +} + std::unique_ptr CameraCapabilities::requestTemplateManual() const { if (!capabilities_.count(ANDROID_REQUEST_AVAILABLE_CAPABILITIES_MANUAL_SENSOR)) { diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index 6f66f221d33f..c3e6b48ab91d 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -30,6 +30,9 @@ public: CameraMetadata *staticMetadata() const { return staticMetadata_.get(); } libcamera::PixelFormat toPixelFormat(int format) const; + bool needConversion(int format) const; + std::pair + conversionFormats(int format) const; unsigned int maxJpegBufferSize() const { return maxJpegBufferSize_; } std::unique_ptr requestTemplateManual() const; @@ -77,6 +80,7 @@ private: std::vector streamConfigurations_; std::map formatsMap_; + std::map> conversionMap_; std::unique_ptr staticMetadata_; unsigned int maxJpegBufferSize_; diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index d34bae715a47..842cbb06d345 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -635,8 +635,12 @@ int CameraDevice::configureStreams(camera3_stream_configuration_t *stream_list) continue; } + CameraStream::Type type = CameraStream::Type::Direct; + if (capabilities_.needConversion(stream->format)) + type = CameraStream::Type::Internal; + Camera3StreamConfig streamConfig; - streamConfig.streams = { { stream, CameraStream::Type::Direct } }; + streamConfig.streams = { { stream, type } }; streamConfig.config.size = size; streamConfig.config.pixelFormat = format; streamConfigs.push_back(std::move(streamConfig)); diff --git a/src/android/camera_stream.cpp b/src/android/camera_stream.cpp index 4fd05dda5ed3..961ee40017f1 100644 --- a/src/android/camera_stream.cpp +++ b/src/android/camera_stream.cpp @@ -95,6 +95,7 @@ int CameraStream::configure() switch (outFormat) { case formats::NV12: + case formats::YUYV: postProcessor_ = std::make_unique(); break; @@ -107,6 +108,16 @@ int CameraStream::configure() return -EINVAL; } + needConversion_ = + cameraDevice_->capabilities()->needConversion(camera3Stream_->format); + + if (needConversion_) { + auto conv = cameraDevice_->capabilities()->conversionFormats(camera3Stream_->format); + LOG(HAL, Debug) << "Configuring the post processor to convert " + << conv.first << " -> " << conv.second; + output.pixelFormat = conv.second; + } + int ret = postProcessor_->configure(input, output); if (ret) return ret; @@ -183,7 +194,12 @@ int CameraStream::process(Camera3RequestDescriptor::StreamBuffer *streamBuffer) streamBuffer->fence.reset(); } - const StreamConfiguration &output = configuration(); + StreamConfiguration output = configuration(); + if (needConversion_) { + output.pixelFormat = + cameraDevice_->capabilities()->conversionFormats(camera3Stream_->format).second; + } + streamBuffer->dstBuffer = std::make_unique( *streamBuffer->camera3Buffer, output.pixelFormat, output.size, PROT_READ | PROT_WRITE); @@ -205,6 +221,39 @@ void CameraStream::flush() worker_->flush(); } +Size CameraStream::computeYUYVSize(const Size &nv12Size) +{ + /* + * On am62x platforms, the receiver driver (j721e-csi2rx) only + * supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY. + * + * However, the gralloc implementation is only capable of semiplanar + * YUV420 such as NV12. + * + * To trick the kernel into believing it's receiving a YUYV buffer, we adjust the + * size we request to gralloc so that plane(0) of the NV12 buffer is long enough to + * match the length of a YUYV plane. + * + * for NV12, one pixel is encoded on 1.5 bytes, but plane 0 has 1 byte per pixel. + * for YUYV, one pixel is encoded on 2 bytes. + * + * So apply a *2 factor. + * + * See: + * https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/pixfmt-packed-yuv.html + * https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/pixfmt-yuv-planar.html + */ + constexpr unsigned int YUYVfactor = 2; + + unsigned int width = nv12Size.width; + unsigned int height = nv12Size.height; + + if (needConversion_) + width = width * YUYVfactor; + + return Size{ width, height }; +} + FrameBuffer *CameraStream::getBuffer() { if (!allocator_) @@ -222,8 +271,9 @@ FrameBuffer *CameraStream::getBuffer() * \todo Store a reference to the format of the source stream * instead of hardcoding. */ + const Size hackedSize = computeYUYVSize(configuration().size); auto frameBuffer = allocator_->allocate(HAL_PIXEL_FORMAT_YCBCR_420_888, - configuration().size, + hackedSize, camera3Stream_->usage); allocatedBuffers_.push_back(std::move(frameBuffer)); buffers_.emplace_back(allocatedBuffers_.back().get()); diff --git a/src/android/camera_stream.h b/src/android/camera_stream.h index 4c5078b2c26d..52a5606399c5 100644 --- a/src/android/camera_stream.h +++ b/src/android/camera_stream.h @@ -128,10 +128,13 @@ public: int configure(); int process(Camera3RequestDescriptor::StreamBuffer *streamBuffer); + libcamera::Size computeYUYVSize(const libcamera::Size &nv12Size); libcamera::FrameBuffer *getBuffer(); void putBuffer(libcamera::FrameBuffer *buffer); void flush(); + bool needConversion() const { return needConversion_; } + private: class PostProcessorWorker : public libcamera::Thread { @@ -184,4 +187,6 @@ private: std::unique_ptr postProcessor_; std::unique_ptr worker_; + + bool needConversion_; };