From patchwork Fri Sep 15 07:57:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattijs Korpershoek X-Patchwork-Id: 19006 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 689B7BD160 for ; Fri, 15 Sep 2023 07:57:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D887462916; Fri, 15 Sep 2023 09:57:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1694764650; bh=3SdiyO1pw58kqEjFRk+FHmHH3cxV+dL3L5UsMOla668=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=DzWo8kq5BusJMqXVfGSWCx1oxvICyxUKv46zMF2N183ag2Z71dK7tihi6/ju0EYdu t2iivcjlsmxz17V5aNxzvizwrXHWL0MqyvjX1ui8dmRY9jPbWf33KEs1bC5r/uWL0k oTbRtfFapKaddJHuEbxGk/zpPtS2dGPRhLgnifCRGr5F+ieJay/nKx5iuJOBz0NggE jTdOLlon1d3+VUzmC9BfSbyjTkr5F+OlEEg0V13m+PNW+FLLI80sELiKwvaYhE1z7F YmxjucFGyW3eXI+IHGN+GKwMaSlhj3Ihb6xq6JYp5PpiWww2MYXXDtSku9651XH9Jz d9YIQEeQ5GmiQ== Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8519A628D8 for ; Fri, 15 Sep 2023 09:57:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.b="JJ6FlN6l"; dkim-atps=neutral Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-307d20548adso1730976f8f.0 for ; Fri, 15 Sep 2023 00:57:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1694764648; x=1695369448; darn=lists.libcamera.org; h=cc:to:content-transfer-encoding:mime-version:message-id:date :subject:from:from:to:cc:subject:date:message-id:reply-to; bh=JAs6y7nr8Bf0vaOzekou9Pww6kWBxfb8lW+XzGjSqvc=; b=JJ6FlN6lsf7l1l5PpADww934At38zpVw7jEBa/foJgs8jLyfFkYnnA9F7z5kMiJFiG p2oRlTnl0AwJSPQFpZOh4GW9Tpe+bdpoHXN9FH+u5hp492BG8En4nkM1vfmF8s6DbE88 Q/GVbEwE2CTWX6NiPkNBPeSoEBE1kQF6KJSav9K/LYIdcSqS31tLltt6+VbgHIub5ip0 rUuqbL3UWTt4n0Ymp9LMn+NEq25uRfWoZSrxmHoB7TFYzD0TQwTvnPcuY1ibwukMGRMG ecuNJXvWt+EvL2/aNltq7XgfJ/HMNnRzwmCMGO7D5Cos702sTUOuPU3NiRWuku4mlNK9 TgHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764648; x=1695369448; h=cc:to:content-transfer-encoding:mime-version:message-id:date :subject:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=JAs6y7nr8Bf0vaOzekou9Pww6kWBxfb8lW+XzGjSqvc=; b=dQuo3CxltLn5IkuHOhR4tZXrNnm74n0C2ui1yf3u0zrjXXHyxKkiFs+Ghi1ON1Q4OD OW4J234oFLiCdzjB9WlaTzDa2cFKVvgvbylUcYOH0uxv5ViW85Pp9o6VlacslZQX2PVV 9raKoorILnZuCammeBCHzjd0RyzsUBVt58gKfMyaMM+FHVk4aFpp4Dxxx0JWelZYx4a2 3o/xUC8EQVTmOxbhFg1cUETf2+GShks9F9fg1/XfJcw400J7nDGAmKfqNrjaf1ybHVbF ffWfn+6lMowzhCPoDexsLV/7MUDBl4Kaev2Zyq4d5jE/Ki2NIanPTssTX6c4l69iju01 fhxA== X-Gm-Message-State: AOJu0Yxt2U12PY2dHR0LRJm6qbt6MYCjgFPurFZ5KRJmHvDHZzUn3CT0 VyH1rH77koiWJU5QgrsuEYIfgA== X-Google-Smtp-Source: AGHT+IF5bdVg7Oru8qo/5niLxqtpKvxdyoKR++iWqoNVCCtFXKtdGNvIEdLVJN1jVhk7bxe51pPH8g== X-Received: by 2002:adf:ef0e:0:b0:31d:d58c:9432 with SMTP id e14-20020adfef0e000000b0031dd58c9432mr737083wro.3.1694764647499; Fri, 15 Sep 2023 00:57:27 -0700 (PDT) Received: from [192.168.1.20] ([2a01:cb19:8704:be00:4f55:bd9d:611a:6c8e]) by smtp.gmail.com with ESMTPSA id m2-20020a056000174200b0031fe9a47a87sm2506942wrf.112.2023.09.15.00.57.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 00:57:27 -0700 (PDT) Date: Fri, 15 Sep 2023 09:57:24 +0200 Message-Id: <20230915-libyuv-convert-v1-0-1e5bcf68adac@baylibre.com> MIME-Version: 1.0 X-B4-Tracking: v=1; b=H4sIAGQOBGUC/6tWKk4tykwtVrJSqFYqSi3LLM7MzwNyDHUUlJIzE vPSU3UzU4B8JSMDI2MDS0MT3ZzMpMrSMt3k/Lyy1KIS3bTUZHMDCyNjE+NUEyWgpoKi1LTMCrC B0UpBbs5KsbW1AE1L5PxlAAAA To: libcamera-devel@lists.libcamera.org X-Mailer: b4 0.12.4-dev-6aa5d Subject: [libcamera-devel] [PATCH RFC 0/7] android: add YUYV->NV12 conversion via libyuv X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mattijs Korpershoek via libcamera-devel From: Mattijs Korpershoek Reply-To: Mattijs Korpershoek Cc: Guillaume La Roque Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" On some platforms, it's possible that the gralloc implementation and the CSI receiver cannot agree on a pixel format. When that happens, there is usually a m2m converter in the pipeline which handles pixel format conversion. On platforms without pixel format converters, such as the AM62x, we need to do software conversion. The AM62x platform: * uses a CSI receiver (j721e-csi2rx), that only supports packed YUV422 formats such as YUYV, YVYU, UYVY and VYUY. * Has a gralloc implementation that only supports of semi-planar YUV420 formats such as NV12. This series add support for software conversion using libyuv. Right now, only YUYV->NV12 is supported. This has been send as an RFC because: 1. I don't have much media knowledge, nor C++ knowledge so this is probably not good enough for libcamera's high quality codebase. 2. I'm not sure that transforming the "main Type::Direct" stream into an Internal stream has no side effects. 3. I'm unsure what to do with the gralloc size in case of conversion. Right now, we always allocate HAL_PIXEL_FORMAT_YCBCR_420_888 in CameraStream::getBuffer(). This should be adapted both on: - the gralloc capabilities of the platform - the conversion (if any) This has been tested on a AM62x SK EVM board with an Alinx AN5641 (ov5640) sensor. In Android 13. It might work on BeaglePlay with the same sensor, but I have not tested that. Both preview and still capture are functional with the default com.android.camera2 app. Note that video recording is not supported, because that would require a similar conversion from YUYV->MJPEG which I did not investigate yet. I would appreciate some feedback. Thanks a lot in advance if you have a look! Depends on: - https://patchwork.libcamera.org/cover/18995/ Signed-off-by: Mattijs Korpershoek --- Mattijs Korpershoek (7): android: yuv: separate source destination in length check android: yuv: loop over each plane for size check android: yuv: prepare support for other pixel formats android: camera_device: support when no Direct stream is found android: camera_stream: add add explicit input configuration android: yuv: add YUYV -> NV12 conversion WIP: android: add YUYV->NV12 format conversion via libyuv src/android/camera_capabilities.cpp | 90 ++++++++++++++++++- src/android/camera_capabilities.h | 4 + src/android/camera_device.cpp | 9 +- src/android/camera_stream.cpp | 57 +++++++++++- src/android/camera_stream.h | 5 ++ src/android/yuv/post_processor_yuv.cpp | 156 ++++++++++++++++++++++----------- src/android/yuv/post_processor_yuv.h | 12 ++- 7 files changed, 270 insertions(+), 63 deletions(-) --- base-commit: 58e501c71c47e57f02afde1bd296a037038cd6d5 change-id: 20230914-libyuv-convert-fec7082343e4 Cheers,