From patchwork Sun Aug 6 18:01:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 18924 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A0510BDCBF for ; Sun, 6 Aug 2023 18:02:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C5FA2627E6; Sun, 6 Aug 2023 20:01:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1691344919; bh=bXhdGAzK5/u0CPijq690ikFSLXps+DtU9PV6eLGaICQ=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=Pkcbmly4ejBZC+VrGEZEiUcaa52YcSlqmnWl1ljCWMYP6VSLq4q4+FlkCEK/7FiG+ alg4uec3REJSdw+cQXRw+bnZpeyH+F/ESnq+jwnTeu6n8nCZ1KLps18A+rzurD2ry7 dXnM3eVngyp1I0FFwEKut5aBbYSJRmHFPW/TJdZfsbNaNuafjt/3JBxLimGY5vi5Cb t5zvH0lq+4n4kiMZdvxdDixY+wceEQ1/aajk2miegtobjDZGpxDCrnqNm+VkO7o+K1 pmyK5F8moxJw+50OfR7amvQaqecyrOxEWPm7hpRX1Rk5Th1LNwM/zXhHU/M2D5epTU HGEgFxGr46Rrw== Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com [IPv6:2a00:1450:4864:20::233]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1C2D0627E6 for ; Sun, 6 Aug 2023 20:01:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="TOzhPymI"; dkim-atps=neutral Received: by mail-lj1-x233.google.com with SMTP id 38308e7fff4ca-2b9fa64db41so58958901fa.1 for ; Sun, 06 Aug 2023 11:01:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1691344917; x=1691949717; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=QFQaIPxsNHI8LxtDb9sN70kRx69T/QGqvHCcG/5ZTmI=; b=TOzhPymIHT59Xs0MuOg3B+qye2V7Fl8vnrBputvKX+fSO1grK6LdhvtHqotuWBZg2h /7/9g+9uMaw5zfzVGr9ZxUT/nDwBZmUoM+8KZs5VaXwlXrlw6rmzqh9NxbptEQaiMCNF RlrbydAdeppOJMMPHqEtP/hP9X1pB54bNhjALBUnzED+Av5Nr0JbVXyP9X0UEyIJhnab K5Gs2tQwKP4HeZ2gMzZVP8uCenzFDb5H8E7pZhHjnlh0CDuDDqBIE+nTn89vRsRISvcT 9eQ1lQVXEe1Id9bOs5DtQOLNh7c4F1cq7kdBl2Ct0BnTUcnWs758X3shl9z4fOoEjctU wC8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691344917; x=1691949717; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QFQaIPxsNHI8LxtDb9sN70kRx69T/QGqvHCcG/5ZTmI=; b=RG65E/EDTUhBsePFbYlpshkvBCfAuTfeo/2YwIkM99VBw5ObHyKab4WBt+V9VxBYHd v2tdJveCvsJzEUHg+UfUCbvO5NG/Sq1IHFPO4oPu9g2Gesl5Q8sjwDpW2oOxvlNdTjcP zvUrEqD2zFUna4gsOioGJ41agBnI0KsS+tuHROIpVw+KpwuJHxWxiJ1wN0gKyT51fbHm YQuve/G0ZzbZuzyhr2c4fWOORyeIpVTt/ybP+uTxb3SrwLyPejV5XTZYanP9xBUBCjG2 PXO8j6XCMxLloN8iywnVc4A2oAW2BEHDPGLC1VJIGuTX2xbVQYfscOg0IyltPBTM00OP PgOw== X-Gm-Message-State: AOJu0YzYFdcjNICiEok19FvaLEEptlyr4/FyyER/e/6z1QELCrrdxwQJ gcBT6pkiVz+fVyXa0a0Abxxk3iV0mVlutm9+0GY= X-Google-Smtp-Source: AGHT+IGQ11OuU3FD0S3I2D8QaE2N+Oynk2XsVT+Lf0UWt7Mc5o1f3YW0u286iCfOTDO08G9RhDW5rg== X-Received: by 2002:a2e:8517:0:b0:2b8:3ff3:cd93 with SMTP id j23-20020a2e8517000000b002b83ff3cd93mr4810314lji.7.1691344917105; Sun, 06 Aug 2023 11:01:57 -0700 (PDT) Received: from Lat-5310.. ([87.116.165.37]) by smtp.gmail.com with ESMTPSA id j15-20020a170906430f00b0099b76c3041csm4197396ejm.7.2023.08.06.11.01.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Aug 2023 11:01:56 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Sun, 6 Aug 2023 21:01:34 +0300 Message-Id: <20230806180136.304156-1-andrey.konovalov@linaro.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 0/2] libcamera: converter: introduce software converter for debayering and AWB X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Andrey Konovalov via libcamera-devel From: Andrey Konovalov Reply-To: Andrey Konovalov Cc: bryan.odonoghue@linaro.org, srinivas.kandagatla@linaro.org Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Here is a draft implementation of Bayer demosaicing which follows the Converter interface and runs on CPU. It also includes a naive version of Grey World AWB. Just for demo purposes (to make the final image looking a bit nicer). Otherwise, Converter isn't the right place for AWB - only the statistics should be gathered here, and the rest belongs to an IPA module. Currently this software converter supports single output only, but adding the second stream for statistics is under consideration. As libcamera::Converter currently assumes a media device underneath the convertor, I wasn't able to avoid hacking the simple pipeline handler to make it work with the software converter. For the same reason ConverterFactory is not used for now. Only RAW10P format from the sensor is currently supported, but adding more Bayer formats wouldn't be a problem. Out of 10 bits, only 8 most significant ones are used to lessen the load on CPU. Simple bilinear interpolation is used for the same reason. AWB simplifications: - a naive implementation of "Grey World" algorithm: all pixels are used (no brightest and darkest pixels excluded from the calculations) - to lessen the load on CPU, works on raw Bayer data and takes red values from red pixels, blue values from blue, and green values from green pixels only. - to lessen the load on CPU, the red/green/blue gains calculated from the current frame data are applied to the next frame. These gains are purely in software (no V4L2 controls are set). No performance analysis or tuning have been done yet. On RB5 board this software convertor gives: ~ 5 fps at 3278x2462 (camera sensor runs at 15 fps) ~ 18..19 fps at 1918x1078 (out of 30 fps) ~ 18..19 fps at 1638x1230 (out of 30 fps) ~ 30 fps at 638x478 (out of 30 fps) (The resolutions above are the output ones; demosaic filter drops 1 pixel from each side of the frame, so that 3280x2464 from the camera sensor becomes 3278x2462 etc) Andrey Konovalov (2): libcamera: converter: add software converter [DNI] libcamera: pipeline: simple: a hack to use sotware converter with qcom-camss .../internal/converter/converter_softw.h | 90 ++++ src/libcamera/converter/converter_softw.cpp | 430 ++++++++++++++++++ src/libcamera/converter/meson.build | 3 +- src/libcamera/pipeline/simple/simple.cpp | 22 +- 4 files changed, 542 insertions(+), 3 deletions(-) create mode 100644 include/libcamera/internal/converter/converter_softw.h create mode 100644 src/libcamera/converter/converter_softw.cpp