[{"id":28233,"web_url":"https://patchwork.libcamera.org/comment/28233/","msgid":"<2e268f5d-ecf9-45d8-83c2-f47c5914c141@linaro.org>","date":"2023-12-04T09:40:20","subject":"Re: [libcamera-devel] [RFC PATCH 0/7] libcamera: introduce Software\n\tISP and Software IPA","submitter":{"id":175,"url":"https://patchwork.libcamera.org/api/people/175/","name":"Bryan O'Donoghue","email":"bryan.odonoghue@linaro.org"},"content":"On 04/12/2023 01:10, Andrey Konovalov wrote:\n> Here is a draft implementation of Software ISP and Software IPA\n> which provide debayering and implementation of image processing\n> algorithms for systems without a hardware ISP, or in cases when\n> there are no public drivers for the hardware ISP present in the\n> system.\n> \n> The implementation of the Software ISP is a reference one.\n> A naive AWB alorithm is implemented as part of a function which\n> does debayering and statistics calculations - the algorithm part\n> is to be moved to the IPA in the next version of the patch set.\n> And for debayering itself there is already a more efficient\n> implementation by Hans de Goede. This patch set is currently using\n> the earlier debayering implementation as it is less lines of code\n> and is OK for the initial discussion.\n> Only RAW10P format from the sensor is currently supported, but\n> other debayering functions can be easily added (which of them to\n> call is decided based on the input format).\n> \n> The Software IPA has only auto exposure and AGC. For the AGC\n> the analogue gain control of the camera sensor is used (if\n> available). The algorithm is very much simplified, and is\n> mostly included as a reference code.\n> \n> The 6th patch renames some variables in the simple pipeline\n> handler for the Software ISP to use the same buffer handling\n> code as the Converter currently does. This lets one to\n> avoid adding extra code to the pipeline handler, but also\n> makes the Software ISP and the Converter mutually exclusive.\n> \n> The Software ISP / IPA are intended to be used with the simple\n> pipeline handler. The proper integration isn't done yet, and\n> the last patch in this series unconditionally creates the\n> Software ISP and Software IPA instances if the pipeline is\n> not using a Converter. In the future the Software ISP / IPA\n> would be created if the media pipeline can only produce raw\n> Bayer frames. And a possibility to use different implementations\n> of Software ISP and IPA will be added.\n> \n> This patch set uses SharedMemObject class used by the RPi pipeline\n> handler - the second patch in the series moves the header file\n> to a common directory.\n> \n> This patch set has been tested on Qualcomm RB5 board with\n> a mezzanine board equipped with RPi camera v2 (not the\n> standard RB5 camera mezzanine).\n> \n\nGreat work, thanks for posting.\n\nDo you have a link to a buildable tree from gitlab for this series that \ncan be shared ?\n\n---\nbod","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 580E3C3226\n\tfor ;\n\tMon, 4 Dec 2023 09:41:53 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A8E59629CF;\n\tMon, 4 Dec 2023 10:41:52 +0100 (CET)","from mail-ed1-x529.google.com (mail-ed1-x529.google.com\n\t[IPv6:2a00:1450:4864:20::529])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B207D629BC\n\tfor ;\n\tMon, 4 Dec 2023 10:40:20 +0100 (CET)","by mail-ed1-x529.google.com with SMTP id\n\t4fb4d7f45d1cf-54b545ec229so5341123a12.0\n\tfor ;\n\tMon, 04 Dec 2023 01:40:20 -0800 (PST)","from ?IPV6:2001:1c06:2302:5600:366d:ca8f:f3af:381?\n\t(2001-1c06-2302-5600-366d-ca8f-f3af-0381.cable.dynamic.v6.ziggo.nl.\n\t[2001:1c06:2302:5600:366d:ca8f:f3af:381])\n\tby smtp.gmail.com with ESMTPSA id\n\tm25-20020a170906235900b009fcb10eecb2sm5038259eja.84.2023.12.04.01.40.19\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tMon, 04 Dec 2023 01:40:19 -0800 (PST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1701682912;\n\tbh=mSGwhb1d+CL0aumjhoQCcAEqcN4eMHZAkYsoJK6rqSE=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=pLLqRtELZtRHatPy+DuAjEBSeJt9eke3hSO8MJYhu5LTVA/hi6Gr6Ziw4mISIIpic\n\toimm7wJehIdTYJCyu5AIpyGJuZbA/iO7HiXnbVeARKrwNw5s8ientswosHc8Bfng9/\n\tXHQ+eodUhm2+7qcZpBYmFw1WC6bNt4F05c69p4H7dCf/Sg5xez5toBMkfiEKWlyG6/\n\trn6KeTBMwLROBH7tJ5DA4WeSLowGF06HfmnsjQKBE45OYq8cewPsqL/uQe0UQMygNF\n\t7bAQ0hQ3+7Ic5uhK212q1Ji4KPA5opv2ihiAoA2u1AVXb3Yi9N3w3tvZ/osASyY3UW\n\tln5GS3YC6pZ0Q==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=linaro.org; s=google; t=1701682820; x=1702287620;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:in-reply-to:from:references:cc:to\n\t:content-language:subject:user-agent:mime-version:date:message-id\n\t:from:to:cc:subject:date:message-id:reply-to;\n\tbh=NbEfi/PoI/qhXBJnZ2DSXHX1CPxVn7e1SUhB3kHKjzY=;\n\tb=Q9F7EVzb5Jj5K58heRkNrNRLEmZWH2G/sE15+lLHPmtyN9saNGYqRkCtrp/+uChSaZ\n\tx2eD6TmlHS3J4b5nAzZfbXTaH2vvPZ6mljgi+Ysc0kTKyIYT2VWppOLoypOynWZsWh2F\n\tc7GocoMXK0qxSOpwiiCzG9hfe9Ys1U/FYRLhvXI71LgrqRgHhDow7SGuvJcSCjkGTZEP\n\t4AUSRozuwPRfgb+7b5PZzRIkPynGb3lP4FLuM8unjCyRAkgl/dbM7Xvd2g3tT4K19Chg\n\t1w7n7/sDeojpH+kWkQDPfXDfHJV3EnWCAFMscFidhQcjCZqF/iap78Pmpcfqe6ZQ2r6T\n\tGc5g=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"Q9F7EVzb\"; \n\tdkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1701682820; x=1702287620;\n\th=content-transfer-encoding:in-reply-to:from:references:cc:to\n\t:content-language:subject:user-agent:mime-version:date:message-id\n\t:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;\n\tbh=NbEfi/PoI/qhXBJnZ2DSXHX1CPxVn7e1SUhB3kHKjzY=;\n\tb=C6XpPwy/oi3rV8ZOij19qxLWDSozhOBfmmA8D3C8mK0adIypO2p6B2Dgxl0BcO3Ru7\n\t+eYj1I/qD27TLVwW19yvqUM6ZgF7GYVpyNQ5QSRJwjWeGcdkhbv7dvstr5bRafInemJ+\n\t8VxMYrhHCPE3NyWR47NcmJro8lav/YYKVUeV1czMAJX1UTgpbSulvUpwkGDk4gbr1m1f\n\tCTFZviih0K5IeJX6uVuL5rSdB4UjjNeIPIKSq5eKH0lmEu8NyN5GIiCgzUl2VLjJW2DD\n\tAep2dKxC2zYndbHGta/A8T4DRkrLqRghOQwAZvEqf+zj8HTxOLDICPijLupMm0uYSkxr\n\ttOfw==","X-Gm-Message-State":"AOJu0YzlaZabHWaQmkvKPzHXN0RrSv1733DcLlZ0UtMiPteHegKqaedj\n\t0vRtQd+lJqndV8BQm4PnMC0U8g==","X-Google-Smtp-Source":"AGHT+IFfUYCzYkNflzwystzm7zcOhqIOCoioG0WBy0LBZjIUfl1yL4ya26NfvF3GvjuBpd34NXk/iA==","X-Received":"by 2002:a17:906:6816:b0:a03:90a3:b775 with SMTP id\n\tk22-20020a170906681600b00a0390a3b775mr3194502ejr.27.1701682820095; \n\tMon, 04 Dec 2023 01:40:20 -0800 (PST)","Message-ID":"<2e268f5d-ecf9-45d8-83c2-f47c5914c141@linaro.org>","Date":"Mon, 4 Dec 2023 10:40:20 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Content-Language":"en-US","To":"Andrey Konovalov ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20231204001013.404720-1-andrey.konovalov@linaro.org>","In-Reply-To":"<20231204001013.404720-1-andrey.konovalov@linaro.org>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","X-Mailman-Approved-At":"Mon, 04 Dec 2023 10:41:51 +0100","Subject":"Re: [libcamera-devel] [RFC PATCH 0/7] libcamera: introduce Software\n\tISP and Software IPA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Bryan O'Donoghue via libcamera-devel\n\t","Reply-To":"Bryan O'Donoghue ","Cc":"mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com,\n\tsrinivas.kandagatla@linaro.org, admin@dennisbonke.com","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":28236,"web_url":"https://patchwork.libcamera.org/comment/28236/","msgid":"<8d8652d1-d3bb-4b7d-a4b1-d62ad381b235@linaro.org>","date":"2023-12-04T13:13:05","subject":"Re: [libcamera-devel] [RFC PATCH 0/7] libcamera: introduce Software\n\tISP and Software IPA","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi Bryan,\n\nOn 04.12.2023 12:40, Bryan O'Donoghue wrote:\n> On 04/12/2023 01:10, Andrey Konovalov wrote:\n>> Here is a draft implementation of Software ISP and Software IPA\n>> which provide debayering and implementation of image processing\n>> algorithms for systems without a hardware ISP, or in cases when\n>> there are no public drivers for the hardware ISP present in the\n>> system.\n>>\n>> The implementation of the Software ISP is a reference one.\n>> A naive AWB alorithm is implemented as part of a function which\n>> does debayering and statistics calculations - the algorithm part\n>> is to be moved to the IPA in the next version of the patch set.\n>> And for debayering itself there is already a more efficient\n>> implementation by Hans de Goede. This patch set is currently using\n>> the earlier debayering implementation as it is less lines of code\n>> and is OK for the initial discussion.\n>> Only RAW10P format from the sensor is currently supported, but\n>> other debayering functions can be easily added (which of them to\n>> call is decided based on the input format).\n>>\n>> The Software IPA has only auto exposure and AGC. For the AGC\n>> the analogue gain control of the camera sensor is used (if\n>> available). The algorithm is very much simplified, and is\n>> mostly included as a reference code.\n>>\n>> The 6th patch renames some variables in the simple pipeline\n>> handler for the Software ISP to use the same buffer handling\n>> code as the Converter currently does. This lets one to\n>> avoid adding extra code to the pipeline handler, but also\n>> makes the Software ISP and the Converter mutually exclusive.\n>>\n>> The Software ISP / IPA are intended to be used with the simple\n>> pipeline handler. The proper integration isn't done yet, and\n>> the last patch in this series unconditionally creates the\n>> Software ISP and Software IPA instances if the pipeline is\n>> not using a Converter. In the future the Software ISP / IPA\n>> would be created if the media pipeline can only produce raw\n>> Bayer frames. And a possibility to use different implementations\n>> of Software ISP and IPA will be added.\n>>\n>> This patch set uses SharedMemObject class used by the RPi pipeline\n>> handler - the second patch in the series moves the header file\n>> to a common directory.\n>>\n>> This patch set has been tested on Qualcomm RB5 board with\n>> a mezzanine board equipped with RPi camera v2 (not the\n>> standard RB5 camera mezzanine).\n>>\n> \n> Great work, thanks for posting.\n> \n> Do you have a link to a buildable tree from gitlab for this series that can be shared ?\n\nI've pushed it to\nhttps://gitlab.freedesktop.org/camera/libcamera-softisp/-/commits/SoftwareISP-v02/\n\nThanks,\nAndrey\n\n> ---\n> bod\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4CC39C31E9\n\tfor ;\n\tMon, 4 Dec 2023 13:13:09 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8F56A629CF;\n\tMon, 4 Dec 2023 14:13:08 +0100 (CET)","from mail-wr1-x436.google.com (mail-wr1-x436.google.com\n\t[IPv6:2a00:1450:4864:20::436])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 549E061D9F\n\tfor ;\n\tMon, 4 Dec 2023 14:13:07 +0100 (CET)","by mail-wr1-x436.google.com with SMTP id\n\tffacd0b85a97d-332fd81fc8dso3537846f8f.3\n\tfor ;\n\tMon, 04 Dec 2023 05:13:07 -0800 (PST)","from [192.168.118.20] ([87.116.166.54])\n\tby smtp.gmail.com with ESMTPSA id\n\ti7-20020adffc07000000b0033346408820sm3751546wrr.68.2023.12.04.05.13.05\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tMon, 04 Dec 2023 05:13:06 -0800 (PST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1701695588;\n\tbh=DQbTQNEjGs8ZW3kFjqh4pfS56ufzyPF9geyFLAJ6aJ0=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=GuTBI2NZ/j7gUJSOv97mAzd+VTX3EYoQ+cwCwsjOix+fGkksVhxCdKpb36HVVGUPG\n\tMkbtUzuvmdfyU0nOwl8L1MPIhxOYivFJ1EZEmdKNMdPPXkIhChXb9KCwyHgdI9VDta\n\t8joPEHAPrdFT30+N2AlphMlntFxdzlgMmgR4nvctSjnh3hxyt+JHtdTABaS/MOmRvG\n\tER6OkhMF9eli0Hi31iDMijmQTTmrOo7VwWfot+UYQQLVAP4L5Q7fm9mDg+VTWcybv/\n\t/wJ77i7ak51U98G/sqhX/2YIJ+FCoOMwE7c83IGCeRbPrhGCeWNIwPQuuEzP6P05hQ\n\turHQYlxtN1E9g==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=linaro.org; s=google; t=1701695587; x=1702300387;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:in-reply-to:from:content-language\n\t:references:cc:to:subject:user-agent:mime-version:date:message-id\n\t:from:to:cc:subject:date:message-id:reply-to;\n\tbh=33gY4wH/5IhbtXq3vdoQskDf79LrYaMNjq03xaS68I8=;\n\tb=iqeV23ckaWpP0w2RlHuzNLj2fm9lCY3LugAjkE9WopGl8Mddl3AvsEIPqlOD/RACBD\n\tvrOvXFWjsfcs8vWjAYpNwJ1BMT7kyNwlYfCcM+/+wQTRZK3BFleG5cbhFH0M2uF98bDT\n\tWKUa0IXtIAcQfquLbXrgfIfSfEBr5CHjEcZ1r5DVqlVkQ9rAR1NhXY/tpplA8rUqLY7O\n\tRC/M+4bOSPS8XVgjiFOuy8o1j7fyZQZImznTez2Dg9+LIB6TM4LDyxWF9czXygMxmngl\n\tRf8oS7m+27ihyHe5VERA0kc9vdn7PP0anympeDT2dRIkzrZRe5KD/4WdvUcqZpVha2DN\n\tZF+Q=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=linaro.org\n\theader.i=@linaro.org header.b=\"iqeV23ck\"; \n\tdkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1701695587; x=1702300387;\n\th=content-transfer-encoding:in-reply-to:from:content-language\n\t:references:cc:to:subject:user-agent:mime-version:date:message-id\n\t:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;\n\tbh=33gY4wH/5IhbtXq3vdoQskDf79LrYaMNjq03xaS68I8=;\n\tb=Sgui4FuRztcCtBWGfJl6YLIX+RiXz/hlo9Q6FmTgV/o1sJjDMDQMdN972DChOrStbM\n\twAXPGN9Mz4yfKI9cTJU3qgEo0ZCBEUV9m+fwi+6F6mpttqRREX5NrLM7uoxYTpLE/hne\n\tbkgYjG2UiJIglBTpxX5qUobyShuBRT2u+c+vF2dGUSyItsevVPC0AH0aegDo+B3BxuzS\n\t3v46xMDjj5iKp8aUKegcuOGSPIwtMJqe8XfN/mtQkDOiTepK113aRDnVFMFLSmJmWSJt\n\tmgU4OmniV0l5toKKnSlAXAtxicTTlUXMvvu/AKRq/9l5+NOwT8wV5k9Ouq10Wn4oNK4r\n\tRlOQ==","X-Gm-Message-State":"AOJu0Yx3a1fXp+HGgTkKAxsp3sDmlLoP1Ka9qiitDxWuMj0+L4kS9q9e\n\tsyyGi6i7bggC7K5RrkiANo3xIg==","X-Google-Smtp-Source":"AGHT+IHVpYa1aRax7Y939lNmc61WKdXCozYjGKnQSiEX9Xg/Wnzw5f6xBRwnBLY1uV0gBo+QFU27Ig==","X-Received":"by 2002:a5d:4084:0:b0:333:2fd2:68e2 with SMTP id\n\to4-20020a5d4084000000b003332fd268e2mr2983234wrp.117.1701695586814; \n\tMon, 04 Dec 2023 05:13:06 -0800 (PST)","Message-ID":"<8d8652d1-d3bb-4b7d-a4b1-d62ad381b235@linaro.org>","Date":"Mon, 4 Dec 2023 16:13:05 +0300","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","To":"Bryan O'Donoghue ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20231204001013.404720-1-andrey.konovalov@linaro.org>\n\t<2e268f5d-ecf9-45d8-83c2-f47c5914c141@linaro.org>","Content-Language":"en-US","In-Reply-To":"<2e268f5d-ecf9-45d8-83c2-f47c5914c141@linaro.org>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [RFC PATCH 0/7] libcamera: introduce Software\n\tISP and Software IPA","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Andrey Konovalov via libcamera-devel\n\t","Reply-To":"Andrey Konovalov ","Cc":"mripard@redhat.com, g.martti@gmail.com, t.langendam@gmail.com,\n\tsrinivas.kandagatla@linaro.org, admin@dennisbonke.com","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]