{"id":22081,"url":"https://patchwork.libcamera.org/api/covers/22081/?format=json","web_url":"https://patchwork.libcamera.org/cover/22081/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20241125180310.1254519-1-mzamazal@redhat.com>","date":"2024-11-25T18:03:01","name":"[v2,0/9] Software ISP support for CCM","submitter":{"id":177,"url":"https://patchwork.libcamera.org/api/people/177/?format=json","name":"Milan Zamazal","email":"mzamazal@redhat.com"},"mbox":"https://patchwork.libcamera.org/cover/22081/mbox/","series":[{"id":4820,"url":"https://patchwork.libcamera.org/api/series/4820/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4820","date":"2024-11-25T18:03:01","name":"Software ISP support for CCM","version":2,"mbox":"https://patchwork.libcamera.org/series/4820/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/22081/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 955CABD160\n\tfor ;\n\tMon, 25 Nov 2024 18:03:26 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id ACAE265F74;\n\tMon, 25 Nov 2024 19:03:25 +0100 (CET)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.129.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9251865F66\n\tfor ;\n\tMon, 25 Nov 2024 19:03:23 +0100 (CET)","from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com\n\t(ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63])\n\tby relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3,\n\tcipher=TLS_AES_256_GCM_SHA384) id us-mta-639-B6HKTX5RNPSM7C6AkEL-Hg-1;\n\tMon, 25 Nov 2024 13:03:21 -0500","from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com\n\t(mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com\n\t[10.30.177.15])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\tkey-exchange X25519 server-signature RSA-PSS (2048 bits)\n\tserver-digest SHA256) (No client certificate requested)\n\tby mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix)\n\twith ESMTPS id 12C5419560A2; Mon, 25 Nov 2024 18:03:20 +0000 (UTC)","from nuthatch.redhat.com (unknown [10.45.224.109])\n\tby mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix)\n\twith ESMTP id 86004196BC17; Mon, 25 Nov 2024 18:03:18 +0000 (UTC)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"gtw0LUlx\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1732557802;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tcontent-transfer-encoding:content-transfer-encoding;\n\tbh=17rwhWfDoiYTub0o6DEpSsB8P0qScBc1Qj7HqcIXPVE=;\n\tb=gtw0LUlx5jTJXgR8YnBlGQGnYsRtI0f1JEiSyuAb1zH6iNwQTttD8QAfD1RpJ/vNs+vQsl\n\tuuds5rl4+1jfl/gB3LEkS7aVJlHu7heIchQHxTKkiJ9UCbaaf7/dqT1zgJiL0+JNSqho8j\n\tXMWssIXGAv/6OxnRh5fBH5mF4L+EqSg=","X-MC-Unique":"B6HKTX5RNPSM7C6AkEL-Hg-1","X-Mimecast-MFC-AGG-ID":"B6HKTX5RNPSM7C6AkEL-Hg","From":"Milan Zamazal ","To":"libcamera-devel@lists.libcamera.org","Cc":"Milan Zamazal ,\n\tRobert Mader ","Subject":"[PATCH v2 0/9] Software ISP support for CCM","Date":"Mon, 25 Nov 2024 19:03:01 +0100","Message-ID":"<20241125180310.1254519-1-mzamazal@redhat.com>","MIME-Version":"1.0","X-Scanned-By":"MIMEDefang 3.0 on 10.30.177.15","X-Mimecast-Spam-Score":"0","X-Mimecast-MFC-PROC-ID":"th3PRCjYxkRVQSps6I5vPJNKaY0GtRiMLTp2YvZ3oR8_1732557800","X-Mimecast-Originator":"redhat.com","Content-Transfer-Encoding":"8bit","content-type":"text/plain; charset=\"US-ASCII\"; x-default=true","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This series implements application of color correction matrices in\nsoftware ISP.\n\nSupport for color temperature is added to auto white balance algorithm,\nthis is needed to obtain the right CCM. A new Ccm algorithm is added to\ndetermine the matrix. Both roughly follow what the hardware pipelines\ndo.\n\nLut algorithm is modified to incorporate the CCM and perform some\nprecomputations in the form of 10 lookup tables (one per each matrix\nelement and one for gamma).\n\nA tricky part is to support both CCM and the original (no-CCM)\ntransformations. CCM obviously adds a significant overhead; per-frame\ntime increases by ~45% on Debix Model A and ~85% on TI AM69 SK. This\nmeans CCM must be optional, which is determined by presence of Ccm\nalgorithm in the tuning file.\n\nThe performance critical debayering code must not include extra\nconditionals and must work efficiently for both the cases. The macros\nin debayering code are refactored and restructured for this. An added\nbonus is that this also removes some annoying code duplication. And\nsome more, partially already present, templating is used to select\nbetween CCM and non-CCM processing.\n\nPerformance is much influenced by the data structures used in\ndebayering. I tried to define something reasonable and performed only\nbasic performance testing. Suggestions for possible improvements are\nwelcome.\n\nSee the commit messages for more details.\n\nChanges in v2:\n- Fix of the gamma table index upper boundary.\n- Fix of red<->blue being temporarily swapped in an intermediate patch.\n- Fix of CCM transformation when swapRedBlueGains_ is true.\n- A related minor lookup data structures rearrangement, with an\n unintended but welcome effect of ~10% speedup with CCM in my\n environment. \n\nMilan Zamazal (9):\n libcamera: software_isp: Determine color temperature\n libcamera: software_isp: Store color temperature to metadata\n libcamera: software_isp: lut: Remove maybe_unused on a used argument\n libcamera: software_isp: Use common code to store debayered pixels\n libcamera: software_isp: Use a macro to assign debayering methods\n libcamera: software_isp: Add CCM algorithm\n libcamera: software_isp: Add an example CCM to uncalibrated.yaml\n libcamera: software_isp: Track whether CCM is enabled\n libcamera: software_isp: Apply CCM in debayering\n\n .../internal/software_isp/debayer_params.h | 20 ++-\n include/libcamera/ipa/soft.mojom | 2 +-\n src/ipa/simple/algorithms/awb.cpp | 18 ++-\n src/ipa/simple/algorithms/ccm.cpp | 86 ++++++++++++\n src/ipa/simple/algorithms/ccm.h | 45 ++++++\n src/ipa/simple/algorithms/lut.cpp | 62 +++++---\n src/ipa/simple/algorithms/lut.h | 1 +\n src/ipa/simple/algorithms/meson.build | 1 +\n src/ipa/simple/data/uncalibrated.yaml | 7 +\n src/ipa/simple/ipa_context.h | 22 ++-\n src/ipa/simple/soft_simple.cpp | 8 +-\n src/libcamera/software_isp/debayer.cpp | 53 ++++++-\n src/libcamera/software_isp/debayer.h | 3 +-\n src/libcamera/software_isp/debayer_cpu.cpp | 132 ++++++++++--------\n src/libcamera/software_isp/debayer_cpu.h | 31 ++--\n src/libcamera/software_isp/software_isp.cpp | 11 +-\n 16 files changed, 393 insertions(+), 109 deletions(-)\n create mode 100644 src/ipa/simple/algorithms/ccm.cpp\n create mode 100644 src/ipa/simple/algorithms/ccm.h"}