[{"id":17639,"web_url":"https://patchwork.libcamera.org/comment/17639/","msgid":"","date":"2021-06-18T22:18:08","subject":"Re: [libcamera-devel] [PATCH v3 0/3] Support monochrome raw sensors","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"On 18/06/2021 23:04, David Plowman wrote:\n> Hi again\n\nHello ;-)\n\n> Here's a v3 set which addresses Kieran's comments, so thanks for\n> those! Otherwise everything is functionally the same as before.\n> \n> To answer Kieran's other questions:\n> \n> * Yes, a monochrome raw sensor is very different to a monochrome\n> non-raw sensor! No black levels, vignetting to worry about on those,\n> among other things.\n\nUnderstood.\n\n\n> * On the question of warning about missing AWB results only for\n> monochrome sensors. Yes, that's a reasonable thought, though at the\n> moment I don't think the IPA side of the world even knows if a\n> sensor is monochrome. We'd have to pass that somehow, maybe in the\n> IPACameraSesnsorInfo? That's perhaps a slightly longer discussion,\n> I'm not entirely sure I want to open that right now...\n\nI don't mind either at the moment. I wasn't sure if it was directly\nfeasible, so it could be something to optionally consider later if it\nbecame helpful. The print is still there - it might just then needlessly\nspam the debug logs for monochrome sensors. But I don't think that will\nbe a big impact currently.\n\n\n\n> * We need to be very nice to Dave, but then the plan would be to start\n> upstreaming new drivers, as we've been doing previously. There are a\n> couple more I'd like to get going too. Some of them would need a\n> \"Y12P\" format, which seems to be missing from the kernel, currently.\n\nI'll leave Laurent to the decisions there, but as long as we know\nthere's intent to get the support upstream, I think the IPA can get in\nplace earlier. I don't think there's particular risk on this of things\nchanging on the way in, as we use the standard interfaces ...\n\n\n> * The tuning I've given for the OV9281 is for an Innomaker module, or\n> maybe not, I'm not sure what it is. But it actually makes\n> surprisingly little difference as there is no colour shading, AWB or\n> colour matrices to worry about.\n\nWell I don't think anyone can complain about less things to do. The ISP\ngets to take a break ;-)\n\nI guess things like lens shading will still apply here and be specific\nto a module - but that needs to be per-device anyway ... and I think I\nrecall you mentioned that the ALSC has a good go at it anyway.\n\n\n> Thanks\n> David\n> \n> \n> David Plowman (3):\n> libcamera: Add support for monochrome sensors\n> libcamera: ipa: raspberrypi: Demote warnings about lack of AWB results\n> libcamera: ipa: raspberrypi: Add support for ov9281 sensor\n> \n> include/libcamera/internal/bayer_format.h | 3 +-\n> src/ipa/raspberrypi/cam_helper_mov9281.cpp | 65 +++++++++++++++\n> src/ipa/raspberrypi/controller/rpi/agc.cpp | 2 +-\n> src/ipa/raspberrypi/controller/rpi/alsc.cpp | 4 +-\n> src/ipa/raspberrypi/data/meson.build | 1 +\n> src/ipa/raspberrypi/data/mov9281.json | 92 +++++++++++++++++++++\n> src/ipa/raspberrypi/meson.build | 1 +\n> src/libcamera/bayer_format.cpp | 14 +++-\n> src/libcamera/camera_sensor.cpp | 3 +\n> src/libcamera/property_ids.yaml | 4 +\n> 10 files changed, 183 insertions(+), 6 deletions(-)\n> create mode 100644 src/ipa/raspberrypi/cam_helper_mov9281.cpp\n> create mode 100644 src/ipa/raspberrypi/data/mov9281.json\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 3C00EBD78E\n\tfor ;\n\tFri, 18 Jun 2021 22:18:13 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A982468942;\n\tSat, 19 Jun 2021 00:18:12 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 082EC6050C\n\tfor ;\n\tSat, 19 Jun 2021 00:18:12 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8180655C;\n\tSat, 19 Jun 2021 00:18:11 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"UgwI7qiQ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1624054691;\n\tbh=IWe7NBMJoPSSMDESKPixdxs7GPbnLC3IL315nX7tacw=;\n\th=Reply-To:To:References:From:Subject:Date:In-Reply-To:From;\n\tb=UgwI7qiQJWhJodRtwdNPIVS3bXZLGcUbl6/AOMYXKZN62z3u3TzXOCsxeFGTENqLU\n\t+y1nu3Hwl+dWQPXwFF7Ii2/Fc7SouNgcqlYdsWdlGuM0nZi77mRs+QuIvenTZssApp\n\tFQDvMzOV7Phl6Gvz35VPTXnzTuXNMUH9M7o+mo/A=","To":"David Plowman ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20210618220430.31457-1-david.plowman@raspberrypi.com>","From":"Kieran Bingham ","Organization":"Ideas on Board","Message-ID":"","Date":"Fri, 18 Jun 2021 23:18:08 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.8.1","MIME-Version":"1.0","In-Reply-To":"<20210618220430.31457-1-david.plowman@raspberrypi.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-GB","Content-Transfer-Encoding":"8bit","Subject":"Re: [libcamera-devel] [PATCH v3 0/3] Support monochrome raw sensors","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":17656,"web_url":"https://patchwork.libcamera.org/comment/17656/","msgid":"","date":"2021-06-21T11:19:43","subject":"Re: [libcamera-devel] [PATCH v3 0/3] Support monochrome raw sensors","submitter":{"id":27,"url":"https://patchwork.libcamera.org/api/people/27/","name":"Dave Stevenson","email":"dave.stevenson@raspberrypi.com"},"content":"Hi Kieran\n\nOn Fri, 18 Jun 2021 at 23:18, Kieran Bingham\n wrote:\n>\n> On 18/06/2021 23:04, David Plowman wrote:\n> > Hi again\n>\n> Hello ;-)\n>\n> > Here's a v3 set which addresses Kieran's comments, so thanks for\n> > those! Otherwise everything is functionally the same as before.\n> >\n> > To answer Kieran's other questions:\n> >\n> > * Yes, a monochrome raw sensor is very different to a monochrome\n> > non-raw sensor! No black levels, vignetting to worry about on those,\n> > among other things.\n>\n> Understood.\n>\n>\n> > * On the question of warning about missing AWB results only for\n> > monochrome sensors. Yes, that's a reasonable thought, though at the\n> > moment I don't think the IPA side of the world even knows if a\n> > sensor is monochrome. We'd have to pass that somehow, maybe in the\n> > IPACameraSesnsorInfo? That's perhaps a slightly longer discussion,\n> > I'm not entirely sure I want to open that right now...\n>\n> I don't mind either at the moment. I wasn't sure if it was directly\n> feasible, so it could be something to optionally consider later if it\n> became helpful. The print is still there - it might just then needlessly\n> spam the debug logs for monochrome sensors. But I don't think that will\n> be a big impact currently.\n>\n>\n>\n> > * We need to be very nice to Dave, but then the plan would be to start\n> > upstreaming new drivers, as we've been doing previously. There are a\n> > couple more I'd like to get going too. Some of them would need a\n> > \"Y12P\" format, which seems to be missing from the kernel, currently.\n>\n> I'll leave Laurent to the decisions there, but as long as we know\n> there's intent to get the support upstream, I think the IPA can get in\n> place earlier. I don't think there's particular risk on this of things\n> changing on the way in, as we use the standard interfaces ...\n\nYou know the way I tend to work is to write patches that should be\nacceptable to upstream, but rarely have the energy to do battle there.\nYou're aware that Gordon (software boss at Pi Towers) is generally\nhappy to contract out upstreaming work if there are people willing to\ndo it....\n\n\nThe two patches for Y12P and Y14P exist in our rpi-5.10.y tree with\ndocs, so it should just be a case of firing them off.\nc79333a30676 media: Add a pixel format for MIPI packed 14bit luma only.\nec127d471e84 media: Add a pixel format for MIPI packed 12bit luma only.\n\nSensor drivers are a little more complex, but hopefully this OV9281\ndriver is in reasonable shape for upstream.\nThere are also:\nOV7251 (VGA mono global shutter) is very similar. I meant to drop my\nmodule into the office as another one for David to do a rough tune on,\nbut forgot.\nOV2311 (1600x1300 mono global shutter) isn't merged to our tree quite\nyet as it needs some more work.\nIMX290/IMX327 (1080p 60fps in colour or mono variants) I sorted last\nweek to have all the relevant controls working correctly for\nlibcamera, extending the existing mainline driver.\n\n> > * The tuning I've given for the OV9281 is for an Innomaker module, or\n> > maybe not, I'm not sure what it is. But it actually makes\n> > surprisingly little difference as there is no colour shading, AWB or\n> > colour matrices to worry about.\n\nAlmost all of the ones we have are Innomaker (available via Amazon [1]).\nI do have 2 Arducam ones [2] [3], and a Vision Components[4] one, but\nI have all those variants in my big box of sensors. [2] only has a\nsingle data lane wired up and currently doesn't work with the driver,\nbut the other 2 work fine.\n\n[1] https://www.amazon.co.uk/Raspberry-Industrial-Monochrome-Shutter-1MPixel/dp/B085VQ9BBY\n[2] Integrated lens\nhttps://www.arducam.com/product/arducam-ov9281-mipi-1mp-monochrome-global-shutter-camera-module-raspberry-pi/\n[3] M12 lens https://www.arducam.com/product/arducam-ov9281-1mp-mono-global-shutter-mipi-camera-with-130degree-850nm-only-m12-mount-for-raspberry-pi/\n[4] https://www.vision-components.com/en/products/oem-embedded-vision-systems/mipi-camera-modules/\n\n> Well I don't think anyone can complain about less things to do. The ISP\n> gets to take a break ;-)\n>\n> I guess things like lens shading will still apply here and be specific\n> to a module - but that needs to be per-device anyway ... and I think I\n> recall you mentioned that the ALSC has a good go at it anyway.\n\nLens shading will apply here as most of these OV9281 modules have M12\nmounts so could have any lens. I have a selection of M12 lenses from a\n1.7mm fisheye up to I think it's a 25mm.\nAt least there isn't colour variation due to the lens, just overall vignetting.\n\n Dave\n\n> > Thanks\n> > David\n> >\n> >\n> > David Plowman (3):\n> > libcamera: Add support for monochrome sensors\n> > libcamera: ipa: raspberrypi: Demote warnings about lack of AWB results\n> > libcamera: ipa: raspberrypi: Add support for ov9281 sensor\n> >\n> > include/libcamera/internal/bayer_format.h | 3 +-\n> > src/ipa/raspberrypi/cam_helper_mov9281.cpp | 65 +++++++++++++++\n> > src/ipa/raspberrypi/controller/rpi/agc.cpp | 2 +-\n> > src/ipa/raspberrypi/controller/rpi/alsc.cpp | 4 +-\n> > src/ipa/raspberrypi/data/meson.build | 1 +\n> > src/ipa/raspberrypi/data/mov9281.json | 92 +++++++++++++++++++++\n> > src/ipa/raspberrypi/meson.build | 1 +\n> > src/libcamera/bayer_format.cpp | 14 +++-\n> > src/libcamera/camera_sensor.cpp | 3 +\n> > src/libcamera/property_ids.yaml | 4 +\n> > 10 files changed, 183 insertions(+), 6 deletions(-)\n> > create mode 100644 src/ipa/raspberrypi/cam_helper_mov9281.cpp\n> > create mode 100644 src/ipa/raspberrypi/data/mov9281.json\n> >\n>\n> --\n> Regards\n> --\n> Kieran","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id ECD4EC3218\n\tfor ;\n\tMon, 21 Jun 2021 11:20:02 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 28BB960296;\n\tMon, 21 Jun 2021 13:20:02 +0200 (CEST)","from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com\n\t[IPv6:2a00:1450:4864:20::42c])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 99F0260295\n\tfor ;\n\tMon, 21 Jun 2021 13:19:59 +0200 (CEST)","by mail-wr1-x42c.google.com with SMTP id m18so19212373wrv.2\n\tfor ;\n\tMon, 21 Jun 2021 04:19:59 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"moHfwQRJ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=pCa4tJyIiDXWKnqCLsK+gagsjvj0ohfgZR7Y17lGpEg=;\n\tb=moHfwQRJp1rcDDJ9bfAXwwTitM1CV2otgxHzhSvQB2KMuQknn6Y6/xX4ZgOUgDO8pI\n\tAMhIhIJOZqnxlK0EhanjWftbcP5fQPQ/UACvIjHrKL3p3EQTIZlZaDBHKDmaLd7IkKS3\n\tG2D7tC/2G27mg+JMByBULCzLhzySntC/5fOTGFNU8G82xla0fQaeUHgowkVnFGk5kTPg\n\ttVX1VzNJGYXCVsBY5b+JiPzfjXI7zIWl2Zh59FVE4GgcL3SAXekIPdwA4V6CChGgY5jd\n\t8GwSqj22I3tZcmDe2OlpFnr0mpZiO5eWxMLIXsBp3v5CaTAtyJ/HD8PF4FtLcDAdX9Wx\n\tXp5Q==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=pCa4tJyIiDXWKnqCLsK+gagsjvj0ohfgZR7Y17lGpEg=;\n\tb=roxmBqa94Lzv0AzemvM5DbmjsP2c8ippWAOoFeb56V/L/NyGREVYYEu+7h2X761O/H\n\tEZjOLSPMDZmCE3MHWBLFJSJ3EVXIVeze1u6/Plsy9aORaFws57gM8x07Z4rogauiefZi\n\tj0GCeYjqqllxaIpElxfU66950OXDMT7V1ztWHMJZBAQWE9euIS8s1pfD/QBJhH0ihl1w\n\t7XJwmOE3AN50xhOqphX+4hHpCn26Z2eaT0Gmn49cwCaVHycYsVg6y6SrXWV9rZKDWo2Y\n\tC+AV9XGCSMBqD7i4Jv8/dHZ2XenIQcH0FPVfcuy8BsvydTGr3qnuiE2ptMrBQ6ph0Z/1\n\ttsug==","X-Gm-Message-State":"AOAM5301z8ut04/HrPvHYTtY7m182K1BSWuZ0f2Dqi8ve/kb9acY17Od\n\th/rFIBkUSHaeGkDKyYMTju2qbghLcjQqAupPfuXRbg==","X-Google-Smtp-Source":"ABdhPJz5bBvQ9gyPPPv8U259ZREu+lOxQp+jQQtsATIzmiiy6CmGUjaxBzy+w47ibwwUwI0tm0larPv5yG1nCHZ+Wm8=","X-Received":"by 2002:a05:6000:18ab:: with SMTP id\n\tb11mr28121544wri.42.1624274399283; \n\tMon, 21 Jun 2021 04:19:59 -0700 (PDT)","MIME-Version":"1.0","References":"<20210618220430.31457-1-david.plowman@raspberrypi.com>\n\t","In-Reply-To":"","From":"Dave Stevenson ","Date":"Mon, 21 Jun 2021 12:19:43 +0100","Message-ID":"","To":"\"cc: Kieran Bingham\" ","Content-Type":"text/plain; charset=\"UTF-8\"","Subject":"Re: [libcamera-devel] [PATCH v3 0/3] Support monochrome raw sensors","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]