From patchwork Wed Dec 1 15:44:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14981 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 26F36C3250 for ; Wed, 1 Dec 2021 15:45:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CC605607BD; Wed, 1 Dec 2021 16:45:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="egZLE11i"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 44E3E60820 for ; Wed, 1 Dec 2021 16:45:09 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id o19-20020a1c7513000000b0033a93202467so1478776wmc.2 for ; Wed, 01 Dec 2021 07:45:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VJw90Ju80i+WaMP0k5HDW9tQHz/oLLZEeXbCbFwltEk=; b=egZLE11ijGhRUxvpsICptPjzDRmOqV7byRQKeHG69VPEclD5NJdq5uUTZLWode4W3l WR8JQeWjlsJcFoObqu/lgM8wMIoxF1RriFtew5Sll9opzDculqo0mlTygQbB1Z1Jjkkr Uq+68914b6W6m0zpAOr1lOLysQu6iYhjrNYTXgb7vObUNOCKtuJw/VkCUfsRrSQjcZ2P GqbaZI8IjE1fnWhREYlRdCc/tTwANthobrROejZOmlKZ7w1XKX/dOrltmAEbAZKPgkDa 6OMnm6SM2WLov/LOQxHlOBgjWdESgLKoi2C78rzDmVVYw9kUb07gl/SkOoyX+pylMRT1 SroA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VJw90Ju80i+WaMP0k5HDW9tQHz/oLLZEeXbCbFwltEk=; b=4t4P36pSlGG4sIj28J5YI5ayPnNFvs0xLNbcPgkd+ODljKE7WIIvA8bhDLBg6EqnUM y6ZFjhv03bgHdi9sTVk09kYf1cWcdTVWI4CLepXwLt7E+39pFd0ef9/mt2KjXzC+0Y6A 4kiLJ0jUA/fMzQ3VWZXuW5Z942Ta+fCnSuwIIp86FfRrv0HxzL8q9+VYWTenJMyGv+Pm +zjtiLgmcptA7aqCj8f7o+QqT2ONTa7jdLNgeqSWEeR284Ax4hQ0vcU7sTJfaI3hM5E3 DirDIizDwl1XNiMhtZas80Mz72FiCzgguAkf7K7k1mzWfDGBV7MtA1FeTcowN3R20W5E cUcg== X-Gm-Message-State: AOAM533korm/abMnODt/SQ479PbqmrwBt7S797zcPul+BTZ/j94bgUCd qYgwhXDvS6SNnlYhLyTNzYmndnjB2g1qK2TB X-Google-Smtp-Source: ABdhPJzfigTm4sVxp4bmGACoE3u9kKir946KgdiZHxqlYLwAJWM23sg02WAvudmgh/5OscGfeEUPOg== X-Received: by 2002:a1c:f20e:: with SMTP id s14mr8198200wmc.186.1638373508671; Wed, 01 Dec 2021 07:45:08 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm148209wrw.55.2021.12.01.07.45.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 07:45:08 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org, Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Hans Verkuil , Tomasz Figa , Naushir Patuck Date: Wed, 1 Dec 2021 15:44:34 +0000 Message-Id: <20211201154434.23127-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211201154434.23127-1-david.plowman@raspberrypi.com> References: <20211201154434.23127-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v8 8/8] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index ad526a8b..19e7addb 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -93,6 +93,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -129,6 +130,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -335,6 +337,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -497,11 +501,23 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + cfg.colorSpace = format.colorSpace; + cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -525,6 +541,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -539,6 +556,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -546,6 +564,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -563,6 +582,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -571,6 +592,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -602,6 +624,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -706,6 +729,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -721,6 +745,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space (" + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace) << ") in stream " + << stream->name(); + cfg.colorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -745,6 +786,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)