From patchwork Fri Nov 26 10:40:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14803 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BCF63BF415 for ; Fri, 26 Nov 2021 10:41:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5B49560566; Fri, 26 Nov 2021 11:41:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="H3eLdUck"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BA58760568 for ; Fri, 26 Nov 2021 11:41:04 +0100 (CET) Received: by mail-wm1-x32f.google.com with SMTP id n33-20020a05600c502100b0032fb900951eso10356058wmr.4 for ; Fri, 26 Nov 2021 02:41:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jf1EaSSTDD8mkESejGEpzJhNIYmQJlpPGr+i0A7KLkA=; b=H3eLdUck+l7IOyZmU42uvsEs2Uj3yHg+e91VMb2NY+La+Va/6uLuXwYUBFnxnx/kTN wslZ8lj7Mds3G5NLT0bdHtWVPs0yTl3+qHryJ9jd/eDrN1d6ukqGgUe/OOY6AwJgA675 f4RSE2+J/rMqK9Ot55yUbloGF3pIqC7B9lF+csKO28B74ohzoEufBERQhzeF7eBorDjL /Sblr/fJ8pMO9YrypUPjNm31XRUC2hP40egYQ6uFl89/B4EdG30tnYUukNZ7jeWVRme/ 9Ks4+LjO0MWW5VjCOHsQWWcmoX2qYka9vVkuvhTsIPzPhPR1T4Kx0OEqK/ZOj0NXqIoV Jg+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jf1EaSSTDD8mkESejGEpzJhNIYmQJlpPGr+i0A7KLkA=; b=tGWNiZ1jA2FPealNrWTfdBzzFHN/GLs9zZTYJmF+YQh3ZTzerwrTbXSgILW+tpGcGS jfj0zwILB4kFy+AgwCRcMmbZp7nxDbLpAYPEus3V5kAyYjFHrxNSXAWIPB2uUSBpTHy1 Hai7B7GV51u3AFlKMhHp+puqtTlwnnfXhZicfsBjPn0hOcRAzXkuFM2ZlIcvmLVWktQ2 7DnqYcnpJm+HJqH29i0ckY8kpNvD2APkZtUJeUYVR+14GiXAKKXl1X3UBOF6iYkJxDqa eYf512ysiB1MRHb0b9iGQXwklWTo34+ARo54HNdBDdctl3d3Tz9iDlIwhTlnobqS6u85 4Iuw== X-Gm-Message-State: AOAM531wzRSxdx5udaYCPAORIfmpBt+nMC53kPzSa2sgrKOh8j3ydgSu Y0dFX+z5G2j8j7YQLPxT2nvbPQ== X-Google-Smtp-Source: ABdhPJyfZXRqd1O4hjR5Dv19abR2ojU1sEX/X7whho/VoR65pw4n1uQLzSg9gprCUY4LaZElC0jksQ== X-Received: by 2002:a05:600c:21cd:: with SMTP id x13mr14906909wmj.52.1637923264385; Fri, 26 Nov 2021 02:41:04 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:e4a2:3070:eea4:e434]) by smtp.gmail.com with ESMTPSA id y7sm5106795wrw.55.2021.11.26.02.41.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Nov 2021 02:41:03 -0800 (PST) From: David Plowman To: Laurent Pinchart , Kieran Bingham , Hans Verkuil , Tomasz Figa , Jacopo Mondi , Naushir Patuck , libcamera-devel@lists.libcamera.org Date: Fri, 26 Nov 2021 10:40:45 +0000 Message-Id: <20211126104045.4756-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211126104045.4756-1-david.plowman@raspberrypi.com> References: <20211126104045.4756-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v7 7/7] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index ad526a8b..f0ec23cb 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -93,6 +93,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -129,6 +130,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -335,6 +337,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -497,10 +501,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + cfg.colorSpace = format.colorSpace; cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -525,6 +540,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -539,6 +555,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -546,6 +563,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -563,6 +581,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -571,6 +591,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -602,6 +623,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -706,6 +728,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -721,6 +744,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space (" + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace) << ") in stream " + << stream->name(); + cfg.colorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -745,6 +785,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)