From patchwork Mon Nov 15 16:12:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14613 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7267BBF415 for ; Mon, 15 Nov 2021 16:13:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0F50C603EE; Mon, 15 Nov 2021 17:13:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="pkafFUku"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE06D603BE for ; Mon, 15 Nov 2021 17:13:02 +0100 (CET) Received: by mail-wm1-x32c.google.com with SMTP id o29so14465858wms.2 for ; Mon, 15 Nov 2021 08:13:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MWi8mj5S4W6cGhtr7B2BNpqXeeXBc2vqiVreCzJ+MGg=; b=pkafFUkuNsZmry6XV2Z3L1635rN0NUm46EEalWr8Y4ft1buSpHbllgzIv/HpD3O8YL c519zKZaSlJHMuMIkoFw3OOO3Ky+wRpY8gJUUq1DxjNvVq2LJV/N48jiTmqy8pU8wv9I TzTpIf6iZoiWDVvGIFq7YuxTBS3G1TL5dlDi44WFtENXDsQEGvwzIrnaZoAThTSIm61X KBmuBJwgcl0rcnoKZUZ+/u67XOmDQ3ibUsj+vw+0A0s+C4QPtS17nDP4MIFmEodWfNCw pJ1uIhdPYCQCIKsE/jl/63Okv9KxrwCHVXZasR3rdwsZuHAwmJaNJVU06enapOgrh8l1 AEbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MWi8mj5S4W6cGhtr7B2BNpqXeeXBc2vqiVreCzJ+MGg=; b=5suObWhDYUlYq2/Es69cVpnsgeVCd63mZP3JJyxGNggwc+oMHNx9+pPU834lZrArns 9wLyEq8MJ9554rG3tduk22ZxI83XhCUC15RKAUq50fnKMMKxJk8cOGV3Q1xfo7BpRzQg GS1X6Xec8gWeKnLYzjQlc+/dEfd/AJuQEnAB5iMrCzRdkoGmzE057zjXbeX6RBVjHvSk +aKVsPBYGr3PK/cPRPAoceMeNbzyETwJan8IcUfjOfBWof33eHTHuQ7Oyt7BKUjEMi7Z 7QNp1B0MhDnPshvFOZOjSJqoDTunARG2uketpSKYRXharw+9T4Az06eXv7UCR9eKLNyf B1oQ== X-Gm-Message-State: AOAM530zFq3hlZ4AzMge34q1mwVCD9rjfebOAXrkxw7KIfBYWmB1JRnN yZ20d/VU22pw2dCRQYMWbZJZhQuNuoQZEilg X-Google-Smtp-Source: ABdhPJwOwf5AXDcB5BwQalHOCOiBNJBq3ftV4r+n3CsV9NpA6zPVut7C2g5Cf66XJDvKZZro+4QMwA== X-Received: by 2002:a1c:cc09:: with SMTP id h9mr45736360wmb.191.1636992782633; Mon, 15 Nov 2021 08:13:02 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id j40sm16315615wms.16.2021.11.15.08.13.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 08:13:02 -0800 (PST) From: David Plowman To: hverkuil-cisco@xs4all.nl, laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, jacopo@jmondi.org, tfiga@google.com, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Mon, 15 Nov 2021 16:12:05 +0000 Message-Id: <20211115161205.24335-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211115161205.24335-1-david.plowman@raspberrypi.com> References: <20211115161205.24335-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 7/7] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- .../pipeline/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5e1f2273..bbb21e9b 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -93,6 +93,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -126,6 +127,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -332,6 +334,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -494,10 +498,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.requestedColorSpace; + LOG(RPI, Debug) + << "Try color space " << cfg.requestedColorSpace.toString(); int ret = dev->tryFormat(&format); if (ret) return Invalid; + cfg.actualColorSpace = format.colorSpace; + if (cfg.actualColorSpace != cfg.requestedColorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << cfg.requestedColorSpace.toString() << " to " + << cfg.actualColorSpace.toString(); + } cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -522,6 +537,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + ColorSpace colorSpace; if (roles.empty()) return config; @@ -536,6 +552,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -543,6 +560,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -560,6 +578,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -568,6 +588,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -599,6 +620,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.requestedColorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -703,6 +725,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.requestedColorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -718,6 +741,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (cfg.actualColorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space change (" + << cfg.actualColorSpace.toString() << " to " + << format.colorSpace.toString() << ") in stream " + << stream->name(); + cfg.actualColorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << cfg.actualColorSpace.toString(); + cfg.setStream(stream); stream->setExternal(true); @@ -742,6 +782,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)