From patchwork Thu Nov 18 15:19:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 14632 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E16D2C324F for ; Thu, 18 Nov 2021 15:20:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7CB7C60231; Thu, 18 Nov 2021 16:20:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Y1n1m+94"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A73960231 for ; Thu, 18 Nov 2021 16:20:31 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id c71-20020a1c9a4a000000b0032cdcc8cbafso5084066wme.3 for ; Thu, 18 Nov 2021 07:20:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zJKMksua6c6xIYVePhIiVLpfcsB/8lKEFhsflNfPxDc=; b=Y1n1m+943lV/X3THWku0FecTuLVN3nsOxp/ssfkLmmDVtY0r2roIj4JrQf5cBVjfo8 8WLO//2NYjm1rdSgoaOGyIaxN6Vog5OtHzmu/vy2histdt5oaOjfbfcwhLkOzDC4Ndq9 xJqJQRAs7m1ioSayUwnLpIjyRELwbvq0EVvTzBZszC0puVV9jBNKgIrRyLqoQ5dksABe TKuU2Rx8feV5MK71J7TMaf2zblqtwMaGlXGWXz7gjXtNjP5Uvt9WGZrLJvCdSKr7ANpK XPeDtQZxhsIiHYp/oEq94gJnX3JL4tdi4N+4qdC8AyKH5QvtISw3qYOxxc0727aCoxzy yXZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zJKMksua6c6xIYVePhIiVLpfcsB/8lKEFhsflNfPxDc=; b=RiJxaJjSMLRXrxJIWrbFKIf3c/a7IGEWsB+n3dRqm1WYPU/TN1x2GAuVB3PY8BLZZf gW7GqNtT+fetG6DuC3Eso0Y/KDkWjcHMaPLz76dooAoVz0JRcr5HoQUYCW0TpLlqVeDm mlIal8FmIxKLUEp2pH1ksT23aOsValb4JlTanp054WFb+RJGhRLqb//zx0AidFqNb3Xc ZOXnk7E0YbDHZH8dq0kPRa1QsVkiNJqmW/eHzt1YSXIhnp4I3gToJLz7EtotpKZ3d8hO nbrhaMfhDkWScfIKkJ5I+ekdMtwRy9FMvAkWL9ohdmH3AgUsjOtzRb4XGuuuamTtLiJc Dsqw== X-Gm-Message-State: AOAM533EVncVpmvqh9G9MMwSdb1gclqbPByUgKyM3mqLnR2rNhY0cVZ5 P9iRpgZrlHL/6SfW3Thb0Zp4iw== X-Google-Smtp-Source: ABdhPJwl+3cpG2F6EqvyQtYon45NsKTdtVczGQI+9OHBrQdQdrXKLRdNhn4NtS+0yuB4CQ30rpZfCA== X-Received: by 2002:a05:600c:224a:: with SMTP id a10mr10815542wmm.154.1637248830874; Thu, 18 Nov 2021 07:20:30 -0800 (PST) Received: from pi4-davidp.pitowers.org ([2a00:1098:3142:14:1ce1:9965:4328:89c4]) by smtp.gmail.com with ESMTPSA id p12sm147367wro.33.2021.11.18.07.20.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Nov 2021 07:20:30 -0800 (PST) From: David Plowman To: laurent.pinchart@ideasonboard.com, kieran.bingham@ideasonboard.com, hverkuil-cisco@xs4all.nl, tfiga@google.com, jacopo@jmondi.org, naush@raspberrypi.com, libcamera-devel@lists.libcamera.org Date: Thu, 18 Nov 2021 15:19:33 +0000 Message-Id: <20211118151933.15627-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20211118151933.15627-1-david.plowman@raspberrypi.com> References: <20211118151933.15627-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 7/7] libcamera: pipeline: raspberrypi: Support color spaces X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi pipeline handler now sets color spaces correctly. In generateConfiguration() it sets them to reasonable default values based on the stream role. validate() now calls validateColorSpaces() to ensure that the requested color spaces are sensible, before proceeding to check what the hardware can deliver. Signed-off-by: David Plowman --- .../pipeline/raspberrypi/raspberrypi.cpp | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 5e1f2273..5f09ba59 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -93,6 +93,7 @@ V4L2DeviceFormat toV4L2DeviceFormat(const V4L2SubdeviceFormat &format, deviceFormat.fourcc = V4L2PixelFormat::fromPixelFormat(pix); deviceFormat.size = format.size; + deviceFormat.colorSpace = format.colorSpace; return deviceFormat; } @@ -126,6 +127,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & { double bestScore = std::numeric_limits::max(), score; V4L2SubdeviceFormat bestFormat; + bestFormat.colorSpace = ColorSpace::Raw; #define PENALTY_AR 1500.0 #define PENALTY_8BIT 2000.0 @@ -332,6 +334,8 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() if (config_.empty()) return Invalid; + status = validateColorSpaces(true); + /* * What if the platform has a non-90 degree rotation? We can't even * "adjust" the configuration and carry on. Alternatively, raising an @@ -494,10 +498,21 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() V4L2DeviceFormat format; format.fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; + format.colorSpace = cfg.colorSpace; + LOG(RPI, Debug) + << "Try color space " << ColorSpace::toString(cfg.colorSpace); int ret = dev->tryFormat(&format); if (ret) return Invalid; + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + status = Adjusted; + LOG(RPI, Warning) + << "Color space changed from " + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace); + } + cfg.colorSpace = format.colorSpace; cfg.stride = format.planes[0].bpl; cfg.frameSize = format.planes[0].size; @@ -522,6 +537,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, PixelFormat pixelFormat; V4L2VideoDevice::Formats fmts; Size size; + std::optional colorSpace; if (roles.empty()) return config; @@ -536,6 +552,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, pixelFormat = mbusCodeToPixelFormat(sensorFormat.mbus_code, BayerFormat::Packing::CSI2); ASSERT(pixelFormat.isValid()); + colorSpace = ColorSpace::Raw; bufferCount = 2; rawCount++; break; @@ -543,6 +560,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::StillCapture: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::NV12; + colorSpace = ColorSpace::Jpeg; /* Return the largest sensor resolution. */ size = data->sensor_->resolution(); bufferCount = 1; @@ -560,6 +578,8 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, */ fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::YUV420; + /* This will be reasonable for many applications. */ + colorSpace = ColorSpace::Rec709; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -568,6 +588,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, case StreamRole::Viewfinder: fmts = data->isp_[Isp::Output0].dev()->formats(); pixelFormat = formats::ARGB8888; + colorSpace = ColorSpace::Jpeg; size = { 800, 600 }; bufferCount = 4; outCount++; @@ -599,6 +620,7 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, StreamConfiguration cfg(formats); cfg.size = size; cfg.pixelFormat = pixelFormat; + cfg.colorSpace = colorSpace; cfg.bufferCount = bufferCount; config->addConfiguration(cfg); } @@ -703,6 +725,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) V4L2PixelFormat fourcc = V4L2PixelFormat::fromPixelFormat(cfg.pixelFormat); format.size = cfg.size; format.fourcc = fourcc; + format.colorSpace = cfg.colorSpace; LOG(RPI, Debug) << "Setting " << stream->name() << " to " << format.toString(); @@ -718,6 +741,23 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) return -EINVAL; } + if (!format.colorSpace || cfg.colorSpace != format.colorSpace) { + /* + * We should have been through validate() before so this + * shouldn't be possible, but we mustn't sweep color space + * problems under the carpet. + */ + LOG(RPI, Warning) + << "Unexpected color space (" + << ColorSpace::toString(cfg.colorSpace) << " to " + << ColorSpace::toString(format.colorSpace) << ") in stream " + << stream->name(); + cfg.colorSpace = format.colorSpace; + } + LOG(RPI, Debug) + << "Stream " << stream->name() << " has color space " + << ColorSpace::toString(cfg.colorSpace); + cfg.setStream(stream); stream->setExternal(true); @@ -742,6 +782,8 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format = {}; format.size = maxSize; format.fourcc = V4L2PixelFormat::fromPixelFormat(formats::YUV420); + /* No one asked for output, so the color space doesn't matter. */ + format.colorSpace = ColorSpace::Jpeg; ret = data->isp_[Isp::Output0].dev()->setFormat(&format); if (ret) { LOG(RPI, Error)