From patchwork Tue Mar 2 23:53:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 11472 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A830DBD1F1 for ; Tue, 2 Mar 2021 23:54:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1DE8568A93; Wed, 3 Mar 2021 00:54:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cCE83gyH"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A44A602E8 for ; Wed, 3 Mar 2021 00:54:20 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E00EF29A for ; Wed, 3 Mar 2021 00:54:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1614729260; bh=GUKTlNiZI/vA0fpo13JNA9kZgdqUSg1SBBN/lKY5Ywg=; h=From:To:Subject:Date:From; b=cCE83gyHlUtTKLuVjq/6aKuwkVmS8ELvuhk9brBPGS3z42XjMBCS87rqjZ7+OzcdA j3mKdGHldn4Q36aMU6iDTwVYMr2qBi2xwuaXcVFXSSz24+9NvhQ6yzoPib9D5tEicY Lmk7iXNSFJ3SxJx9VzEArnVC2x/AVWXATywZbKqo= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Wed, 3 Mar 2021 01:53:48 +0200 Message-Id: <20210302235348.2878-1-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2] libcamera: pipeline: simple: Support camera sensors that contain an ISP X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Camera sensors can include an ISP. For instance, the AP1302 external ISP can be connected to up to two raw camera sensors, and the combination of the sensors and ISP is considered as a (smart) camera sensor from libcamera's point of view. The CameraSensor class has limited support for this already. Extend the simple pipeline handler to support such sensors, by using the media entity corresponding to the ISP instead of the raw camera sensor's entity. We don't need to handle the case where an entity in the SoC would expose the MEDIA_ENT_F_PROC_VIDEO_ISP function, as pipeline containing an ISP would have a dedicated pipeline handler. The implementation is limited as it won't support other multi-entity camera sensors (such as CCS). While this would be worth supporting, we don't have a test platform with a CCS-compatible sensor at this point, so let's not over-engineer the solution. Extending support to CCS (and possibly other sensor topologies) will likely involve helpers that can be used by other pipeline handlers (such as generic graph walk helpers for instance) and extensions to the CameraSensor class. Signed-off-by: Laurent Pinchart --- Changes since v1: - Fix link handling --- src/libcamera/pipeline/simple/simple.cpp | 88 ++++++++++++++++++++---- 1 file changed, 75 insertions(+), 13 deletions(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 36f55e8709f1..aa6155f0596f 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -245,6 +245,8 @@ private: PipelineHandler::cameraData(camera)); } + std::vector locateSensors(); + void bufferReady(FrameBuffer *buffer); void converterInputDone(FrameBuffer *buffer); void converterOutputDone(FrameBuffer *buffer); @@ -869,6 +871,78 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) * Match and Setup */ +std::vector SimplePipelineHandler::locateSensors() +{ + std::vector entities; + + /* + * Gather all the camera sensor entities based on the function they + * expose. + */ + for (MediaEntity *entity : media_->entities()) { + if (entity->function() == MEDIA_ENT_F_CAM_SENSOR) + entities.push_back(entity); + } + + if (entities.empty()) + return {}; + + /* + * Sensors can be made of multiple entities. For instance, a raw sensor + * can be connected to an ISP, and the combination of both should be + * treated as one sensor. To support this, as a crude heuristic, check + * the downstream entity from the camera sensor, and if it is an ISP, + * use it instead of the sensor. + */ + std::vector sensors; + + for (MediaEntity *entity : entities) { + /* + * Locate the downstream entity by following the first enabled + * link from a source pad. + */ + const MediaPad *pad = nullptr; + const MediaLink *link = nullptr; + + for (const MediaPad *p : entity->pads()) { + if (p->flags() & MEDIA_PAD_FL_SOURCE) { + pad = p; + break; + } + } + + if (!pad) + continue; + + for (const MediaLink *l : pad->links()) { + if (l->flags() & MEDIA_LNK_FL_ENABLED || + !(l->flags() & MEDIA_LNK_FL_IMMUTABLE)) { + link = l; + break; + } + } + + if (!link) + continue; + + MediaEntity *remote = link->sink()->entity(); + if (remote->function() == MEDIA_ENT_F_PROC_VIDEO_ISP) + sensors.push_back(remote); + else + sensors.push_back(entity); + } + + /* + * Remove duplicates, in case multiple sensors are connected to the + * same ISP. + */ + std::sort(sensors.begin(), sensors.end()); + auto last = std::unique(sensors.begin(), sensors.end()); + sensors.erase(last, sensors.end()); + + return sensors; +} + bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) { const SimplePipelineInfo *info = nullptr; @@ -892,19 +966,7 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) } /* Locate the sensors. */ - std::vector sensors; - - for (MediaEntity *entity : media_->entities()) { - switch (entity->function()) { - case MEDIA_ENT_F_CAM_SENSOR: - sensors.push_back(entity); - break; - - default: - break; - } - } - + std::vector sensors = locateSensors(); if (sensors.empty()) { LOG(SimplePipeline, Error) << "No sensor found"; return false;