[libcamera-devel,v2,03/14] libcamera: pipeline: Move IPA from pipeline to camera data

1890 diff mbox series

The IPA acts on a camera and not on a pipeline which can expose more
then one camera. Move the IPA reference to the CameraData and move the
loading of an IPA from the specific pipeline handler implementation to
base PipelineHandler.

It's still possible to expose a camera without an IPA but if an IPA is
request the camera is not valid and will not be registered in the system
if a suiting IPA module can't be found.

Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
---
 src/libcamera/include/pipeline_handler.h | 12 +++++--
 src/libcamera/pipeline/vimc.cpp          |  8 +----
 src/libcamera/pipeline_handler.cpp       | 40 +++++++++++++++++++++++-
 3 files changed, 50 insertions(+), 10 deletions(-)

Hi Niklas,

Thank you for the patch.

On Fri, Aug 30, 2019 at 01:26:42AM +0200, Niklas Söderlund wrote:
> The IPA acts on a camera and not on a pipeline which can expose more
> then one camera. Move the IPA reference to the CameraData and move the
> loading of an IPA from the specific pipeline handler implementation to
> base PipelineHandler.

This means that two cameras sharing part of the pipeline will have one
IPA interface instance each, sharing resources. The pipeline handler
should implement mutual exclusion, but I wonder how this will work on
the IPA side, if IPAs will need to be aware of each other.

> It's still possible to expose a camera without an IPA but if an IPA is
> request the camera is not valid and will not be registered in the system
> if a suiting IPA module can't be found.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund@ragnatech.se>
> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> ---
>  src/libcamera/include/pipeline_handler.h | 12 +++++--
>  src/libcamera/pipeline/vimc.cpp          |  8 +----
>  src/libcamera/pipeline_handler.cpp       | 40 +++++++++++++++++++++++-
>  3 files changed, 50 insertions(+), 10 deletions(-)
> 
> diff --git a/src/libcamera/include/pipeline_handler.h b/src/libcamera/include/pipeline_handler.h
> index 1fdef9cea40f1f0a..ffc7adb802215313 100644
> --- a/src/libcamera/include/pipeline_handler.h
> +++ b/src/libcamera/include/pipeline_handler.h
> @@ -15,6 +15,7 @@
>  #include <vector>
>  
>  #include <libcamera/controls.h>
> +#include <libcamera/ipa/ipa_interface.h>
>  #include <libcamera/stream.h>
>  
>  namespace libcamera {
> @@ -33,8 +34,11 @@ class Request;
>  class CameraData
>  {
>  public:
> -	explicit CameraData(PipelineHandler *pipe)
> -		: pipe_(pipe)
> +	explicit CameraData(PipelineHandler *pipe,
> +			    uint32_t minIPAVersion = 0,
> +			    uint32_t maxIPAVersion = 0)

I'd rather add an explicit loadIPA() (or similar) method to this class
instead of adding parameters to the constructor, to allow the caller to
handle errors more explicitly. You won't have to store the min and max
version in that case.

If you want to make this happen more magically, storing the min/max
version inside the PipelineHandler class would allow the versions to be
passed to the REGISTER_PIPELINE_HANDLER() macro, which could also be an
option, but I think I'd prefer an explicit call.

> +		: pipe_(pipe), ipa_(nullptr), minIPAVersion_(minIPAVersion),
> +		  maxIPAVersion_(maxIPAVersion)
>  	{
>  	}
>  	virtual ~CameraData() {}
> @@ -44,6 +48,10 @@ public:
>  	std::list<Request *> queuedRequests_;
>  	ControlInfoMap controlInfo_;
>  
> +	std::unique_ptr<IPAInterface> ipa_;
> +	const uint32_t minIPAVersion_;
> +	const uint32_t maxIPAVersion_;
> +
>  private:
>  	CameraData(const CameraData &) = delete;
>  	CameraData &operator=(const CameraData &) = delete;
> diff --git a/src/libcamera/pipeline/vimc.cpp b/src/libcamera/pipeline/vimc.cpp
> index 86aefc08a9563d44..be6507cd4bc0d1b9 100644
> --- a/src/libcamera/pipeline/vimc.cpp
> +++ b/src/libcamera/pipeline/vimc.cpp
> @@ -38,7 +38,7 @@ class VimcCameraData : public CameraData
>  {
>  public:
>  	VimcCameraData(PipelineHandler *pipe)
> -		: CameraData(pipe), sensor_(nullptr), debayer_(nullptr),
> +		: CameraData(pipe, 1, 1), sensor_(nullptr), debayer_(nullptr),
>  		  scaler_(nullptr), video_(nullptr), raw_(nullptr)
>  	{
>  	}
> @@ -100,8 +100,6 @@ private:
>  		return static_cast<VimcCameraData *>(
>  			PipelineHandler::cameraData(camera));
>  	}
> -
> -	std::unique_ptr<IPAInterface> ipa_;
>  };
>  
>  VimcCameraConfiguration::VimcCameraConfiguration()
> @@ -361,10 +359,6 @@ bool PipelineHandlerVimc::match(DeviceEnumerator *enumerator)
>  	if (!media)
>  		return false;
>  
> -	ipa_ = IPAManager::instance()->createIPA(this, 1, 1);
> -	if (ipa_ == nullptr)
> -		LOG(VIMC, Warning) << "no matching IPA found";
> -
>  	std::unique_ptr<VimcCameraData> data = utils::make_unique<VimcCameraData>(this);
>  
>  	/* Locate and open the capture video node. */
> diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp
> index 3e54aa23d92b9a36..558b4b254d111e31 100644
> --- a/src/libcamera/pipeline_handler.cpp
> +++ b/src/libcamera/pipeline_handler.cpp
> @@ -12,6 +12,7 @@
>  #include <libcamera/camera_manager.h>
>  
>  #include "device_enumerator.h"
> +#include "ipa_manager.h"
>  #include "log.h"
>  #include "media_device.h"
>  #include "utils.h"
> @@ -49,13 +50,20 @@ LOG_DEFINE_CATEGORY(Pipeline)
>   */
>  
>  /**
> - * \fn CameraData::CameraData(PipelineHandler *pipe)
> + * \fn CameraData::CameraData(PipelineHandler *pipe, uint32_t minIPAVersion,
> + * uint32_t maxIPAVersion)
>   * \brief Construct a CameraData instance for the given pipeline handler
>   * \param[in] pipe The pipeline handler
> + * \param[in] minIPAVersion Minimum acceptable version of IPA module
> + * \param[in] maxIPAVersion Maximum acceptable version of IPA module
>   *
>   * The reference to the pipeline handler is stored internally, the caller shall
>   * guarantee that the pointer remains valid as long as the CameraData instance
>   * exists.
> + *
> + * The IPA maximum and minimum version numbers are used to match with an IPA
> + * interface that would be compatible with the Camera. If no IPA interface
> + * is needed for the camera both parameters should be set to 0.
>   */
>  
>  /**
> @@ -96,6 +104,24 @@ LOG_DEFINE_CATEGORY(Pipeline)
>   * creating the camera, and shall not be modified afterwards.
>   */
>  
> +/**
> + * \var CameraData::ipa_
> + * \brief The IPA module used by the camera
> + *
> + * Reference to the Image Processing Algorithms (IPA) operating on the camera's
> + * stream(s). If no IPAs are in operation this should be set to nullptr.
> + */
> +
> +/**
> + * \var CameraData::minIPAVersion_
> + * \brief Minimum acceptable version of IPA module
> + */
> +
> +/**
> + * \var CameraData::maxIPAVersion_
> + * \brief Maximum acceptable version of IPA module
> + */
> +
>  /**
>   * \class PipelineHandler
>   * \brief Create and manage cameras based on a set of media devices
> @@ -425,6 +451,18 @@ void PipelineHandler::registerCamera(std::shared_ptr<Camera> camera,
>  				     std::unique_ptr<CameraData> data)
>  {
>  	data->camera_ = camera.get();
> +
> +	if (data->minIPAVersion_ || data->maxIPAVersion_) {
> +		data->ipa_ = IPAManager::instance()->createIPA(this,

I think we'll have to remove the IPAManager singleton, for the same
reason the CameraManager singleton was removed, but we can handle this
later.

> +							       data->minIPAVersion_,
> +							       data->maxIPAVersion_);
> +		if (!data->ipa_) {
> +			LOG(Pipeline, Warning) << "Skipping " << camera->name()
> +					       << " no IPA found";
> +			return;
> +		}
> +	}
> +
>  	cameraData_[camera.get()] = std::move(data);
>  	cameras_.push_back(camera);
>  	manager_->addCamera(std::move(camera));