| Message ID | 20190320163055.22056-16-jacopo@jmondi.org |
|---|---|
| State | Superseded |
| Headers | show |
| Series |
|
| Related | show |
Hi Jacopo, Thank you for the patch. On Wed, Mar 20, 2019 at 05:30:39PM +0100, Jacopo Mondi wrote: > As the lenghty comment reports, link enable/disable is not trivial, as > links in one ImgU instance interfere with capture operations in the > other one. As I struggle to find where to disable links selectively, > as of now reset the media graph and enable the required links at > streamConfiguration time. Do we need this additional complexity ? Given that we currently have to link all but the params video nodes, couldn't you just disable the params link and enable all the other ones at match time, without touching links when configuring the streams ? > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> > --- > src/libcamera/pipeline/ipu3/ipu3.cpp | 110 +++++++++++++++++++++++++++ > 1 file changed, 110 insertions(+) > > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp > index ed2409907cf0..43fa0e4a7f9d 100644 > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp > @@ -52,6 +52,10 @@ public: > } > > void init(MediaDevice *media, unsigned int index); > + int linkSetup(const std::string &source, unsigned int sourcePad, > + const std::string &sink, unsigned int sinkPad, > + bool enable); > + int enableLinks(bool enable); > > unsigned int index_; > std::string imguName_; > @@ -264,6 +268,14 @@ int PipelineHandlerIPU3::configureStreams(Camera *camera, > return -EINVAL; > } > > + /* > + * \todo: Enable links selectively based on the requested streams. > + * As of now, enable all links unconditionally. > + */ > + ret = data->imgu->enableLinks(true); > + if (ret) > + return ret; > + > /* > * Pass the requested output image size to the sensor and get back the > * adjusted one to be propagated to the CIO2 device and to the ImgU > @@ -589,6 +601,30 @@ bool PipelineHandlerIPU3::match(DeviceEnumerator *enumerator) > return false; > } > > + /* > + * FIXME: enabled links in one ImgU instance interfere with capture > + * operations on the other one. This can be easily triggered by > + * capturing from one camera and then trying to capture from the other > + * one right after, without disabling media links in the media graph > + * first. > + * > + * The tricky part here is where to disable links on the ImgU instance > + * which is currently not in use: > + * 1) Link enable/disable cannot be done at start/stop time as video > + * devices needs to be linked first before format can be configured on > + * them. > + * 2) As link enable has to be done at the least in configureStreams, > + * before configuring formats, the only place where to disable links > + * would be 'stop()', but the Camera class state machine allows > + * start()<->stop() sequences without any streamConfiguration() in > + * between. > + * > + * As of now, disable all links in the media graph at 'match()' time, > + * to allow testing different cameras in different test applications > + * runs. A test application that would use two distinct cameras without > + * going through a library teardown->match() sequence would fail > + * at the moment. > + */ > if (imguMediaDev_->disableLinks()) > goto error_close_mdev; > > @@ -625,6 +661,80 @@ void ImgUDevice::init(MediaDevice *mediaDevice, unsigned int index) > mediaDevice_ = mediaDevice; > } > > +/** > + * \brief Enable or disable a single link on the ImgU instance > + * > + * This method assumes the media device associated with the ImgU instance > + * is open. > + * > + * \return 0 on success or a negative error code otherwise > + */ > +int ImgUDevice::linkSetup(const std::string &source, unsigned int sourcePad, > + const std::string &sink, unsigned int sinkPad, > + bool enable) > +{ > + MediaLink *link = mediaDevice_->link(source, sourcePad, sink, sinkPad); > + if (!link) { > + LOG(IPU3, Error) > + << "Failed to get link: '" << source << "':" > + << sourcePad << " -> '" << sink << "':" << sinkPad; > + return -ENODEV; > + } > + > + return link->setEnabled(enable); > +} > + > +/** > + * \brief Enable or disable all media links in the ImgU instance to prepare > + * for capture operations > + * > + * \todo This method will probably be removed or changed once links will be > + * enabled or disabled selectively. > + * > + * \return 0 on success or a negative error code otherwise > + */ > +int ImgUDevice::enableLinks(bool enable) > +{ > + std::string inputName = imguName_ + " input"; > + std::string outputName = imguName_ + " output"; > + std::string viewfinderName = imguName_ + " viewfinder"; > + std::string statName = imguName_ + " 3a stat"; > + int ret; > + > + /* \todo Establish rules to handle media devices open/close. */ > + ret = mediaDevice_->open(); > + if (ret) > + return ret; > + > + ret = linkSetup(inputName, 0, imguName_, PAD_INPUT, enable); > + if (ret) { > + mediaDevice_->close(); > + return ret; > + } > + > + ret = linkSetup(imguName_, PAD_OUTPUT, outputName, 0, enable); > + if (ret) { > + mediaDevice_->close(); > + return ret; > + } > + > + ret = linkSetup(imguName_, PAD_VF, viewfinderName, 0, enable); > + if (ret) { > + mediaDevice_->close(); > + return ret; > + } > + > + ret = linkSetup(imguName_, PAD_STAT, statName, 0, enable); > + if (ret) { > + mediaDevice_->close(); > + return ret; > + } > + > + mediaDevice_->close(); > + > + return 0; > +} > + > int PipelineHandlerIPU3::mediaBusToCIO2Format(unsigned int code) > { > switch (code) {
Hi Laurent, On Sat, Mar 23, 2019 at 03:09:34PM +0200, Laurent Pinchart wrote: > Hi Jacopo, > > Thank you for the patch. > > On Wed, Mar 20, 2019 at 05:30:39PM +0100, Jacopo Mondi wrote: > > As the lenghty comment reports, link enable/disable is not trivial, as > > links in one ImgU instance interfere with capture operations in the > > other one. As I struggle to find where to disable links selectively, > > as of now reset the media graph and enable the required links at > > streamConfiguration time. > > Do we need this additional complexity ? Given that we currently have to > link all but the params video nodes, couldn't you just disable the > params link and enable all the other ones at match time, without > touching links when configuring the streams ? Enable links on both the ImgU instances you mean? As I tried to explain in the comment, this is not really possible, because if you enable links on the ImgU instance, the system expects frames to be queued to its input, otherwise the processing on the the other instance, which is in use, stall. This can be easily triggered by enabling links on imgu1 in the ipu3-process.sh script, which uses imgu0. Thanks j > > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> > > --- > > src/libcamera/pipeline/ipu3/ipu3.cpp | 110 +++++++++++++++++++++++++++ > > 1 file changed, 110 insertions(+) > > > > diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp > > index ed2409907cf0..43fa0e4a7f9d 100644 > > --- a/src/libcamera/pipeline/ipu3/ipu3.cpp > > +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp > > @@ -52,6 +52,10 @@ public: > > } > > > > void init(MediaDevice *media, unsigned int index); > > + int linkSetup(const std::string &source, unsigned int sourcePad, > > + const std::string &sink, unsigned int sinkPad, > > + bool enable); > > + int enableLinks(bool enable); > > > > unsigned int index_; > > std::string imguName_; > > @@ -264,6 +268,14 @@ int PipelineHandlerIPU3::configureStreams(Camera *camera, > > return -EINVAL; > > } > > > > + /* > > + * \todo: Enable links selectively based on the requested streams. > > + * As of now, enable all links unconditionally. > > + */ > > + ret = data->imgu->enableLinks(true); > > + if (ret) > > + return ret; > > + > > /* > > * Pass the requested output image size to the sensor and get back the > > * adjusted one to be propagated to the CIO2 device and to the ImgU > > @@ -589,6 +601,30 @@ bool PipelineHandlerIPU3::match(DeviceEnumerator *enumerator) > > return false; > > } > > > > + /* > > + * FIXME: enabled links in one ImgU instance interfere with capture > > + * operations on the other one. This can be easily triggered by > > + * capturing from one camera and then trying to capture from the other > > + * one right after, without disabling media links in the media graph > > + * first. > > + * > > + * The tricky part here is where to disable links on the ImgU instance > > + * which is currently not in use: > > + * 1) Link enable/disable cannot be done at start/stop time as video > > + * devices needs to be linked first before format can be configured on > > + * them. > > + * 2) As link enable has to be done at the least in configureStreams, > > + * before configuring formats, the only place where to disable links > > + * would be 'stop()', but the Camera class state machine allows > > + * start()<->stop() sequences without any streamConfiguration() in > > + * between. > > + * > > + * As of now, disable all links in the media graph at 'match()' time, > > + * to allow testing different cameras in different test applications > > + * runs. A test application that would use two distinct cameras without > > + * going through a library teardown->match() sequence would fail > > + * at the moment. > > + */ > > if (imguMediaDev_->disableLinks()) > > goto error_close_mdev; > > > > @@ -625,6 +661,80 @@ void ImgUDevice::init(MediaDevice *mediaDevice, unsigned int index) > > mediaDevice_ = mediaDevice; > > } > > > > +/** > > + * \brief Enable or disable a single link on the ImgU instance > > + * > > + * This method assumes the media device associated with the ImgU instance > > + * is open. > > + * > > + * \return 0 on success or a negative error code otherwise > > + */ > > +int ImgUDevice::linkSetup(const std::string &source, unsigned int sourcePad, > > + const std::string &sink, unsigned int sinkPad, > > + bool enable) > > +{ > > + MediaLink *link = mediaDevice_->link(source, sourcePad, sink, sinkPad); > > + if (!link) { > > + LOG(IPU3, Error) > > + << "Failed to get link: '" << source << "':" > > + << sourcePad << " -> '" << sink << "':" << sinkPad; > > + return -ENODEV; > > + } > > + > > + return link->setEnabled(enable); > > +} > > + > > +/** > > + * \brief Enable or disable all media links in the ImgU instance to prepare > > + * for capture operations > > + * > > + * \todo This method will probably be removed or changed once links will be > > + * enabled or disabled selectively. > > + * > > + * \return 0 on success or a negative error code otherwise > > + */ > > +int ImgUDevice::enableLinks(bool enable) > > +{ > > + std::string inputName = imguName_ + " input"; > > + std::string outputName = imguName_ + " output"; > > + std::string viewfinderName = imguName_ + " viewfinder"; > > + std::string statName = imguName_ + " 3a stat"; > > + int ret; > > + > > + /* \todo Establish rules to handle media devices open/close. */ > > + ret = mediaDevice_->open(); > > + if (ret) > > + return ret; > > + > > + ret = linkSetup(inputName, 0, imguName_, PAD_INPUT, enable); > > + if (ret) { > > + mediaDevice_->close(); > > + return ret; > > + } > > + > > + ret = linkSetup(imguName_, PAD_OUTPUT, outputName, 0, enable); > > + if (ret) { > > + mediaDevice_->close(); > > + return ret; > > + } > > + > > + ret = linkSetup(imguName_, PAD_VF, viewfinderName, 0, enable); > > + if (ret) { > > + mediaDevice_->close(); > > + return ret; > > + } > > + > > + ret = linkSetup(imguName_, PAD_STAT, statName, 0, enable); > > + if (ret) { > > + mediaDevice_->close(); > > + return ret; > > + } > > + > > + mediaDevice_->close(); > > + > > + return 0; > > +} > > + > > int PipelineHandlerIPU3::mediaBusToCIO2Format(unsigned int code) > > { > > switch (code) { > > -- > Regards, > > Laurent Pinchart
diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index ed2409907cf0..43fa0e4a7f9d 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -52,6 +52,10 @@ public: } void init(MediaDevice *media, unsigned int index); + int linkSetup(const std::string &source, unsigned int sourcePad, + const std::string &sink, unsigned int sinkPad, + bool enable); + int enableLinks(bool enable); unsigned int index_; std::string imguName_; @@ -264,6 +268,14 @@ int PipelineHandlerIPU3::configureStreams(Camera *camera, return -EINVAL; } + /* + * \todo: Enable links selectively based on the requested streams. + * As of now, enable all links unconditionally. + */ + ret = data->imgu->enableLinks(true); + if (ret) + return ret; + /* * Pass the requested output image size to the sensor and get back the * adjusted one to be propagated to the CIO2 device and to the ImgU @@ -589,6 +601,30 @@ bool PipelineHandlerIPU3::match(DeviceEnumerator *enumerator) return false; } + /* + * FIXME: enabled links in one ImgU instance interfere with capture + * operations on the other one. This can be easily triggered by + * capturing from one camera and then trying to capture from the other + * one right after, without disabling media links in the media graph + * first. + * + * The tricky part here is where to disable links on the ImgU instance + * which is currently not in use: + * 1) Link enable/disable cannot be done at start/stop time as video + * devices needs to be linked first before format can be configured on + * them. + * 2) As link enable has to be done at the least in configureStreams, + * before configuring formats, the only place where to disable links + * would be 'stop()', but the Camera class state machine allows + * start()<->stop() sequences without any streamConfiguration() in + * between. + * + * As of now, disable all links in the media graph at 'match()' time, + * to allow testing different cameras in different test applications + * runs. A test application that would use two distinct cameras without + * going through a library teardown->match() sequence would fail + * at the moment. + */ if (imguMediaDev_->disableLinks()) goto error_close_mdev; @@ -625,6 +661,80 @@ void ImgUDevice::init(MediaDevice *mediaDevice, unsigned int index) mediaDevice_ = mediaDevice; } +/** + * \brief Enable or disable a single link on the ImgU instance + * + * This method assumes the media device associated with the ImgU instance + * is open. + * + * \return 0 on success or a negative error code otherwise + */ +int ImgUDevice::linkSetup(const std::string &source, unsigned int sourcePad, + const std::string &sink, unsigned int sinkPad, + bool enable) +{ + MediaLink *link = mediaDevice_->link(source, sourcePad, sink, sinkPad); + if (!link) { + LOG(IPU3, Error) + << "Failed to get link: '" << source << "':" + << sourcePad << " -> '" << sink << "':" << sinkPad; + return -ENODEV; + } + + return link->setEnabled(enable); +} + +/** + * \brief Enable or disable all media links in the ImgU instance to prepare + * for capture operations + * + * \todo This method will probably be removed or changed once links will be + * enabled or disabled selectively. + * + * \return 0 on success or a negative error code otherwise + */ +int ImgUDevice::enableLinks(bool enable) +{ + std::string inputName = imguName_ + " input"; + std::string outputName = imguName_ + " output"; + std::string viewfinderName = imguName_ + " viewfinder"; + std::string statName = imguName_ + " 3a stat"; + int ret; + + /* \todo Establish rules to handle media devices open/close. */ + ret = mediaDevice_->open(); + if (ret) + return ret; + + ret = linkSetup(inputName, 0, imguName_, PAD_INPUT, enable); + if (ret) { + mediaDevice_->close(); + return ret; + } + + ret = linkSetup(imguName_, PAD_OUTPUT, outputName, 0, enable); + if (ret) { + mediaDevice_->close(); + return ret; + } + + ret = linkSetup(imguName_, PAD_VF, viewfinderName, 0, enable); + if (ret) { + mediaDevice_->close(); + return ret; + } + + ret = linkSetup(imguName_, PAD_STAT, statName, 0, enable); + if (ret) { + mediaDevice_->close(); + return ret; + } + + mediaDevice_->close(); + + return 0; +} + int PipelineHandlerIPU3::mediaBusToCIO2Format(unsigned int code) { switch (code) {
As the lenghty comment reports, link enable/disable is not trivial, as links in one ImgU instance interfere with capture operations in the other one. As I struggle to find where to disable links selectively, as of now reset the media graph and enable the required links at streamConfiguration time. Signed-off-by: Jacopo Mondi <jacopo@jmondi.org> --- src/libcamera/pipeline/ipu3/ipu3.cpp | 110 +++++++++++++++++++++++++++ 1 file changed, 110 insertions(+)