Camera view is slow with QML but not with Qt widgets

Hello,
We have an issue with QML multimedia to display camera stream. The result is not smooth while with Qt widgets it works fine. Gstreamer pipeline

gst-launch-1.0 -v v4l2src ! "video/x-raw, width=640, height=480, framerate=(fraction)30/1, format=(string)UYVY" ! videoconvert ! videoscale ! video/x
-raw,width=480,height=800 ! kmssink driver-name=stm connector-id=32 force-modesetting=true

is smooth as well.

The hardware is a STM32MP157C SoC which provides a Vivante GCnano as GPU. Everything is compiled with a specific Yocto layer along with meta-qt5 for Qt 5.14.2. GSTreamer release is 1.16. The camera output is VGA in UYVY format and the screen resolution is 480x800. Everything was tested in Weston session or EGLFS without differences.

QML applications are started by default with this environment

QT_QPA_EGLFS_ALWAYS_SET_MODE=1 QT_QPA_EGLFS_INTEGRATION=eglfs_kms QT_QPA_PLATFORM=eglfs QT_QPA_EGLFS_KMS_ATOMIC=1

(because EGLFS is our reference).

We have this issue with official examples, declarative-camera (or qmlvideofx example) is displaying around 7-10 fps when camera widgets example is around 25-30 fps. We tried to analyze and to test some workarounds without understanding what is going on. And it does not seems normal to have good results in Qt widgets but not in QML.

We also try to play several video files (H264 in VGA or 720p) without issues with official examples for both methods: QML and Qt widgets. The problem is not related to video but camera stream only. And according to our monitoring with top and sysfs (for GPU), memory, GPU and CPU are not overused.

We tried to get more logs and according to renderer loop debug for QML example, a lot of frames are generated in 100-250 ms which seems to be the issue.

Firstly, we try in our application to use Camera -> VideoOutput items directly. Camera item is also configured with those instructions:

QCameraViewfinderSettings settings = camera->viewfinderSettings();
settings.setPixelFormat(QVideoFrame::Format_UYVY);
 settings.setResolution(640, 480);
 settings.setMaximumFrameRate(30);
 settings.setMinimumFrameRate(10);
 camera->setViewfinderSettings(settings);

Then using MediaPlayer instead, to be able to use custom pipeline feature:

source: "gst-pipeline: v4l2src ! video/x-raw,width=640,height=480 ! videoconvert ! qtvideosink"

but it does not work better.

Finally, we try to use qmlglsink in connection with GstGLVideoItem item. It works on the PC, but on the target it triggers a segmentation fault into gcoTEXTURE_GetMipMap function (related to OpenGL library provided by Vivante) according to gdb backtrace. We don't have access to the source code so it is difficult to identify the issue and how to fix it.

Setting the Qt::AA_ShareOpenGLContexts value before starting QML engine does not change the results.

On official examples we tried also several workaround:

QT_QUICK_NO_TEXTURE_VIDEOFRAMES environment variable set to 1. No changes for QML examples.
QT_GSTREAMER_USE_OPENGL_PLUGIN environment variable set to 1. Camera generates an error, it can not stream (probably due to a required conversion). And BTW we tried to play a video with qmlvideofx example with this environment variable value and the video is not smooth. Probably due to uploads/downloads from/to GPU. So it is probably not relevant to continue in this direction.

We tried to change QSG_RENDER_LOOP environment variable, by default it seems threaded is used, but other values don't improve the situation.

Playing with QT_GSTREAMER_WINDOW_VIDEOSINK or QT_GSTREAMER_WIDGET_VIDEOSINK seemed useless in our case also.

So, the hardware allow us to do what we want according to Qt widgets and Gstreamer results. It seems to be a bug to have this big difference between QML and Qt widgets for this kind of tasks. And we don't see how to fix it in QtMultimedia code in fact.

Do you have any suggestions?