The attached test-case sets up a QVideoWidget and a QMediaPlayer. It calls setMedia() and calls setVideoOutput() and play() afterwards. When closing the window the size hint is queried, and the return value is invalid.

When calling setVideoOutput() after setMedia() and before returning to the event-loop (i.e. disabling the timer), then the sizehint is returned correctly.

As far as I can see it is safe to query gstreamer for the native size from within void QGstreamerPlayerSession::busMessage(const QGstreamerMessage &message)
when rendering the GST_STATE_PAUSED state. That's where the code also calls m_renderer->precessNewStream();

In the test-case however there is no renderer at this point, and therefore the native size is never queried and therefore never returned.

I've tried calling updateNativeSize() later on in the code when initializing the overlay, but that doesn't seem to work.

If you do not intend to support calling setVideoOutput() after setMedia(), then I suggest that this is documented prominently in the documentation of the setVideoOutput() overloads.