logo

Live Production Software Forums


Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Options
Go to last post Go to first unread
bgadient  
#1 Posted : Thursday, January 13, 2022 4:53:19 AM(UTC)
bgadient

Rank: Newbie

Groups: Registered
Joined: 1/13/2022(UTC)
Posts: 1
United States
Location: Minnesota

For quite some time we have had issues with Vmix where about once a stream, the internet upload would be cut to a speed that would either cripple the stream or would stop the stream. It would only last for about a minute and then go back to normal. We've had no issues after our IT guy updated things. (see below) Anyone else have the same issue with the outdated FFMPEG and Youtube?

We have a 1gig upload and a high end pc. Here are the main specs of the pc...

Precision 5820 Tower
Precision 5820 Tower XCTO Base
Processor
Intel Core i9-10900X (3.7GHz, 4.7GHz Turbo, 10C, 19.25MB Cache, HT, (165W)), DDR4-2933 Non-ECC
Operating System
Windows 11 Pro, English, French, Spanish
Chassis Options
Precision 5820 Tower Core X 950W PCIe FlexBay Chassis CL FMX
Video Card
Nvidia Quadro RTX4000, 8GB, 3DP, VirtualLink (XX20T)
Memory
32GB, DDR4 UDIMM non-ECC memory

"I think I have a solution for your issue. I have replaced the FFMPEG2 binary in vMix with one I built which so far has resolved the issue. To use it select FFMPEG2 when setting up streaming, the FFMPEG option will use the default version included with vMix. The FFMPEG I've included a more complete trip down the rabbit hole below if you are interested below.

image.png (107kb) downloaded 0 time(s).

Here is an overview of the steps I took to get to this point:
1. I set up vMix to stream to YouTube's primary and secondary servers at the same time. This is mentioned as a "fix" on several of the vMix forums along with using the Flash Media Encoder from Adobe. During a ~18 hour run both the primary and secondary streams restarted due to buffer errors around 8 times but neither stream restarted within an hour or two of the other stream. Any network congestion or packet loss issues between the computer and the point in Google's network where their paths split should show up on both streams at the same time. This pointed to either the issue being on the computer or a remote possibility of something going on randomly in Google's network.

2. I completed a full network packet capture on the computer during a series of 4-5 test streams and observed traffic up until the point of failure. An analysis of the TCP traffic flows didn't show any packet loss or loss of ACKs during the time leading up to the failure. In all of the cases Google's server sent an ACK and then there was ~30s of silence from the vMix computer. When the connection timed out Google's server would send a FIN packet to close the connection after which the vMix computer would try to send a bunch of traffic before stopping. This pointed to some sort of race condition where FFMPEG on the vMIx computer and YouTube's server are both waiting on eachother for data.

3. I then examined FFMPEG on the vMix computer. vMix has ffmpeg.exe and ffmpeg2.exe which are stored in the streaming directory of the vMix application in Program File (x86). ffmpeg.exe appears to have been built from the FFMPEG 2.8 master branch in 2014. ffmpeg2.exe appears to have been built from the FFMPEG 3.3 master branch in 2017. I performed a stream with FFMPEG2 but it exhibited the same issue as FFMPEG1. I downloaded several past versions of vMix hoping to see a rebuild in recent history that is causing this error but file checksums are the same for vMix versions going back to 2015/2017 indicating that nothing has changed. At this point I suspected the issue could be something in the local network stack or a bug in the old version of FFMPEG. I then used OBS on the vMix computer, which uses modern FFMPEG libraries for streaming, to complete a 16 hour stream to YouTube without any dropped frames or quality issues which seemed to confirm that even if there is some issue in the network stack that the modern FFMPEG libraries are able to handle it gracefully.

4. I looked online for more modern FFMPEG binaries that incorporated both the NVIDIA decoding plugin and Windows specific DirectShow support but due to proprietary licensing issues they are non-existent from reputable sources. Given that, I downloaded the latest NVIDIA SDK, MinGW-W64 build environment, and the latest stable FFMPEG version and produced a modern 64-bit FFMPEG 4.4 binary to be a drop in replacement for the FFMPEG2 binary. The original binary was built using the MinGW-W64 gcc compilers so I opted to do the same rather than try using Microsoft's Visual C compilers which they do for the GUI components of vMix but not the streaming command line version. I did a couple of short tests with the new binary before moving on to longer tests. So far I have completed a 24.5 hour stream over the weekend and a 15 hour stream from late last night up until the elementary concert started without a crash or the network hang issue occurring. I can't declare victory yet, but so far the results are promising. If I were working for vMix I'd work further to determine the exact issue so that a patch could be backported, but for now if it works or masks the issue so I'll leave it at that. My binary is 32 vs 64 bit, uses a much newer NVIDIA SDK, and has different GCC compilers and libraries so the root cause could be in a whole bunch of different places.

It would be nice if Vmix would keep their FFMPEG versions up to date with the latest maintenance branch. Version 2.8 and 3.3 are both still getting critical fixes and patches but it doesn't appear like they've ever really updated things. I didn't try building the latest 2.8 or 3.3 versions to see if a backported patch is already in those. If they'd add FFMPEG 4 they would have a version with HLS support which Google has added to YouTube to increase reliability and add HDR support. "
Users browsing this topic
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.