logo

Live Production Software Forums


Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Options
Go to last post Go to first unread
Ittaidv  
#1 Posted : Saturday, October 1, 2016 9:10:19 PM(UTC)
Ittaidv

Rank: Advanced Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 600
Man
Belgium
Location: Belgium

Thanks: 75 times
Was thanked: 91 time(s) in 75 post(s)
Lately we had some huge troubles with our newest pc's. We dropped audio samples, our focusrite 18i20 usb soundcard went nuts, our decklink quad2 lost frames, and vmix had serious stability problems which resulted in latency spikes. I think this is normal, since we bought them in a rush, to do some jobs, but never really had time to configure them properly.

The most noticable were the instability of the soundcard: after a few hours it would start dropping samples with thousands per minute sometimes. This really made our job almost impossible.
Also the blackmagic capture cards could drop lots of frames from time to time, making our video look choppy.

The hardware we use is:

Motherboard: Asus x99 e-ws
RAM: 32gb ddr4
GPU: Nvidia 1080
CPU: E5 2630 V4
C drive: Samsung evo 850 250gb
Recording drive: Samsung evo 1TB
Multicorder/video playback drives: 2 x 500 gb kingston
Soundcard: Focusrite 18i20 V1
Capture cards: Quad2, dekclink sdi, Decklink intensity 4k, Intensity pro
OS: windows 10 x64

As you can see it's some beefy hardware, but still, that was not enough to run a stable vmix. With latencymon, i could see the latency of some processes was super high. Wdf01000.sys and dxgkrnl.sys both introduced unacceptable latency. Let's say we bought the best of the best, but the system wasn't optimised at all.. Dropping frames and samples with such a system shouldn't happen, that's why I locked myself up for a week and tried to learn as much about my hardware as possible.

So I went on a hunt, and googled the hell out of every component. To find out every piece of hardware in the pc seemed to have troubles reported on fora. Gamers were complaining about the pascal GPU's latency, intel NIC appeared to be super unstable, x99 appeared to be the worst chipset, .. If everything that is on the internet somewhere would be true and apply to our setup, it looked like we made the worst hardware choices ever...

Still I continued reading a lot, and found many usefull guides. Most of them are for audio editing, but they seemed usefull for vmix systems as well. After gathering as many tips as possible, i made a list, and followed it. After every tweak i ran vmix with latencymon, and checked if there were noticable differences. Also the statistics window as open all the time. The aim was to lose 0 frames or samples.

Not all tweaks delivered noticable results in vmix or in latencymon. Still I applied some, thinking that as long as it doesn't influence performance in a negative way, it might be better for long term stability. Especially the Hpet timer, the pcie cards and the disabling of windows services seemed to have 0 effect. I do think though, they might prevent some problems in the future. On lower end hardware I think the results with this could be more drastic.

These are the changes and tweaks I made:

Windows
- I uninstalled the anniversary update and went back to 1511. I remained on windows 10, since all my hardware has good stable driver releases for the OS.

Hardware
- Installed a pcie usb 3.0 card: used to connect the focusrite, it makes sure the card is always on a usb roothub, and doesn't share any recourses with mouse, keyboard or other usb hardware (used the pcie usb port only for the audio interface).
- Installed a seperate network card in a pcie slot and disabled onboard network hardware in the bios: intel has some latency issues on some systems hich have not been resolved according to their forum.
- Bought a higher quality usb cable with 2 ferrite cores to replace the one which was supplied with the soundcard.

Bios
- Disabled all power management features : Cstates, turbo boost, speedstep: all off
- Fastboot also off

Clocking:
-Overclocked the cpu slightly to 2400 mhz, since the cpu is watercooled, it doesn't affect temperatures that much, and higher clockspeeds make sure all I/O is processed slightly faster.

Drivers
- Downloaded all the latest drivers from the ASUS website, then went to manifacturer websites of every piece of hardware and updated to the latest
- Not all motherboard drivers on the ASUS site appeared to be the very latest, so I got some from the Intel website
- downloaded the latest nvidia drivers, and only installed the graphics drivers, left 3d vision and audio drivers out of the installation.
- There appeared to be a driver for the focusrite on their normal website which was not the latest, it is on a seperate page with beta drivers and includes lots of fixes. There's also a firmware update linked to these new beta drivers. After these beta drivers, focusrite never released a stable alpha driver, shame...

Tweaks
- I installed a program called fidelizer, which prioritises audio processes. It costs a few dollars, but in all modes, except purist (which slows down non audio processes), it appears to really enhance the stability of the audiodrivers on windows. After applying this 'secret tweak sauce' i never had a single dropped sample in vmix anymore.
- I used a tool called harmonic, to enable hpet (system clock) on windows 10. Not sure if it changed a lot, but it didn't hurt neither, and the tick drift of the system clock looked a bit more stable (might be my imagination as well)
- I used proces lasso to prioritise the vmix and ffmpeg proces in windows to realtime (not sure neither if this changed something, but didn't hurt neither). I also used it to make a dirty core: with 20 virtual cores it didn't seem like a bad idea to make vmix, ffmpeg, all audio related processes (svchost) and decklink and gpu related processes run on 19 cores only. The 20th core i used for windows background processes which are less important, but could potentially cause spikes when they are activated, which now can't influence the important ones anymore.
- Used a registry tweak to set all network adapters (including the ethernet) to metered, so no more updates can be installed
-Forced resolutions with EDID overrides which are fixed in the computer, so no accidental changes can be made with nvidia driver updates or by using a different screen, using custom Resolution Utility (CRU). This to make sure all screens attached are always 50hz (synced with 50i inputs).

Settings:
- Set the 18i20 soundcard to it's highest latency settings (mixing). This made the Wdf01000.sys process cause higher dpc latency, but according to Martin in another thread the soundcard should be set to it's maximum latency, so think this is the best choice.
- Used samsung Magician to set the recording drive to rapid mode, and enabled the performance mode from there (rapid mode really made the drive faster, performance mode is just some tweaks which didn't change that much)
- set windows to high performance (cpu 100% at all times) and disabled all power saving features (usb, sleep mode,..)
- went into the nvidia control panel and set the 1080 to maximum performance (highest clockspeed at all times)
-Disabled system sounds
-Set processor scheduling to "Background services"
-Disabled Windows Firewall, anti-virus and spyware software

Windows services:
-disabled all windows services such as phone, fax, printer, smartcard,... services from the system, don't need them anyway.

Conclusion:

I'm not sure if all tweaks and changes were equally effective, but now vmix runs stable as never before. No more lost frames or samples at all! It took me a lot of hours, but I'm sure my system runs a lot better after tweaking it. Perhaps some tweaks could be slightly counterproductive, so Off course all opinions are welcome, my aim is to learn more about it and share the info so all vmix pc's can run better :)

I found a lot of info to tweak windows for audio, and surprisingly these tweaks also made the stability of the capture cards better. Still it would be cool if there was any info out there for decklink cards specifically.

A sidenote: in my opinion there should be some kind of general optimisation thread on vmix forums, which should be stickied. Right now there is some info, but it's spread all over the forums. It would be cool if all members could contribute to a guide which helps troubleshooting slow, unresponsive or unstable systems.

Some good guides and information you can find via the following links:

https://support.focusrit...-for-Audio-on-Windows-10
http://www.sweetwater.co...ving-dpc-latency-issues/
https://support.native-i...ips-for-Audio-Processing
http://www.blackviper.com/







thanks 6 users thanked Ittaidv for this useful post.
stigaard on 10/2/2016(UTC), sinc747 on 10/4/2016(UTC), richardgatarski on 10/19/2016(UTC), fLipz on 10/23/2016(UTC), hellokitty on 10/25/2016(UTC), Peter B on 10/29/2016(UTC)
doggy  
#2 Posted : Sunday, October 2, 2016 4:29:32 AM(UTC)
doggy

Rank: Advanced Member

Groups: Registered
Joined: 12/27/2012(UTC)
Posts: 5,218
Belgium
Location: Belgium

Thanks: 291 times
Was thanked: 955 time(s) in 790 post(s)
Quote:
A side note: in my opinion there should be some kind of general optimization thread on vmix forums, which should be stickied. Right now there is some info, but it's spread all over the forums. It would be cool if all members could contribute to a guide which helps troubleshooting slow, unresponsive or unstable systems.


+1
richardgatarski  
#3 Posted : Wednesday, October 19, 2016 4:33:02 PM(UTC)
richardgatarski

Rank: Advanced Member

Groups: Registered
Joined: 2/18/2014(UTC)
Posts: 1,837
Location: Stockholm

Thanks: 144 times
Was thanked: 297 time(s) in 250 post(s)
lttaidv, thanks for sharing all that! And congrats for succeding :)

Given my own bad experience with X99, still not solved. And what Martin and Tim said in the October Fun Time Live Show, I think I'm gonna throw out my X99 and go for another mobo-cpu combo. I simply cant (wont ;) tweak on the detailed level you do.

One of your tweaks made me think it might solve another problem we have on many of our PC:s. When we connect another display output, eg for IMAG or ISO recording (like an Atomos Ninja), and turn Fullscreen on - our whole production stutters. I suspect it is because some monitors are at 60 Hz, while we produce in 50p.

Quote:
Forced resolutions with EDID overrides which are fixed in the computer, so no accidental changes can be made with nvidia driver updates or by using a different screen, using custom Resolution Utility (CRU). This to make sure all screens attached are always 50hz (synced with 50i inputs).

Could you please elaborate on that tweak?

In any case, thanks again.
Ittaidv  
#4 Posted : Thursday, October 27, 2016 3:41:06 PM(UTC)
Ittaidv

Rank: Advanced Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 600
Man
Belgium
Location: Belgium

Thanks: 75 times
Was thanked: 91 time(s) in 75 post(s)
richardgatarski wrote:
lttaidv, thanks for sharing all that! And congrats for succeding :)

Given my own bad experience with X99, still not solved. And what Martin and Tim said in the October Fun Time Live Show, I think I'm gonna throw out my X99 and go for another mobo-cpu combo. I simply cant (wont ;) tweak on the detailed level you do.

One of your tweaks made me think it might solve another problem we have on many of our PC:s. When we connect another display output, eg for IMAG or ISO recording (like an Atomos Ninja), and turn Fullscreen on - our whole production stutters. I suspect it is because some monitors are at 60 Hz, while we produce in 50p.

Quote:
Forced resolutions with EDID overrides which are fixed in the computer, so no accidental changes can be made with nvidia driver updates or by using a different screen, using custom Resolution Utility (CRU). This to make sure all screens attached are always 50hz (synced with 50i inputs).

Could you please elaborate on that tweak?

In any case, thanks again.


This tweak I use, to make sure that we always send out the desired resolution, without having changes happening in the background. I used to change stuff in the nvidia panel, and see it changed silently in the background whenever I changed the screen. Since I want total control, this cannot happen. Beware however! During testing, I tested the software with an older screen and forced 1920x1080. Since the screen didn't support this resolution, it simply started crashing after displaying a 'this resolution is not supported' message for a few minutes :)

I originally got to know this trick last summer when we were vj'ing in a big screen on a festival. The screen (28*5 meters) was so big in resolution, that they had to split it up in 2 hd feeds. In one of the connectors between us and the screen there was a device with only limited EDID's programmed into it, which made that this signal was always forced to 60hz, which caused the big screen to have a nasty line in the middle because one half was 50hz, the other 60 hz.

Since I got to know this tweak, I thought it might be a good idea to install it, and force my screens to 50 hz. I have no idea if it's really giving any performance gains, but I thought there would probably be some recourses used to convert a 50i signal to display on a 60hz monitor. Since all of our monitors support 50hz, it's no problem to do this.

I never saw one of these stutters, and I also don't know much about the internals of vmix, but it's logical to assume that there is some kind of processing going on when you connect a 60hz device to a 50i production. For safety I would for sure recommend this little tool, it's free anyway!


you can download it here: http://www.monitortests.com/cru-1.2.6.zip







richardgatarski  
#5 Posted : Friday, October 28, 2016 11:27:24 AM(UTC)
richardgatarski

Rank: Advanced Member

Groups: Registered
Joined: 2/18/2014(UTC)
Posts: 1,837
Location: Stockholm

Thanks: 144 times
Was thanked: 297 time(s) in 250 post(s)
thanks lttaidv, but no good fix for us as the stuttering problem is mainly on our laptops. From the CRU Requirements "Laptops with switchable graphics are not supported"
doggy  
#6 Posted : Saturday, October 29, 2016 9:53:50 AM(UTC)
doggy

Rank: Advanced Member

Groups: Registered
Joined: 12/27/2012(UTC)
Posts: 5,218
Belgium
Location: Belgium

Thanks: 291 times
Was thanked: 955 time(s) in 790 post(s)
Quote:
A side note: in my opinion there should be some kind of general optimization thread on vmix forums, which should be stickied. Right now there is some info, but it's spread all over the forums. It would be cool if all members could contribute to a guide which helps troubleshooting slow, unresponsive or unstable systems.


Please would love this

Myself I loose at times like between 15-20% of CPU power due for example Service Host items (W10)
admin  
#7 Posted : Saturday, December 10, 2016 8:13:40 PM(UTC)
admin

Rank: Administration

Groups: Administrators
Joined: 1/13/2010(UTC)
Posts: 5,208
Man
Location: Gold Coast, Australia

Was thanked: 4290 time(s) in 1520 post(s)
I've started a new dedicated X99 thread here so we can discuss potential solutions and fixes as they come up:

http://forums.vmix.com/d....aspx?g=posts&t=8456
Users browsing this topic
Guest (3)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.