logo

Live Production Software Forums


Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Options
Go to last post Go to first unread
admin  
#1 Posted : Saturday, December 10, 2016 8:09:29 PM(UTC)
admin

Rank: Administration

Groups: Administrators
Joined: 1/13/2010(UTC)
Posts: 5,207
Man
Location: Gold Coast, Australia

Was thanked: 4284 time(s) in 1519 post(s)
This is a new thread to discuss potential issues with the Intel X99 Chipset and any potential fixes we have found.

What is the Intel X99

Intel X99 is a chipset targeting the high end desktop PC market.

The two key features it has over the mainstream Intel chipsets is support for more than 16 PCI express lanes and
support for processors with more than 4 cores.

What is the main advantage of the X99 for vMix users

The increased number of PCI Express lanes makes it possible to install more capture cards in a single system
allowing for 16 or more HD inputs and even 4 or more 4K inputs!

So what's the problem?

When building our own studio PC based around the X99, we encountered the following challenges:

* Audio crackling and latency, both from USB audio interfaces, and from the audio output
from the NVIDIA GeForce HDMi ports

* Running into PCI Express bandwidth limits much sooner than expected leading to high GPU render times
when using more than approximately 12 HD or 3 4K cameras

Does this only affect vMix?

No. The audio crackling issue in particular appears to be a common issue on various online forums affecting
everything from professional audio recording software to games.

In fact, we have seen the issue crop up by merely playing a high quality MP4 video in Windows Media Player with no other software running.

Is there a solution? (Updated 2018)

The first step is always to make sure Windows and all drivers are up to date. (In particular NVIDIA, Network, WiFi and Sound drivers).
If you have a X99 system and everything is working well for you then there is no need for you to take any action!

Unfortunately, for others there isn't a clear solution or fix so far. Some forums users have reported success with updating the BIOS and various drivers (particularly for sound and network devices)
and disabling any onboard motherboard devices that are not being used (such as onboard sound) has worked well for them.

Some have had success using LatencyMon to identify problem drivers and search for replacements.
http://www.resplendence.com/latencymon

One possible solution is to try the new High Input Performance Mode in the latest version of vMix. This can be enabled in Settings -> Performance.

What alternatives are there?

We recommend the new X299 chipset from Intel.
Our Obsidian reference PC uses this and achieves great results when working with 4K and productions with a large number of inputs.
https://www.vmix.com/pro...ce-systems.aspx#obsidian






This thread is for those that have come across potential solutions, or would like to discuss their own progress in solving the problem.
There have also been some encouraging reports of users experiencing no issues with their X99 system, if that is you please post your system specs
here so we can see if the working systems share anything in common.

Regards,

Martin
vMix

thanks 1 user thanked admin for this useful post.
Ittaidv on 12/11/2016(UTC)
al4video  
#2 Posted : Sunday, December 11, 2016 1:31:53 PM(UTC)
al4video

Rank: Advanced Member

Groups: Registered
Joined: 4/13/2015(UTC)
Posts: 59

Thanks: 45 times
Was thanked: 3 time(s) in 3 post(s)
I don't believe I have encountered the problems described. I'd be interested if anyone is having trouble with the Gigabyte version of this board?

My System:

VMIX HD Latest version as of 12/10/2016

Motherboard Gigabyte GA-X99 SLI

CPU Intel 17-5820K 3.8GHz

Thermaltake 750 watt power supply

GRAPHICS EVGA NVIDIA 960 4GIG MEMORY

Memory CORSAIR DDR4 16 GIG (2X8) 3200mhZ Dual
Vengeance 1.35V 288pin

Samsung EVO 250 system drive.

Onboard sound with Realtek Audio Sound Driver.

Toshiba Xtreme series 4Tb Media Drive
*****

Inside my system are:

3 Decklink Mini Recorders

1 Decklink Mini Monitor

1 Epiphan VGA2USB VGA capture device

***
Other software on machine:

Adobe Premiere CC 2017

Adobe After Effects CC

Adobe Lightroom CC

Xsplit Broadcaster

Vidcoder

Red Giant Plural Eyes 4.4

Apple Quicktime 7.79 ?

***Note. I run my system with 2 monitors. One for system interface and one for program out etc. I also use a Video Devices Pix-E7 (Monitor/Recorder) as a record monitor off the Decklink Mini Monitor. It takes both HDMI or SDI signals and can loop the signal out to another monitor as well.



James Bond  
#3 Posted : Sunday, December 11, 2016 8:48:06 PM(UTC)
James Bond

Rank: Newbie

Groups: Registered
Joined: 10/7/2016(UTC)
Posts: 4
Man
Location: Knoxville, Tennessee

Thanks: 3 times
Reposting from another thread:
I recently build a new streaming computer with an ASUS x-99-A 3.1 USB motherboard (LGA 2011-V3 socket) along with an Intel i7-5820K processor, 32 GB RAM, 500 GB SSD, etc. Was able to overclock the processor to 4.5ghz... worked very well for the first few months.

Our video set up includes a Sony HD prosumer camcorder, Blackmagic ATEM-4K with video output going to computer via a Blackmagic internal capture card. Audio input goes into a Behringer X18 digital mixer and into computer via USB. We did the separate audio routing because we found the BM capture card became overloaded trying to send audio and video through it... lots of cracking, popping, crashing, etc. That's another issue that we resolved.

Anyway, about a month ago I started having issues with the audio input to VMIX from the Behringer mixer USB - the computer stopped recognizing that the USB driver was installed after reboot. By checking Windows Sound menu, I found the recording input from the mixer was missing, as if it was not installed. But the computer still recognized the driver software being 'properly installed' when I tried to 'troubleshoot' it.

Long story short, I found several articles and YouTube videos about this same issue with X99 ASUS motherboards, suggesting that if any devices were plugged into the USB hub, that the computer would 'fail to post' correctly and the USB devices attached would not be recognized during boot-up.

Solution that worked: UNPLUG all USB hubs and use the motherboard mounted USB ports only! I moved the mixer USB cable to the back of the computer, using a 3.0 USB port and it booted/posted without any issues.

Long term solution: Need to replace motherboard with something other than ASUS as they continue to acknowledgment the problem but failed to provide any viable solution.

Hope this helps someone. I would NOT recommend ASUS motherboards!
admin  
#4 Posted : Tuesday, December 13, 2016 9:26:37 PM(UTC)
admin

Rank: Administration

Groups: Administrators
Joined: 1/13/2010(UTC)
Posts: 5,207
Man
Location: Gold Coast, Australia

Was thanked: 4284 time(s) in 1519 post(s)
I've received a report that setting the memory to use XMP in the Bios setting worked for them.
If those with X99 systems haven't tried that yet give it a go and report back here your results.
Yamato  
#5 Posted : Friday, January 20, 2017 9:53:19 PM(UTC)
Yamato

Rank: Member

Groups: Registered
Joined: 11/5/2016(UTC)
Posts: 28
Man
Location: Paris

Thanks: 7 times
Was thanked: 4 time(s) in 4 post(s)
Hello,
I've been reading the forum since quite some time, maybe it's time to participate a little bit :)

I have been using two custom made rigs based on X99 chipset.
First one with a MSI Motherboard X99 raider and Intel i7-6900K with BM Decklink Quad 2 and Duo 2 for video input, Focusrite Scarlet 2i2 for sound input.
Video works fine, Render time is around 15ms when i have a lot of sources and inputs with a master frame rate at 1080p60 but it is really smooth and CPU usage relatively low while streaming, local recording and outputting a feed simultaneously.

My only concern was about sound : From time to time, the sound become metallic with lots of crackling and i have to unplug it and plug it back. I did a drivers check, updated bios and always plug the Scarlet directly on motherboard. The problem is happening less often but it's still here. After some reading on Internet, it's the same for Gigabyte motherboards also, so it's definitely not an asus problem.

Second setup is brand new, it's a Micro ATX RIG with my "old" i7-5820K and an Asus Mobo x99m-ws/se.
I have exactly the same problem with a different soundcard.

I've read here and there it's a compatibility problem between X99 chipset and Nvidia Card. So i'm tempted to get an AMD card and test with it for a while.
Or i could go for a new i7-7700K rig maybe, even though it might be limited with PCIE Lanes ?

Regards,
Valentin
Ittaidv  
#6 Posted : Friday, February 3, 2017 12:19:13 PM(UTC)
Ittaidv

Rank: Advanced Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 600
Man
Belgium
Location: Belgium

Thanks: 75 times
Was thanked: 91 time(s) in 75 post(s)
We have many systems with x99, and really had some big trouble since last summer. I really tried to figure out what is going on, and i'm working on a fix. For now we use dual systems, which both output the program to NDI to a third system, where we can quickly switch in case of problems. Also we now do backuprecordings of audio in a tascam recorder, in order to fix audio problems afterwards.

The main problem is the system losing connection to the soundcard, which results in massive sample drops and eventually a disconnect. Also crackling, and unstable render times, resulting in framedrops.

I didn't see it back then, but we even had huge troubles during our most important gig this year. Only saw it afterwards:
If you look well, you see sometimes the video is really choppy, this because of framedrops. I could reproduce this problem with our best x99 system, with 8 inputs sometimes the render times start jumping up and down, resulting in choppy video. It was a very troublesome gig anyway (almost no space in the front of house, cable troubles, building up over night..) so that's why were already happy it worked :)

We have a lot of systems:

System 1&2:

Asus x99 e ws
Xeon E5-2630 v4
Gtx 1080
32 gb ddr 4 ram
Decklink quad 2

System 3:
MSI X99S XPOWER AC
Radeon r9 270x
16gb ddr 4ram
4 -5 x mini recorder or intensity pro or decklink sdi (depending on the situation)

System 4:
Gygabyte z97 gaming g1
intel 4970s
GTX 950
16gb ddr3 ram
4 x mini recorder or intensity pro or decklink sdi (depending on the situation)

System 5:
don't know the exact specs, but it's an older i5, 16gb ram,.. We use it as a backup, because it still accepts those yuan cards.


Soundcards:
2x focusrite 18i20
2x focusrite 2i2
1x yamaha MG16XU
1x XENYX Q1202USB PA

We tried the x99 systems with all soundcards, and all of them will start dropping samples. There is no difference between the msi x99 and the asus, both have the same troubles. The older z97 works flawless, but is not suited for more then 4-5 cam captations/livestreams.

There are a few scenario's where the soundcards go nuts which are reproducable. I know they're not real life situations, but it might give some useful info:

situation 1:
When you record prores or another heavy coded to ssd, and the disk is full: soundcard loses connection

situation 2:
When you start 4-5 multicorders at the same time, in super heavy codec (too hard for the harddisk to handle it): soundcard loses connection.

Off course these are not real life situations, but it might be useful info to find the root of the problem.


I also could come up with a few tweaks the last weeks, which have resulted (for now) in zero problems. It's not the first time we had no problems a few gigs in a row though, so i'm not crying victory yet.

Tweak 1: change recording codec.

We used to record in hvec codec, switching to mpeg ts (same bitrate) made the render times less jumpy.

Tweak 2: change the process priority of ffmpeg with nvidia cards to below normal: made the soundcard keep it's connection (using process lasso)

Tweak 3: using processor lasso ProBalance to prevent micro lags. This made the render times even less jumpy it seems.

Tweak 3: disabling hardware encoding during livestreaming seems to help a bit with an nvidia card


With these 3 tweaks we had a few gigs in a row without any troubles. Let's hope this solves it for us :)

FIY:

An example of sample drops of the soundcard:

https://www.facebook.com...videos/1179276618834507/

An example of the soundcard dropping huge amounts of frames:

https://www.facebook.com...videos/1220374621341200/

Of course we have 100's of examples where it all went perfect (we do now 3-4 livestreams a week), but we often had to go the extra mile in order to fix dropouts we had during testing, or to provide all kinds of backup solutions like a second or third pc, sound recorders,..

I now did hundreds of hours of testing, and can confirm 100% sure that only on x99 we have these issues.
When it comes down to nvidia or radeon cards: both have similar issues.
All usb soundcards seem to be affected. I'm not really keen on using a built in soundcard (bad preamps) or a pcie soundcard (losing a slot).

Hope this info can help someone with similar issues a bit!















richardgatarski  
#7 Posted : Friday, February 3, 2017 5:23:52 PM(UTC)
richardgatarski

Rank: Advanced Member

Groups: Registered
Joined: 2/18/2014(UTC)
Posts: 1,837
Location: Stockholm

Thanks: 144 times
Was thanked: 297 time(s) in 250 post(s)
@lttaidv,
thank you for sharing your insights, tips, and tweaks!!!

I understand that, besides you, many users with X99-based systems for various reasons want to fix these issues.

Personally (with only one X99), I decided to throw it out, or rather replace the MoBo and CPU. Not worth the time trying to fix, nor the risks involved. Everything points to that this is certainly not a vMix issue. Rather an Intel/nVidia flaw. Perhaps combined with how vMix is using their hardware.
Ittaidv  
#8 Posted : Thursday, February 9, 2017 6:49:51 PM(UTC)
Ittaidv

Rank: Advanced Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 600
Man
Belgium
Location: Belgium

Thanks: 75 times
Was thanked: 91 time(s) in 75 post(s)
richardgatarski wrote:
@lttaidv,
thank you for sharing your insights, tips, and tweaks!!!

I understand that, besides you, many users with X99-based systems for various reasons want to fix these issues.

Personally (with only one X99), I decided to throw it out, or rather replace the MoBo and CPU. Not worth the time trying to fix, nor the risks involved. Everything points to that this is certainly not a vMix issue. Rather an Intel/nVidia flaw. Perhaps combined with how vMix is using their hardware.


Well, we could also throw our x99 machines out, but it would be a big cost. Besides that our productions are also growing (we now sometimes need 8 HD cams + 2 external outputs) and we're now experimenting with sports, where 4 replay camera's would be added to the mix as well ;) So besides x99 I don't think there's a chipset which could handle this. Splitting up the tasks across multiple computers might be an option, but makes it a lot of work to set up and debug the system in case of troubles.

We're now patiently waiting and hope to upgrade once the X299 from Intel sees the light. Buying into z170 or z270 would be a bad move timing whise at this moment I guess.

If it's a vmix issue I don't know. I did recently notice livestream.com sells x99 based hardware to there costumers. I can't imagine they would sell it if they would have issues with it as well. But yeah, they only rely on the cpu, which makes their system capable of an insane amounts of HD or 4K inputs (I think because the pcie traffic is not adressed back and forth to the graphics card), but it also has it's downsides software whise. Plus I don't like their company, with those annoying sales guys who harass people to buy into their crappy streaming site ;)


Yamato  
#9 Posted : Friday, February 10, 2017 10:22:57 PM(UTC)
Yamato

Rank: Member

Groups: Registered
Joined: 11/5/2016(UTC)
Posts: 28
Man
Location: Paris

Thanks: 7 times
Was thanked: 4 time(s) in 4 post(s)
@lttaidv;

thanks for the tips and tweaks, i will test it ASAP !
Wrighty  
#10 Posted : Friday, February 17, 2017 8:06:06 PM(UTC)
Wrighty

Rank: Newbie

Groups: Registered
Joined: 2/17/2017(UTC)
Posts: 4
Location: Tavistock, Devon, UK

Thanks: 1 times
Was thanked: 2 time(s) in 1 post(s)
We've had a vMix system running largely fine with an X99 motherboard for over a year now... Spec here:

Asus X99-S S2011-V3 motherboard
Intel Core i7 5930K 6-core 3.5GHz CPU
Crucial 8GB DDR4 PC4-19200 RAM kit (2x4GB)
Asus GTX760 2GB DDR5 HDMI/DP/2xDVI graphics card
Crucial BX100 250GB SSD (system and working drive)
Seagate ST1000NM0023 1TB HDD (archive drive)
Pioneer DVR-221LBK DVD-RW drive
2 x Magewell XI400DE 4xHDMI PCIe capture cards
2 x Magewell XI100XE-Pro 1xHD-SDI/HDMI/DVI/VGA/YUV/CV PCIe capture cards
Windows 10 Professional 64-Bit

I did find that it's quite important which slots you use for cards, as some do share PCIe lanes with each other and probably with other hardware on the motherboard, such as the USB, Ethernet, audio and SATA interfaces.

We frequently run all 10 capture inputs with a variety of resolutions - some are only 1024x768 but some are 1920x1080. Master vMix rate is 1080p50. If we capture everything at 1080p50, it does drop frames and render time occasionally goes up over 20ms. I've also typically got around 20 other inputs with various PiPs and other stuff set up, along with 2 full screen monitor outputs. On the majority of jobs, we're just using it as a vision mixer - not doing recording or streaming. I have successfully used it for recording though and I'm about to build another lower spec machine with just 2 inputs for doing basic recording and streaming.

The only issue I have had is random frame stuttering on the output sometimes, which I haven't fully got to the bottom of. I think it only happens when I'm capturing audio via the on-board sound card. If I capture audio via one of the XI100 cards instead, I don't get the problem. Other than that, I haven't had any audio break-up or crackling or anything.
thanks 2 users thanked Wrighty for this useful post.
Squier on 2/25/2017(UTC), Bluecruza on 3/11/2017(UTC)
admin  
#11 Posted : Tuesday, March 28, 2017 7:17:25 AM(UTC)
admin

Rank: Administration

Groups: Administrators
Joined: 1/13/2010(UTC)
Posts: 5,207
Man
Location: Gold Coast, Australia

Was thanked: 4284 time(s) in 1519 post(s)
A brand new NVIDIA Quadro P4000 dropped by our office today so I was able to put it through its paces
comparing performance on a X99 system vs Z270

The vMix testing results put the P4000 right in between the GeForce 1060 and 1070.
Here are the specifications of these cards for reference:

P4000
1792 Cores
243 GB/s Memory Bandwidth
5.3 TFLOPS

1060 3GB
1280 Cores
182 GB/s Memory Bandwidth
4.0 TFLOPS

1070 8GB
1920 Cores
256 GB/s Memory Bandwidth
6.5 TFLOPS

So our results match up with the specifications well.

The bad news is the Quadro driver does not overcome the weaknesses on the X99 platform and works pretty much exactly as a similarly specced Geforce would.
I suspect NVIDIA have unified the drivers including the DMA memory optimisations that use to be reserved for the Quadro.

The good news is the Z270 is stable for 16 HD inputs and there a couple of motherboards available allow additional PCI-E lanes through the use of a chip from PLX.
Bandwidth shouldn't be a problem in this case as x16 3.0 can be split to x8 3.0 + x8 2.0 + x8 2.0 which is more than enough for 2x 8 input cards.

An example of a motherboard that can handle this setup:
http://www.gigabyte.com/...GA-Z270X-Gaming-9-rev-10

This is the updated version of the Z170X-Gaming G1 motherboard we use in the studio pc.
So this currently is our recommended platform for 16 HD or 2 4K production setups for the time being.

We are now looking forward to the X299 release in the second half of this year which will bring 40 lanes to the proven Skylake platform.

Regards,

Martin
vMix





thanks 1 user thanked admin for this useful post.
Ittaidv on 4/13/2017(UTC)
DWAM  
#12 Posted : Tuesday, March 28, 2017 7:47:41 AM(UTC)
DWAM

Rank: Advanced Member

Groups: Registered
Joined: 3/20/2014(UTC)
Posts: 2,721
Man
France
Location: Bordeaux, France

Thanks: 243 times
Was thanked: 794 time(s) in 589 post(s)
Hi!

Does this NVIDIA Quadro P4000 support more than 2 simultaneous encodes?
As the case may be, will vMix take advantage of this?
Would you recommend this card over GTX models for general use with vMix (not only for encodes) ?

Thanks
Guillaume
admin  
#13 Posted : Thursday, March 30, 2017 8:18:52 PM(UTC)
admin

Rank: Administration

Groups: Administrators
Joined: 1/13/2010(UTC)
Posts: 5,207
Man
Location: Gold Coast, Australia

Was thanked: 4284 time(s) in 1519 post(s)
Yes, the Quadro P series supports many more simultaneous encodes than 2. (No fixed limit, up to what the card is capable of)
Other than that performance is equal to a GeForce with the same core count.
thanks 1 user thanked admin for this useful post.
DWAM on 3/31/2017(UTC)
BloodyIron  
#14 Posted : Tuesday, May 2, 2017 5:47:40 PM(UTC)
BloodyIron

Rank: Advanced Member

Groups: Registered
Joined: 9/25/2013(UTC)
Posts: 48
Location: Canada

Thanks: 8 times
Was thanked: 3 time(s) in 3 post(s)
What about workstation chipsets like c602 or c612? This seems to only be talking about X99, but I am interested in how this stacks up for workstation chipsets too. Considering the kind of workload that vMix is tailored for, I really would like to see more information about workstation class hardware too in regards to performance and issues like this.

But curious info despite that ;o
admin  
#15 Posted : Wednesday, February 7, 2018 11:09:52 PM(UTC)
admin

Rank: Administration

Groups: Administrators
Joined: 1/13/2010(UTC)
Posts: 5,207
Man
Location: Gold Coast, Australia

Was thanked: 4284 time(s) in 1519 post(s)
As a final update, we recommend the X299 build specs below as a replacement to X99 builds:

https://www.vmix.com/pro...ce-systems.aspx#obsidian

Those on older X99 systems reporting high render times can try turning on High Input Performance mode in Settings -> Performance
in vMix, but moving to X299 is recommended for all those considering purchasing a new system.

Regards,

Martin
vMix
thanks 1 user thanked admin for this useful post.
yimmyo on 11/13/2020(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.