I use an automixer for having a lot of mics open continously. That is extra hardware. Maybe that helps.
Or just very stupid make the sound louder and just use the studio mic. (This might come with a slight off lipsync though)

are you using cardioid or omni microphones ? stage mics like SM58 are designed to avoid spill from speakers and mostly only pickup sound in front of them..that sort of mic isnt as clean as a condensor for speech but if you had speakers directly behind them and the volume isnt too high you should hear very little spill and you could possibly EQ it to still sound OK..

another option is use a gate plugin on the hosts audio buss and set the threshold just above the background level of the speakers.. with a decent gate plugin you can set attack to ease in the volume so when the hosts start speaking the gate shouldnt truncate the beginning of their words.. its not ideal but if you did this in conjunction with directional mics you should get a decent result.. whether or not the quality of the hosts audio is still good enough though would need to be tested..

i think dugan automixing is the preferred option for this sort of application.. waves do a plugin but its not cheap and i think it need sidechaining from the host software which i dont think vmix is capable of.. i might be wrong on this though ? otherwise you could look at something like Behringer XAir 18 mixer which has automixing. depending on your setup that may work ?

Thanx for thinking with me, but I've found the name for the 'thing' I want to do: Acoustic Echo Cancellation. So to filter out a sound from the mic-source, based on a certain input.

I've seen it happen, it's been used in phonecall and meetingsoftware, but now I want to find a plugin that can do it with a certain input. In this case: to filter out the sound of the vmix-call (coming through the speakers) in the signal of the microphones.