There are a couple ways to do this:

1) The preferred method is to see if your BIOS has the option to disable the iGPU and use the dGPU only. Depending on what system you have, this can show up in different places or named different things like "initial display" or "Optimus" or "igpu" or "multimonitor". whatever the option is, you want to carefully read the description to see which one will enable the Discrete GPU only and turn off the integrated GPU. This is preferred as you get maximum performance out of this mode. The downside is that your laptop will get hotter due to using the higher performance processor causing the fans to spin more often, and your battery life will be reduced, but I asssume if you are running vMix you are plugged in anyways. I should also note that doing this may cause thunderbolt 4 connections to be disabled in some laptops as those are direct connected to the iGPU.

2) The second method is to set the program to use the 3080 at a software level. To do this, you want to open up the NVIDIA control panel and select the "Manage 3D settings" in the left hand menu. In the options list to the right, Select the "Program Settings" tab. and in "select a program to customize" drop down select vMix (vMix64.exe). If you do not see this in the list, go to the "add" button right next to the drop down list and try to find it there, If its not there, click the "browse" button to add it. Usually it's under "c:\ProgramFiles(x86)\vMix" by default. Once you have the program selected or added, you should see a "Preferred Graphics Processor" drop down. Select the "High Performance NVIDIA Processor" (Make sure not to use Auto, or Integrated). Close the control panel and vMix if it is running, then restart vMix (you may have to restart the machine in some cases). This should set vMix to utilize the 3080 which you can verify in the vMix settings.