smartPTZ app in the Apple App Store: https://apps.apple.com/u...pp/smartptz/id6741871488

I am a long time vMix user/fan. Until recently my team and I have been using Canon CR-N300 PTZ cameras with NDI-HX, a cheap POE switch, and a laptop with vMix to capture, produce, and stream live events and podcasts. This current system produces great video. But, it’s not a super small kit. It’s not a quick and easy set up. Even with POE for power, ethernet cabling still needs to be run to each camera, which takes time and attention to install/test. And, the cost for replacing all the equipment in one of these kits is around $20,000.

- - - - -

In 2024, Belkin introduced a new iPhone stand and Insta360 introduced a new pro level gimbal, both devices were Apple-DockKit-enabled. DockKit is a protocol that allows iPhone apps to connect to the attached stand/gimbal wirelessly (using a protocol called NFC). Once connected, iPhone apps can be programmed to pan and tilt the phone. These devices are cheap, around $150 each.

We were already fans of using NDI-HX on a private LAN, and were comfortable using live streams from cameras to feed and switch through vMix. We had heard good things about SRT. One thing in particular that we thought was an under-recognized and relatively unique feature of SRT was that the latency from encoder-to-vMix could be measured, and FIXED, in milliseconds.

With SRT you send multiple camera feeds back to vMix over a “good network” (does not need to be the same LAN, does not need to be wired, could even be over the public Internet) and when the SRT streams show up in the vMix session (maybe ¼ of a second later) all the cameras are in perfect sync. There is no problem switching between them in real time.

At the same time, the iPhone camera is excellent and getting significantly better every year. And, while the phone may not have the same level of zoom as a mid/top tier PTZ, at close range it looks better than cameras 5x times the price. And, to accommodate for lesser zoom we just set up our iPhone cameras closer to the action (they're so small they are hardly noticed).

There are numerous additional advantages to using an iPhone-camera–based-PTZ vs a traditional PTZ. The iPhone version has (1) a built-in battery, (2) no cables are required, (3) its a very small small form factor, (4) there's an on-board video monitor (screen), (4) an on-board 4K video recorder, and (5) cellular back-up. And, again, let's not forget about the quality of the iPhone camera/tech itself. (The iPhone 15 Pro Max camera was used to capture the 'over-the-shoulder shots' in the IMAX movie F1, it's that good.)

- - - - -

So, we built an iPhone app… That has the following levels of functionality:


Level 1 - SRT Encoding/Streaming, with Color Control

As soon as it is downloaded, here is the feature set:

- streams SRT to vMix (or any other SRT Listener). User can adjust encoder settings.

- records audio/video locally to the iPhone at the same time.

- user can control the camera’s white balance and exposure



Level 2 - Add Remote Control for SRT Streaming with Color Control

User can link their smartPTZ iphone app to the free account they can create at smartPTZ.com

- Once registered, the user can control the smartPTZ app from our secure web user interface, streaming, recording and color control.



Level 3 - Full Remote Control, Including Pan and Tilt

- iPhones on DockKit-enabled stands or gimbals, can then be PTZ controlled remotely.



Level 4 - Multiple Camera Remote Control using API

- Using the smartPTZ API, users can control multiple smartPTZ cameras from their own control surface.


- - - - -


The final result of one year of effort… The smartPTZ app, and a vMix and Companion Template for a three camera shoot.

In addition to developing the smartPTZ app, we also prepared a vMix template and accompanying Bit Focus Companion app that can be used to produce a very high quality three camera switched program.

Our new 3 iphone based kit costs around $5,000 (our comparable Canon system was around $20,000). Our three camera kit is less than ½ the size, and takes less than ½ the time to set up.

If you are interested in getting a copy of our vMix template and Bit Focus Companion App configuration files, please reach out. We are happy to share our latest build with you. We’ll even help you set it up and test it.


Smart Station - Apple AI based Speaker Tracking

In this initial build we have included a touch of AI, we call it the “smart station”. When this is feature is enabled, the camera tracks and follows the primary speaker in the frame. The application will, however, turn off tracking when the speaker leaves the “station”. This “station” is defined by the user. If/when another user enters the station (camera frame), it will re-engage and track that speaker.

We envision the smart station feature being used where there is a position on a set or stage that people speak from regularly. We don’t want to track them on/off the set/stage. But, we do want to track and follow them, ever so gently, as they might move a touch to left and right as they speak (either sitting or standing).

There is a lot to unpack and understand here. I hope some of you find the time to download the iPhone app and give it a try with vMix out of the box. And, perhaps some of you will give the PTZ controls a try, using it with a DockKit enabled device. Maybe a few of you will use the app to produce multiple camera events, I hope so. Regardless of what you do with it, let us know how it goes.

smartPTZapp@gmail.com

- - - - - -

Links for Reference:


smartPTZ app in the App Store:

https://apps.apple.com/u...pp/smartptz/id6741871488


Website to set up and control your smartPTZ camera remotely:

https://www.smartPTZ.com


smartPTZ FAQs:

https://app2.spaceli.io/...Fejg591BpUN2UNg46k30jmCg


Real Use Case: Two smartPTZ cameras in a High School Auditorium:




Real Use Case: Three Camera Live Podcast:




Unboxing the Belkin DockKit Enabled Stand: