The maximum resolution is limited for a bunch of reasons (e.g. H.264 has a maximum frame size), but we’ve implemented some improvements allowing us to be more complaisant. 🙂 Thanks to this, in our testing version we’ve doubled the maximum resolution.
The change is documented here so you can subscribe to be informed when it’ll be released:
If you’re connected to a virtual desktop and if your client supports a resolution width bigger than 4096 you can enable the remote resize (from Display toolbar) and increase the window size up to the resolution you need. The client will try to reach that resolution to work around the custom resolution size constraints.
Switching to VP8 would not help because, as I said, the H.264 codec is only one of the reasons why there is a limit. We’ll release shortly (couple of weeks) the version with the improvement I’ve explained.