I can confirm that the “Samsung Keyboard” is mostly to blame. With “Simple Keyboard” I no longer have the situation that all alphabetical keys are stuck, input feels a bit sluggish, which is acceptable in a remote session (however some keystrokes seem to get missed entirely with fast typing).
On another note, the right mouse button of a real mouse connected via USB is really difficult to trigger (something like: hold both buttons for two seconds). It seems to be related to right button emulation in touch mode. Is there a way to disable that?