I suspect this is happening because of coordinate mismatches between your screen and the VM, or the VM's inability to completely accurately know when/where you're clicking, and for how long.
I was able to resolve, I had to plug in a second mouse and then i had to give control of that mouse to the VM