Mouse Pull on GunGame server

Every few minutes while playing, the mouse will randomly get pulled to the lower right side of the screen. Completely random, sometimes it will happen twice in a match, sometimes never in a match. My coworker sees the same issues and we are playing on two completely different computers, him with a wireless mouse, me with a wired mouse. Both of us with wired LAN on 1Gb/s Internet connection at work playing during our lunch break. This does not happen on other servers ever.