Without other usb devices the keyboard works immediately. With them it takes 5 ish seconds for it to start responding. This is regardless of which device is discovered first (which I can manipulate), it seems to just wait for all of them. Is there a way to make it not do that?
This worked. Apparently I had all the usb devices connected to the same controller and it seems linux initialises them controller by controller. Thanks