Is the sampling frequency set to 44.1 KHz for everything?
I’m actually also surprised you managed to connect them directly without issues. In my case i have to go through a USB Hub for things to even show up 
There’s a thread about this as well.
I’ve personally had good experience with the recent VB-Matrix Beta (only available on their discord at the moment) instead of Asio4All. But that is in early stages and i only recommend this for the adventurous types
Once it becomes officially released, i’ll probably make a tutorial on how to use it. So far i’ve had good results with it. Going even as far as mixing different sample rates across various devices.