I have this issue since I'm not even sure when, probably almost always.
I did recently an upgrade to a 3800x from a i4770k and I had hopes that would improve but it's still the same.
The problem arise while the mouse is going over different fences: I have 10 with 220 shortcuts and 8 folders. It's a lot but that's the point having fences.
If I hover the mouse pointer only over the icons in a single fence there's a slight lag before the icon is selected and that's what I'd normally expect.
Instead hovering the mouse over two or more fences make this tragic; if I make a circle for 5 seconds I can see the icons are selected with a 3 seconds lag.
Moving in or out an icon between fences or from the desktop needs delay otherwise is a hit or miss.
Is there a way to fix it?