The Folder Portal is one of the most useful features for the desktop.
But when you go to the second or third sub-folder to drop an item and want to go up again, it can become a mild annoyance.
I would have 2 suggestions here:
1. make the "folder up"-button stationary
or
2. have another button that restores the folder portal to the original state/the original folder.