Okay, the title is slightly misleading ..
You can not move layers without touching them, but you can move them from the layers palette without clicking on them using the following keyboard shortcuts:
To move a layer down to the layers palette, press: Command-left bracket (PC: Ctrl-left bracket)
To move a layer to the palette of layers, press: Command-right bracket (PC: Ctrl-right bracket)
Now, if …

When you have multiple objects in different layers on your desktop (e.g. many stars) it’s annoying trying to guess the object’s layer by its position in order to select and move it. Simply select the move tool, left on its properties check the Auto select layer by clicking on the desktop and the desired object’s layers are automatically selected.