to make the icons into .ico format, go to this site: http://www.instructables.com/id/Make-Windows-Icon-Files-With-Photoshop/
Download the file. move the file into the folder described on the site and start photoshop!
the icon you make has to be 64x64 though or it will not work!!!!!! (the file type wont show up when you save as if its not 64x64)