To get started soldering the first thing you'll need is a soldering iron. I recommend as a first soldering iron that you get something that has temperature control. The package should say temperature control on it. This will enable you to begin working with soldering without worrying about whether you're iron is too cold or too hot and all the problems associated with that temperature. Now an iron like that can run a wide variety of prices but you might be able to find something on the order of twenty dollars or less.
The next thing you'll need is a stand. It's essential that you have some kind of stand to hold your iron because it takes a long time for your iron to cool down and you want to make sure that you have somewhere to put it when you don't need it at that particular moment. So even a one dollar folded piece of metal or something is really necessary. The third thing you're going to need is something to hold your work piece and the best way to do that is with a vice. Finally, you'll need some solder, preferably flex core solder.
A roll of solder like this can be purchased at Radio Shack for probably six or seven dollars. There is a lot of solder here. This is a three ounce package and it's enough for tens of boards. This will last a long, long time and perhaps a to help you along the way. This actually holds quite a bit of material. So, since you're only using a tiny little bit for each solder that amount could last for years and years, even a decade or more. A tin like this costs maybe six or seven dollars. A piece of sandpaper and you should be all set. That's all you need to get started.