There are two basic options:
There is a third "option" - use a resistor to drop the voltage, but the resistor will have to get rid of the extra energy as heat and when using a power source like batteries, the voltage will drop as the batteries go flat. Both the voltage regulator and switchmode power supply will keep the output voltage (5V) constant even when the input voltage fluctuates.
Initially I used a LM7805 voltage reg (hot and inefficient) which I built into a USB cable.
Due to that being hot and inefficient, I ordered an Rx-Regulator or battery eliminator which is used on battery powered remote control models to eliminate the need for a separate Rx-battery pack.
The Rx-Regulator takes your input voltage and can output either 5V or 6V at up to 3A. Weight is only 9g. The particular type that I got also has a LED indicator that glows green when its powered and red when the supply voltage drops below 6V (indicating your LiPo pack is flat).
I soldered a suitable male plug to the power input and a femae USB connector the output.
Here's a view of the other side:
To get this power supply working properly with my Motorola Defy+ phone, you need to short the D- / D+ lines (or connect with 220Ohm resistor). If you only power the GND and +5V lines then the phone will not charge, it requires the D- / D+ lines to be connected to start charging.
Here's the complete setup, with 4 x 18650 bike light battery pack, USB breakout cable box with USB-A to USB-B micro, mini and Nokia round connectors, RX-Regulator wired to USB socket: