Always connect a resistor !
By convention, a red cable plugs into 5V and a black cable plugs into GND (ground).
An LED must be plugged in the correct way round; the long leg is for 5V, the short leg is for GND. If you get it wrong, there's a small popping sound and a melting electronics smell. When this happens, whip out the power supply quickly, then replace the LED with a new one.
You must plug in a resistor. If you don't, there's a circuit from 5V to GND, and something bad happens.
|A simple circuit; 5V to a resistor; resistor to an LED; LED to ground.|