Cycle Bags
There are hundreds of different cycle bags to suit all kinds of bikes and people.
Pannier bags
This is the most common cycle bag and is perhaps the best way to carry items on a bike. You let the bike carry the weight and when evenly balanced you should not destabilise the bike.
For ease of use a quick release version of these bags can be preferable. Alternatively you can click them onto the side of the rack and secure the rack with bungee cords.
These bags come in all shapes and sizes. Having two bags, one large and one small, can be ideal to allow for different cycling circumstances. Alternatively look for bags that can be expanded.
Handlebar Bags
Another good way of carrying items on your bike. These bags are usually fixed to the bike so can get wet if left outside. The advantage of these bags is that you can keep a map on the top where you can always see it.
Other Bags
The obvious other way to carry items is to use a rucksack. This will mean you are carrying the weight rather than the bike. The disadvantage is that you will sweat more however for short rides where you need to carry extra items these can provide a great short term solution.