There are two schools of thought in the art world; one says that you should buy cheap materials until you improve and the second says you should go with the more expensive quality from the start.
I must agree with buying good quality equipment.
How can you possibly improve your painting skills when the tools won't perform properly?
If you were asked to paint your front room would you buy a dodgy 10 litre bucket of watery paint and a pack of six cheap brushes? Don't scrimp and scrape on the tools you need. The old saying 'a bad workman blames his tools' is certainly justified in this case.
Going back to the front room scenario; you imagine trying to paint those four walls while the watery paint splashes everywhere and the hairs fall out of your cheap brushes. Stepping back to look at your work you see that, not only are their brush hairs stuck to the wall, but it'll need another coat!
The same is true of art. Good quality paint is labeled as Artist's Quality paint. If its not labeled as such then it is cheaper quality paint, sometimes referred to as Student Quality. What's the difference? Student quality paints use cheaper colour pigments and can fade badly through time. Also the general texture and behaviour of the paint is different.
Before you paint your four walls you need to prepare the walls. Wipe them down, maybe sand the walls, scrape them even? Again, the same is true in art. Cheap paper may warp or buckle and/or not hold the paint properly. For watercolour, the minimum paper weight should be 300gsm, the texture of the paper is entirely up to personal preference (rough or smooth) and for oils or acrylic you can either use pre-prepared (or make your own) canvases or use special canvas textured paper.
Ditto for brushes. Cheap brushes will quickly lose their shape and be cumbersome to use. Personally I have several good quality brushes that I use for my work but do have several cheap brushes that I use for various rough jobs and for reshaping via a quick hair-cut.
The one place you can save money is with things like water pots, I use (empty!) yogurt pots. For lighting you can use a cheap lamp but do use a good quality artificial sun-light bulb, this will give much truer lighting than the yellow light emitting house bulbs. For a palette you can use an old piece of perspex plastic or an old piece of plywood. Just make sure that the wood won't absorb the water or oil from the paint. I remember watching English watercolourist Keith Fenwick on his Art Doctor TV show, his palette was an old tray covered with cling film!
One last thing; if your painting oils... use odoucless turps, otherwise you'll lose your friends very quickly!