In the early 19th century there were various forms of horse drawn railway and the wheels were held on the track by big flanges. These worked but they would rub against the rails creating friction and making a horrible noise. A neat solution was found, probably by accident when they started casting the wheels. When you cast a wheel it will naturally have a slight slope on it to get it out of the mould.
They tried putting the wheels on the axels both ways round. If the wheel is smaller on the outside it means that if the axle moves slightly to the left the wheel is bigger on that side so it steers back onto the track.
 |  |
|---|---|
Wheels sloping outward
|
Wheels sloping inward
|
But if the wheel is bigger on the outside and moves to the left, the axle will turn to the left and quickly fall off the track as you found.
If the track is too sharply curved the cone shape of the wheels can't cope and the flanges of the wheels do rub against the rails making a horrible noise and wearing out the track. Most tracks don't corner this sharply.
If you deliberately push the wheels at an odd angle it can overshoot and take a wobbly swaying path down the rails, this is one reason why old trains often sway gently, though modern suspension is a lot better at coping with this.
Read more here:
http://www.thenakedscientists.com/HTML/content/kitchenscience/exp/train-tracks-how-trains-go-round-corners/
