Mount Vernon has the reputation as the rich part of town, but there's so many young professionals and grad students here that there are tons of affordable places and cheap stuff to do. Ridiculous socio-economic diversity with super wealthy and the city jail 5 blocks away on the other side of the freeway. It's the gay neighborhood as well - not in your face though and don't know if you care either way.
Fells & fed hill are cooler parts of the city. More bars, restaurants and right by the water. But not as classy or historic as mt vernon, the parking and traffic is a nightmare, and their public events and festivals suck comparatively (and there are tons of them like free concerts in mt vernon park every first thursday during spring and summer with a few thousand people). But if you like lots of restaurants bars and pretty girls - fells is the place. Just a hunch, but probably more expensive than mt vernon.
Hampden is nice but, its much less "urban" meaning there are more small homes and yards and stuff and less apartments - has a cool strip though. Far fewer places to walk to. Less options means that you might end up paying more. There are ridiculous deals on purchasing a home subsidized through the city if you commit to living for a certain number of years and you put a certain amt of money in improvement - true for the whole city but not available really in the other neighborhoods.
Towson...meh. It's a suburb. You may end up moving there eventually, but live in the city. There are Towsons around every city in America. Baltimore is too fun for a young person to live in Towson.
Outside of cost of living for rent, Baltimore's not that bad for the east coast and its the affordable option compared to DC. Expect your insurance to go up though because your car will be broken into. It's a Baltimore right of passage.
[This message has been edited by craigernaught (edited 10/15/2012 5:50p).]