I live in downtown Madison and love it! My building is nice, new, and priced just right so that we don't have very many undergrads. I find it's mostly grad students and working professionals. If you think you might want ot live downtown, I'd recommend where I live (The Renaissance) or Main Depot. Living downtown allows you to be within walking distance of the farmer's market, Concert on the Square, both lakes, campus, dining, and nightlife. Picking the right downtown location can spare you the noise. Also, if choosing downtown make sure you get an underground parking spot for the snowy winters!
If looking for something away from downtown, I like the Wingra/Vilas area (near the zoo, arboretum, and Lake Wingra), downtown Middleton (peaceful little town), or the Willy Street/Atwood area.
Let me know if you have more questions!
I have a small group of A&M friends in Madison, so let me know if you are looking to meet some fellow Ags! I'm actually moving back to Texas at the end of the summer, but I am ensuring that the group continues after I'm gone!