I'm in my mid 50's and got into running about 5 years ago. I spend most of my time trail racing, mostly at the Half Marathon to 25K distance, with occasional hops up to longer distances.
Get a decent pair of daily trainers (or two) from a running shoe store. They can make recommendations based on your goals, seeing you walk, expected mileage, etc.
Consider making a training plan, there's a lot of free resources (Hal Higdon, Garmin, etc) out there that can help recommend the ideal balance/approach to training. Start slow (low miles, slow pace, etc.) and resist the urge to increase miles, pace, etc. for now.
Find places to run that you enjoy. For me, it's trails, but I know others who spend almost all of their time on the 'dreadmill' and running circles around the track.
Consider joining a casual run club, they are everywhere. The majority of them are focused on fitness, support and fun (rather than racing). I'm hot and cold on running clubs though, depending on what I'm training for.