Take the tour of both and see what you like. Like the previous comment said, if you live on the south edge of town, proximity may make the difference.
I can only speak for Willowbrook as I have never been to the pool at Hollytree, but the pool at Willowbrook is fantastic. It's very large, and has a big shallow area for the kids. Typically lots of young families there in the morning and early afternoon. They normally clear out by mid afternoon. Lots of tables/umbrellas. Private cabanas. Plenty of waiters. On holidays, they have DJ's by the pool with games all day. They have family movie night by the pool on Fridays during the summer. They have adult swim nights on occasion.
I don't play golf, but my friends who play at Willowbrook really like it, but the Hollytree golf course is very nice too. The tennis scene at Hollytree used to be far superior, but that may have changed. Willowbrook has been trying hard to improve their tennis program, but I don't know if they have caught up or not.
Willowbrook has the sports grill that we really like. Casual atmosphere on the gold course. Good bartenders, and TV's for football. Menu at the sports grill is limited (compared to the main dining room), but it's good.