I have been a member of private clubs in the past and have been a member at a public course in my hometown for the past 2 years. The primary reason I joined was because my regular group all decided to join, and I either had to join or make new friends (I like the ones I have).
I like it because it means I get to practice whenever I can squeeze it in, and as long as I play 4+ times per month the value is there.
Secondary to the other items was quality of course (high), facilities, and ability to reserve advance tee times.
Having been a member at private and public courses I will say that the private club experience is better - the staff at our current club treats us great, but it isn't the same as being at a private club. The course covers their greens for ~1 month every winter, and closes several weekends in the summer for VSGA events, which can be frustrating. At a public course you don't get the same amenities as you do at many private clubs - my former club had a pool, tennis courts, restaurant, and the public course is essentially just a really nice golf course.