Sam, sorry to hear of your disappointing experience when you visited Big Caliente. I wonder if Little Caliente would have been better? Most people don't seem willing to make the trip over the rougher/steeper road. In my opinion, the road to Big Caliente is practically a "freeway" compared to the road to Little Caliente. Those drunks could have been headed to one of the campgrounds way before you reach the spring. One drawback about Little Caliente is that the pools are all in the sun all day long.
Your experience sounds similiar to ours ... we will have to plan a trip later in the fall when hunting season is finished.