Joel ,you may want to consider driving in to Middle Santa Ynez campground and take a mountain bike. Park at the campground and you bike or walk the short distance to Big Caliente. You can also mountain bike or hike along the dirt road to Little Caliente. I have a high clearance two wheel drive Toyota and I've always been able to drive right up to both springs unless the access gate to Little Caliente is closed.
In the Santa Barbara area, you can also soak at Las Cruces (Gaviota) Hot Springs and also hit the excellent nude beach at Gaviota as well. Someone already mentioned Sespe and Wiletts. Long hikes but worth it. Sespe is best visited as an overnighter, although if you come down the shorter trail from Frazier Park.Mutau Flat, you can do it in a long day. Enjoy!