Is it doable to hike to the springs around 1:00 pm in Aug?

I am going to be near the springs on a business trip Wednesday but unfortunately I will not be able to get to the Bowen Ranch parking lot until 12:30 pm. I usually visit the hot springs during the spring and fall seasons and go to Blacks beach during the summer months but I really want to stop by the springs on Wednesday afternoon. I will be free hiking in as usual so I will bring plenty of sunscreen and I will have my hydration backpack on and full of water. Am I crazy for going down during the hottest part of the day or should I be okay as long as I take my time? I was thinking that since I was going downhill I should be okay. I will wait until later in the day before I attempt hiking back out. I am in my early 50's and I workout on an elliptical machine regularly so I think I am in pretty decent shape. Any opinions?

Thanks.