I've also been down there when the Marines were preparing to do an exercise and moved us all down to the regular beach. They really don't mind people on their beach (sex is a different subject) or else there would be a barrier and more signs.
I've even been on the base ... it's really no big deal.