Hmm.. Most hotel lobbies have a place to sit and watch tv.. even the cheaper ones, I would say don't give up on your plans for fun trips to have some down time, but that is just me.

A coffee shop, an open air restaurant.. taking a drive in the car, will any of those things do for down time for you, or do you need to have it in the place you are staying?

It is going to be a memorable trip, no matter how it pans out.