I'm a much smaller than average single female.
I also live near an area with a REALLY high crime rate (I work IN that really high crime rate area AND went to school in it.)
100% of my public restroom decisions (which one to use, where should I go, should I "hold it") have to do with my safety.
Where I live, the sight of a man following a women into a restroom (unless it looks like an OBVIOUS mistake) alerts witnesses to a possible violent crime.
HOWEVER, I had a co-worker once who most people thought was a woman who used to be a man. I would feel unbelievably safer in a public restroom if I knew she was there. But that is to say I feel safer in a public restroom when I am with another person whom I know ------ I don't care about their gender.
As for public policy - I don't know? If a large scary man is allowed to follow me into a public restroom, I'm going to have to watch the amount of liquid I consume.
But then again, I had a friend (male) who used to have to ask "strangers" for help when he thought his 6 year old daughter might be taking too long in the public restroom at the beach.