"It's unusual to to have sole legal custody. This usually only happens when there has been severe abuse or neglect, the parent is a danger to the child, and they never get to see or the child.
Also, you don't have sole physical custody if he has any court ordered parenting time. Instead, you would be known as the primary parent."
I guess state laws vary from state to state. I have sole legal custody and the father has parenting time. He "gets to" see the child as much as he wants to actually.
Also, my court order states I have sole physical custody, just before it spells out the father's parenting time.
To the OP - I've never given my ex official permission to seek medical care. I have always sent a copy of the insurance card with for longer stays. But if dad was to seek medical treatment, it would only be an issue if you made it one; any medical provider he would seek out does not know the court order granting you sole legal even exists, unless you or dad tell them. If you are nervous about it anyway, I would call a clinic in the area he is looking to stay at and ask them what documentation they would need, and then get him that. In reality, a piece of paper officially granting him permission protects him more than you - if you are okay with this, you have already given him permission; the piece of paper would be proof for him if you were to change your mind later. Also, I am fairly sure no ER will decline treating your kids in an emergency, regardless of who brings them in.
I have also never given my ex any document showing I consented to the trip. Will they be driving or flying? If they are flying, I believe the would still only need documentation for international travel, but you can always check with the airline to see if they have any requirements.
If you do end up giving him written consent, be sure to have it notarized.