Can having a man stay at times all night deem you unfit and cause you to lose your children in a divorce even if the children are not aware he stays over night.