Does a divorce take effect if the wife goes to her family's house after the husband said, "By divorce, if you go to my family's house" three times, and what should the husband do to retract this matter?

If your intention by swearing an oath of divorce was to prevent your wife from visiting your family, then her visit would divorce her, and you may take her back as long as it is not the third divorce. Some scholars, like Ibn Taymiyyah, hold the view that if an oath of divorce is intended for prevention or emphasis, it is considered like an oath sworn by Allah, in which case the one who breaks it must offer an expiation for an oath and the divorce does not occur. Thus, you may permit her to visit along with offering expiation. The lawful oath is by Allah Almighty.