Is it permissible to marry another Muslim woman while retaining the non-Muslim wife who has not embraced Islam—despite the previous agreement that she would embrace Islam—and what should I do to stabilize my life?

If the girl is from the People of the Book, there is no harm in continuing with her and inviting her to Islam. However, if she does not respond and you fear she may corrupt the children, then separate from her and choose a righteous, believing woman.