How can I atone for my ill-treatment of my mother, who is possessed, and whom I sometimes lay hands on to prevent disturbing the neighbors?

You should seek treatment for your mother through legitimate ruqyah (definition) [Islamic exorcism/healing] from people of virtue, and turn to Allah the Almighty, while staying away from charlatans. It is not permissible to lay a hand on her with a beating in any way, despite your assertion that you can preserve her by means other than striking. This is a trial, and you must be patient, for Allah the Almighty says in Surat Al-Isra: "And your Lord has decreed that you shall not worship except Him, and to parents, good treatment. Whether one or both of them reach old age [while] with you, say not to them [uff] and do not repel them but speak to them a noble statement. And lower to them the wing of humility out of mercy and say, 'My Lord, have mercy upon them as they nurtured me when I was small.'" Cursing one's parents and disobeying them are among the major sins. A mother is entitled to three-quarters of righteous conduct, as stated in the noble hadith (definition) narrated by Abu Hurairah. You must be gentle with your mother, just as she was gentle with you.