Was the brothers' action of assaulting and beating someone who insulted religion during a quarrel correct, or should the situation have been handled differently, especially since the Hudud are not applied in their country?

Cursing the religion is one of the greatest major sins and an act of disbelief that expels one from the faith. What you did in terms of reconciling and resolving the dispute is a good deed for which you will be rewarded. As for striking the man, it would have been better to handle the matter with wisdom, calmness, and advice. This is not the establishment of a Sharia penalty, because the establishment of penalties belongs to the ruler or his representative, and individuals do not establish penalties. One who curses the religion does not deserve dignity or respect, and what some did was a reaction out of zeal for the religion. Confronting this type of situation should be done with wisdom, good admonition, and inviting to Allah, because many Muslim children have not been raised with a proper Islamic upbringing. We advise the questioner and his brothers to fear Allah and to act with wisdom.