Empowering Safe Online Spaces: AI in Gender Violence Detection and Prevention
DOI:
https://doi.org/10.46243/jst.2025.v10.i02.pp39-50Keywords:
Gender-based online violence (GBOV),, Artificial intelligence (AI), Hate speech detection, Content moderation, Algorithmic biasAbstract
Gender-based online violence (GBOV) is a pervasive issue that disproportionately impacts women and marginalized genders, leading to psychological distress, economic consequences, and restricted participation in digital spaces. This article examines how Artificial Intelligence (AI) offers innovative solutions to detect and mitigate GBOV through tools such as sentiment analysis, hate speech detection, image recognition, and behavioral analysis. AI-powered interventions have significantly enhanced the ability to identify harmful content, automate moderation, and empower victims by providing real-time safeguards. Despite these advancements, challenges such as algorithmic bias, privacy concerns, and the evolving tactics of online abuse remain critical obstacles. This study highlights the importance of developing ethical AI systems, fostering multi-stakeholder collaboration, and implementing robust regulatory frameworks to create safer, more equitable online environments for all.
References
European Commission. 2021. Proposal for a Regulation Laying Down Harmonized Rules on Artificial Intelligence (AI Act). https:// eur-lex.europa.eu/legal-content/ EN/ TXT/?uri=CELEX:52021PC0206
United Nations Broadband Commission. 2021. Towards a Resilient, Safe, and Inclusive Digital Future. https://broadbandcommission.org
Australian eSafety Commissioner. 2021. Australia’s eSafety Act: Enhancing Online Safety. https://www.esafety.gov.au
Nicola Henry, Asher Flynn, and Anastasia Powell. 2020. Technology-facilitated domestic and sexual violence: A review. Violence against women 26, 15-16 (2020), 1828–1854. https:// doi.org/10.1177/1077801219875821
StopNCII.org Initiative. 2021. StopNCII.org: An AI-driven solution for combating image-based abuse. https://stopncii.org
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6), 1-35. https://doi. org/10.1145/3457607
The Economist Newspaper. 2021. Measuring the prevalence of online violence against women. https://onlineviolencewomen.eiu.com
WITNESS Organization. 2022. WITNESS: Addressing Online Harms through AI and Collaboration. https://witness.org
World Health Organization. 2021. Violence Against Women Prevalence Estimates. https:// www.who.int/publications/i/item/violenceagainst-women
Meta AI Research. 2022. Our New AI System to Help Tackle Harmful Content. https://about. fb.com/news/2021/12/metas-new-ai-systemtackles-harmfulcontent
Sarah T. Roberts. 2019. Behind the Screen: Content Moderation in the Shadows of Social Media. Yale University Press. https://doi. org/10.2307/j.ctvhrcz0v
Sayar Ghosh Roy, Ujwal Narayan, Tathagata Raha, Zubair Abid, and Vasudeva Varma. 2021. Leveraging multilingual transformers for hate speech detection. arXiv preprint arXiv:2101.03207 (2021). https://doi. org/10.48550/arXiv.2101.03207
Raiswa Saha, Sakshi Ahlawat, Umair Akram, Uttara Jangbahadur, Amol S Dhaigude, Pooja Sharma, and Sarika Kumar. 2024. Online abuse: a systematic literature review and future research agenda. International Journal of Conflict Management (2024). https://doi. org/10.1108/IJCMA-09-2023-0188
Maarten Sap et al. 2020. The Risk of Racial Bias in Hate Speech Detection. Proceedings of the ACL (2020),1668–1679. https://doi.org/10.18653/ v1/P19-1163
Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. ACL Anthology (2017), 1–10. https://doi.org/ 10.18653/v1/W17-1101
UNESCO. 2022. Media and Information Literacy: UNESCO’s Framework for Digital Literacy. https://www.unesco.org/en/mediainformation-literacy
UNESCO. 2022. UNESCO Guidelines for Ethical AI Development. https://unesdoc.unesco.org/ ai-guidelines
Myakala, P. K., Jonnalagadda, A. K., & Bura, C. (2025). The Human Factor in Explainable AI Frameworks for User Trust and Cognitive Alignment. https://dx.doi.org/10.2139/ ssrn.5103067
Myakala, P. K. (2024). Consciousness in Machines: A Critical Exploration. International Journal of Multidisciplinary Research and Analysis, 7(12), 10-47191. https://doi. org/10.47191/ijmra/v7-i12-18
Published
How to Cite
Issue
Section
