German lawmakers last week called for a ban on deepfake pornography after two prominent actors revealed details of the AI-enabled violence they had suffered. In Denmark, a law safeguarding a person’s voice and appearance as intellectual property enters into force this week, addressing a similar gap. At EU level, the European Parliament recently proposed amendments to the AI Act to specifically ban AI ‘nudifier’ apps used to create sexually explicit images without consent, largely targeting women.
These developments signal that legislators are beginning to treat AI-generated gender violence as a design and governance issue, not merely a content moderation one. While a welcome step, gender mainstreaming across EU digital policies is needed beyond the actions set out in the recently adopted Gender Equality Strategy.
Since AI reflects the ethics, values and biases of those who fund and develop it, it is no surprise that real-world inequalities are embedded within it. Digital gender-based violence disproportionately affects women – particularly female politicians, an estimated 29% of whom are victims of cyberviolence – raising important policy questions: how can such harms be mitigated effectively? Further, how can new technologies avoid perpetuating violence against women in the first place?
Where current frameworks still fall short
To adequately address the rise of AI-generated violence, however, a clear distinction is needed. Gender-based digital violence – such as harassment, stalking and doxing – predates widespread AI and is broadly covered by existing legislation. AI-generated harmful content is qualitatively different: it manufactures harm at scale and lowers barriers to real-life violence, as actors exploit grey areas that existing instruments are not equipped to address.
The EU’s Directive on Combating Violence against Women (VAW Directive), the Victims' Rights Directive, the Anti-Trafficking Directive and the Action Plan Against Cyberbullying provide a starting point to treat new AI-enabled forms of violence. In 2024, the VAW Directive criminalised non-consensual AI-generated deepfakes. The Anti-Trafficking Directive acknowledges how digital tools are used to recruit, advertise and control victims – risks that AI intensifies. The Action Plan Against Cyberbullying also recognises the growing threat posed by generative AI in the context of gendered violence.
However, the EU’s new Gender Equality Strategy still frames AI primarily as a tool used to produce and disseminate harmful content, rather than as a systemic issue that can reproduce and amplify gender inequalities by design. Anchored in the Digital Services Act (DSA), the EU’s platform moderation toolkit, the current approach focuses on treating the outcomes of AI-facilitated gender violence rather than preventing it upstream.
Two years after the DSA’s entry into force, six member states still lack trusted flaggers – independent bodies tasked with alerting platforms to illegal content. The Gender Equality Strategy therefore calls for boosting their capacity alongside a “structured regulatory dialogue with very large online platforms on gender-based cyber violence.” While the DSA can enable the removal of deepfakes, stopping them at the source means governing AI throughout its lifecycle – from data collection to model development, training and deployment.
While the AI Act mentions gender equality, it does not acknowledge gendered power structures as a fundamental influence on how AI systems are designed, trained and deployed, nor AI’s societal implications. The Gender Equality Strategy missed an opportunity to call for gender mainstreaming in the Act's implementation, particularly given that implementation and standard-setting processes are ongoing.
Elements of the AI Act Code of Practice on Transparency, which governs the labelling of AI-generated content, could better address gender equality: the second draft does not mention gender at all. Similarly, the risk taxonomy under the Code of Practice on General-Purpose AI fails to classify gender-based discrimination and gender-based violence as systemic risks. While model providers must consider discrimination and fundamental rights impacts, they themselves are responsible for deciding, on a case-by-case basis, whether these impacts pose systemic risk – with no external review until the EU AI Office reviews the Model Report. This tiered approach drew criticism throughout the drafting process. Ongoing standard-setting procedures offer an opening: gender-sensitive standards should introduce audit methodologies to check for bias, in line with the EU Charter on Fundamental Rights.
The AI Act's structural limitations also matter here. Unlike the General Data Protection Regulation, which is rooted in human rights law, the AI Act is grounded in product safety legislation, limiting its capacity to shape societal outcomes. Addressing this gap requires stronger transparency measures, allowing users and regulators to challenge AI-generated content or decisions that perpetuate discrimination before harm occurs. Building explainable – and thus contestable – AI, rather than black-box systems, serves broader democratic goals by uncovering the power structures embedded in algorithmic systems.
Another opportunity lies in high-quality data, as recognised by the Common European Data Spaces. Representative data norms and intersectional impact assessments, beyond narrowly defined ‘high-risk’ applications, may help reduce AI-facilitated gender violence upstream. Standard-setting procedures are developing shared methodologies for dataset quality and governance to detect bias, yet their implementation remains uncertain.
Beyond digital policy
Attitudinal backsliding on gender equality, particularly among boys and young men, underscores this issue’s urgency. 30% of EU citizens believe women should accept sexist or abusive online responses, while 25% believe women exaggerate claims of rape or abuse. These trends are boosted by AI-driven recommender systems that reward attention and create echo chambers of misogyny. This reinforces the case for gender-sensitive upstream interventions in AI and data governance, alongside broader equality policies that shape the societal context in which technology develops. Only by addressing both can technology become more inclusive.
Viktoria Henkemeier is a Junior Policy Analyst in the European Policy Centre’s Health and Social Resilience Programme.
Samuel Goodger is a policy analyst in the European Policy Centre’s Health and Societal Resilience Programme.
The support the European Policy Centre receives for its ongoing operations, or specifically for its publications, does not constitute an endorsement of their contents, which reflect the views of the author only. Supporters and partners cannot be held responsible for any use that may be made of the information contained therein.
