The Evolution of Propaganda Detection
What started as manual fact-checking and rule-based keyword detection has grown into a sophisticated research field incorporating machine learning, natural language processing (NLP), and large language models (LLMs).
Early studies focused on simple classification tasks, distinguishing propaganda from neutral text.
Over time, researchers moved toward more advanced AI-driven methods that could identify specific propaganda techniques, cross-linguistic patterns, and multimodal disinformation (text, images, and video).
This article follows the progression of propaganda detection research, starting from traditional machine learning approaches in 2020, through the rise of Transformer-based NLP models like BERT in 2023, to the emergence of GPT-4 and multimodal detection in 2024.
By looking at key findings and technological breakthroughs, we’ll examine what works, what doesn’t, and what’s next for this rapidly evolving field.
Early Machine Learning Approaches (2020)
The first attempts at automated propaganda detection relied on traditional machine learning models that categorized news articles as either propaganda or non-propaganda based on linguistic features.
These early models were built using supervised learning techniques, where researchers trained algorithms on labeled datasets of known propaganda and non-propaganda articles.
A 2020 study on propaganda detection in text data by Oliinyk et al. focused on building binary classification models for propaganda identification.
The researchers worked with a dataset of 35,993 English-language news articles, labeled as either propaganda or non-propaganda.
Their approach involved:
- Preprocessing text data: Removing stop words, punctuation, and unnecessary symbols to clean the text.
- Feature extraction: Using TF-IDF (Term Frequency-Inverse Document Frequency) to quantify the importance of words in a document.
- Machine learning classification: Applying logistic regression and support vector machines (SVMs) to classify articles based on their linguistic patterns.
Key Findings
- The best-performing model achieved an accuracy of 94.3%, suggesting that basic machine learning techniques could effectively distinguish propaganda from regular news.
- However, the model struggled with subtle propaganda techniques such as loaded language and emotional framing, which required contextual understanding beyond simple word frequencies.
- The dataset was imbalanced, with far more non-propaganda articles (31,972) than propaganda articles (4,201). This imbalance meant that the model was better at identifying neutral news than propaganda, limiting its real-world effectiveness.
Challenges & Limitations
- Lack of contextual awareness: These models couldn’t understand the deeper meaning of phrases, sarcasm, or rhetorical devices.
- Limited adaptability: The approach was heavily dependent on word choice, making it vulnerable to paraphrased or reworded propaganda.
- English-only focus: The study worked exclusively with English-language content, leaving non-English propaganda detection unaddressed.
Despite these limitations, this study laid the groundwork for later, more sophisticated approaches.
As propaganda tactics became more advanced, researchers realized they needed stronger AI models that could analyze language in a more nuanced way.
This led to the next major breakthrough: Transformer-based NLP models like BERT.
While these early models laid the foundation for automated propaganda detection, they struggled to capture the full complexity of persuasive manipulation.
This led researchers to seek a more advanced solution—one that could understand not just words, but the deeper meaning behind them.
The Rise of Transformer-Based Models (2023)
By 2023, researchers had moved beyond traditional machine learning and adopted Transformer-based NLP models, which revolutionized propaganda detection.
Unlike earlier approaches that relied on word frequency and basic classification, Transformers allowed models to understand the deeper context and structure of language, making them far more effective at spotting propaganda techniques.
A 2023 study on multilingual propaganda detection by Solopova et al. introduced a BERT-based approach to identifying pro-Kremlin narratives across multiple languages (English, Russian, Ukrainian, Romanian, and French).
This was a major step forward, as it addressed one of the biggest weaknesses of earlier models: the inability to detect propaganda across different linguistic and cultural contexts.
How BERT Improved Propaganda Detection
- Contextual Understanding: Unlike TF-IDF models, which treated words as independent units, BERT could analyze entire sentences in context, making it much better at detecting subtle propaganda tactics like misleading framing and emotional appeals.
Multilingual Capabilities: The study used a multilingual BERT model trained on propaganda from multiple countries, allowing it to identify patterns that transcended language barriers.
Feature-Based vs. Neural Network Approach: The researchers compared two different methods:\n 1. Handcrafted linguistic features (negations, pronouns, emotional markers). \n 2. BERT-based classification, which learned patterns directly from text.
Key Findings
- BERT significantly outperformed traditional feature-based models, achieving an F1-score of 92%, compared to 73% for decision trees and 87% for support vector machines (SVMs).
- The model successfully identified propaganda in multiple languages, proving that propaganda detection was no longer an English-only task.
- However, it struggled with short-form content, such as Telegram posts, which lacked enough textual context for accurate classification.
- When tested on French-language news, performance dropped to 50% F1-score, showing that BERT still required more fine-tuning for underrepresented languages.
Challenges & Limitations
- Data Dependency: While BERT models are powerful, they require large amounts of labeled propaganda data, which can be difficult to collect.
- Struggles with Short Texts: Many propaganda messages on social media and Telegram are brief and ambiguous, making them harder for models trained on full-length articles.
- Computational Cost: Transformer-based models require significant computing power, limiting accessibility for smaller research teams.
Despite these challenges, this research marked a turning point in the field. For the first time, AI was able to detect propaganda across multiple languages, with high accuracy and contextual awareness.
However, researchers were already looking ahead: could even more advanced AI models, like GPT-4, push the field further?
The Era of Large Language Models (2023-2024)
As Transformer-based models like BERT demonstrated their ability to identify propaganda across multiple languages, researchers began experimenting with even more powerful AI systems—Large Language Models (LLMs) like GPT-3 and GPT-4.
Unlike previous models, which relied on predefined classification tasks and labeled training data, LLMs could adapt dynamically to new content, making them promising candidates for detecting evolving propaganda tactics.
A 2023 study by Sprenkamp et al. examined whether GPT-3 and GPT-4 could outperform traditional NLP models in detecting propaganda.
Instead of training a model on labeled datasets, the researchers used prompt engineering—designing instructions that guided the LLM to analyze news articles and flag propaganda techniques.
How GPT-3 and GPT-4 Changed Propaganda Detection
- Zero-shot & Few-shot Learning: Unlike BERT, which needed a large training dataset, GPT-4 could classify propaganda with little or no prior training simply by being given well-structured prompts.
"Chain of Thought" Reasoning: One of the key innovations tested was prompting the model to explain why a news article might contain propaganda, making its decisions more transparent and interpretable.
Adaptability: GPT-4 showed the ability to detect propaganda in real-world news articles, even those it had never seen before, thanks to its broad training data.
Key Findings
- GPT-4 matched or outperformed previous state-of-the-art models (like RoBERTa) in detecting propaganda techniques across 14 categories.
- Performance varied based on prompt design: When given clear step-by-step reasoning instructions (a "Chain of Thought" prompt), GPT-4 achieved higher precision (56.86%) but lower recall (57.82%).
- GPT-3 struggled compared to GPT-4, showing issues like hallucinations (making up false claims) and inconsistency in labeling articles.
- GPT models performed worse than fine-tuned BERT-based systems on structured datasets but were more flexible in analyzing real-world content.
Challenges & Limitations
- Lack of Stability: Unlike fine-tuned models, which produce consistent classifications, GPT models sometimes gave different answers when asked the same question multiple times.
- Hallucination Risks: GPT-3 in particular had a tendency to invent propaganda claims, making it unreliable without human oversight.
- Bias & Ethical Concerns: Since LLMs are trained on vast amounts of internet data, they can absorb and reinforce biases, leading to potential misclassification.
- Computational Cost: Running GPT-4 for large-scale detection remains expensive, making it inaccessible for many researchers.
Despite these challenges, LLMs introduced a new way of thinking about propaganda detection.
Rather than relying on rigid, pre-trained classifiers, GPT-4 showed that AI could be guided to analyze propaganda dynamically.
However, even with this advancement, a major problem remained: the rise of multimodal disinformation, where propaganda was no longer just in text but also in images and videos.
The Future of Multimodal Propaganda Detection (2024)
As propaganda techniques continue to evolve, researchers are facing a new challenge: propaganda is no longer confined to text-based narratives.
Increasingly, state-backed disinformation campaigns are using images, videos, deepfakes, and AI-generated content to manipulate audiences.
In response, researchers have begun integrating multimodal AI models, capable of analyzing text, visuals, and metadata simultaneously.
A 2024 study by Danylyk and Vysotska explored how machine learning, sentiment analysis, and multimodal AI could be used to detect propaganda in real-time.
Their research marked a significant shift from purely textual analysis to an integrated approach combining Natural Language Processing (NLP), image forensics, and video analysis.
How Multimodal AI is Changing the Game
- Text + Image + Video Analysis: Rather than analyzing propaganda in isolation, new AI models can cross-check narratives across multiple formats, detecting inconsistencies between text-based claims and visual content.
Sentiment & Emotion Detection: Using sentiment analysis, the study found that propaganda articles were more emotionally charged than non-propaganda content.- Propaganda news had an average emotional intensity of 0.151, compared to 0.116 for neutral news.
- Subjectivity scores were higher in propaganda (0.365) vs. non-propaganda (0.283).
Deepfake & Image Manipulation Detection: AI tools are being developed to detect digitally altered content and flag synthetic media used in disinformation campaigns.
Key Findings
- Multimodal analysis significantly improves detection rates, allowing AI to identify propaganda even when the text appears neutral but the accompanying images or videos are misleading.
- Real-time AI detection is possible but requires high computational power, making widespread deployment challenging.
- Propaganda detection is becoming more of a cybersecurity issue, requiring cross-platform collaboration between social media companies, governments, and AI researchers.
Challenges & Limitations
- Computational Costs: Multimodal models are far more expensive to train and run, making real-time detection difficult for small organizations.
- Misinformation Adaptation: Propaganda networks are already adapting by using human-in-the-loop AI, which allows disinformation to bypass automated detection systems.
- Privacy & Ethical Concerns: Expanding real-time surveillance AI raises concerns about free speech, misclassification, and potential misuse of detection technology.
The Road Ahead
To fully combat modern propaganda, AI researchers, policymakers, and tech companies must collaborate.
Future propaganda detection efforts will need:
- Stronger AI models trained on multimodal datasets (text, image, video).
- Real-time monitoring systems capable of scaling across platforms like Telegram, TikTok, and WhatsApp.
- Better transparency in AI decision-making, ensuring detection models do not introduce bias or false positives.
Propaganda is no longer just a battle of words—it is a battle of information ecosystems, where AI-driven narratives, synthetic media, and real-time misinformation spread faster than ever.
The next frontier of research must address how to counteract these evolving tactics while protecting digital freedoms.
Conclusion & What Comes Next
Over the past few years, propaganda detection research has rapidly evolved, moving from basic machine learning classifiers to sophisticated multimodal AI systems capable of analyzing text, images, and video.
While each technological leap has improved our ability to detect propaganda, the disinformation landscape continues to adapt, forcing researchers to stay ahead of increasingly complex tactics.
Key Takeaways from the Evolution of Propaganda Detection
- 2020: Early Machine Learning Models
- Used logistic regression and SVMs to classify news as propaganda or non-propaganda.
- Limitations: Struggled with contextual understanding, lacked multilingual capabilities.
2023: Transformer-Based NLP (BERT)- Improved contextual detection and allowed for multilingual analysis.
- Limitations: Required large datasets, struggled with short-form content like Telegram posts.
2023-2024: Large Language Models (GPT-4)- Introduced prompt engineering and zero-shot learning, reducing reliance on labeled data.
- Limitations: Prone to hallucinations, inconsistent classification, and computationally expensive.
2024: The Rise of Multimodal AI- Combined text, image, and video analysis to detect deepfake-enhanced disinformation.
- Limitations: Requires high computational power, raises privacy concerns.
The Road Ahead
Despite these advances, the fight against propaganda is far from over.
To build effective, scalable solutions, researchers and policymakers must:
✅ Improve Real-Time Detection – AI systems must process information as it spreads, not just analyze it after the fact.
✅ Enhance Multimodal AI Capabilities – Future models must integrate text, images, video, and metadata for more accurate propaganda detection.
✅ Address Ethical & Privacy Concerns – Avoid overreach in surveillance while ensuring bias-free AI decision-making.
✅ Encourage Collaboration Across Platforms – Social media companies, governments, and research institutions must work together to track coordinated disinformation campaigns.
Final Thoughts
Propaganda detection has come a long way in just a few years, but the battle between AI-driven misinformation and AI-driven detection is only beginning.
The next decade will determine whether technological progress can outpace the rapidly evolving tactics of disinformation networks—and whether society can strike the right balance between mitigating propaganda and preserving free speech.
"As AI advances, so too will the tactics of those who seek to manipulate information. The battle between disinformation and detection is a technological arms race—one that will define the future of truth itself."
Sources & Further Reading
For those interested in deepening their understanding of propaganda detection, here are the key studies referenced in this article:
- Propaganda Detection in Text Data Based on NLP and Machine Learning (2020) – Oliinyk et al.
- Focus: Early machine learning models (logistic regression, TF-IDF) for binary classification of propaganda.
- Findings: 94.3% accuracy, but struggled with contextual meaning and subtle propaganda techniques.
- Full paper
- Automated Multilingual Detection of Pro-Kremlin Propaganda (2023) – Solopova et al.
- Focus: BERT-based NLP model for detecting pro-Kremlin narratives across English, Russian, Ukrainian, Romanian, and French.
- Findings: 92% F1-score for BERT models, but struggled with short-form content and some languages.
- Full paper
- Large Language Models for Propaganda Detection (2023) – Sprenkamp et al.
- Focus: Using GPT-3 and GPT-4 for propaganda detection with prompt engineering.
- Findings: GPT-4 matched state-of-the-art models, but suffered from hallucinations and inconsistent labeling.
- Full paper
- Information Technology for Detecting Fakes and Propaganda (2024) – Danylyk & Vysotska
- Focus: Multimodal AI, combining text, image, and video analysis for real-time disinformation detection.
- Findings: Propaganda content had higher emotional intensity and subjectivity; AI struggled with deepfake detection at scale.
- Full paper
