Socioeconomic Threats of Deepfakes and the Role of Cyber-Wellness Education in Defense

Due to the limits of science and its steep learning curve, we must rely on the expertise of others to develop our knowledge and skills.²⁶ Toward this end, social media platforms have revolutionized how netizens—users who are actively engaged in online communities—gain knowledge and skills by facilitating the exchange of costless information with the public (for example, followers or influencers). Businesses around the world also use these platforms along with tools based on generative artificial intelligence (GenAI) to craft synthetic media, hoping to grow revenue by attracting more customers and improving their online experience.²⁸

These GenAI tools can also be leveraged by malicious actors, who create deepfakes—content that has been digitally manipulated to convincingly imitate people or things, often to portray events that have not actually happened. Such synthetic media may also contain partial facts to mislead viewers, often called misinformation. But public awareness of these threats is lagging. In a 2022 survey conducted by iProov¹⁵ of consumers (n = 16,000) from eight countries, only 71% knew what a deepfake video is and only 57% could recognize the difference between a real and an artificially crafted video. According to a poll report published by the New York Post,³¹ a survey of 2,000 registered U.S. voters showed not only that participants are increasingly pessimistic about the use of deepfakes in political advertising, but also that they could not distinguish between AI-generated and human-created content.

To highlight the danger of mass-crafted spurious information, in June 2023 a group of two engineers used open source GenAI models to develop CounterCloud,^a an experimental “propaganda machine.” In an exemplary test case, the project’s GenAI-crafted deepfake texts^1,32 (for example, 50 tweets and 20 news articles daily) were seen as convincing 90% of the time. The primary goal behind this project was to enlighten people about how effortless it is to weaponize GenAI to propagate deliberately manipulated content, called disinformation, on a global scale. In addition, the creators claimed that by spending approximately $4,000 per month, anyone can generate more than 200 articles per day. This is enough content to counter the efforts of more than 40 news outlets—without requiring any human interaction—and could be enough to influence an election.

In addition to disinformation and misinformation, there is also malinformation, forms of deepfake threats³³ based on manipulated facts intended to harm targeted victims (for example, cyberscamming¹⁹ and cyberbullying³⁴). Netizens must therefore be alert to these suspicious activities, whereby malicious actors can use malinformation to target them through vulnerabilities in their decision making processes (for example, cognitive biases, which can lead to poor judgment).

Socioeconomic Threats of Deepfakes

As shown in recent cases reported by mass media and law enforcement agencies,³⁴ scammers can deceive victims by creating and deploying deepfake content (for example, voice and/or video) that poses as a colleague, family member, or a celebrity known to the target group or individual. When netizens lack the necessary awareness, knowledge, and skills to identify deepfakes, they may eagerly trust and be fooled by this synthetic content, leading to unwise actions and/or financial losses. For example, in a recent cyberscamming incident using GenAI tools, $25.6 million was stolen from a multinational firm in Hong Kong by tricking an employee into believing that the CEO requested the funds transfer via a video call.⁸ Similarly, if a victim clicks on the link connected to a deepfake ad and transfers some money or cryptocurrency, they may fall into a scammer’s phishing trap.¹⁹ Recently, in a consumer alert issued by the U.S. Federal Trade Commission (FTC),²⁷ an education specialist stated that voice-cloning scams in the guise of family emergency calls are rising dramatically. To gain the victim’s trust, a scammer simply needs to find a short voice recording of their family member—often available on a social media profile—and then apply a GenAI tool (for example, the LivePerson voice AI chatbot¹⁴) to create a deepfake voice recording that is virtually indistinguishable from the real one. To avoid these voice-cloning traps, potential victims should not trust these messages and should confirm their authenticity³⁴ by applying cyber-wellness knowledge, such as calling the claimed person and asking for verification.

In another form of cyber scam, crafted deepfake videos of celebrities were widely spread on social media platforms to promote services linked to phishing websites (for example, the Quantum AI Elon Musk trading bot¹⁹). Moreover, in a June 2023 public alert,³⁴ the U.S. Federal Bureau of Investigation (FBI) announced that they had received reports on cybercriminals deploying GenAI tools to create artificial porn videos and photos of underage victims. The offenders sent the generated content directly to victims for cyberbullying or sextortion purposes. To combat these unprecedented threats, the U.S. National Center for Missing and Exploited Children has built an online platform called Take It Down,^b which provides free services to help victims stop and prevent the spread of these videos and images, particularly sexually explicit content with underage children. 

These cyber threats are an ongoing worldwide phenomenon with significant implications for many areas, including crypto market trading, elections, health, and education. In a recent regulatory action,³⁵ the U.S. Federal Communications Commission (FCC) declared that unwanted GenAI-crafted robocalls and robotexts should be banned by law. It also closed a comment period on a public citizen’s petition for a new rule concerning the application of deepfakes in election advertisements. This legislative action was initiated after the propagation of a robocall on January 21, 2024, in which GenAI-created audio impersonating President Joe Biden discouraged citizens from voting. It is expected that once this petition is approved at the national level, such regulations will be recommended internationally under the aegises of organizations such as the European Union and the United Nations, as well as tech-related consortiums like the Christchurch Call.^c

Educational Aspects

While the full dangers of these technologies are beyond anyone’s imagination, GenAI tools also have some legitimate uses. Netizens can easily access GenAI software for free or pay a subscription fee for various applications (see Table 1). These GenAI tools provide opportunities to innovate and reform industries, such as education and advertising. At the same time, they can negatively affect individuals’ lifelong learning and can have severe consequences for their critical and creative thinking skills. A recent study showed that OpenAI’s GPT-4 could achieve high scores on standardized tests, such as 99% on the GRE Verbal and 89% on the SAT Math,²⁴ due to it applying more collaborative, creative, and efficient transformers compared to previous versions. In addition, a survey study showed that more than 51% of students believed applying GenAI software such as ChatGPT to pass exams or prepare multimedia assignments is technically cheating.¹² 

Public usage of such unaccountable and unexplainable²⁸ GenAI tools has given rise to societal and governmental concerns over why they are easily accessible to everyone. For instance, several countries recently banned ChatGPT in public universities by blocking access to it through their networks. They also ruled that using GenAI tools to prepare assignments would be considered academic misconduct. This includes at least five Australian states and eight elite universities in the Russell Group in the U.K., including Oxford and Cambridge.⁶ But since GenAI tools are rapidly evolving, crafted outcomes are becoming impossible to distinguish from actual human-generated media. Consequently, regulatory agencies and industrial sectors must jointly develop educational programs to guide people in making safe decisions when dealing with deepfake content. In an exemplary policy action, Italy’s Data Protection Agency temporarily restricted ChatGPT’s services. Subsequently, to incorporate this criticism, OpenAI agreed to perform a series of updates to its online privacy policies and notices, such as optionality, security, and transparency.² However, these regulatory policies do not provide sufficient guidelines for netizens to learn how to decide and act safely when facing suspicious media.

To address this issue, the science of cyber-wellness education (CWE) uses standardized guidelines to teach netizens how to protect themselves and stay safe while interacting in cyberspace.¹⁶ Over the past few decades, the European Union^d and many individual countries (for example, the U.S.^e and the U.K.^f) have developed digital literacy and/or CWE programs (see Table 2), actively using them in educational institutions to reduce the risk of cyber threats. Numerous online scams recently caused netizens to lose millions of dollars,¹⁹ mainly due to various forms of deepfake cyber threats. The current CWE programs are not sufficiently updated to protect against these threats, making them inadequate for netizens, even if they do receive CWE training. Moreover, these programs often focus on training students and pay less attention to regular consumers of digital content on the Internet.

As outlined in Table 1, GenAI tools and social media platforms allow people to easily craft and spread propaganda and manipulate information, often using intentionally crafted deepfakes to deceive their target audiences. In addition to these intentional deepfakes, traditional media outlets, online libraries, and social media platforms can be both perpetrators and victims of unintentionally shared fabricated content. Sometimes users of GenAI tools will write prompts, over-trust the result, and unintentionally use it for sensitive purposes.³³ This is not a safe practice, as all output from GenAI tools should be validated due to the possible biases and hallucinations²⁸ of the underpinning models. For instance, a New York attorney used ChatGPT to conduct legal research to represent a client’s injury claim. While overseeing the suit, the federal judge noticed that six citations quoted in the attorney’s brief were falsified.^g In this hallucination scenario, ChatGPT made up the cited cases and even asserted that they were accessible in major legal databases.

GenAI Distortion Risks in Society

GenAI tools’ ability to craft seemingly realistic media by combining fact with fiction may distort netizens’ perceptions of the content’s reality and its associated dangers—a phenomenon called GenAI distortion risks.³⁸ One way to reduce these risks is to learn how to apply prompting knowledge and skills correctly, as well as understand decision-making biases that help people design and refine effective prompts for crafting more accurate outcomes from GenAI tools and avoiding deepfakes.³⁶ Technically, this involves crafting or refining a query: one or more connected sentences given to GenAI tools that may result in producing more accurate and relevant content. The validity and quality of output from GenAI tools are influenced by two independent factors, the lack of which can negatively affect accuracy:¹⁷

• Coherence: the quality of a prompt being logically close to what a user expects to gain as content.

• Relevance: the availability of relevant data that can be processed and collected by Web-scraping approaches through GenAI tools.

Enhancing netizens’ knowledge and skills on how to craft and refine prompts can lead to more accurate results and, eventually, better content being produced by large language models (LLMs), large vision language models (LVLMs), music-specialized large language models (MLLMs), speech large language models (SLLMs), and large multimodal models (LMMs). Since GenAI tools deploy Web-scraping techniques to collect relevant data from resources accessible on the Internet, they are somewhat limited. Therefore, if the prompts fed to the GenAI models contain insufficient or incorrect data, they may produce biased content containing irrelevant concepts. Technically, these biased outcomes are caused by three data-driven biases:³⁷

• Problematic training data unintentionally perpetuates biases that may link inaccurate or incorrect data to specific subjects.

• Accessibility to real-time data is a significant limitation, which can cause recency bias.

• Underpinning models can be biased by design and then embed biases into the associated data, which inevitably produces unfair outcomes.³⁹

In addition to these data-driven biases, there are three decision-making biases to which netizens are vulnerable:

• The tendency to over-trust AI tools, which leads to a false confirmation.

• Optimizing prompts, which might steer GenAI models to adapt their replies toward netizens’ objectives and can cause feedback-loop bias.²⁸

• Using GenAI-crafted content for commercial purposes (for example, journalism or scientific publications), which might violate the ownership rights of similar content and can be interpreted as anti-copyright bias.

Practically, data-driven biases may cause GenAI hallucinations that result in unintentional deepfakes if netizens trust this content while also being subject to decision-making biases.

To reduce the possibility of creating invalid content using GenAI tools, we will now discuss a prompting protocol that can guide netizens to understand the validity and reliability of GenAI-crafted information. Crafting content requires well-structured human-GenAI interactions to operate accurately and achieve high-standard results. Therefore, to write an effective prompt, the following five elements must be considered:

• Relevance: An effective prompt must be relevant to the expected task by providing adequate information to direct the GenAI tool to make precise predictions.²¹

• Diversity: An effective prompt should contain a range of specific details to ensure the generalization of the integrated GenAI models to new data.⁹

• Consistency: An effective prompt should follow a correct, consistent format to guide the GenAI models to successfully perform the requested tasks.⁹

• Simplicity: An effective prompt should be concise and coherent to reduce the possibility of confusion for GenAI models when processing the requested tasks.²⁸

• Clarity: An effective prompt should be clearly and unambiguously defined so that the task concept meets the best possible quality of transparency that the GenAI model can perform.

Effective prompts can result in more informative and accurate GenAI outcomes, while defectively designed ones may result in irrelevant and confusing responses.⁴ In addition to considering the above key features, netizens must learn that using GenAI services responsibly is crucial if they wish to apply the full capabilities of these tools without forfeiting their integrity and ingenuity. But since GenAI tools technically generate content by interpreting a given prompt and reproducing new information based on the trained data, there is a possibility they may construct incorrect or misleading results (hallucinations).²⁵ This is a common problem with GenAI tools, as they are developed to craft new forms of content and can sometimes create plausible-sounding content that might actually be incorrect.²³ Surprisingly, most corporate systems (or service providers) are aware of such constraints and are gradually advancing and optimizing their GenAI models to enhance the accuracy of their outputs. Since the capability of GenAI tools depends on interpreting netizens’ prompts, they should be treated as collaborative entities in a knowledge co-construction process.³⁰ To enrich human-GenAI knowledge co-construction, we extended the prompting protocol introduced by Robertson et al.²⁸ It helps netizens to write more detailed and effective prompts and includes a question-based framework to validate their accuracy.

In constructivist theory, learners construct knowledge instead of passively receiving information. Commonly, they will experience the environment around them and reflect upon their observations, and eventually form their perceptions by interpreting and incorporating new concepts into their preexisting information. Similarly, when given a prompt, a GenAI tool crafts relevant content by predicting the new concept through the interpretation process.²⁸ In the context of human-GenAI prompting, an iterative refinement process is necessary to improve the dialogue between GenAI tools and netizens, formulating a cohesive reply to an inquiry as a problematic task. Considering the above constructivist perspectives,²⁸ the prompting process can therefore be formed as an iterative, circular-based protocol with three correlated dimensions and nine steps that can coherently facilitate knowledge co-construction and its validation by empowering human-GenAI dialogue (see Figure 1).

Dimension 3 (evaluational).  Evaluating the prompt drafted in the previous dimension requires three more challenging steps (7–9) to validate, refine, and reassess it to gain the desired outcome.²⁸ During the testing and validation step, the user must verify and ensure the reliable accuracy of the GenAI-crafted content according to their expected objectives in the prompt, considering possible biases.³ In the refinement and reclarification step, the user is involved in fine-tuning the structure of the prompt by increasing the clarity of the objectives and elaborating on complex expectations. When (re)assessing GenAI-crafted content based on a given prompt for accuracy and validity,²⁰ the user must not only determine whether it is the best match for their expectations, but also consider the possibility of biases caused by human-GenAI interactions.³⁷ To validate the content’s reliability, netizens can obtain its rationale by asking six questions to address those biases (see Figure 2). However, with some sensitive topics (for example, scientific tasks¹⁸), the trustworthiness of GenAI results cannot be confirmed unless relevant experts review them.²⁹ For example, in a recent study,²⁹ 91 dichotomous (yes/no) questions about endodontics were designed and divided into three difficulty classifications. First, researchers randomly selected 20 questions from each class. Then they asked ChatGPT for answers to those 60 questions, while also getting responses from two endodontic experts, who answered the 60 questions separately. Finally, they conducted a statistical analysis via the SPSS tool to evaluate the consistency and accuracy of the experts’ answers compared with the replies generated using ChatGPT. They concluded that ChatGPT had an average accuracy of 57.33% on the selected questions. This experiment highlights the fact that netizens must validate the accuracy of GenAI outcomes when using them for specialized applications while not being overseen by relevant experts.

As depicted in Table 3, we used the prompting protocol to construct a well-designed and well-structured prompt to find and validate an efficient way of implementing quantum-resistant TLS 1.3 in a Java-based Android application. In our experiments, despite posing a straightforward question in dimension 1, ChatGPT-4o created two responses, of which the second one, which considered the experts’ point of view, was correct. By following the steps of the next two dimensions, we optimized and validated the results of ChatGPT-4o while revising the prompt two more times to obtain valid and unbiased content. Note that the validation questions are customized based on the question-based framework according to expectations from the GenAI tool. After that, we recruited n = 10 volunteers (for example, DevOps engineers) to test the sample in Table 3 by following the prompting protocol’s three dimensions. Eventually, all volunteers pointed out that they learned not to trust the first output of ChatGPT-4o and to verify it by optimizing the prompt and validating the results (for example, source code). Although our test focused on a specific topic, it can be extended by creating more generalized samples in CWE programs to simplify the understanding of the prompting protocol for all netizens. Technically, the GenAI tools still have negative cyberpsychological impacts (for example, distortion of netizens’ critical thinking and digital trust) and technological limitations, such as hallucinations, as well as content integrity, privacy, safety, copyright, and ownership issues that have yet to be addressed.⁴⁰ At the same time, as depicted in Table 2, current CWE programs partially cover the knowledge and skills necessary for all types of netizens to mitigate the emerging risks in society (for example, cryptocurrency heists and deepfake-driven social engineering attacks). Therefore, educational institutions must upgrade existing CWE programs by integrating more relevant creative-thinking knowledge and skills (such as the above-suggested prompting protocol) and defense mechanisms (such as Take It Down) to help netizens address emerging cybersecurity risks.

Table 3.  A sample of a well-designed prompt using the three-dimensional prompting protocol.

Steps	Prompting Processes	Summary of ChatGPT-4o Output	Validation Question
Dimension 1 (Conceptual; steps 1,2,3)	How can I implement a quantum-resistant TLS protocol in a Java- based Android application?	It generated a list of guidelines and source codes that suggest two libraries, such as OQS-OpenSSL and Bouncy Castle.	Which library is the best option for implementing the quantum-resistant TLS 1.3?
Dimension 2 (Structural; steps 4,5,6)	What Java library can be used as an efficient way to implement a quantum-resistant TLS protocol in an Android application? Please give me optimized sample source codes to implement it.	It crafted a list of instructions and source codes on how to implement the quantum-resistant TLS protocol using the Bouncy Castle library.	Why is the Bouncy Castle Library the most efficient one?
Dimension 3 (Evaluational; steps 7,8,9)	Why can the Bouncy Castle library efficiently implement a quantum-resistant TLS 1.3 protocol using Java in an Android app? Please give me optimized source codes to implement it and provide me with some references from developer.android.com to validate why it is efficient.	It provided five reasons for why the Bouncy Castle Library is an efficient way to implement a quantum-resistant TLS 1.3 protocol in an Android application. Also, it refers to three references that support the efficiency factors from Google’s official source (developer.android.com) for Android application development.	Have my prompt and subsequent interactions effectively influenced ChatGPT’s output toward better knowledge co-construction accuracy?
Note that validation questions vary for different prompting tasks. These questions must clear possible biases in the GenAI-crafted content, especially when the output involves sensitive scientific or industrial concepts.

Adaptive E-Governance

Understanding why netizens trust deepfakes is a complex issue.²² The current literature on public engagement emphasizes that people refuse to accept scientific evidence when it risks their profits or questions their beliefs.²⁶ Over the past few years, much damage has been caused by proof-of-work blockchains (for example, Bitcoin) that negatively affect the climate, human mortality, and personal finances (for example, crypto heists via deepfake phishing attacks¹⁹). For instance, according to the Chainalysis Crypto-Crime report,⁷ more than $24.2 billion was received from illicit cryptocurrency addresses in 2023. This was a significant drop compared with 2022, during which $39.6 billion was transferred, highlighting that criminal activities have declined despite dark web markets and ransomware attacks increasing dramatically. Nevertheless, these numbers provide further evidence that these unprecedented cyber threats must be investigated and integrated into CWE programs. This, in turn, involves developing, upgrading, and teaching defense mechanisms and safe, accountable ways of conducting human-GenAI interactions (for example, the suggested prompting protocol outlined earlier) at all levels of society: individuals, families, and schools. Enhancing the efficiency and effectiveness of CWE programs therefore requires more proactive and strategic action from educational institutions as well as society, industry, and government.¹⁶

Figure 3 depicts a circular-based puzzle representing collaboration on upgrading the effectiveness of CWE programs among various groups. In this holistic, usable cybersecurity management framework, six groups of actors play a significant role by sitting at a decision-making table, investigating the problem, and contributing to the primary goal: mitigation of emerging cybersecurity risks, where players’ actions (in)directly affect their neighbors’ decisions and subsequent activities. The roles of each of these six groups are discussed below.

Figure 3.  Circular puzzle-based framework for enhancing the effectiveness of CWE programs.

Third-party companies provide innovative GenAI-based services typically by only partially considering their impacts on the public, most likely because their priority is attracting consumers to increase revenue. For example, GenAI tools are easily accessible to everyone, enabling netizens to construct content without considering ethical and social consequences,²⁴ which require regulation and control by standards organizations. At the same time, these companies are responsible for deploying explainable GenAI models and ensuring fairness while processing netizens’ data according to AI laws. As such, they must update and incorporate e-governance policies to reform their services toward shaping safer human-GenAI interactions.

Social media platforms are owned by private companies like Meta and ByteDance, which offer free social networking services to billions of netizens around the world while storing and processing consumers’ sensitive data and activities.²⁶ Companies’ usage of netizens’ data must comply with international laws, such as the EU’s General Data Protection Regulation (GDPR)¹¹ and China’s Data Security Law (DSL)¹⁰ if they want to continue to do business in those regions. This highlights the need for research and development to reduce the potential risks of their services and prevent them from being fined under the laws. They should also develop and integrate deepfake detection algorithms in their platforms to enhance netizens’ awareness of GenAI-fabricated media.

Netizens are Internet users who surf and engage through online communities, where intruders target them via cyberattacks. They should be exposed to CWE programs and trained accordingly, as it is their social responsibility to learn possible mechanisms and tools to deal with today’s cybersecurity risks.¹⁶ Learning defense mechanisms and critical thinking skills from these programs can help netizens increase their awareness of deepfakes and their socioeconomic impacts as well as actively report them to relevant e-governance agencies. Similarly, parents must regularly learn these techniques by participating in CWE programs and proactively guide their children to make the right choices and practice safer Web-surfing activities.

Educational systems are the executive and research centers for organizing CWE programs and training teachers and potential netizens. They are also responsible for gradually developing effective resources and practices to mitigate possible cyberspace risks by following CWE policies. In some cases, they can propose adaptive policies for standards organizations to incorporate around the unprecedented risks of emerging GenAI-based technologies.

Standards organizations are responsible for developing enforcement policies in all technical and nontechnical fields and creating uniformity across corresponding agencies, producers, and consumers. They also publish standards guidelines for individuals and organizations to follow as responsible societal actors. Governmental agencies have statutory duties to enforce adaptive e-governance policies and standards proactively by investigating the public usage of GenAI tools and controlling the accountability of their risky services. They must also regularly adapt and introduce CWE policies to be integrated and carried out by educational systems or through mass media campaigns.

Cybercrime agencies are executive governmental organizations, such Europol’s EC3 or the EU’s EDPB, that are responsible for investigating cybercrimes, identifying the latest cyberspace risks, and addressing them with regional/international laws. In addition, they must collect netizens’ reports on their experiences with privacy violations and cooperate with educational systems to develop CWE programs based on recently discovered cyberthreats.

Robust Oversight

Here, we present three proactive recommendations for concerned policymakers that could best contribute to mitigating emerging cybersecurity risks.

Conclusion and Outlook

In this article, we investigated the global-scale role of CWE (or digital media literacy) to help netizens better understand and prepare for the threats of deepfakes and their associated risks. Unfortunately, when it comes to the development of up-to-date CWE programs and the legislation of enforcement policies and actions, policymakers and educational institutions appear to be a step behind malicious actors. To overcome this problem, regulatory agencies and educational organizations should take a number of actions. First, they should investigate the copyright and ownership issues of deepfakes more broadly to enhance defense mechanisms for netizens viewing misinformation, disinformation, and malinformation. They should also integrate recent policies, laws, and actions (for example, the EU’s AI Act) into the CWE curriculum regularly, upgrade existing programs (for example, including the prompting protocol suggested earlier), and create educational outlets for all types of netizens. Further, as a crucial part of these programs, they should expand the training of CWE curricula and monitor the regular integration of emerging socioeconomic threats of deepfakes. Also important is reducing the time between the legislation of enforcement policies and proactive actions that can be taken by any involved administrative or educational organizations. Finally, these organizations must engineer practical solutions that provide public services to recognize deepfakes and ensure the reliability of GenAI-generated content for anyone inquiring about such content.

Acknowledgments

This study was partially supported by the Ulam research grant No. BPN/ULM/2022/1/00069, funded by the Polish National Agency for Academic Exchange (Narodowa AgencjaWymiany Akademickiej (NAWA), and by U.S. NSF awards #1820609, #2114824, and #2131144.