Assessing the Quality of COVID-19 Messages on Ahvaz Jundishapur University of Medical Sciences Website: A Content Analysis

1. INTRODUCTION

The COVID-19 pandemic originated in Wuhan, China, in 2019 and quickly became a global health crisis [1-4]. Concurrently, advancements in technology have led to the development of new tools for detecting and preventing COVID-19. For instance, a study has examined innovative methods for identifying adherence to health protocols during multi-person interactions [5], while other research has focused on introducing new techniques to enhance the quality of CT images in COVID-19 patients [6, 7]. The dissemination of these findings to the public is facilitated by news media. This situation has impacted all aspects of daily life [4, 8-10] and increased media consumption as people seek reliable information to protect themselves from the virus [11]. Therefore, the news media played a critical role in shaping people's perceptions and reactions to the outbreak [12]. In this regard, a study by Vatsa et al. examining the impact of media on public perception and behavior in dealing with the Omicron variant showed that media can influence people's health attitudes and behaviors [13].

The media provided a lot of new, up-to-date, and contradicting information about different aspects of the COVID-19 disease, leading to a heterogeneous public health response [12, 14]. Reliable information is critical to motivate people to take appropriate action against the virus, promote science-based public health guidelines, and prevent the spread of false information [15]. Around 75-80% of Internet users worldwide actively search for health information online [16].

A study in the Journal of Medical Internet Research found that 2% of websites are devoted exclusively to health topics, underscoring the importance of reliable sources of information. More than 70,000 websites publish health-related information, and surveys show that most internet users rely on online resources for health information [17, 18]. In other words, virtual space plays a crucial role in our social lives and in our efforts to stay healthy [19].

Due to inadequate health systems, authorities have taken measures such as school closures, remote work, and social distancing regulations to combat COVID-19 [20]. Under such dire conditions, the public has been looking for trusted information and guidance based on scientific evidence to respond to the outbreak and slow the spread of the virus in these difficult times [21].

The COVID-19 pandemic has led to more people accessing free health information online [22]. However, it is important to evaluate the reliability and accuracy of online sources because misinformation can have serious consequences. Online sources can be valuable for learning about the pandemic, but it is critical to rely on reputable sources and critically evaluate the information received. The Internet has become one of the most important sources of health information due to its widespread distribution and use [20, 21, 23].

Although the Internet offers a wealth of health information, it is important to know that not all of it is trustworthy. Some online sources can help individuals stay healthy and manage the disease, but their reliability and accuracy are debatable [23].

Local media play a critical role in informing the public about health issues. However, they must adapt to changing media trends and provide an effective content structure that meets the changing needs of society [24]. In this way, they can fulfill their civic responsibility by providing valuable information to their communities. However, a recent study has shown that the language factor is often overlooked in the design of health messages [25]. The language used in health messages can significantly influence their effectiveness in promoting healthy behaviors and preventing the spread of COVID-19.

Media literacy is critical in today's digital age, and educators should incorporate it into educational programs to help students develop the skills necessary to create and evaluate effective media messages, including those related to health [24]. In Iran, since the COVID-19 outbreak, medical universities have provided a wealth of health information by playing a role in disseminating accurate and reliable information about COVID-19 to the public, including students, medical professionals, and the general population. The role of the media in shaping public opinion and influencing individuals' thoughts and feelings about health-related issues is crucial in our interconnected world. Quality media can lead us to meaningful conversations that benefit everyone. Therefore, it is critical to evaluate the quality of health-focused news media for accuracy, impartiality, and, relevance. It is important to analyze and critique the news we produce not only to ensure its current accuracy and effectiveness but also to learn how we acquire and use knowledge in the future.

The quality of online health information content is critical due to the varying accuracy levels of sources. Users may make uninformed decisions without a reliable method to assess their accuracy and credibility. Standardized tools like the CRAAP Test [26], AIMQ Methodology [27], information quality tool [IQT], and quality scale [QS] [28] offer significant advantages in evaluating the quality of online health information. These methods allow users to systematically assess the credibility, relevance, accuracy, timeliness, and purpose of information using a scientific approach. For instance, the CRAAP Test evaluates criteria related to the currency, relevance, authority, accuracy, and purpose of information, which can lead to more informed decision-making. Similarly, the AIMQ methodology focuses on assessing the quality of health information, helping users identify credible sources, and preventing the spread of inaccurate information. These tools not only improve the quality of health information but also contribute to enhancing media and health literacy. The choice of tool depends on the context and specific requirements of the evaluation process. Therefore, considering the prevalence of misinformation, contradictory, and non-standard information regarding COVID-19, this study assessed the quality of health messages published on the website of AJUMS. This research aimed to ensure the accuracy and credibility of the information provided to the community, assisting physicians, health professionals, and decision-makers in improving the quality of health-related information disseminated to the public, thereby enhancing public health education. Additionally, the results of this research could lead to improved communication strategies in public health and increased public awareness through reliable informational resources.

2. MATERIALS AND METHODS

3. RESULT

The percentage distribution of responses to the questionnaire items is shown in Table 1. Also, other results show that all media evaluators have experience in creating, producing, and analyzing health media. One member has a master's degree, and five others were faculty members with doctorates in health education and promotion and health promotion and educational technology.

Only two of the websites studied had archives of news related to COVID-19 media coverage. The archives were organized by topic, downloadable content, newsiness of the message, and other things.

Data analysis revealed that 73.4% (80/109) of news messages focused on the prevention of COVID-19 transmission, while only 34.9% (38/109) addressed the management and control of the disease (Fig. 1). Fig. (2) shows that 89% (97/109) of the message content was aimed at the general public, while 16.5% (18/109) was related to patients and only 14.7% (16/109) was focused on people in contact with the patient.

Approximately 42.2% (46/109) and 38.5% (42/109) of messages contained recommendations for social distan- cing and proper hand and facial hygiene, respectively. Of concern, only 16.5% (18/109) of messages recommended vaccination, and 17.4% (19/109) contained no recommendations at all (Fig. 3). Only 11% (12/109) of messages targeted guilds, while 92.7% (101/109) were general messages that did not address a specific situation (Fig. 4).

Evaluators' analysis revealed that 45% (49/109) of the messages were found to have a news tone, while 23.9% (26/109) had a persuasive tone. In addition, 9.2% (10/109) of the messages had a threatening tone, and 21% (23/109) kept a balance between persuasive and threatening elements.

The analysis shows that 46.8% (51/109) of the messages were about physical health, while 41.3% (45/109) were about social health and 22% (24/109) were about mental health. About 17.4% (19/109) of the messages were related to emotions management. Of note, about 27.5% (30/109) of the messages did not relate to any specific dimension of health.

Fleiss' kappa coefficient was utilized to assess inter-rater agreement. It extends Cohen's kappa to account for more than two raters. Based on this measure, there is a slight level of agreement regarding the clarity and comprehensibility of the questions, as well as the overall instrument score. These questions exhibit the required level of stability and certainty. In contrast, the questions on the nature of the content, scientific documents, and evidence show moderate agreement, but cannot demonstrate the required level of stability and certainty.

The results indicate that there was reasonable agree-ment among raters for questions on visual presentation and appearance of content design, suggesting a high level of confidence and consistency.

Although the Fleiss-Kappa agreement coefficient for the questions in the Strength and Scientific Precision sections indicated good agreement, the stability of the raters' level of certainty was questionable (Table 2).

4. DISCUSSION

The study evaluated the quality of COVID-19 news on the websites of Ahvaz Jundishapur University of Medical Sciences and its affiliated units. It found that only two media websites had COVID-19 news archives, which are organized historical records of an organization's performance over time. These archives are comprehensive and regularly updated, indicating a commitment to providing accurate and up-to-date information to its audience. This is important during a pandemic, as having accessible and well-maintained archives contributes to transparency and accountability in reporting COVID-19 news. The study underscores the importance of preserving and organizing information for easy retrieval and reference, especially during global health crises like the COVID-19 pandemic.

The Iranian media industry lacks proper archiving systems, forcing people to rely on memory and online searches for past news. This poses a challenge for those seeking reliable information and underscores the need for a comprehensive archiving system to preserve history and ensure accuracy. Establishing digital and physical archives can improve efficiency, facilitate research, and prevent data loss. Proper archiving is critical to historical accuracy and supporting progress.

Effective health messages can encourage healthy behaviors, but most current messages focus on COVID-19 prevention and management, with insufficient attention to disease treatment and rehabilitation methods. This is concerning, as individuals may resort to self-medication due to a lack of knowledge. Overall, more attention should be paid to proper treatment methods to improve the health and well-being of those affected by a disease [32, 33].

Self-medication can result in serious consequences such as misidentifying illnesses, delaying medical treatment, harmful drug interactions, severe adverse effects, incorrect dosage, concealing ailments, and an increased risk of substance abuse. To ensure the health and well-being of the community, it is important to address these negative consequences of self-medication [34].

The analysis found that the majority of health messages (89%) were directed at the general public, with only a small proportion targeting patients, caregivers, or those who interact with them. This is concerning as even though there is a high risk of anonymous spread of COVID-19, close contacts and caregivers of infected individuals are not getting tested for the virus. This can easily result in the virus being spread within the community, leading to an increase in the number of patients [35].

Upon analyzing the website's recommendations, it becomes clear that the majority of advice focuses on social distancing and proper hand hygiene practices. Meanwhile, nutrition, drug treatment, rehabilitation, and vaccination recommendations are considered less significant by comparison.

To prevent COVID-19 spread, all relevant guidelines must be included in educational materials. Neglecting critical strategies can harm pandemic control, so comprehensive information dissemination is crucial. Emphasizing all aspects of preventive actions maximizes impact and builds public trust [36]. Designers should consider all recommendations to increase messaging effectiveness and avoid conspiracy theories.

A lack of knowledge of proper treatment practices can lead to harmful behaviors like self-medication or reliance on unverified drugs [37]. Educating individuals on appropriate techniques is crucial to prevent such practices and promote evidence-based medical practices. Reliable information on treatment options must be available. Educating people on effective ways to address health concerns will mitigate potential risks. Improved health literacy within communities can significantly improve well-being.

In terms of the specificity of the messages for particular groups and situations, a small percentage of the examined messages targeted industries, factories, educational centers, and transportation. However, the content of the majority of the messages was not related to specific contexts or conditions. Even though experts in health psychology emphasize the importance of providing accurate information about the context and purpose of behavior and messaging when communicating health information [38]. However, many people fail to acknowledge this crucial aspect. It is imperative to understand that the context in which health information is disseminated has a significant impact on how it is perceived and adopted. Therefore, providing accurate context and messaging is crucial to improving health outcomes.

On a positive note, the published messages were strong in that they covered almost all situations, including ceremonies, celebrations, Ramadan, guilds, organizations, offices, and industries. This feature made them efficient and effective in their reach.

The analysis of the website messages found that the majority of the published content (45%) had a news-oriented aspect, focusing on presenting disease and mortality statistics and adopting a threatening approach. However, only a very small percentage of the messages integrated a combination of news, encouragement, and warnings.

Experts found that 23% of reviewed health-related messages were disrespectful. To increase effectiveness, messages should incorporate mixed content instead of being only threatening, persuasive, or respectful. A variety of different types of content make messages more engaging for audiences. Using various techniques can make our messages more impactful and effective in captivating and retaining attention [39].

The study found that the majority of messages focused on physical and social health, with less than 20% addressing psychological health. Unfortunately, the spiritual aspect was largely disregarded. Recent research has shown that the pandemic has hurt individuals' spiritual well-being [40, 41]. Therefore, it is vital to prioritize the spiritual aspect of health. Neglecting this dimension can hinder us from achieving complete physical, mental, and emotional wellness.

Regarding the credibility of the message, coordinated and consistent message content, accuracy in writing phrases, understanding and awareness of the audience, simplicity of the message, and readability, the published messages had a favorable situation.

The published messages lacked reliability, accuracy, and consistency with other media. They should have included up-to-date scientific sources, avoided contra- dictions, and used simple language. Key concepts should have been presented at the beginning, and appropriate images could have been used. The layout should have been done appropriately, with a clear section and paragraph separation for better readability. Improving these aspects could have made the message more effective (Table 2).

Effective communication through media requires more than just scientific accuracy. Clear language, attention-grabbing headlines, visuals, relatable stories, personal anecdotes, and repetition increase success and prevent belief in conspiracy theories or pseudoscience [42]. COVID-19 posed a challenge for crafting messages due to limited evidence on the novel topic, making accurate communication difficult.

According to the results of a review article, it is important to consider several factors when designing messages for behavior change. These include assessing if the target audience is already engaging in the recommended behavior, emphasizing desired outcomes and advantages of changing behaviors, raising awareness of potential harm for those who do not see the risk as high, and tailoring messages to unique needs and challenges of the target audience using an exploratory approach. By following these principles, messages can be designed that are applicable, clear, useful, precise, and effective in promoting behavior changes [43]. Therefore, it is important to consider people's perceptions and cultural backgrounds when designing and implementing health promotion interventions [44].

In general, it can be said that the scientific power and precision of the published messages are not desirable, more importantly, they often lack practical tips and fail to make a significant impact on the audience. This is even though according to the guidelines of effective communi-cation, the validity and practicality of the message are of great importance [43]. Therefore, to ensure trustworthy, timely, and factual information for the audience, decision-makers should consult with experts when crafting messages. Experts provide accurate and reliable communication that is easy to understand. They can be particularly helpful in designing public information campaigns and crisis communications. Decision-makers should consider hiring experts to help them craft messages that effectively reach their intended audience and achieve their goals.

The Fleiss kappa showed that questions about content type, scientific documentation, evidence, and accuracy lacked stability. This means the website assessment instrument needs improvement for better accuracy.

The Fleiss kappa coefficient of agreement can have a low value due to several reasons, including non-random selection of raters and the use of incompatible evaluators. These factors can lead to biased results and inconsis- tencies, which can hinder the accuracy of conclusions. To obtain accurate results when using Fleiss kappa analysis, it is crucial to adhere to specific requirements and assumptions that are fundamental and cannot be overlooked.

5. LIMITATIONS OF THE STUDY

The present study only considered the perspectives of specialists and experts and did not take into account the perceptions of message recipients. Additionally, the message design only included elements listed in Table 1 and did not explore other essential factors such as perceptive aspects, fear, and message framing [45]. To gain a more comprehensive understanding of the topic, additional research is advisable while considering the constraints of the current study.

6. STRENGTHS OF THE STUDY

This study had several strengths, including the ability to measure the impact of messages through web-based services and involvement and consultation with social and behavioral science experts, resulting in an efficient response to the pandemic. Additionally, creating a set of indicators to measure policy effectiveness in preventing and controlling COVID-19 was invaluable in ensuring that policies were yielding excellent results in curbing the spread of the virus.

CONCLUSION

The majority of messages lack acceptable standards in terms of content types, scientific documentation and evidence, scientific strength and precision, fluidity and simplicity of the message, as well as overall appearance and design. Effective science communication requires adherence to established standards to ensure information is correctly conveyed and easily accessible. Decision-makers must collaborate with experts to create messages that are supported by empirical evidence while catering to their target market's needs for effective communication. Creating reliable educational content in the health field can positively impact the health of individuals and improve their overall outcomes. The tool developed in this study offers a valuable opportunity to assess the quality of messages on different health-related topics, and future studies should explore the psychometric properties of this tool.

REFERENCES