All published articles of this journal are available on ScienceDirect.
Online Training and Supervision to Improve Immunization Programs in Thailand
Abstract
Introduction:
Healthcare providers are regarded as one of the key forces in the implementation of standards for building up the immune system. This research thus makes use of e-training systems to develop the capacity of immunization providers to comply with the current format of learning.
Objective:
To study the effectiveness of an online training system on the knowledge and skills of immunization services.
Methods:
This experimental research was conducted on 62 providers (31 interventions and 31 control groups) at the immunization service in Thailand, which was an experimental group receiving an Expanded Program on Immunization (EPI) e-course training with regular supervision and a control group receiving regular supervision. A knowledge scale on immunization and demographic data were completed by providers, and the skill of immunization used observation forms. Data were analyzed using SPSS-16 software. Analysis was performed by means of a t-test, and Chi-square.
Results:
After a four-week training period, the knowledge scores of the experimental group were higher than those of the control group, with a statistical significance of .01. However, the skill of immunization displayed no significant difference between groups.
Conclusion:
From these findings, online training has a positive effect on knowledge because the key benefits of online training are convenience, self-paced nature, and the ability to replay material as needed. While the skill of immunization did not differ, it seems that practice training must necessarily depend on the follow-up methods of supervision for routine monitoring and further recommendation according to the standards of immunization.
1. INTRODUCTION
Developing immunization providers is considered to be one of the strategies of manpower development of the National Vaccine Institute. Currently, there are more than 20,000 immunization providers. They are the main force in the vaccine drive to full vaccination and achieving higher vaccine coverage [1]. In Thailand, there is a systematic approach to human resource development, beginning with the orientation stage, in which the administration of vaccinations is introduced to new workers. In-service training is also provided, with the result that they constantly expand their knowledge base and review their various skills and principles [2, 3]. Worker supervision may be directed by an expert under the supervision of an Immunization network to develop their service [4]. At the same time, those engaged in the work can be refreshed in their academic knowledge through practical training based on the standard curriculum of immunization, which has been implemented each year from 2011 to 2019. Thus far, those who have completed their training number more than 3,000 [5]. Organizing the training according to the original format requires three days for each course, and there are various techniques for passing on the acquired knowledge, including lecturing, presentation of hypothetical roles, brainstorming, seminars, and demonstrations [6]. Each of these methods stimulates the interest of the provider, and the provider has accomplished an achievement after the completion of training and has accumulated some additional knowledge points [7, 8].
Nonetheless, practical training in the original format (face-to-face training) has a limitation in setting the number of meeting attendees at no more than 40 per graduating class, making it impossible to train all those involved in the work and the expenditure of a rather high budget [9]. As a means of reducing this gap in human resource development, the researchers decided on teaching technology in the form of online training that would enhance the ability of students or trainees to access a body of knowledge on a wide range of immunizations [10-12]. From a literature review [13], it was found that a few agencies in the country had used human resource development systems in their agencies through a diversity of online systems, such as the online training of the BMA Training and Development Institute, the online training system of the Department of Skill Development, the training management system of the Royal Irrigation Department, and the online self-learning system of the Department of Corrections. The purposes of using the online professional development system differ and may include the use of the online system to clarify the newly installed officers to create a basic understanding of the organization [10, 14]. Some projects, logging into the organization’s learning management system, have used it to set appointments; and the last of the groups used the online system to gauge the level of knowledge relevant to certain professional standards, such as a massive open online course (MOOCs) training in healthcare education, including nurses, medicine, and pharmacy [15]. It may be observed that most of the government-sector online systems in use for professional development are unidirectional in nature (for offline learning). These systems offer the advantage of granting students the flexibility of freely apportioning their study time. Students can thus manage their own time. Regarding these results, online course training in a developing country has emerged as an innovative educational tool in higher education and professional development. The potential contribution of E-course training for healthcare providers has become numerous in a few years during the COVID-19 pandemic [16, 17] or example, the online course in health promotion, community challenges, and public health education. It seems like an effective tool for scaling-up education. Due to the limitation of communication between the immunization providers and the Covid-19 vaccination course coordinator, they perceived their weak performance to access the training package. Hence, before launching an online training program, the developer should consider the demonstrated accessibility of online learning activities and the required baseline level of learners’ digital literacy [8, 18-20]. In this way, the use of online systems reduces the past limitations of the practical training format. Their use also promotes an increase in the number of workers who will be able to access the training. During this research, we applied the EPI e-course training version 1.0 in conjunction with the concept of quality cycle development, or the PDCA (plan, do, check, and act) of Deming [21]. It would be used to determine the procedure of concretely implementing the online training system and to monitor and evaluate the knowledge and skills of those who provide immunization services.
2. MATERIALS AND METHODS
2.1. Design and Setting
The present research consisted of a study on the effectiveness of the EPI e-course training system of the immunization curriculum in use by regional service representatives. It took the form of a two-group measurement quasi-experimental research, which was carried out from January to August 2022.
2.2. Sample/Participants
The experimental group consisted of immunization providers and was considered according to the following inclusion criteria: They were immunization providers who were at least 20 years of age, were able to communicate fluently in the Thai language, and had basic computer knowledge. Exclusion criteria included transfer to another healthcare service where they undertook other responsibilities which differed from the work of immunization, or previously having passed the experimental learning test in the EPI e-course training system.
2.3. Sampling Method
The experimental group was selected by stratified random sampling. The group was subdivided by the regional locations of the service units that were under the affiliation of the Ministry of Public Health. Researchers would then select one random province to represent one region. A search was conducted for a list of primary care to tertiary care units that would be inclined to become the sample groups for our study. The sizes of the sample groups were determined by opening the test power table at a level of 80% and setting the effect size at 0.5, with a statistical significance level of 0.05 [22]. The sample groups were randomly selected as two groups of 30 persons each. Since we were conducting an online study, there was a tendency for some of our participants to disappear. So, we increased the sample groups by 30% to offset the attrition rate, thus bringing the group size to 40 persons each. After the research, there were a total of 31 persons in each of the two groups.
2.4. Intervention: the EPI e-course Training System
The researchers took the EPI e-course training system of Rutchanagul et al. (2019) [23], together with the system operation manual, for the research work being conducted on the research topic of developing a self-learning model (via e-course training) for immunization providers. Its content consisted of seven categories as based on immunization standards, including Module 1: Basic Knowledge of immunization; Module 2: Vaccine and preventable Diseases; Module 3: Immunization information systems; Module 4: Vaccination records; Module 5: Vaccines and cold-chain System; Module 6: Vaccine administration; and Module 7: Preventing and managing adverse reactions, in which each content category consists of a variety of online lesson components, such as images (.jpg), movies (.mp3), short clips (.mp4), animations, as well as text (.pdf). Content categories passed the quality testing of the research tools of seven established experts, with an IOC value of 0.78. The online training system was tested for use in phase-1 research with 80 immunization providers affiliated with the Department of Health (Bangkok Metropolitan Administration) and was modified in accordance with certain proposals and recommendations prior to being applied for use in this project. The EPI e-course online training system can be accessed at www.guruvaccine.com/e-learning. Pre-training questionnaires were self-completed in electronic form during orientation sessions in both groups, while post-training questionnaires, also in electronic form, were administered every week. This took place one to five weeks following the orientation sessions for the experimental group and control group, as the researcher conducted 2-3 categories/week. To avoid habitual responses and plagiarism among participants, the items of post-training questionnaires were shuffled. An e-course training program was conducted over 4 sessions. During the 1st week, the researcher generated a participant code for the experimental group to facilitate self-learning with the e-course training system, while participants in the control group received this code at the end of the program (5th week). For the evaluation session, participants in both groups were reassessed with the post-training questionnaires 15 items/week (during the 2nd week to the 4th week). Finally, during the 5th week, all participants submitted a clip video about an immunization service skill online.
2.5. Data Collection Tools
Data collection tools consisted of the compilation of personal data consisting of the variables of gender, education level, type of profession, immunization-work experience, and a two-part questionnaire comprised:
2.5.1. Part 1 was a Knowledge Measurement Form Pertaining to the Standards of Immunization Work
It was used to assess the knowledge of those who had undergone training in the seven content categories by taking measurements before and after the training. This tool was a measurement protocol consisting of a multiple-choice form with four options for each of the 60 questions and a total of 60 points. The researchers obtained the measurement form from the study conducted by Rutchanagul et al. [24]. In the previous research, this same measurement form had a content validity index of 0.80 and was used for testing on a sample group of 30 cases with feature-matching immunization providers. It was then used to calculate the reliability of this tool set by using Cronbach’s alpha coefficient at a level of 0.05, which was found to be 0.63. They were divided into 7 sections: Section 1 contains 5 questions regarding basic knowledge of immunization. Section 2 contains 10 questions about vaccines and preventable diseases. Section 3 contains 5 questions about immunization information systems. Section 4 contains 10 questions about vaccination records. Section 5 contains 10 questions about vaccines and cold-chain systems. Section 6 contains 10 questions about vaccine administration. Finally, section 7 contains 10 questions about preventing and managing adverse reactions. The alpha coefficients of this measurement of section 1 to section 7 were 0.82, 0.72, 0.77, 0.60, 0.88, 0.68, and 0.62, respectively.
2.5.2. Part 2 An Immunization Service Skills Observation Form
Part 2, an immunization service skills observation form was a form containing 15 topics for which the criteria for assigning points were at two levels, namely, did not perform/performed, or performed incorrectly (0 points) and performed correctly (1 point). There was a total of 15 points on the observation form, with the requirement of achieving 12 or more points (or a minimum of 80% of the total). Subjects under observation were then considered to have passed the criteria. Researchers drew from the invention development project of Rutchanagul et al. [24] to practice vaccinating children. They proceeded according to the framework of standards for immunization on the quality of vaccine services. They had an IOC determined by five qualified individuals of 0.85. The test-retest reliability of an immunization service skills observation form was 0.84. The level of exactness was tested by all three of the researchers who evaluated the vaccination skills training of 10 practitioners in the immunization service unit.
2.6. Statistical Analysis
The personal attributes of the participants in the control and experimental groups were compared by use of the t-test and chi-square test. Knowledge scores of the experimental and control groups, both before and four weeks after training, were compared by an independent t-test. Comparisons of knowledge scores, both before and four weeks after training, were made within the control and experimental groups by use of paired t-tests; and the immunization service skills of the control and experimental groups were compared by an independent t-test.
3. RESULTS
3.1. Sample Characteristics
Table 1 illustrated that 62 immunization providers were enrolled in this study and found that the two groups did not differ regarding their age, gender, education level, types of professions, or immunization-work experiences.
3.2. Results of Testing the Effectiveness of Using the EPI E-course Training
Prior to the beginning of this research, there was no difference in the test scores between the experimental group and the control group. Four weeks after the research, it was found that the workers who had received the online training had average knowledge scores that were higher than the group that had received regular supervision (p = 0.001). The average scores for immunization services, however, did not differ, as shown in Table 2. After sorting out the knowledge test scores for the content categories 1 to 7, it was found that the workers who had received online training for immunization had an increase in their average knowledge scores after four weeks of training in every category (p < 0.01), as shown in Table 2.
Demographic | Experimental Group | Control Group | p-value | |
Age | - | align="center"No (%) | No (%) | - |
20:<30 | 10 (25) | 9 (22.5) | - | |
31:<40 | 17( 42.5) | 22 (55) | - | |
40:<50 | 8 (20) | 9 (22.5) | - | |
50:<60 | 5 (12.5) | 0 (0) | - | |
Mean and SD | 36.8±9.19 | 35.35 ±7.9 | .756a | |
Gender | Male | 2 (5) | 2 (5) | 1.00b |
Female | 38 (95) | 38 (95) | - | |
Education level | Bachelor’s degree | 33 (82.5) | 37 (92.5) | .176b |
Master’s degree | 7 (17.5) | 3 (7.5) | - | |
Types of professions | Diploma Nurse | 31 (77.5) | 36 (85) | .390b |
Public health technical officer | 9 (22.5) | 4 (15) | - | |
Immunization-work experiences | Vaccine administration | 35 (87.5) | 36 (90) | 1.00b |
Vaccine storage &handling | 5 (12.5) | 9 (22.5) | .239b | |
Continuing education for immunization | 6 (15) | 8 (20) | .556b |
Variables | Experimental Group (n=31) | Control Group (n=31) | t | p-value | ||
Mean | S.D. | Mean | S.D. | |||
Immunization provider’s knowledge | - | - | - | - | - | |
Knowledge score before intervention (pre-test) | 22.06 | 4.96 | 23.00 | 5.25 | 0.720 | .474 |
Knowledge score post-intervention (posttest) | 33.06 | 6.38 | 26.90 | 6.93 | 4.352 | .001* |
Skill of immunization service | 14.66 | 0.61 | 13.40 | 1.26 | 2.077 | .390 |
Experimental Group | Pre-test | Post-test | t | p-value | ||
Module | Mean | S.D. | Mean | S.D. | ||
Module 1 Basic knowledge of immunization | 1.64 | 1.01 | 2.54 | 1.17 | 3.478 | .002* |
Module 2 Vaccine and preventable disease | 3.54 | 1.62 | 5.54 | 1.56 | 4.678 | .001* |
Module 3 Immunization information systems | 1.77 | 0.92 | 2.74 | 0.68 | 4.991 | .001* |
Module 4 Vaccination records | 3.00 | 1.36 | 4.38 | 1.60 | 3.748 | .001* |
Module 5 Vaccines and cold-chain system | 4.41 | 1.82 | 6.09 | 1.88 | 3.892 | .001* |
Module 6 Vaccine administration | 4.61 | 1.76 | 6.19 | 1.72 | 4.302 | .001* |
Module 7 Preventing and managing adverse reactions | 3.06 | 1.48 | 5.54 | 2.09 | 6.702 | .001* |
Total score | 22.06 | 4.96 | 33.06 | 6.38 | 8.816 | .001* |
Control Group | Pre-test | Post-test | t | p-value | ||
Module | Mean | S.D. | Mean | S.D. | ||
Module 1 Basic knowledge of immunization | 1.93 | 1.43 | 3.25 | 0.96 | 4.001 | .001** |
Module 2 Vaccine and preventable disease | 3.38 | 1.43 | 4.16 | 1.48 | 2.322 | .027** |
Module 3 Immunization information systems | 1.93 | 1.12 | 3.16 | 0.63 | 5.115 | .001* |
Module 4 Vaccination records | 2.93 | 1.09 | 3.12 | 1.60 | .512 | .612 |
Module 5 Vaccines and cold-chain system | 4.54 | 2.01 | 5.06 | 1.69 | 1.361 | .184 |
Module 6 Vaccine administration | 4.54 | 1.28 | 5.03 | 1.58 | 1.878 | .070 |
Module 7 Preventing and managing adverse reactions | 3.70 | 1.41 | 4.32 | 1.66 | 1.804 | .081 |
Total score | 23.00 | 5.25 | 26.90 | 6.93 | 2.971 | .006* |
The average knowledge scores of the workers in the control group who had received regular supervision may have had average scores, even after its expansion before receiving supervision (p = 0.006). Yet, when considered in terms of categorical content, it was then found that Module 1: Basic Knowledge of Immunization; Module 2: Vaccine and preventable disease; and Module 3: Immunization information Systems show increases in average knowledge scores (p < 0.01, p < 0.05, and p < 0.01) with, respectively. Meanwhile, Module 4: Vaccination Records; Module 5: Vaccines and cold-chain system; Module 6: Vaccine administration; and Module 7: Preventing and managing adverse Reactions increased with no statistical significance (p > 0.05) (Tables 3 and 4).
4. DISCUSSION
From the results of the study on the effectiveness of using the online immunization training system for healthcare providers, it was found that the providers who had received their training through the online EPI e-course training system had higher average knowledge scores than the group that had received ordinary supervision (p = 0.001). One explanation is that online learning or training assists in the expansion of knowledge by reviewing the body of knowledge through various activities, such as quizzes and playing games during the lessons in each content category. It helps the learner to review his knowledge in various subjects from time to time, as noted by AlMarzooqi et al. (2018), who stated that an online study often focuses on gauging knowledge and attitudes in the performance of work as important outcomes [25]. The use of the online learning system through virtual case studies about promoting the safety of patients aids the ability of the student to analyze a situation and to practice problem-solving according to the context [13, 25]. When testing the knowledge gained after studying, it has been found that medical students have higher average knowledge scores after their study than they had before their study, with a statistical significance of .01. They also have higher positive attitude scores on the prevention of mistakes in their practice with humans, with a statistical significance of .01 [26, 27]. Currently, the World Health Organization supports the use of the online training system for the development of a diversity of medical personnel, such as basic courses on evidence-based medicine e-course, vaccine safety, basic e-learning course, cervical cancer prevention e-learning course, and gender and humanitarian action. Not only does the online training system help to prepare basic knowledge in vaccine work for immunization providers, but is also capable of practical use for the expansion of knowledge in situations in which there is a spread of contagious-disease groups [20]. For instance, in a study conducted by Baggier et al. (2020), online training through an e-learning course for immunization service providers was found to be statistically significant in its effectiveness [28]. The course, which was centered on a campaign of measles and rubella prevention that occurred in Italy, aided the trainees in acquiring more knowledge than they had before their training. Trainees were also capable of correctly managing any outbreaks of measles or rubella they may encounter.
However, regular supervision can also contribute to their developing body of knowledge in their routine work since coaching as they work the practitioners to undergo constant improvement [29]. It was found from a literature review that the factors that influence immunization service skills depend on the nature of the work that has been assigned and a training experience that provides knowledge related to vaccine work, including supervisors characterized by their knowledge and skills in vaccine work [30]. However, from the nature of the health care services of the primary care units, it was found that the aspects of health reinforcement cover many areas of work, including the urgent mission of the campaign to build coverage for the anti-COVID vaccine. As a result, none of the practitioners will have every possible occasion to access the training system [31, 32]. Currently, various formats of training are being paired together with routine work to develop a body of knowledge that will comply with the standard criteria for immune system enhancement [33, 34]. In addition to the three-day workshops that comply with the curriculum of practice for immunization providers in Thailand [6], the work of immunization takes on other formats as well, including weekly two-hour vaccine knowledge reviews prior to service over a continual eight-week period (in-service training). The results of the study demonstrated, with statistical significance, that the knowledge test scores after training were higher than they were before training [35, 36], which is consistent with the results of this present study which used the online training system to develop the knowledge base of the practitioners. Moreover, it was found that the average knowledge test scores after learning with the EPI e-course training were higher than they were before learning in each of the content categories. The reason for these higher scores is that the training process in each content category entails a diversity of media formats that incentivize the student to review the content repeatedly [37, 38]. The online training system is even capable of displaying a self-learning progress bar for the student to view in each content category. The student is thus freely able to lay out a plan of learning. With each weekly evaluation, it has been found that test result scores after learning with the online training system continue to increase, matching the results of a study conducted by Brown et al. (2017), who studied the results of a program of immunization provider development in the nation of Nigeria by organizing two-day courses in academic rehabilitation training. Results of the research showed that the experimental group [39] that had undergone the training had knowledge test scores after their training that were higher than those of the control group. After gauging the results three months and six months after training, the healthcare providers still had average scores after their training that were higher than the group that had received only regular supervision. It can thus be seen that short-term training or training with the features of academic rehabilitation training can assist the development of a body of knowledge in vaccine work, but with an emphasis on basic vaccines that comply with the immunization plan of work [34, 40].
Currently, it has been found that new vaccines are being introduced into the plan of work for immunization services from time to time [41]. The aforementioned training program has thus been forced to modify its content completely, increasing the amount of content already embedded in the online system and causing long delays that have impeded learning through the online system [42]. A new format of learning was thus developed by integrating virtual reality technology into the training program for medical personnel. It was found from the results of our study that once we started the program that enabled the medical students to begin training on the online system and create their mock avatars, they were delighted. Members of the sample groups possessed both knowledge and understanding. After their training, they were able to offer some appropriate advice to the people on possibly getting new vaccines. There has been a statistically significant increase in the rate of coverage for anti-cervical cancer vaccines [43]. From the standpoint of systematically providing information pertaining to vaccines, it has been found that immunization providers who underwent the training were able to make practical use of the medical data in their decision of a correct choice of a vaccine. They were able to provide information to their service recipients. However, they were not able to respond to questions on complicated issues. For instance, there would be limitations in administering a vaccine in the case of a child who had contracted cancer [44]. Accordingly, while the online training program may take on various formats, it has already helped to develop a body of knowledge for immunization providers. In carrying out the work of immunization, online training should be provided alongside follow-up supervision of the immunization work at a primary level to develop a concrete operational standard over the long term for the future.
4.1. Limitations and Suggestions
In view of the limitations of work performance imposed by the circumstances of the spread of the coronavirus, volunteers must accelerate the fulfillment of their urgent tasks to take control of the pandemic situation, which is preventing continual participation of thethe research program. Future research should take on an online study format consistent with the type of vaccines that are under the plan of work for immunization but separate from emergency vaccines and pilot vaccines of other types. The next issue should be a study of an online training format consistent with the features of an immunization service provider; various types of which include nurses, public health officials, academicians, physicians, and pharmacists, among others.
CONCLUSION
The results of this research indicate that the use of the online training system coupled with supervision in the work of immunization helps to increase the knowledge of immunization providers. Online training should be embedded as a basic method of personnel development, especially for those who work at the primary care level, which is the first line of service and important in driving the work of immunization for people.
LIST OF ABBREVIATIONS
PDCA | = Plan, Do, Check, and Act |
MOOCs | = Massive Open Online Course |
ETHICS APPROVAL AND CONSENT TO PARTICIPATE
The study was approved by the Human Research Ethics Committee of Thammasat University (Science), (HREC-TUSC) #COA No. 081/2563.
HUMAN AND ANIMAL RIGHTS
No animals were used for the studies that are the basis of this research. All human procedures followed were following the guidelines of the Helsinki Declaration of 1975.
CONSENT FOR PUBLICATION
Participation in this study was voluntary and informed consent was obtained from all participants.
AVAILABILITY OF DATA AND MATERIALS
The data that support the findings of this study are available from the corresponding author, [P.K], upon reasonable request.
FUNDING
This article has been performed with the financial support of the Thammasat University Research Fund (Funder Id/Grant No. TUFT 23/ 2564).
CONFLICT OF INTEREST
The authors declare no conflict of interest or otherwise.
ACKNOWLEDGEMENTS
The authors would like to thank the healthcare team for their collaborative efforts in making this study success. The authors also thank the Thammasat University for their financial support by Thammasat University Research Fund, Contract No. TUFT 23/2564.