RESEARCH ARTICLE


Online Training and Supervision to Improve Immunization Programs in Thailand



Patcharaporn Kaewwimol1, *, Pregamol Rutchanagul1, Syamol Rompipat1
1 Faculty of Nursing, Thammasat University, Pathum Thani 12120, Thailand


Article Metrics

CrossRef Citations:
0
Total Statistics:

Full-Text HTML Views: 610
Abstract HTML Views: 365
PDF Downloads: 236
ePub Downloads: 171
Total Views/Downloads: 1382
Unique Statistics:

Full-Text HTML Views: 399
Abstract HTML Views: 210
PDF Downloads: 175
ePub Downloads: 121
Total Views/Downloads: 905



Creative Commons License
© 2023 Kaewwimol et al.

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

* Address correspondence to this author at the Faculty of Nursing, Thammasat University, Pathum Thani 12120, Thailand; Tel: +662-986-9214, ext. 7343; E-mail: champ_rans@hotmail.com


Abstract

Introduction:

Healthcare providers are regarded as one of the key forces in the implementation of standards for building up the immune system. This research thus makes use of e-training systems to develop the capacity of immunization providers to comply with the current format of learning.

Objective:

To study the effectiveness of an online training system on the knowledge and skills of immunization services.

Methods:

This experimental research was conducted on 62 providers (31 interventions and 31 control groups) at the immunization service in Thailand, which was an experimental group receiving an Expanded Program on Immunization (EPI) e-course training with regular supervision and a control group receiving regular supervision. A knowledge scale on immunization and demographic data were completed by providers, and the skill of immunization used observation forms. Data were analyzed using SPSS-16 software. Analysis was performed by means of a t-test, and Chi-square.

Results:

After a four-week training period, the knowledge scores of the experimental group were higher than those of the control group, with a statistical significance of .01. However, the skill of immunization displayed no significant difference between groups.

Conclusion:

From these findings, online training has a positive effect on knowledge because the key benefits of online training are convenience, self-paced nature, and the ability to replay material as needed. While the skill of immunization did not differ, it seems that practice training must necessarily depend on the follow-up methods of supervision for routine monitoring and further recommendation according to the standards of immunization.

Keywords: Expanded program on immunization (epi), Online training, Immunization providers, Knowledge, Skills, Health service.



1. INTRODUCTION

Developing immunization providers is considered to be one of the strategies of manpower development of the National Vaccine Institute. Currently, there are more than 20,000 immunization providers. They are the main force in the vaccine drive to full vaccination and achieving higher vaccine coverage [1]. In Thailand, there is a systematic approach to human resource development, beginning with the orientation stage, in which the administration of vaccinations is introduced to new workers. In-service training is also provided, with the result that they constantly expand their knowledge base and review their various skills and principles [2, 3]. Worker supervision may be directed by an expert under the supervision of an Immunization network to develop their service [4]. At the same time, those engaged in the work can be refreshed in their academic knowledge through practical training based on the standard curriculum of immunization, which has been implemented each year from 2011 to 2019. Thus far, those who have completed their training number more than 3,000 [5]. Organizing the training according to the original format requires three days for each course, and there are various techniques for passing on the acquired knowledge, including lecturing, presentation of hypothetical roles, brainstorming, seminars, and demonstrations [6]. Each of these methods stimulates the interest of the provider, and the provider has accomplished an achievement after the completion of training and has accumulated some additional knowledge points [7, 8].

Nonetheless, practical training in the original format (face-to-face training) has a limitation in setting the number of meeting attendees at no more than 40 per graduating class, making it impossible to train all those involved in the work and the expenditure of a rather high budget [9]. As a means of reducing this gap in human resource development, the researchers decided on teaching technology in the form of online training that would enhance the ability of students or trainees to access a body of knowledge on a wide range of immunizations [10-12]. From a literature review [13], it was found that a few agencies in the country had used human resource development systems in their agencies through a diversity of online systems, such as the online training of the BMA Training and Development Institute, the online training system of the Department of Skill Development, the training management system of the Royal Irrigation Department, and the online self-learning system of the Department of Corrections. The purposes of using the online professional development system differ and may include the use of the online system to clarify the newly installed officers to create a basic understanding of the organization [10, 14]. Some projects, logging into the organization’s learning management system, have used it to set appointments; and the last of the groups used the online system to gauge the level of knowledge relevant to certain professional standards, such as a massive open online course (MOOCs) training in healthcare education, including nurses, medicine, and pharmacy [15]. It may be observed that most of the government-sector online systems in use for professional development are unidirectional in nature (for offline learning). These systems offer the advantage of granting students the flexibility of freely apportioning their study time. Students can thus manage their own time. Regarding these results, online course training in a developing country has emerged as an innovative educational tool in higher education and professional development. The potential contribution of E-course training for healthcare providers has become numerous in a few years during the COVID-19 pandemic [16, 17] or example, the online course in health promotion, community challenges, and public health education. It seems like an effective tool for scaling-up education. Due to the limitation of communication between the immunization providers and the Covid-19 vaccination course coordinator, they perceived their weak performance to access the training package. Hence, before launching an online training program, the developer should consider the demonstrated accessibility of online learning activities and the required baseline level of learners’ digital literacy [8, 18-20]. In this way, the use of online systems reduces the past limitations of the practical training format. Their use also promotes an increase in the number of workers who will be able to access the training. During this research, we applied the EPI e-course training version 1.0 in conjunction with the concept of quality cycle development, or the PDCA (plan, do, check, and act) of Deming [21]. It would be used to determine the procedure of concretely implementing the online training system and to monitor and evaluate the knowledge and skills of those who provide immunization services.

2. MATERIALS AND METHODS

2.1. Design and Setting

The present research consisted of a study on the effectiveness of the EPI e-course training system of the immunization curriculum in use by regional service representatives. It took the form of a two-group measurement quasi-experimental research, which was carried out from January to August 2022.

2.2. Sample/Participants

The experimental group consisted of immunization providers and was considered according to the following inclusion criteria: They were immunization providers who were at least 20 years of age, were able to communicate fluently in the Thai language, and had basic computer knowledge. Exclusion criteria included transfer to another healthcare service where they undertook other responsibilities which differed from the work of immunization, or previously having passed the experimental learning test in the EPI e-course training system.

2.3. Sampling Method

The experimental group was selected by stratified random sampling. The group was subdivided by the regional locations of the service units that were under the affiliation of the Ministry of Public Health. Researchers would then select one random province to represent one region. A search was conducted for a list of primary care to tertiary care units that would be inclined to become the sample groups for our study. The sizes of the sample groups were determined by opening the test power table at a level of 80% and setting the effect size at 0.5, with a statistical significance level of 0.05 [22]. The sample groups were randomly selected as two groups of 30 persons each. Since we were conducting an online study, there was a tendency for some of our participants to disappear. So, we increased the sample groups by 30% to offset the attrition rate, thus bringing the group size to 40 persons each. After the research, there were a total of 31 persons in each of the two groups.

2.4. Intervention: the EPI e-course Training System

The researchers took the EPI e-course training system of Rutchanagul et al. (2019) [23], together with the system operation manual, for the research work being conducted on the research topic of developing a self-learning model (via e-course training) for immunization providers. Its content consisted of seven categories as based on immunization standards, including Module 1: Basic Knowledge of immunization; Module 2: Vaccine and preventable Diseases; Module 3: Immunization information systems; Module 4: Vaccination records; Module 5: Vaccines and cold-chain System; Module 6: Vaccine administration; and Module 7: Preventing and managing adverse reactions, in which each content category consists of a variety of online lesson components, such as images (.jpg), movies (.mp3), short clips (.mp4), animations, as well as text (.pdf). Content categories passed the quality testing of the research tools of seven established experts, with an IOC value of 0.78. The online training system was tested for use in phase-1 research with 80 immunization providers affiliated with the Department of Health (Bangkok Metropolitan Administration) and was modified in accordance with certain proposals and recommendations prior to being applied for use in this project. The EPI e-course online training system can be accessed at www.guruvaccine.com/e-learning. Pre-training questionnaires were self-completed in electronic form during orientation sessions in both groups, while post-training questionnaires, also in electronic form, were administered every week. This took place one to five weeks following the orientation sessions for the experimental group and control group, as the researcher conducted 2-3 categories/week. To avoid habitual responses and plagiarism among participants, the items of post-training questionnaires were shuffled. An e-course training program was conducted over 4 sessions. During the 1st week, the researcher generated a participant code for the experimental group to facilitate self-learning with the e-course training system, while participants in the control group received this code at the end of the program (5th week). For the evaluation session, participants in both groups were reassessed with the post-training questionnaires 15 items/week (during the 2nd week to the 4th week). Finally, during the 5th week, all participants submitted a clip video about an immunization service skill online.

2.5. Data Collection Tools

Data collection tools consisted of the compilation of personal data consisting of the variables of gender, education level, type of profession, immunization-work experience, and a two-part questionnaire comprised:

2.5.1. Part 1 was a Knowledge Measurement Form Pertaining to the Standards of Immunization Work

It was used to assess the knowledge of those who had undergone training in the seven content categories by taking measurements before and after the training. This tool was a measurement protocol consisting of a multiple-choice form with four options for each of the 60 questions and a total of 60 points. The researchers obtained the measurement form from the study conducted by Rutchanagul et al. [24]. In the previous research, this same measurement form had a content validity index of 0.80 and was used for testing on a sample group of 30 cases with feature-matching immunization providers. It was then used to calculate the reliability of this tool set by using Cronbach’s alpha coefficient at a level of 0.05, which was found to be 0.63. They were divided into 7 sections: Section 1 contains 5 questions regarding basic knowledge of immunization. Section 2 contains 10 questions about vaccines and preventable diseases. Section 3 contains 5 questions about immunization information systems. Section 4 contains 10 questions about vaccination records. Section 5 contains 10 questions about vaccines and cold-chain systems. Section 6 contains 10 questions about vaccine administration. Finally, section 7 contains 10 questions about preventing and managing adverse reactions. The alpha coefficients of this measurement of section 1 to section 7 were 0.82, 0.72, 0.77, 0.60, 0.88, 0.68, and 0.62, respectively.

2.5.2. Part 2 An Immunization Service Skills Observation Form

Part 2, an immunization service skills observation form was a form containing 15 topics for which the criteria for assigning points were at two levels, namely, did not perform/performed, or performed incorrectly (0 points) and performed correctly (1 point). There was a total of 15 points on the observation form, with the requirement of achieving 12 or more points (or a minimum of 80% of the total). Subjects under observation were then considered to have passed the criteria. Researchers drew from the invention development project of Rutchanagul et al. [24] to practice vaccinating children. They proceeded according to the framework of standards for immunization on the quality of vaccine services. They had an IOC determined by five qualified individuals of 0.85. The test-retest reliability of an immunization service skills observation form was 0.84. The level of exactness was tested by all three of the researchers who evaluated the vaccination skills training of 10 practitioners in the immunization service unit.

2.6. Statistical Analysis

The personal attributes of the participants in the control and experimental groups were compared by use of the t-test and chi-square test. Knowledge scores of the experimental and control groups, both before and four weeks after training, were compared by an independent t-test. Comparisons of knowledge scores, both before and four weeks after training, were made within the control and experimental groups by use of paired t-tests; and the immunization service skills of the control and experimental groups were compared by an independent t-test.

3. RESULTS

3.1. Sample Characteristics

Table 1 illustrated that 62 immunization providers were enrolled in this study and found that the two groups did not differ regarding their age, gender, education level, types of professions, or immunization-work experiences.

3.2. Results of Testing the Effectiveness of Using the EPI E-course Training

Prior to the beginning of this research, there was no difference in the test scores between the experimental group and the control group. Four weeks after the research, it was found that the workers who had received the online training had average knowledge scores that were higher than the group that had received regular supervision (p = 0.001). The average scores for immunization services, however, did not differ, as shown in Table 2. After sorting out the knowledge test scores for the content categories 1 to 7, it was found that the workers who had received online training for immunization had an increase in their average knowledge scores after four weeks of training in every category (p < 0.01), as shown in Table 2.

Table 1. Demographic characteristics of the immunization providers (N=62).
Demographic Experimental Group Control Group p-value
Age - align="center"No (%) No (%) -
20:<30 10 (25) 9 (22.5) -
31:<40 17( 42.5) 22 (55) -
40:<50 8 (20) 9 (22.5) -
50:<60 5 (12.5) 0 (0) -
Mean and SD 36.8±9.19 35.35 ±7.9 .756a
Gender Male 2 (5) 2 (5) 1.00b
Female 38 (95) 38 (95) -
Education level Bachelor’s degree 33 (82.5) 37 (92.5) .176b
Master’s degree 7 (17.5) 3 (7.5) -
Types of professions Diploma Nurse 31 (77.5) 36 (85) .390b
Public health technical officer 9 (22.5) 4 (15) -
Immunization-work experiences Vaccine administration 35 (87.5) 36 (90) 1.00b
Vaccine storage &handling 5 (12.5) 9 (22.5) .239b
Continuing education for immunization 6 (15) 8 (20) .556b
Note:a p-value obtained from the t-test; bp-value obtained from the chi-square test.
Table 2. Test results of the two groups about their knowledge and skills in providing immunization services, as compared by the use of independent t-test statistics.
Variables Experimental Group (n=31) Control Group (n=31) t p-value
Mean S.D. Mean S.D.
Immunization provider’s knowledge - - - - -
Knowledge score before intervention (pre-test) 22.06 4.96 23.00 5.25 0.720 .474
Knowledge score post-intervention (posttest) 33.06 6.38 26.90 6.93 4.352 .001*
Skill of immunization service 14.66 0.61 13.40 1.26 2.077 .390
Note: *Significance at level .01.
Table 3. Knowledge test results both before and after online training of the experimental group from a comparison of test score differences before and after training in each lesson by paired t-test statistics.
Experimental Group Pre-test Post-test t p-value
Module Mean S.D. Mean S.D.
Module 1 Basic knowledge of immunization 1.64 1.01 2.54 1.17 3.478 .002*
Module 2 Vaccine and preventable disease 3.54 1.62 5.54 1.56 4.678 .001*
Module 3 Immunization information systems 1.77 0.92 2.74 0.68 4.991 .001*
Module 4 Vaccination records 3.00 1.36 4.38 1.60 3.748 .001*
Module 5 Vaccines and cold-chain system 4.41 1.82 6.09 1.88 3.892 .001*
Module 6 Vaccine administration 4.61 1.76 6.19 1.72 4.302 .001*
Module 7 Preventing and managing adverse reactions 3.06 1.48 5.54 2.09 6.702 .001*
Total score 22.06 4.96 33.06 6.38 8.816 .001*
Note: *Significance at level .01.
Table 4. Knowledge test results before and after receiving the supervision of the control group through a comparison of test score differences before and after the supervision followed by a statistical paired t-test.
Control Group Pre-test Post-test t p-value
Module Mean S.D. Mean S.D.
Module 1 Basic knowledge of immunization 1.93 1.43 3.25 0.96 4.001 .001**
Module 2 Vaccine and preventable disease 3.38 1.43 4.16 1.48 2.322 .027**
Module 3 Immunization information systems 1.93 1.12 3.16 0.63 5.115 .001*
Module 4 Vaccination records 2.93 1.09 3.12 1.60 .512 .612
Module 5 Vaccines and cold-chain system 4.54 2.01 5.06 1.69 1.361 .184
Module 6 Vaccine administration 4.54 1.28 5.03 1.58 1.878 .070
Module 7 Preventing and managing adverse reactions 3.70 1.41 4.32 1.66 1.804 .081
Total score 23.00 5.25 26.90 6.93 2.971 .006*
Note: *Significance at the level .01, ** Significance at level .05.

The average knowledge scores of the workers in the control group who had received regular supervision may have had average scores, even after its expansion before receiving supervision (p = 0.006). Yet, when considered in terms of categorical content, it was then found that Module 1: Basic Knowledge of Immunization; Module 2: Vaccine and preventable disease; and Module 3: Immunization information Systems show increases in average knowledge scores (p < 0.01, p < 0.05, and p < 0.01) with, respectively. Meanwhile, Module 4: Vaccination Records; Module 5: Vaccines and cold-chain system; Module 6: Vaccine administration; and Module 7: Preventing and managing adverse Reactions increased with no statistical significance (p > 0.05) (Tables 3 and 4).

4. DISCUSSION

From the results of the study on the effectiveness of using the online immunization training system for healthcare providers, it was found that the providers who had received their training through the online EPI e-course training system had higher average knowledge scores than the group that had received ordinary supervision (p = 0.001). One explanation is that online learning or training assists in the expansion of knowledge by reviewing the body of knowledge through various activities, such as quizzes and playing games during the lessons in each content category. It helps the learner to review his knowledge in various subjects from time to time, as noted by AlMarzooqi et al. (2018), who stated that an online study often focuses on gauging knowledge and attitudes in the performance of work as important outcomes [25]. The use of the online learning system through virtual case studies about promoting the safety of patients aids the ability of the student to analyze a situation and to practice problem-solving according to the context [13, 25]. When testing the knowledge gained after studying, it has been found that medical students have higher average knowledge scores after their study than they had before their study, with a statistical significance of .01. They also have higher positive attitude scores on the prevention of mistakes in their practice with humans, with a statistical significance of .01 [26, 27]. Currently, the World Health Organization supports the use of the online training system for the development of a diversity of medical personnel, such as basic courses on evidence-based medicine e-course, vaccine safety, basic e-learning course, cervical cancer prevention e-learning course, and gender and humanitarian action. Not only does the online training system help to prepare basic knowledge in vaccine work for immunization providers, but is also capable of practical use for the expansion of knowledge in situations in which there is a spread of contagious-disease groups [20]. For instance, in a study conducted by Baggier et al. (2020), online training through an e-learning course for immunization service providers was found to be statistically significant in its effectiveness [28]. The course, which was centered on a campaign of measles and rubella prevention that occurred in Italy, aided the trainees in acquiring more knowledge than they had before their training. Trainees were also capable of correctly managing any outbreaks of measles or rubella they may encounter.

However, regular supervision can also contribute to their developing body of knowledge in their routine work since coaching as they work the practitioners to undergo constant improvement [29]. It was found from a literature review that the factors that influence immunization service skills depend on the nature of the work that has been assigned and a training experience that provides knowledge related to vaccine work, including supervisors characterized by their knowledge and skills in vaccine work [30]. However, from the nature of the health care services of the primary care units, it was found that the aspects of health reinforcement cover many areas of work, including the urgent mission of the campaign to build coverage for the anti-COVID vaccine. As a result, none of the practitioners will have every possible occasion to access the training system [31, 32]. Currently, various formats of training are being paired together with routine work to develop a body of knowledge that will comply with the standard criteria for immune system enhancement [33, 34]. In addition to the three-day workshops that comply with the curriculum of practice for immunization providers in Thailand [6], the work of immunization takes on other formats as well, including weekly two-hour vaccine knowledge reviews prior to service over a continual eight-week period (in-service training). The results of the study demonstrated, with statistical significance, that the knowledge test scores after training were higher than they were before training [35, 36], which is consistent with the results of this present study which used the online training system to develop the knowledge base of the practitioners. Moreover, it was found that the average knowledge test scores after learning with the EPI e-course training were higher than they were before learning in each of the content categories. The reason for these higher scores is that the training process in each content category entails a diversity of media formats that incentivize the student to review the content repeatedly [37, 38]. The online training system is even capable of displaying a self-learning progress bar for the student to view in each content category. The student is thus freely able to lay out a plan of learning. With each weekly evaluation, it has been found that test result scores after learning with the online training system continue to increase, matching the results of a study conducted by Brown et al. (2017), who studied the results of a program of immunization provider development in the nation of Nigeria by organizing two-day courses in academic rehabilitation training. Results of the research showed that the experimental group [39] that had undergone the training had knowledge test scores after their training that were higher than those of the control group. After gauging the results three months and six months after training, the healthcare providers still had average scores after their training that were higher than the group that had received only regular supervision. It can thus be seen that short-term training or training with the features of academic rehabilitation training can assist the development of a body of knowledge in vaccine work, but with an emphasis on basic vaccines that comply with the immunization plan of work [34, 40].

Currently, it has been found that new vaccines are being introduced into the plan of work for immunization services from time to time [41]. The aforementioned training program has thus been forced to modify its content completely, increasing the amount of content already embedded in the online system and causing long delays that have impeded learning through the online system [42]. A new format of learning was thus developed by integrating virtual reality technology into the training program for medical personnel. It was found from the results of our study that once we started the program that enabled the medical students to begin training on the online system and create their mock avatars, they were delighted. Members of the sample groups possessed both knowledge and understanding. After their training, they were able to offer some appropriate advice to the people on possibly getting new vaccines. There has been a statistically significant increase in the rate of coverage for anti-cervical cancer vaccines [43]. From the standpoint of systematically providing information pertaining to vaccines, it has been found that immunization providers who underwent the training were able to make practical use of the medical data in their decision of a correct choice of a vaccine. They were able to provide information to their service recipients. However, they were not able to respond to questions on complicated issues. For instance, there would be limitations in administering a vaccine in the case of a child who had contracted cancer [44]. Accordingly, while the online training program may take on various formats, it has already helped to develop a body of knowledge for immunization providers. In carrying out the work of immunization, online training should be provided alongside follow-up supervision of the immunization work at a primary level to develop a concrete operational standard over the long term for the future.

4.1. Limitations and Suggestions

In view of the limitations of work performance imposed by the circumstances of the spread of the coronavirus, volunteers must accelerate the fulfillment of their urgent tasks to take control of the pandemic situation, which is preventing continual participation of thethe research program. Future research should take on an online study format consistent with the type of vaccines that are under the plan of work for immunization but separate from emergency vaccines and pilot vaccines of other types. The next issue should be a study of an online training format consistent with the features of an immunization service provider; various types of which include nurses, public health officials, academicians, physicians, and pharmacists, among others.

CONCLUSION

The results of this research indicate that the use of the online training system coupled with supervision in the work of immunization helps to increase the knowledge of immunization providers. Online training should be embedded as a basic method of personnel development, especially for those who work at the primary care level, which is the first line of service and important in driving the work of immunization for people.

LIST OF ABBREVIATIONS

PDCA = Plan, Do, Check, and Act
MOOCs = Massive Open Online Course

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

The study was approved by the Human Research Ethics Committee of Thammasat University (Science), (HREC-TUSC) #COA No. 081/2563.

HUMAN AND ANIMAL RIGHTS

No animals were used for the studies that are the basis of this research. All human procedures followed were following the guidelines of the Helsinki Declaration of 1975.

CONSENT FOR PUBLICATION

Participation in this study was voluntary and informed consent was obtained from all participants.

AVAILABILITY OF DATA AND MATERIALS

The data that support the findings of this study are available from the corresponding author, [P.K], upon reasonable request.

FUNDING

This article has been performed with the financial support of the Thammasat University Research Fund (Funder Id/Grant No. TUFT 23/ 2564).

CONFLICT OF INTEREST

The authors declare no conflict of interest or otherwise.

ACKNOWLEDGEMENTS

The authors would like to thank the healthcare team for their collaborative efforts in making this study success. The authors also thank the Thammasat University for their financial support by Thammasat University Research Fund, Contract No. TUFT 23/2564.

REFERENCES

[1] National Vaccine Institute. Twenty-year National Vaccine Institute Strategic Plan. 2017. Available from: http://www.nvi.go.th/index.php/blog/2017/05/s013
[2] Rowe AK, Rowe SY, Peters DH, Holloway KA, Ross-Degnan D. The effectiveness of training strategies to improve healthcare provider practices in low-income and middle-income countries. BMJ Glob Health 2021; 6(1): e003229.
[3] Nicol E, Turawa E, Bonsu G. Pre- and in-service training of health care workers on immunization data management in LMICs: A scoping review. Hum Resour Health 2019; 17(1): 92.
[4] Division of Vaccination. Department of Disease Control. Vaccine preventable diseases action plan. 2019. Available from: http://dvpd.ddc.moph.go.th/content/view/165
[5] Division of Vaccination. Department of Disease Control Expanded program on immunization standards manual year 2019. Nonthaburi: Department of Disease Control, Ministry of Public Health 2019.
[6] Siripitayakunkit U, Varinsathien P, Jiamsiri S, Rutchanagul P, Sonthichai C, Chobtum P, Eds. Training course for EPI workers 2561. 3rd ed. Bangkok: Amarin Printing and Publishing 2018.
[7] Younge VL, Borycki EM, Kushniruk AW. On-the-job training of health professionals for electronic health record and electronic medical record use: A scoping review. Magyar Nyelv 2015; 7(3): 436-69.
[8] Butura M. Advantages and Shortcomings of Online Training in the University Environment. Revista Universitara de Sociologie 2021; 2021(2): 196-205.
[9] Beinicke A, Bipp T. Evaluating training outcomes in corporate e-learning and classroom training. Vocat Learn 2018; 11(3): 501-28.
[10] Arogundade L, Akinwumi T, Molemodile S, et al. Lessons from a training needs assessment to strengthen the capacity of routine immunization service providers in Nigeria. Biomed Central 2019; 19(1): 664.
[11] Watkins KE, Sandmann LR, Dailey CA, Li B, Yang SE, Galen RS, et al. Accelerating problem-solving capacities of sub-national public health professionals: an evaluation of a digital immunization training intervention. Biomed Central 2022; 22(1): 736.
[12] Sarmiento K, Daugherty J, Waltzman D. Effectiveness of the CDC HEADS UP online training on healthcare providers’ mTBI knowledge and self-efficacy. J Safety Res 2021; 78: 221-8.
[13] Shalansky RA, Wu M, Shen SC, et al. Evaluation of a pilot immunization curriculum to meet competency training needs of medical residents. BMC Med Educ 2020; 20(1): 442.
[14] Noppanatwongsakorn R, Rodpleng P, Noiwilai A. Training:heart of human resource development. The 3rd SAU National Interdisciplinary Conference.
[15] Amit S, Karim R, Kafy AA. Mapping emerging massive open online course (MOOC) markets before and after COVID 19: A comparative perspective from Bangladesh and India. Spat Inf Res 2022; 30(5): 655-63.
[16] Zhu H, Xu J, Wang P, et al. The irreplaceable role of medical massive open online courses in China during the COVID-19 pandemic. BMC Med Educ 2023; 23(1): 323.
[17] Shahriar SHB, Akter S, Sultana N, Arafat S, Khan MMR. MOOC-based learning for human resource development in organizations during the post-pandemic and war crisis: A study from a developing country perspective. J Res Innov Teach Learn 2023; 16(1): 37-52.
[18] Bettiol S, Psereckis R, MacIntyre K. A perspective of massive open online courses (MOOCs) and public health. Front Public Health 2022; 10: 1058383.
[19] Emerson LC, Berge ZL. Microlearning: Knowledge management applications and competency-based training in the workplace. KM&EL 2018; 10(2): 125-32.
[20] Wu S, Roychowdhury I, Khan M. Evaluating the impact of healthcare provider training to improve tuberculosis management: a systematic review of methods and outcome indicators used. Int J Infect Dis 2017; 56(C): 105-10.
[21] Archiwaranguprok S. Practical PDCA: A step by step for an effective work improvement and problem solving. 4th ed. Bangkok: Thailand Productivity Institute 2009.
[22] Burns N, Grove SK. The practice of nursing research: Conduct, critique and utilization. 5th ed. Missouri: Elsevier Saunders 2005.
[23] Rutchanagul P, Kaewwimol P, Rompipat S. Developing an E-course training for EPI workers (research report). Pathum Thani: Faculty of Nursing, Thammasat University 2019.
[24] Rutchanagul P, Kaewwimol P, Rompipat S. Developing a model for vaccine injection in children (research report). Pathum Thani: Faculty of Nursing, Thammasat University 2020.
[25] AlMarzooqi LM, AlMajidi AA, AlHammadi AA, AlAli N, Khansaheb HH. Knowledge, attitude, and practice of influenza vaccine immunization among primary healthcare providers in Dubai health authority, 2016-2017. Hum Vaccin Immunother 2018; 14(12): 2999-3004.
[26] Samadbeik M, Yaaghobi D, Bastani P, Abhari S, Rezaee R, Garavand A. The applications of virtual reality technology in medical groups teaching. J Adv Med Educ Prof 2018; 6(3): 123-9.
[27] Ekstrand C, Jamal A, Nguyen R, Kudryk A, Mann J, Mendez I. Immersive and interactive virtual reality to improve learning and retention of neuroanatomy in medical students: A randomized controlled study. CMAJ Open 2018; 6(1): E103-9.
[28] Baggieri M, Barbina D, Marchi A, et al. Measles and rubella in Italy, e-learning course for health care workers. Ann Ist Super Sanita 2019; 55(4): 386-91.
[29] Ward K, Stewart S, Wardle M, et al. Building health workforce capacity for planning and monitoring through the Strengthening Technical Assistance for routine immunization training (START) approach in Uganda. Vaccine 2019; 37(21): 2821-30.
[30] Park E, Yoon J, Choi E-k, Kim IR, Kang D, Lee S-K. A train the trainer program for healthcare professionals tasked with providing psychosocial support to breast cancer survivors. BMC Cancer 2018; 18
[31] Thaivalappil A, Young I, MacKay M, Pearl DL, Papadopoulos A. A qualitative study exploring healthcare providers’ and trainees’ barriers to COVID-19 and influenza vaccine uptake. Health Psychol Behav Med 2022; 10(1): 695-712.
[32] Jungsomjatepaisal P. Thailand’s primary health care in global health context. J Health Sci 2020; 29(1): 152-6.
[33] Jibo AM, Karaye RS, Gajida AU, Abulfathi AA. Knowledge of healthcare workers on immunization: new and under-utilized and towards future vaccines. J Biomed Res Clin Pract 2019; 2(4): 202-7.
[34] Bertsch TG, McKeirnan KC. Perceived benefit of immunization-trained technicians in the pharmacy workflow. Pharmacy 2020; 8(2): 71.
[35] Thomas MP, Kozikott S, Kamateeka M, et al. Development of a simple and effective online training for health workers: Results from a pilot in Nigeria. BMC Public Health 2022; 22(1): 551.
[36] Sato APS. National Immunization Program: Computerized system as a tool for new challenges. Rev Saude Publica 2015; 49(0): 39.
[37] Traicoff D, Tchoualeu DD, Opare J, et al. Applying adult learning best practices to design immunization training for health care workers in Ghana. Glob Health Sci Pract 2021; 9(3): 487-97.
[38] Pahud B, Elizabeth Williams S, Lee BR, et al. A randomized controlled trial of an online immunization curriculum. Vaccine 2020; 38(46): 7299-307.
[39] Brown VB, Oluwatosin OA, Ogundeji MO. Impact of training intervention on immunization providers’ knowledge and practice of routine immunization in Ibadan, South-western Nigeria: A primary health care experience. Pan Afr Med J 2017; 26: 1-15.
[40] Shehata HB, El-Samman GA, Ahmed SM. Effect of Educational program for nurses on their knowledge regarding obligatory vaccination for children. MSNJ 2019; 6(1): 175-82.
[41] Donadel M, Panero MS, Ametewee L, Shefer AM. National decision-making for the introduction of new vaccines: A systematic review, 2010–2020. Vaccine 2021; 39(14): 1897-909.
[42] Fauville G, Luo M, Queiroz ACM, Bailenson JN, Hancock J. Zoom exhaustion & fatigue scale. Comp Hum Behav Rep 2021; 4: 100119.
[43] Real FJ, Ollberding NJ, Meisman AR, et al. Impact of a virtual reality curriculum on human papillomavirus vaccination: a pilot trial. American journal of preventive medicine. Am J Prev Med 2022; 63(5): 865-73.
[44] Hastings TJ, Ha D, Fox BI, Qian J, Lakin J, Westrick SC. Increasing use of immunization information systems for routine vaccinations in independent community pharmacies: A randomized controlled trial. J Am Pharm Assoc 2022; 62(4): 1270-9.