RESEARCH ARTICLE


Development of a Biosecurity Index for the Districts of India for the Assessment of Capacity to Cope with Infectious Disease Outbreak



Navya Vyas1, *, Prakash Narayanan1, Vishwanath Shashidhar2, Shah Hossain1
1 Department of Health Policy, Prasanna School of Public Health, Manipal Academy of Higher Education, Manipal, Karnataka, 576 104-India
2 Department of Microbiology, Kasturba Medical College, Manipal Academy of Higher Education, Manipal, Karnataka, 576104-India


Article Metrics

CrossRef Citations:
0
Total Statistics:

Full-Text HTML Views: 719
Abstract HTML Views: 344
PDF Downloads: 240
Total Views/Downloads: 1303
Unique Statistics:

Full-Text HTML Views: 390
Abstract HTML Views: 226
PDF Downloads: 190
Total Views/Downloads: 806



Creative Commons License
© 2021 Vyas et al.

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

* Address corresponding to this author at Department of Health Policy, Prasanna School of Public Health, Manipal Academy of Higher Education, Manipal, Karnataka, India; Tel: 9538126121; E-mail: navya.vyas@manipal.edu


Abstract

Background:

In the context of newly emerging diseases, global health security has gained importance in recent years. The pandemic COVID-19 has reemphasized the significance. Despite increased attention to enhance health security, the existing capacities were not optimally prepared to respond to public health threats posed by emerging infectious diseases.

Objective:

The objective of the study was to develop a simple tool that can help monitor and build up the critical capacity to prevent, detect, respond and help identify the gaps in the preparedness of the health system.

Methods:

The cross-sectional study was conducted from January 2018 – December 2018 amongst the district level health functionaries like the disease surveillance, laboratory, and the animal health focal points in the selected districts across the three South Indian states.

Results:

The responses received from the participants of the study were scored and categorized by domains into a biosecurity index, which was easily adaptable to all districts and easy to implement by a district official or state health officer for the initial assessment. Data acquired from this tool could be analysed to build and inform readiness and response plans for preventing and controlling health emergencies.

Conclusion:

The biosecurity index developed for the Indian districts was an appropriate instrument for epidemic preparedness assessment.

Keywords: Biosecurity index, Outbreaks, Emerging infectious diseases, Health system preparedness, Pandemic preparedness, Pandemic.



1. INTRODUCTION

Coronavirus disease 19 (COVID-19) pandemic has reinforced the importance of global health security and the need to strengthen frontline services for an efficient response. The worldwide toll of cases crossed approximately 110 million, and the deaths surpassed approximately 24.4 million by the end of February 2021 [1]. The impact of the pandemic on livelihood, the health system, and the economy was huge. The significant progress made in reducing the extreme poverty in the world since the 1990s has been crippled by the pandemic, affecting low-income households [2, 3]. For the low- and middle-income countries, the increased level of debt and the additional impact of the pandemic have increased their struggle to finance their public health and social and economic responses to COVID-19 [4, 5]. All countries across all income groups were overwhelmed by the demanding situation because of the pandemic. In the recent past, public health emergencies such as the plague in Madagascar and Ebola virus disease in West Africa required external support from World Health Organization (WHO) and international organizations to scale operational efforts. In response to COVID-19, countries like Taiwan, South Korea, Hong Kong, and Mongolia which were quick to initiate appropriate response and control operations, were those that had invested in developing capacity for implementation of the International Health Regulations (IHR) and also had prior experience of outbreak management against Severe Acute Respiratory Syndrome (SARS) in 2003 [6-9].

Before the current pandemic, SARS pushed the change for IHR; Ebola in West Africa brought in the question of adequate global preparedness for health emergencies [10]. Ebola in West Africa brought in Global Health Security Agenda (GHSA), and GHSA has been made part of the new IHR for overall pandemic and epidemic preparedness readiness [11-13]. The capacity for responding to outbreaks of international concern was monitored as part of IHR preparedness in the context of the incessant appearance of a novel pathogen practically every year or two, for which most of the world was ill-prepared [14-16]. Thus pandemic and epidemic preparedness readiness has become synonymous with the capacity for coping up with any of the public health emergencies, in detecting an outbreak of the epidemic-prone communicable disease in an area, to one public health emergency of international concern and or one bio event that is human-induced [15]. Disease outbreaks can occur anywhere, anytime, with consequences that can shatter communities [17].

The need of the hour is to develop a critical determinant for identifying and containment of outbreaks by the health systems to prevent, detect and respond [18]. A simple tool that can help assess the critical capacity for this prevention, detection, and response capability, if available, can help identify gaps in preparedness and help appropriately directed response. With this background, we developed a tool with 10 questions that can be administered to the district public health officials, and the results would indicate the level of overall epidemic/pandemic preparedness and point out specific gaps.

2. MATERIALS AND METHODS

The preparedness for public health emergencies is characteristically considered in terms of response, health capacity, and surveillance. These functions, in turn, rely on various components of the health system. Accordingly, we developed an initial framework with 89 questions across the major 3 domains (prevent, detect and respond) (Appendix). These covered the areas like emerging infectious diseases prevention, zoonosis, biosafety, biosecurity, IHR; surveillance system, laboratory capacity, vector surveillance, Integrated Disease Surveillance Programme (IDSP) training; medical countermeasures, coordinated response, surge capacity, rapid response team, and uniform guideline (Table 1). For the early detection, alleviation, and control of infectious disease outbreaks, the role of an effective public health system was vital. The presence of a sensitive surveillance system and laboratory for timely identification and tracking of emerging infectious disease outbreaks were also vital. This capacity requires health infrastructure, human resources, intersectoral coordination, surveillance experience, and training among the health workers. The preparedness for response against public health emergencies was measured in terms of the presence of emergency operation centres, medical countermeasures, stockpiling policy, and the surge capacity of the health system. The existence of a response team with multisectoral representation and an effective system for risk communication is vital for the response operation in an emergency situation. After finalising the questions in the tool, the validation was done by experts in the field of infectious disease and surveillance.

Table 1. Domains and sub-domains in the questionnaire.
Prevent
Sub Domains Assessed Number of Items in Questionnaire
National Legislation and Policy 1
Population Density 3
Emerging Infectious Diseases Prevention 4
Zoonosis 4
Immunization 3
Biosafety 2
Biosecurity 2
IHR _Knowledge 3
IHR_Experience 6
Detect
IDSP 3
Surveillance System 3
Surveillance Capacity 4
Surveillance Analytic Capacity 1
Surveillance Training 2
Surveillance Experience 4
Surveillance Coordination 3
Lab Capacity 6
Vector Surveillance 2
Respond
Disaster Mitigation 4
Uniform Guidelines 2
Data Driven Approach 4
Emergency Operation Centre 4
Medical Counter Measures 3
Stockpile Policy 4
Surge Capacity 6
Rapid Response Team 3
Coordinated Response 3

In the cross-sectional study conducted from January 2018 - December 2018, the initial assessment of the tool was performed among the district officials in 18 districts across three south Indian states, which was almost one-third of the districts across the three south Indian states. After obtaining the ethical clearance from the Institutional Ethics Committee of Kasturba Medical College and Kasturba Hospital, Manipal (IEC 621/2017), the study was initiated. The tool was administered in the English language to the three officials within the district who were the focal point of contact for response to an infectious disease outbreak in the district. They included the District Surveillance Officer (DSO) and the District Microbiologist (DM) from the Department of Health and Family Welfare and the District Animal Husbandry Officer (DAHO) from the Department of Animal Husbandry and Veterinary Services, who was the focal point of contact for outbreaks of zoonotic origin. The testing of the internal consistency of the questionnaire was done via Cronbach’s alpha coefficient, which indicates the extent to which items in a tool measure various aspects of the same characteristic or construct [19]. Cronbach’s alpha (α) greater than 0.7 was indicative of good internal consistency [20].

In order to enhance the usage of the tool for the assessment of the preparedness of the health system among the public sector officials, an index was developed from the tool. The index was finalised in a three-step process. First, certain questions were recategorized based on their ease of response and the score secured by the district officials and based on the comprehensibility of the responsibilities of the district officials in the field. Secondly, the other criteria considered for reducing the components were based on the relevance of the components to public health security. Lastly, some capacities and technical areas having similar types of responses were combined, notably IHR coordination and reporting, national laboratory system, and biosafety and biosecurity.

3. RESULTS

In the study, the internal consistency of the tool was α = 0.84. This indicates that all items in the study were internally consistent to assess the preparedness of the health system. (Table 2). The index developed from the tool (Table 3) after re-categorization had ten questions. The tool developed constituted of components that look into the capacity of the district and the existence of a functional system for stopping outbreaks at the source. The questions were constituted to assess the collaborative efforts to address the zoonotic activities and the preventive activities against outbreaks of emerging infectious diseases in the district. It was also intended to map important outbreak-prone diseases, strengthen the early warning alert and response system, and build the capacity of rapid response teams. The questions were designed to explain the capacity of the health system for confirmation, assessment of risk, intersectoral coordination, and analysis, which were vital elements like the prevention, detection, and control of infectious disease outbreaks.

For each of the ten questions, we developed an index score that can be calculated with the following steps:

(1) Each of the responses of the officials was assigned zero for no or wrong response and one for a correct or affirmative response.

(2) The final score was calculated after taking an average of the scores of the three officials.

(3) The scores calculated were categorised, and an optimal score of above 75th percentile is coded green, between 50th and 75th percentile as orange and those below the 50th percentile as red, which operationally indicates the task ahead for the district public health office (Table 4).

Initially, district X is chosen to assess the preparedness of the health system for a public health emergency. Subsequently, the district officials need to be identified for the assessment. After an interview with the identified officials, the information is compiled. Scores are assigned for the responses, like 1 for correct response and zero for no or wrong response. For example, if the score of District Surveillance Officer is 10, District Microbiologist is 4, and District Animal Husbandry Officer is 5 then the total will be 19. Computation of the average of the scores of the 3 officials makes it 6.3. This score is then categorised according to the criteria of the biosecurity index score. As the average score is 6.3, the district will fall in the orange category.

Table 2. The reliability statistics of the questionnaire.
Domains Number of Items Cronbach’s
Alpha
Prevent 28 0.60
Detect 28 0.71
Respond 33 0.79
Total 89 0.84
Table 3. The components of the Biosecurity Index for the Indian districts.
Sl. Question DSO DM DAHO
1 Name 5 diseases causing an outbreak in your district (over the last 3 years). X X X
2 Name a disease outbreak that happened in your district in the aftermath of a natural disaster (over the last 3 years). X X X
3 Are you formally trained in IDSP and or IHIP? X X 0
4 In your official stock, are there vaccines that can be used in public health emergencies? X 0 X
5 For diagnosis of Diphtheria, do you have facilities in the district or outside? X X 0
6 Is there a designated laboratory to send specimens for diagnosis of unknown disease? X X X
7 Is there one or more private laboratories that can help epidemic disease detection in the district? X X X
8 When is the last time your office was involved in an outbreak investigation in the district? X X X
9 Are trained entomologists (one or more) available in the district? X 0 0
10 Any instance when health personnel from outside the district were deployed for disaster /epidemic response? X 0 0
Table 4. Criteria and Definition for the Biosecurity index score.
SNO: Score Range Colour Coding Classification for the levels in the Biosecurity index score
1 > 7.5 Green The district’s capacity is well advanced and maintainable at all levels of health systems.
2 5-7.5 Orange The district is functional, but effectiveness is low at the intersectoral coordination level
3 < 5 Red Very little capacity to prevent and control the risk or an event

4. DISCUSSION

The biosecurity index was framed based on the Joint External Evaluation (JEE) Technical Areas and State Parties Self - Assessment Annual Reporting (SPAR) 2016 self-evaluation questionnaire [21, 22]. It broadly covers most of the relevant sections, particularly the 13-point SPAR, as applicable to a district, unlike the mandate of SPAR for nations, states, and provinces. It is distinct from other preparedness indices like the Epidemic Preparedness Index (EPI), which was developed to assess the preparedness at the national level and constitutes five sub-indices measuring a country’s economic assets, public health communications, structure, public health systems, and capacity of the institutions [22]. The unique feature of the index is that it is in line with the SPAR tool, which is a part of the monitoring and evaluation framework for IHR for the country. Besides that the index is locally relevant and easy to use; it is pragmatic to implement in similar settings.

The robustness of this index can be furthered by including a small record-based survey mostly available from the district and census data in the public domain. The index has been designed for the district officials. Few responses of the officials are subjective in nature. Therefore, the inclusion of the data from an open-source helps to validate the district officials’ responses. For example, the validation of information on the availability of the entomologists in the district stated by the district office will be verified from the data in the open-source. This, in turn, helps in realistically capturing the information of the preparedness of the health system at a sub-national level to detect and respond to any disease outbreaks.

The suggested data inputs were: population density of the district, population per Primary Health Centre (PHC) in the district (total number of PHCs and population by National Health Mission (NHM)/Health Management Information System (HMIS)/Census), Public Bed Strength in the district per 10,000 population, Private Bed Strength in the district (including nursing homes and maternity homes) per 10,000 population, the total number of animal health facilities in the district, the total number of veterinary personnel available in the district, and the total district livestock population by the latest animal/livestock census.

The matrix for assessing the capacity of the district that we present has significant benefits compared to other existing tools. First and foremost, the framework is designed for districts and hence differs from existing metrics for epidemic preparedness designed for countries or at the national or federal level. The index developed is easily adaptable as the components of the index are constituted by the critical functions of the health personnel, which are essential in supporting the effective functioning of the system. It is easy to be implemented by a district official or state health officer for initial assessment and to monitor the progress of the capacity building of various public health sectors and response. The information and data acquired from this index can be analysed to build and inform readiness and response plans for preventing and controlling health emergencies. These results and the lessons learned from preceding infectious disease outbreaks will enable the districts and the country to strengthen the capacities to successfully detect and verify an event of public health importance.

5. LIMITATIONS

The index has been designed with a limited number of questions to make it simple and easy to implement. Therefore, analysis of other risk variables, including susceptibilities due to socioeconomic conditions, linking public health with security forces, and food security associated with tackling an infectious disease outbreak, would assist in understanding health security capacity. However, it was not taken into account in this tool. This, in turn, can influence the actual biosecurity index scores. Moreover, the index has few questions which involve the element of subjective responses from the district officials.

CONCLUSION

The pandemic COVID-19 has given an opportunity to assess the preparedness of the states and the districts and apply key learnings from the pandemic and the expert suggestions from other major public health emergencies to be prepared against future health emergencies. Hence, having metrics for district epidemic preparedness is imperative to ensure the responsibility of the state parties under the IHR and address the gaps in capacity to detect and manage infectious disease hazards.

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

The ethical clearance for the conduct of the study was given by the Institutional Ethics Committee of Kasturba Medical College and Kasturba Hospital, Manipal, India (IEC621/2017).

HUMAN AND ANIMAL RIGHTS

Not applicable.

CONSENT FOR PUBLICATION

Written consent was obtained from the participant.

STANDARDS OF REPORTING

STROBE guidelines and methodology were followed.

AVAILABILITY OF DATA AND MATERIALS

The data supporting the findings of the article are available from the corresponding author [N.V] upon reasonable request.

FUNDING

None.

CONFLICT OF INTEREST

The authors declare no conflict of interest, financial or otherwise.

ACKNOWLEDGEMENTS

We would like to extend our gratitude to all officials for their cooperation during the process of data collection.

APPENDIX

  1. Name 5 Emerging Infectious Diseases in your district that were reported over the last 3 years.
    1. How many of these were of animal origin?
    2. How many of these were vector-borne?
    3. How many of these were vaccine-preventable?
    4. Are there any pathogen in these that has bio-warfare or bioterrorism potential?
    5. Are there any pathogens in these that can be controlled in animals to save humans?
    6. How many of these pathogens are reportable under International Health Regulations?
  2. What is the hierarchy of reporting of outbreaks in the Integrated Disease Surveillance Program?
    1. What is the channel through which you reported a recent outbreak?
    2. Are you aware of a similar channel in the animal health sector? Kindly name one?
    3. What are the specialties represented in your district Rapid Response Team?
    4. How many members are there in the District Rapid Response Team?
    5. What is your internal rating of working with the Integrated Disease Surveillance Program in your district?
  3. How do you prioritize disease for public health action?
    1. Mention 3 criteria that were used for the prioritization.
    2. Please name a few diseases that one can effectively control with animal vaccination.
  4. Have you seen a typical case of Kyasanur Forest Disease in the recent past?
    1. What specimen would you collect, and how would you transport it?
    2. If you suspect a case of Kyasanur Forest Disease, where is the sample sent for confirmation?
    3. How do you treat Kyasanur Forest Disease?
    4. In the district, who keeps a stock of specific therapeutic agents?
    5. How many vaccines are there in normal stock that can be used in a disaster situation? Please name a few.
  5. When is the last time a natural disaster or manmade disaster happened in your district?
    1. What was the nature of the disaster?
    2. How did you control it?
    3. Did you have any disease outbreak in the aftermath?
    4. Did it involve laboratory support? If yes, which one?
  6. Did you ever prepare daily reports or reporting situations or status reports on any outbreak for higher-ups?
    1. Do you use your computer for data analysis functions related to outbreaks?
    2. Do you have staff trained in disease surveillance?
    3. Did you have formal training in Integrated Disease Surveillance Program?
    4. Did you have any other training in Emerging infectious diseases /Surveillance?
  7. Have you been involved in an incident of Avian Flu?
    1. Did you experience an outbreak or case of Human Brucellosis?
    2. In your work, how many times have you had human/animal health coordination as the focal point from the public health side?
    3. Did you have a meeting with Animal Health Officer in the recent past on outbreak or disease control?
    4. What would this channel be if the infection is of animal origin?
  8. Is there a surveillance system in place for priority zoonotic diseases/pathogens in the district?
    1. Do you have public health-trained veterinarian in the district?
    2. Do you have a biosafety system in place?
    3. Did you ever use the Integrated Disease Surveillance Program reporting system for outbreak reporting?
    4. Did you in the district use the Integrated Disease Surveillance Program reporting channel for any animal disease?
  9. Do you have a formal incident management system in the district?
    1. Were you directly involved in running an Emergency operation centre in the district?
    2. Please describe the background in which it was done and your role in it.
    3. Did you ever use the Integrated Disease Surveillance Program reporting system for outbreak reporting? Please give an example.
  10. How many of your units’ report in Integrated Disease Surveillance Program Laboratory form?
    1. Who handles disease data in your unit?
    2. By performance measure, Integrated Disease Surveillance Program system in the district is more efficient in Syndromic, Presumptive and Laboratory Surveillance forms reporting or reporting of an outbreak?
    3. When do you report an outbreak? How do you determine it is an outbreak?
    4. If you have an outbreak, who is the person that you report to?
  11. What are the major vectors causing Dengue and Chikungunya in the district?
    1. Do you have lab capacity for the diagnosis of dengue in the district?
    2. What are the major malarial vectors in the district?
    3. How frequently do you have to share data or joint meetings with Vector Borne Disease Control Program Officer?
  12. Is there a private lab in the district that can strengthen disease detection capacity?
    1. How many laboratories for the district, including referrals outside the district, for diagnosis of emerging infectious disease are there?
    2. Was there biosafety training in the laboratories in the last 3 years?
  13. Give an example of an outbreak that you reported quickly with details?
    1. Have you been part of an outbreak investigation involving outside the district or state? Please share the experience.
  14. When is the last time that you interacted with media on an outbreak?
    1. When is the last you addressed a meeting of health professionals in the context of outbreak control? Please narrate the context and the audience.
    2. This year, how many assembly questions came to you for a response?
  15. What is the coverage of measles vaccination per your estimate?
    1. Did you use animal vaccination for prevention and or control of the outbreak in the district?
    2. Do you keep Diphtheria antitoxin in stock for the district?
    3. Do you have a policy of strategic stockpiling of drugs or vaccines in the district? If so, which ones?
  16. Is there a contingency or disaster mitigation plan in the district that addresses controlling the outbreak?
    1. Which, of all specimen, do you collect for the cholera epidemic?
    2. What is the lab report on the basis of which you declare an outbreak of cholera cases?
    3. Which is the lab in or outside the district where you sent a cholera stool sample for confirmation of diagnosis?
    4. What are the clinical findings and lab tests that you confirm before declaring a case of Diphtheria?
    5. In your district, which test do you use to ascertain bacterial water quality?
    6. What are the preventive actions that you would initiate in times of cholera outbreak?
  17. Are there any medical colleges in the district (Government/Private)? How many?
    1. Total number of private hospitals in the district? Total private and public bed strength in the district?
    2. How many hospitals are there in the district which have district hospital level or higher services?
    3. How many rural or taluk hospitals are there in the district?
    4. What is the total storage capacity in your cold chain at district headquarters?
    5. What is the annual consumption of Oral Rehydration Solution in the district?
  18. What is the livestock and poultry population in the district?
    1. What is the number of animal health facilities in the district?
    2. What is the number of veterinary personnel available in the district?
  19. What is the population density in the district?
    1. What is the population per Primary Health Centres and Community Health Centre in the district?
    2. What is the population per health personnel available in the district?
    3. How many physicians are available in the district for clinical and emergency duty?
    4. Total number of additional health personnel available in Anganwadi centre in the district?
    5. What is the total number of accredited social health activists working in the district?

REFERENCES

[1] World Health Organization. Available from: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200728-covid-19-sitrep-190.pdf?sfvrsn=fec17314_2
[2] International Monetary Fund. 2020. Available from: https://www.imf.org/en/Publications/WEO/Issues/2020/06/24/WEOUpdateJune2020[accessed: June 2020].
[3] McKinsey & Company. 2020. Available from: https://www.mckinsey.com/~/media/McKinsey/About%20Us/COVID%20Response%20Center/PDFs/COVID-19-response-checklist-for-contact-centers.pdf
[4] The Organization for Economic Cooperation and Development. Available from: http://www.oecd.org/coronavirus/policy-responses/the-impact-of-the-coronavirus-covid-19-crisis-on-development-finance-9de00b3b/
[5] Global Spending on Health. A World in Transition 2020. Available from: https://www.who.int/health_financing/documents/health-expenditure-report-2019.pdf?ua=1
[6] World Health Organization. International health regulations 2020. Available from: https://www.who.int/health-topics/international-health-regulations#tab=tab_1
[7] Kandel N, Chungong S, Omaar A, Xing J. Health security capacities in the context of COVID-19 outbreak: an analysis of International Health Regulations annual report data from 182 countries. Lancet 2020; 395(10229): 1047-53.
[8] Kwok KO, Li KK, Chan HHH, et al. Community responses during early phase of COVID-19 epidemic, Hong Kong. Emerg Infect Dis 2020; 26(7): 1575-9.
[9] Erkhembayar R, Dickson E, Badarch D, et al. Early policy actions and emergency response to the COVID-19 pandemic in Mongolia: experiences and challenges. Lancet Global Health 2020; 1-8.1-8.doi.org/10.1016/
[10] Ijaz K, Kasowski E, Arthur RR, Angulo FJ, Dowell SF. International Health Regulations--what gets measured gets done. Emerg Infect Dis 2012; 18(7): 1054-7.
[11] Tappero JW, Cassell CH, Bunnell RE, et al. US centers for disease control and prevention and its partners’ contributions to global health security. Emerg Infect Dis 2017; 23(13): S5-S14.
[12] Osterholm MT. Global health security—an unfinished journey 2017. Emerg Infect Dis 2017; 23(13)
[13] Montgomery JM, Woolverton A, Hedges S, et al. Ten years of global disease detection and counting: program accomplishments and lessons learned in building global health security. BMC Public Health 2019; 19(3)(Suppl. 3): 510.
[14] World Health Organization. Public health preparedness and response 2017. Avialable from: http://apps.who.int/gb/ebwha/pdf_files/WHA71/A71_8-en.pdf [Accessed 9 Feb 2020.]
[15] International Health Regulations. 2005. Avialable from: https://extranet.who.int/sph/sites/default/files/document-library/document/WHO-WHE-CPI-2018.16-eng.pdf
[16] Talisuna A, Yahaya AA, Rajatonirina SC, et al. Joint external evaluation of the International Health Regulation (2005) capacities: current status and lessons learnt in the WHO African region. BMJ Glob Health 2019; 4(6): e001312.
[17] Global Preparadness Monitoring Board. Avialable from: https://apps.who.int/gpmb/assets/annual_report/GPMB_annualreport_2019.pdf[accessed 13 July 2020]
[18] Rai NK, Rim KI, Wulandari EW, Subrata F, Sugihantono A, Sitohang V. Strengthening emergency preparedness and response systems: experience from Indonesia. WHO South-East Asia J Public Health 2020; 9(1): 26-31.
[19] Revicki D. Internal consistency reliability.Encyclopedia of quality of life and well-being research 2014; 3305-3306. 30.
[20] Ursachi G, Horodnic IA, Zait A. How reliable are measurement scales? External factors with indirect influence on reliability estimators. Procedia Econ Finance 2015; 20: 679-86.
[21] World Health Organization. Joint external evaluation tool 2005. Avialable from: https://apps.who.int/iris/bitstream/handle/10665/259961/9789241550222eng.pdf?sequence=1
[22] Oppenheim B, Gallivan M, Madhav NK, et al. Assessing global preparedness for the next pandemic: development and application of an Epidemic Preparedness Index. BMJ Glob Health 2019; 4(1): e001157.