Translation and Evaluation of Psychometric Properties of the Questionnaire of Factors Affecting the Use of Clinical Guidelines from the Physicians' Point of View

1. INTRODUCTION

Because of the healthcare environment's complexity and quick changes, clinicians need to emphasize clinical practice based on the best available evidence [1]. Evidence-based performance is employed worldwide. Clinical guidelines emphasize the use of evidence in clinical decisions [2, 3]. The chief benefit of guidance is improving the quality of care patients receive by making guidelines more available to clinicians. Guidelines promote interventions that provide potential benefits and discourage ineffective or potentially harmful interventions. Guidelines also improve the consistency of care, empower patients, affect public policy, mediate disease performance measures and evaluations, and drive the planning of high-value interventions [4, 5]. It should be ensured that guidelines are developed correctly and scientifically, involving the target community, which can lead to the improvement in the entire healthcare system’s quality and quantity [6]. To offer full guidance, hundreds of organi- zations across the globe have invested substantial resources. Guides that are based on determinants (facilitators or barriers to guide usage) are more likely to be utilized [7]. Although the guidelines are accurately and comprehensively synthesized by the members of specialized working groups, physicians seldom completely implement them [8, 9]. The results of the studies conducted in the United States and the Netherlands have shown that about 30-40% of patients do not receive evidence-based care [10]. Studies in Iran have stated inadequate familiarity of physicians as the most important factor in not implementing clinical guidelines [11-13]. Insurance factors, trusteeship of the health system, organizational culture, organizational factors, economic factors, and characteristics of clinical guidelines are effective in the implementation of clinical guidelines in Iran [13]. Different studies at a national level have identified barriers to the implementation of hypertension guidelines, particularly in low- and middle-income country primary care settings, citing poor adherence and awareness of hypertension guidelines as a major area of concern [14-16]. One of the important issues impacting the use of guidelines lies within the healthcare system, health infrastructure, or is related to patient factors. Time pressures of primary care appointments, weak primary care health infrastructure, physician inertia, and additional workload created by guideline recommendations have been described as barriers [17]. One of the tools used in this field is the questionnaire “Identifying the effective factors in using clinical guidelines from the physicians' point of view”, which is the most compre- hensive tool. This questionnaire was developed by Gagliardi et al. in 2018 and its reliability and validity were evaluated and confirmed. For this purpose, the inter- national multidisciplinary team (representatives of six countries, including Australia, Canada, the Netherlands, Spain, Sweden, and the United States) created a questionnaire of factors affecting physicians' use of clinical guidelines [18]. The uniformity of the tools used in different parts of the world can be ensured only by conducting methodological and psychometric studies, so if a tool is used to assess a population that speaks another language, there is a need for cultural adaptation and validity evaluation for the questionnaire [19]. Despite the efforts of health organizations to encourage the use of clinical guidelines, there is very limited use in practice [13, 20]. Although clinical guidelines are widely deemed important, there is only a small body of evidence investigating clinicians’ perceptions and use of guidelines. Recognizing and examining the barriers to physicians' use of clinical guidelines for improving the quality of health services seem necessary. Therefore, this study aimed to translate and evaluate the psychometric properties of the questionnaire of factors affecting the use of clinical guidelines from the physicians' point of view.

2. MATERIALS AND METHODS

This was a tool-making study conducted to translate and evaluate the psychometric properties of the questionnaire of factors affecting the use of clinical guidelines from the physicians' point of view designed by Gagliardi et al. in 2018 [18]. The mentioned questionnaire consists of four parts: the first part includes the physician's demographic information, the second part contains 26 questions about the determinants for using clinical guidelines on the Likert scale, the third part includes four open-ended questions specifically for other determinants, and the fourth includes three questions related to physicians' preferred styles of learning clinical guidelines [18]. After obtaining permission and receiving the instructions for the questionnaire from the authors, first, the backward-forward process was performed to translate the questionnaire on the factors affecting the use of clinical guidelines from the perspective of physicians from English to Persian. In the first stage, the questionnaire was translated into Persian independently while maintaining literal translation by two people within health service management who were fluent in English. Then, while comparing the two translated versions, a temporary version of the questionnaire was prepared. The temporary Persian version was translated to English by two people who did not know the English text of the questionnaire, and who also had sufficient proficiency and experience in translating Persian to English. In the next step, the obtained texts were matched with the original text, and during a meeting in the presence of translators, the translation was agreed upon and the final version of the questionnaire was prepared. The population in the qualitative face validity stage was 10 faculty members of health services management, faculty members of social medicine, physicians, and tool-making specialists. To determine the validity of the content qualitatively, 10 experts with knowledge and experience in the field of study (faculty members of health services management, faculty members of social medicine, general physicians, and tool-making specialists) were asked to study the tools and present their corrective views in terms of quality, grammar, and the use of appropriate words. These people were selected based on the opinion of the research team based on the richness of information in the studied area and having sufficient knowledge and experience. The study population in the content validity stage and quantitative face validity included general physicians working in government departments of Universities of Medical Sciences. They participated in the study after obtaining their informed consent. Inclusion criteria were physicians with at least 5 years of occupational therapy experience and physicians selected from hospitals affiliated with Mashhad University of Medical Sciences. To check the validity of the questionnaire content, the translated version was provided to the experts, and the Content Validity Index (CVI) and Content Validity Rate (CVR) were calculated [21]. In this study, the number of specialists was 10; thus, 0.62 was considered acceptable for each item [22]. In examining the Content Validity Index (CVI), the items were examined from the perspective of experts in terms of “relevance”, “clarity”, and “simplicity and fluency”. If the calculated content validity index score was higher than 0.79, the desired item was considered appropriate and maintained, items with a score of 0.70-0.79 were corrected and revised, and items with scores lower than 0.70 were deemed unacceptable and deleted [23]. In the qualitative evaluation of face validity, the opinions of 10 subjects (physicians) were taken to examine the questions in terms of the level of difficulty, appropriateness, and ambiguity. The impact score method was used to quantitatively evaluate the face validity. The questionnaire was given to 10 physicians. For each item of the tool, the 5-point Likert scale of quite important (5), important to some extent (4), moderately important (3), slightly important (2), and not important at all (1) was considered. Physicians were asked to determine the importance of each item based on their experience. Afterward, the impact score of each item was calculated by the formula, and items with an impact score greater than 1.5 were considered appropriate items [24-26]. Cronbach's alpha coefficient was used to evaluate the internal consistency of the questionnaire on factors affecting the use of clinical guidelines from the physicians' point of view; an alpha equal to or greater than 0.7 was considered acceptable [24]. The repeatability (test-retest stability) of the questionnaire was evaluated using an Intra-class Correlation Coefficient (ICC) with an interval of two weeks on 34 general practitioners. This criterion indicated the ratio of interpersonal change to total change. Therefore, the large ratio indicated small changes within the individual. Items with an ICC of more than 0.7 were kept [24].

3. RESULTS

The results of the face validity showed that in all items of the questionnaire, 10 physicians answered options one and two of the designed scale; therefore, all items were confirmed in terms of face validity. To qualitatively evaluate the content validity, the questionnaire was provided to 10 experts, and their corrective views on the grammar and the use of appropriate words were applied. In the quantitative content validity of the questionnaire, the Content Validity Ratio (CVR) and Content Validity Index (CVI) of all questionnaire items were greater than 0.8, being slightly larger than the Lawshe table (0.62). Two methods of internal consistency and stability were used to evaluate the reliability of the questionnaire. The value of Cronbach's alpha coefficient for evaluating the internal consistency of the questionnaire was obtained to be 0.77, which has been found to be an acceptable value. The stability of the questionnaire was evaluated by retesting and calculating the intracluster correlation coefficient with an interval of two weeks on 34 physicians. The mean age of physicians was 47 ± 4.56 years and 73% (25) were female. The intracluster correlation coefficient of the questionnaire was obtained to be 0.98 (p-value <0.001). Table 1 presents the results of the content and face validity of the questionnaire for each item. The qualitative questions were: What is the single most important factor noted above that does/will enable your use of this guideline? What is the single most important factor noted above that does/will challenge your use of this guideline? Physicians participating in this study providing ongoing education, patient admission, available resources, patient needs, and appropriate decision-making about patient status were cited as the most important factors in using this guideline.

4. DISCUSSION

This study was conducted to translate and psycho- metrically assess the factors affecting the use of clinical guidelines from physicians' points of view.

In the present study, the results of the face and content validity of the tool were confirmed by experts using CVI and CVR. In this study, the reliability of the tool of “factors affecting the use of clinical guidelines from the physicians' point of view” was calculated to be 0.96 using the internal consistency test (Cronbach's alpha coefficient), which indicated the appropriate reliability of the tool. The tool has shown good reliability when Cronbach's alpha coefficient was greater than or equal to 0.7 [27]. This coefficient can be between zero and one, and the higher and closer it is to one, the higher the levels of internal consistency of the tool under study [28]. An intracluster correlation coefficient equal to or higher than 0.7 was considered an acceptable limit for repeatability level [29]. In the present study, the intracluster correlation coefficient by test-retest was 0.71, which confirmed the time stability of the tool.

The questionnaire used in this study was designed by seven members of the Guideline International Network Implementation Working Group, representing six countries, including Australia, Canada, the Netherlands, Spain, Sweden, and the United States. All members of the group, specialized in research and health service guidelines and development and systematic review, reviewed 178 questionnaires [30]. By systematically reviewing 178 unique questionnaires that were not comprehensive or valid, the questionnaire identifying the effective factors in using clinical guidelines from the physicians' point of view was revealed [30]. As mentioned, there are several questionnaires to examine the determinants of physicians, including the questionnaire on determinants of executive behavior [31] and the evaluation of determinants of the use of dietary instructions for children [32]. The questionnaire “Factors Affecting the Use of Clinical Guides from Physicians' Point of View”, which we translated and examined its psychometric properties in the present study, is a comprehensive questionnaire that increases the response rate as it is completed more easily and quickly. This questionnaire can be used before and after the intervention to demonstrate, promote, or support the use of a clinical guide to demonstrate the effectiveness of the intervention. Also, this questionnaire can be performed months or years after the clinical guidelines are published when research or audits show that the guidelines are not being implemented. This questionnaire can also be used to predict barriers to a guideline that may be due to changes in its recommendations [33].

The questionnaire on the factors affecting the use of clinical guidelines, in English, provides brief instructions, offering users choices for each section and each item. For example, the Likert scale has been considered by questionnaire developers as the most useful and meaningful answer option. However, the authors of the questionnaire have stated that questionnaire users can choose other types of answer options, such as yes, no, and not sure. According to the unit of analysis, the questionnaire is currently set for questions for all clinical guides, but users can make similar items for each recommendation in a clinical guide. Of course, this may require more time to complete the questionnaire [18].

Limitations: In this study, the reliability testing of the questionnaire was time-consuming due to the busy schedule of the target group, which was physicians, and it led to the lack of cooperation of physicians, which was removed over time in the researchers' follow-up. Another limitation of this study is that this study did not measure convergent validity and discriminant validity.

CONCLUSION

The analysis of this study has shown all items of the tool “factors affecting the use of clinical guidelines from the physicians' point of view” to have good reliability and validity, and this tool can be used by developers, executives, or researchers to assess the determinants of clinical guidance use and the knowledge necessary to plan and implement interventions that support the use of clinical guidance recommendations in practice. Also, it is recommended that managers and decision-makers in the field of health in the country benefit from the results of this type of study to increase the use of clinical guidelines and consequently improve services in this field.

This questionnaire can be utilized by designers, implementers, or analysts to survey the determinants of clinical rule utilization and the information vital to arranging and executing intercessions that bolster the utilization of clinical rule proposals in practice.

AUTHORS’ CONTRIBUTION

S. T., S. F., E. S., R. R., and M. V.: Contributed to the study concept or design; J. J. and P. A.: Performed data analysis and interpretation; J. M. and E. H.: Wrote the paper.

REFERENCES