doi: 10.56294/dm2024.401

METHODOLOGICAL ARTICLE

INTEGRA methodology for the development of integrative reviews: origins, guidelines, and recommendations

Metodología INTEGRA para el desarrollo revisiones integrativas: origen, directrices y recomendaciones

Miguel Valencia-Contrera¹ *, Naldy Febré¹ , Daniella Cancino Jiménez¹ , Flérida Rivera-Rojas² , Jenifer Villa-Velasquez³ , Solange Vallejos Vergara⁴ , Alba Lozano-Romero⁵ , Rodrigo-Alejandro Ardiles-Irarrazabal⁶ , José Castro-Bastidas⁷ , María Quintana-Zavala⁸ , Maria Lucia do Carmo Cruz Robazzi⁹ , Vivian Vilchez-Barboza¹⁰ , Sandra Valenzuela-Suazo¹¹

¹Universidad Andrés Bello, Facultad de Enfermería. Santiago, Chile.

²Universidad Católica del Maule, Departamento de Enfermería. Curicó, Chile.

³Universidad Austral de Chile, Escuela de Enfermería. Puerto Montt, Chile.

⁴Universidad Autónoma de Chile, Facultad de Ciencias de la Salud. Santiago, Chile.

⁵Universidad de Chile, Departamento de Enfermería. Santiago, Chile.

⁶Universidad de Antofagasta, Departamento de Enfermería. Antofagasta, Chile.

⁷Castro & Castro Enfermería de Práctica Avanzada SAS. Barranquilla, Colombia.

⁸Universidad de Sonora, Departamento de Enfermería. Hermosillo, México.

⁹Universidade de São Paulo, Escola de Enfermagem. Ribeirão Preto, Brasil.

¹⁰Centro de Investigación en Cuidado de Enfermería y Salud (CICES), Universidad de Costa Rica. San José, Costa Rica.

¹¹Universidad de Concepción, Facultad de Enfermería. Concepción, Chile.

Cite as: Valencia-Contrera M, Vilchez-Barboza V, do Carmo Cruz Robazzi ML, Quintana-Zavala M, Castro-Bastidas J, Ardiles-Irarrazabal R-A, et al. INTEGRA methodology for the development of integrative reviews: origins, guidelines, and recommendations. Data and Metadata. 2024; 3:.401. https://doi.org/10.56294/dm2024.401

Submitted: 10-02-2024                   Revised: 28-05-2024                   Accepted: 07-09-2024                Published: 08-09-2024

Editor: Adrián Alejandro Vitón-Castillo

Autor para la correspondencia: Miguel Valencia-Contrera *

ABSTRACT

Introduction: the “INTEGRA” methodology represents an updated approach for integrative reviews, emphasizing the quality of outcomes in response to a need expressed by the scientific community.

Objective: to present the INTEGRA methodology and provide guidelines and recommendations for its application.

Method: this methodological study was conducted in two stages: a) Development of the guideline and b) Analysis. The study was carried out by a team of experts from Chile, Colombia, Mexico, Costa Rica, and Brazil, who met at least one of the following criteria: a) holding a doctoral degree or being a doctoral candidate with experience in conducting literature reviews; b) having experience in developing reviews for or with professionals in clinical-care settings, policymakers, government agencies, or other decision-makers.

Results: the “INTEGRA” methodology consists of seven stages: 1. (I) Idea or study problem; 2. (N) Narrowing down the inquiry or objective; 3. (T) Targeting the search strategy; 4. (E) Execution or implementation of the search; 5. (G) Grading and quality control of the results; 6. (R) Reviewing the results; 7. (A) Analysis and discussion.

Conclusions: the application of the “INTEGRA” methodology will provide authors with guidelines for developing integrative reviews and improving the quality of contributions in this field.

Keywords: Integrative Review; Methodology; Critical Analysis of the Literature.

RESUMEN

Introducción: la metodología “INTEGRA” es una actualización metodológica para revisiones integrativas con énfasis en la calidad de los resultados, como respuesta a una necesidad manifestada por la comunidad científica.

Objetivo: presentar la metodología INTEGRA, así como, proporcionar directrices y recomendaciones para su uso.

Método: estudio metodológico, de dos etapas: a) Desarrollo de la directriz y b) Análisis. Este estudio se desarrolló por un equipo de expertos provenientes de Chile, Colombia, México, Costa Rica y Brasil, quienes cumplían con al menos uno de los siguientes requisitos: a) tener el grado académico de doctor o ser candidato al grado de doctor con experiencia trabajando en revisiones de la literatura; b) Con experiencia en el desarrollo de revisiones con o para profesionales del ámbito clínico-asistencial, políticos, organismos gubernamentales u otros tomadores de decisiones.

Resultados: la metodología “INTEGRA” corresponde a una metodología compuesta por siete etapas: 1. (I) Idea o problema de estudio; 2. (N) Interrogante u objetivo; 3. (T) Táctica de búsqueda; 4. (E) Ejecución o empleo de la búsqueda; 5. (G) Grado y control de calidad de los resultados; 6. (R) Resultados filtrados; 7. (A) Análisis y discusión.

Conclusiones: la aplicación de la metodología “INTEGRA” proporcionará a los autores directrices para el desarrollo de revisiones integrativas, así como, mejorar la calidad de las contribuciones de este tipo.

Palabras clave: Revisión Integrativa; Metodología; Análisis Crítico de la Literatura.

INTRODUCTION

There is currently a wide variety of research designs, which can be classified as “primary” or “secondary”.⁽¹⁾ They are primary when an original data collection is carried out, they can follow a qualitative, quantitative or mixed approach; while secondary designs use data that have already been collected.

In this line, considering the “secondary” designs, literature reviews stand out, which correspond to a research design that seeks to collect and analyze data related to a particular phenomenon, this definition seems to be very abstract; however, in essence, it fully represents the complexity of the design; according to the Royal Spanish Academy (RAE), data is “Information on something concrete that allows its exact knowledge, or serves to deduce the consequences derived from a fact”.⁽²⁾ Therefore, a review study seeks to collect and analyze information on a particular phenomenon.

The great variety of sources of information, types of data, and the way of carrying out a review has supported the need to generate review typologies; however, there is no consensus in the area that allows for delimiting the classifications in the area, for example, Sutton, et al.⁽³⁾ identified 48 types of review, while Manterola et al.,⁽⁴⁾ state the existence of 21 types, and 29 variants and associated synonymies. However, considering the way a review is made, without excluding the particularities of each type, they can be grouped into “replicable” and “non-replicable”; they will be replicable when the authors provide the necessary information on each of the decisions taken to reach the final sample, which undoubtedly requires great efforts, and, for this reason, few reviews can be classified in this group, such as integrative reviews.

Integrative reviews are defined as the “synthesis of research or literature with heterogeneous designs”.⁽⁵⁾ They are also known as mixed or mixed methods reviews,⁽⁶⁾ their hallmark being the diversity of the sampling framework.⁽⁴⁾

The integrative review consists of constructing a broad analysis of the literature, contributing to discussions on research methods and results; its initial purpose is to obtain an in-depth understanding of a particular phenomenon from previous studies; it follows standards of methodological rigor and clarity in the presentation of results; the synthesis of knowledge from the studies included reduces uncertainties regarding practical recommendations, allows for accurate generalizations of the phenomenon from the limited information available, and facilitates decision-making on interventions. It is a broad method, allows for the simultaneous inclusion of experimental and quasi-experimental investigations, and it provides a complete understanding of the topic of interest; it makes the combination of data from theoretical and empirical literature possible.⁽⁷⁾

Therefore, in this article, “Integrative Review” will be understood as the generic term that encompasses the types of review that make up a heterogeneous sample.

Along these lines, at the end of 2022, the “INTEGRA” methodology was published.⁽⁸⁾ A methodological update for integrative reviews with an emphasis on the quality of the results, in response to a need expressed by the scientific community.⁽⁹⁾ This concern has not been a recent phenomenon, as it is possible to identify articles inviting discussion of this issue as early as 1976.⁽¹⁰⁾ INTEGRA has been valued by the academic and scientific community, being very useful for the development of scientific articles,^(11,12) as well as in graduate theses.^(13,14)

Despite the relevance and usefulness of the proposed methodology, its original publication does not provide in-depth guidelines, thus reducing its scope. Under the above premise, the present methodological article was developed in response to the objective of presenting the INTEGRA methodology, and to provide guidelines and recommendations for its use.

METHOD

Methodological study, which involved a two-stage process. 1. Development of the guideline: The structure of the original publication was used,⁽⁸⁾ which was complemented with recommendations published in the area.^(15,16,17) These helped to identify which elements could be added, modified or eliminated; also, each team member was able to make recommendations, which were presented in a draft version. 2. Analysis: The elements that made up the preliminary version were discussed among the team of researchers, considering the acceptance of an item when its valuation was at least 70 %. Each member of the team met at least one of the following requirements: a) have the academic degree of doctor or be a candidate for the degree of doctor with experience working on literature reviews; b) have experience in the development of reviews with or for professionals in the clinical-care field, politicians, government agencies or other decision-makers, who present knowledge regarding the information required to expedite the use of the results of integrative reviews.

Regarding ethical considerations, the authors state that the article was developed following the ethical recommendations of good scientific practice, adhering to the principles of honesty, objectivity, integrity, precaution, openness and responsibility.⁽¹⁸⁾ The authors also declare that they ensured proper attribution of authorship.

RESULTS

The “INTEGRA” methodology corresponds to a mnemonic of the steps necessary to develop an integrative review, that is, it corresponds to a methodology composed of seven stages: 1. (I) Idea or problem of study; 2. (N) Question or objective; 3. (T) Search tactics; 4. (E) Execution or use of the search; 5. (G) Grade and quality control of the results; 6. (R) Filtered results; 7. (A) Analysis and discussion.

1. (I) Idea or Problem of Study

The first stage corresponds to the “delineation of an idea, problem, concept or aspect that the researcher wishes to review”.⁽⁸⁾

It is important for researchers to address a phenomenon whose nature requires multiple approaches or perspectives; for example, to perform a concept analysis, it is necessary to conduct a review that allows for identifying all possible uses of the concept, whether in scientific databases, repositories, books, dictionaries, etc.

2. (N) Question or Objective

In the second stage “the polestar of a review will be established, and this will be the question or objective that will guide the searches; from this the search strategies will be developed”.⁽⁸⁾

The objective of a review should be clear, precise, contain the phenomenon of interest of the study, be coherent with the problem statement and, therefore, be in tune with the title and research question (if presented).

Although, in general terms, research objectives should be structured using verbs in infinitive form,⁽¹⁹⁾ the structure of research objectives and questions varies according to the nature of the addressed phenomenon. The structures are generally presented in the form of acronyms, in this line, one of the best known and most used structures corresponds to “PICO”, however, it is not the only one nor the most appropriate in all cases; the main acronyms and the description of the elements that make it up are presented below (table 1).

Table 1. Main acronyms for the construction of review questions
Acronym	Elements
PICO ^(20)	Population or problem of interest (P) + intervention (I) + intervention of comparison (C) + outcome of interest (O)
PICo ^(21)	Population “Population” (P) + Phenomenon of Interest “Fenómeno de interés” (I) + Context “Context” (Co)
PIO ^(22)	Population or problem of interest (P) + intervention (I) + outcome of interest (O)
PICOT ^(22)	Population or problem of interest (P) + intervention (I) + intervention of comparison (C) + outcome of interest (O) + time interval (T)
PICOTT ^(23)	Population or problem of interest (P) + intervention (I) + intervention of comparison (C) + outcomes of interest (O) + type of question (T) + type of study (T)
PEAKS ^(24)	Population or problem of interest (P) + intervention (I) + intervention of comparison (C) + outcomes of interest (O) + study design (S)
PICOC ^(25)	Population “Population” (P) + intervention “Intervention” (I) + comparison “Comparison” (C) + outcome of interest “Outcomes” (O) + context “Context” (C)
PIPOH ^(26)	Population or problem of interest (P) + intervention (I) + professionals targeted (P) + outcomes of interest (O) + health care setting (H) + context of implementation (H)
PECODR ^(27)	Population or problem of interest (P) + exposure (E) + intervention of comparison (C) + outcomes of interest (O) + duration of treatment and/or duration until the outcome is evaluated (D) + results obtained (R)
PECO ^(28)	Population or problem (P) + exposure (E) + Comparator (C) + outcome of interest (O)
PEO ^(29)	Population (P) + exposure (E) + outcomes of interest (O)
PESICO^(30)	Population or problem of interest (P) + environment (E) + people who have an interest in the outcome “stakeholders” (S) + intervention (I) + intervention of comparison (C) + outcome of interest (O)
ECLIPSE^(31)	Expectation (E) + customers (C) + location (L) + impact (I) + professionals involved (P) + services (SE)
SPICE ^(32)	Scenario (S) + Perspective (P) + Intervention (I) + Comparison (C) + Evaluation (E)
SPIDER ^(33)	Sample (S) + phenomenon of interest (PI) + design (D) + evaluation (E) + research type (R).
PIS ^(22)	Population (P) + intervention or problem (I) + situation (S)
PS ^(22)	Population (P) + experience (E)

3. (T) Search Tactic

As for the third stage, this “corresponds to the heart of a review, in which all the characteristics and strategies used to carry it out must be described; one way of knowing if sufficient details have been described is to be able to replicate the search with the information presented and arrive at the same results”.⁽⁸⁾ For a correct presentation of the mentioned characteristics and strategies the present authors recommend contemplating the following statements:

· Point out the sources of information.

· In case of using databases, the search equation, search modes, and filters applied must be presented.

· Indicate the period of study and date of data extraction.

· State inclusion and exclusion criteria used.

· Indicate what information will be extracted from the documents.

· Describe how the data will be analyzed.

3.1 Identifying Sources of Information

The sources of information in an integrative review can be very varied. The main source is usually scientific databases, which enable access to a large number of articles from journals that meet certain criteria established by each database; for this reason, it is possible to identify databases of very specific thematic areas, while others are multidisciplinary; some have highly demanding standards, so it is possible to identify a select group of journals. Below are some examples of databases, the area to which they belong, and their language.

Table 2. Examples of databases, areas to which they belong, and language
Database	Area to which they belong	Language
WoS (Web of Science)	Multidisciplinary	English
SCOPUS	Multidisciplinary	English
SciELO (Scientific Electronic Library Online)	Multidisciplinary	Several languages
PubMed	Biomedical sciences	English
CINAHL (Cumulative Index of Nursing and Allied Literature)	Nursing and related areas	Spanish-English
VHL (Virtual Health Library)	Health sciences	Several languages
Dentistry and Oral Sciences Source	Dentistry	Spanish-English
IEEE XPLORE (Institute of Electrical and Electronics Engineers Xplore)	Engineering	English
Business Source Ultimate	Administration and Economics	English

Other sources that can be used are: books, theses, dictionaries, government reports, reports from international institutions, legislation, among others, which can contribute valuable information to the review. Many of these sources are catalogued as gray, invisible, semi-published, non-conventional, informal or minor literature, defined as “any type of document that is not disseminated through the ordinary channels of commercial publication, and therefore poses problems of access”.⁽³⁴⁾

On the other hand, there are other strategies to strengthen the sample in a review, which consists of performing a manual search, generally by consulting the Google search engine; however, it may also be useful to search for references of other reviews or articles that are related to the phenomenon studied (snowball technique). Finally, bibliometric studies in the area provide information on the most relevant authors in the area and the most cited articles; this information can guide the manual search.

3.2 If Databases Are Used, the Search Equation, Search Modes and Filters Applied Must Be Presented

Databases should preferably be consulted using a controlled language, i.e., the terms used should be validated in thesauri; thesauri correspond to scientific dictionaries, which enable the use of a standardized/systematized language. The thesauri most commonly used in health sciences are: Descriptors in Health Sciences (DeCS) and Medical Subject Headings (MeSH); DeCS is available in Spanish, English, Portuguese, and French, while MeSH only works in English.

Once the descriptors are validated, it is necessary to use Boolean operators; Boolean operators come from Boolean algebra, and they help in the combination of terms in any database, in order to restrict or broaden the search result.⁽³⁵⁾ When it is intended to make the search more specific, “AND” is used, in other words, it is used between different terms to locate studies on two topics (intersection); to broaden the search, “OR” is used, that is, it is used between synonyms to retrieve articles on one topic or another; and finally, when it is desired to exclude terms, “NOT” is used.⁽³⁶⁾ Truncation operators are also useful, which can complement the terms used, among them, one of the most used is the asterisk (*), which allows searching for manuscripts that present the root of the word followed by any character, for example, when using “nurs*”, articles containing terms such as “nurses” or “nursing” will appear in the results. An example of a search equation is shown below:

(((“work-home Interference”) OR (“work-family interface”) OR (“work/family Balance”) OR (“work-family interaction”) OR (“work-family conflict”) OR (“work-family tension”) OR (“work-life conflict”) OR (“work-family spillover”) OR (“work-family spillover”) OR (“home-work Interference”) OR (“family-work interface”) OR (“family/work Balance”) OR (“family-work interaction”) OR (“family-work conflict”) OR (“family-work tension”) OR (“life-work conflict”) OR (“family-work spillover”)) AND Nurs*)

Once the equation is defined, the search modes and filters to be used must be established. The search modes allude to the place where the terms used in the search equation will be searched, for example “title”, “abstract”, “keywords”, among others, although each database has a different interface, it is important to state how the search was performed. As for the filters, these correspond to the targeted selection of articles with the intention of obtaining more precise results, giving answers to the needs of each review, an example of filter can be “year of publication”, “type of document”, “countries/regions”, among others; each database presents different filters, therefore, it is important to state which filters were used in the respective databases consulted. Table 3 below shows an example of database presentation, search strategy (search mode), and filters applied.

Table 3. Example of database presentation, search strategy (search mode) and filters applied
Database	Search strategy/search mode	Filters applied
Web of Science (WoS)	Article title, abstract, keywords	Year: 2019-2023 Document type: Article Language: English, Portuguese, Spanish
...	...	...

3.3 Indicate the Study Period and Date of Data Extraction

It is important to point out the period of study, i.e., the range of years considered in the review and its due justification, generally, researchers use, as a criterion, the “last five years”, based as a relevant time to identify the most recent information in an area, however, this range is not the only criterion nor the most advisable, rather, there must be a justification associated with the phenomenon of interest in each case, for example, “the period analyzed was the last six years, since the last review in the area dates from this date”, or “the present review considered the existing theories in the area since 1950, this time was considered, because the first frameworks in the area emerged at that date”.

Regarding the date of data extraction, it is important to state when the databases were consulted and the identified documents were extracted, since it is not the same to analyze the last five years, considering the last year as a whole, as opposed to only some months when considering a current year, for example, “the last five years were analyzed, i.e. 2015-2019 (up to April), the date on which the data were extracted” as opposed to “the last five years were analyzed, i.e. 2015-2019, the date on which the data were extracted”, i.e. 2015-2019 (up to the month of April), the date when the data extraction was done” as opposed to “the last five years were analyzed, i.e. 2015-2019, the date of data extraction was done in January 2020”, the months of difference in the given examples, in some study phenomena can mean hundreds of articles. In addition, it is recommended that the extraction in the different databases be performed in a single day, since daily additions and/or deletions may occur in these, therefore, they cannot be performed in the course of a month or a week.⁽³⁵⁾

3.4 Declare Inclusion and Exclusion Criteria Used

Eligibility criteria are perhaps one of the sections of the review with most room for improvement, since it is often believed that exclusion criteria are “the opposite of inclusion criteria”, however, the reality is far from this conception, since the inclusion criteria are those elements that the manuscripts analyzed must have in order to be included in the review, for example the “type of study”, “participants”, “geographical location of the study”, “year of publication”, among others; while the exclusion criteria refer to those characteristics present in some included manuscripts that, due to the particularities of the review, should be excluded from the study.

An example of a correct statement of eligibility criteria may be the integrative review by Pursio et al.,⁽³⁷⁾ whose study aimed to “summarize knowledge on professional autonomy in nursing”, in this line they stated the following eligibility criteria: as for the inclusion criteria, empirical studies with quantitative or qualitative designs, in English, and published in peer-reviewed journals with an abstract available between January 2000 and July 2019. While exclusion criteria were studies that met these criteria, if they focused on other types of health care professionals or nursing students, studies that explored patient autonomy, dealt with practice in medical settings other than hospitals, dealt with the practice and role of nurses in prescribing medications, or explored nurse empowerment with no connection to autonomy.

3.5 Indicate Which Information is to Be Extracted from the Documents

Authors should state which data will be extracted, as this will allow clarity on the information that will be sought at the time of reviewing each study, which is closely related to the nature of the review phenomenon; for example, in the integrative review of Silva et al.,⁽³⁸⁾ for example, the authors stated the following: “a Microsoft Excel® spreadsheet version 2013 was developed, registering in it the information corresponding to: authors, year of publication and country of affiliation, title of the article, objective, methodology, level of evidence, characterization of the sample/participants, main interventions, results, and conclusion”.

3.6 Describe How Data Will Be Analyzed

This stage is also known as “data synthesis”, where information must be provided on how the data will be processed, i.e., how the findings will be combined, integrated, or mixed. When the nature of the data is the same, qualitative or quantitative strategies can be used.

Among the qualitative strategies, content analysis (data coded and categorized under thematic headings); thematic synthesis (line-by-line coding, development of descriptive and analytical themes), among others, stand out.⁽³⁹⁾ Quantitative strategies include the use of descriptive statistics, heterogeneity analysis, analysis of average effect, among others.⁽⁴⁰⁾

When the nature of the data is mixed, two main designs for the synthesis stand out:⁽⁶⁾ the convergent design and the sequential design. It will be convergent when quantitative and qualitative studies are analyzed at the same time; their integration may be at the level of data, results or interpretation. Whereas, in sequential design, data are analyzed in stages, the results of which are the basis for continuing the next analysis; that is, the results of the analysis of qualitative data can inform the analysis of quantitative data, or vice versa, the results of the analysis of quantitative data can inform the analysis of qualitative data.

4. (E) Execution or Use of Search

As for the fourth stage, in this “the search is carried out according to the guidelines set out in the previous stage, the results obtained tentatively prior to the evaluation of the quality of these results must be presented”.⁽⁸⁾

At this stage, the authors should explain the review process; one recommendation is the development of flowcharts. A flowchart is a “diagram of the organization of an entity, a program or an activity”.⁽⁴¹⁾ In the context of a review, it is a scheme that presents in an organized manner the information regarding the number of documents identified in a search; this number, as the process progresses, will vary according to the previously defined criteria. Figure 1 below shows examples of review flowcharts.

Figure 1. Examples of review flow charts

Source: Figure (A) corresponds to an adaptation of PRISMA recommendations; Figure (B) is self-made.

5. (G) Extent and Quality Control of Results

“It corresponds to the fifth stage, where the quality filter is performed, the author must indicate which aspects of the articles he/she is evaluating to ensure their scientific rigor”.⁽⁸⁾

Quality corresponds to the “property or set of properties inherent to something, which allow its value to be judged”.⁽⁴²⁾ In the context of the results of an integrative review, this property or properties allude to the decisions made by the authors in the development of the documents included in the review. As previously mentioned, an integrative review can contemplate a wide variety of data in its analysis, which makes the task of establishing analysis criteria that respond to this requirement complex, therefore, it is recommended to use specific guidelines or checklists for each type of document, as shown in table 4 below, which presents some examples of quality assessment guidelines in studies with a qualitative, quantitative, and mixed approach.

Table 4. Examples of quality assessment guidelines in studies with a qualitative, quantitative and mixed approach
Quantitative approach	Qualitative approach	Mixed approach
CASPe (Critical Appraisal Skills Programme Spanish) for randomized clinical trial, diagnostic study, clinical prediction rules, case-control study, cohort study, economic evaluation.^(43)	CASPe (Critical Appraisal Skills Programme Español) for qualitative studies.^(43)	MMAT (Mixed Methods Appraisal Tool).^(44)
Joanna Briggs Institute (JBI) critical appraisal tool for cross-sectional analytical studies, case-control studies, case reports, case series, cohort studies, diagnostic test accuracy, economic evaluations, prevalence studies, quasi-experimental studies, and randomized controlled trials.^(45)	Joanna Briggs Institute (JBI) critical appraisal tool for qualitative research, expert opinion, narrative textual evidence and policy.^(45)	GRAMMS (Good Reporting of a Mixed Methods Study).^(46)
CONSORT (Consolidated Standards of Reporting Trials).^(47)	COREQ (Consolidated Criteria for Reporting Qualitative Research).^(48)
STROBE (Strengthening the Reporting of Observational Studies in Epidemiology).^(49)	SRQR (Standards for Reporting Qualitative Research).^(50)
TREND (Transparent Reporting of Evaluations with Nonrandomized Designs).^(51)

On the other hand, there is a simpler proposal, which focuses on the evaluation of central and common aspects in the different designs, the “assessment scale for articles with heterogeneous methodologies for integrative reviews (EAMH)”, initially proposed in 2022,⁽⁵²⁾ which has proven to be useful in integrative reviews.^(14,53,54) Figure 2 below shows the EAMH scale, an example of application and interpretation of the score.

Figure 2. Example of application of the scale for evaluating articles with heterogeneous methodologies for integrative reviews (EAMH) and its interpretation

6. (R) Filtered results

“In this stage, the articles that met the quality criteria set out in the previous stage are presented, the relevant results associated with the subject that led to the generation of the review should be succinctly presented; it is a stage where the author expresses his creativity in the presentation of the results, it should be sufficiently attractive to the reader and at the same time contain the most pristine information possible”.⁽⁸⁾

7. (A) Analysis and discussion

“Last stage of the review, where the results should be interpreted and contrasted, as well as the limitations, strengths and future implications of the review performed. This stage corresponds to the true contribution of the researcher, although all reviews must be different, the analysis and discussion contains the unique product of the research, where the true contribution to science is reflected, the reason that justifies the need to value each article used in the sample as part of a whole and not independently”.⁽⁸⁾

INTEGRA Checklist

Finally, based on the recommendations presented in this study, an “INTEGRA checklist for presenting integrative review studies” was developed (table 5). This checklist will facilitate the reporting of reviews in the area, being a useful tool for authors, reviewers, and scientific editors. The list presents a total of 18 items, organized according to the INTEGRA stages, and these, in turn, in the sections of an article, which will facilitate the identification of each verification element.

Table 5. Checklist INTEGRA to submit integrative review studies
Stage INTEGRA		Section of an article	Item	Verification element	Location in the article
I	Study idea or problem	Title	1	It states that the article corresponds to an integrative review.
		Introduction	2	It presents the general idea of the phenomenon to be addressed.
			3	It describes the relevance of the phenomenon to be addressed.
			4	It provides background information on the information available in the area.
			5	It exposes the knowledge gap, substantiates the need for a revision.
N	Question or objective	Introduction / Methodology	6	Explicitly state the objective, purpose and/or question of the review.
T	Search tactics	Methodology	7	Point out the sources of information.
			8	In case of using databases, the search equation, search modes and applied filters are presented.
			9	Indicate the period of study and date of data extraction.
			10	Declares the inclusion and exclusion criteria used.
			11	It indicates what information will be extracted from the documents.
			12	Describes how the data will be analyzed according to its nature.
E	Execution or use of the search		13	Describes the search process, indicating the manuscripts that passed the inclusion and exclusion criteria through a review flowchart.
G	Grade and quality control of results		14	Describe the analysis of the quality of the sample (or justify why it is not performed), if guidelines or scales are used, state the standard considered for the study.
R	Filtered results	Results	15	Present the findings according to the analysis strategy described in item 12.
A	Analysis and discussion	Discussion and conclusions	16	Interpret and contrast the results.
			17	Describes strengths, limitations and considerations for future studies.
			18	They establish conclusions that provide answers to the objective and/or research question.

DISCUSSION

Literature reviews, like all research designs, are derived from the scientific method, which is the standard process by which we learn about our world, consisting of five basic steps: defining the research question; making predictions; collecting data; observing; and, finally, drawing conclusions.⁽⁵⁵⁾ In addition to being a way of generating knowledge, reviews constitute a fundamental methodology in the development of other types of research, since they provide valuable and updated information on the state of the art of a phenomenon; in some cases, in addition to being a benefit, they are an obligation, such as in the development of scientific articles, theses or other academic documents.⁽⁵⁶⁾

However, reviews are not always developed with the required standards and meeting a real need, for example, in the case of systematic reviews, some discussions in the area have stated that more than 90 % of the published systematic reviews are clinically “useless”.⁽⁵⁷⁾ In this line, the scientific community has highlighted the need for standards, guidelines, and orientations in the different types of literature reviews, as is the case of integrative reviews; for example, Manterola C et al.,⁽⁴⁾ recently stated that in this type of review “the optimal methods have not yet been determined; therefore, they could produce incoherent data evaluation and analysis”.

Thus, the INTEGRA methodology is presented as a tool, providing technical guidance on how to develop and present an integrative review. It is expected that the use of the INTEGRA checklist will provide a consensus for authors, reviewers, and scientific editors. Future proposals in the area should analyze the application of INTEGRA, its strengths and elements that could be improved, as it is a proposal, its application cannot be assured, however, it constitutes an option that addresses current needs.

CONCLUSIONS

The INTEGRA methodology is a proposal for the development of integrative reviews, and it presents a total of seven stages: 1. (I) Idea or study problem; 2. (N) Narrowing down the inquiry or objective; 3. (T) Targeting the search strategy; 4. (E) Execution or implementation of the search; 5. (G) Grading and quality control of the results; 6. (R) Reviewing the results; 7. (A) Analysis and discussion. To ensure compliance with the standards in the area, the INTEGRA checklist for presenting integrative review studies was proposed, which will facilitate the development and presentation of future proposals.

REFERENCES

1. Campana ÁO. Metodologia da investigação científica aplicada à área biomédica: 2. Investigações na área médica. J Pneumol [Internet]. 1999 Apr;25(2):84–93. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102-35861999000200005&lng=pt&tlng=pt

2. Real Academia Española (RAE). dle.rae.es. 2023. Dato. Available from: https://dle.rae.es/dato

3. Sutton A, Clowes M, Preston L, Booth A. Meeting the review family: exploring review types and associated information retrieval requirements. Heal Inf Libr J [Internet]. 2019 Sep 20;36(3):202–22. Available from: https://onlinelibrary.wiley.com/doi/10.1111/hir.12276

4. Manterola C, Rivadeneira J, Delgado H, Sotelo C, Otzen T. ¿Cuántos Tipos de Revisiones de la Literatura Existen? Enumeración, Descripción y Clasificación. Revisión Cualitativa. Int J Morphol [Internet]. 2023 Aug;41(4):1240–53. Available from: http://dx.doi.org/10.4067/S0717-95022023000401240

5. Knafl K, Whittemore R. Top 10 Tips for Undertaking Synthesis Research. Res Nurs Health [Internet]. 2017 Jun 7;40(3):189–93. Available from: https://onlinelibrary.wiley.com/doi/10.1002/nur.21790

6. Nha Hong Q, Turcotte-Tremblay AM, Pluye P. Revisiones sistemáticas mixtas. Un ejemplo sobre la financiación basada en los resultados. In: Evaluación de las intervenciones sanitarias en salud global Métodos avanzados. Québec: IRD Éditions; 2020. p. 165–94.

7. Tipos de revisão de literatura. Botucatu: Biblioteca Prof. Paulo de Carvalho Mattos. Faculdade de Ciências Agronômicas UNESP, campus de Botucatu; 2015. p. 9.

8. Valencia-Contrera MA. Actualización metodológica de revisiones integrativas: un énfasis en la calidad de los resultados. Salud, Cienc y Tecnol [Internet]. 2022 Dec 21;2:171. Available from: https://revista.saludcyt.ar/ojs/index.php/sct/article/view/171

9. Toronto C, Remington R. A step-by-step guide to conducting an integrative review. Cham: Springer International Publishing; 2020.

10. Yin RK, Bingham E, Heald KA. The Difference That Quality Makes. Sociol Methods Res [Internet]. 1976 Nov 30;5(2):139–56. Available from: http://journals.sagepub.com/doi/10.1177/004912417600500201

11. Carrasco Diaz B, Araya AX. Soledad y fragilidad en personas mayores que viven en comunidad: una revisión integrativa. Horiz Enferm [Internet]. 2023;(NE):332–48. Available from: https://horizonteenfermeria.uc.cl/index.php/RHE/article/view/67319

12. Urgiles Neira AN, Parrales Vique KA, Calderón Calle ME. Importância de um ambiente cirúrgico estéril durante a cirurgia oral. Res Soc Dev [Internet]. 2023 Jun 28;12(6):e26112642354. Available from: https://rsdjournal.org/index.php/rsd/article/view/42354

13. Burgos Robles G. Herramientas y estrategias utilizadas en atención ambulatoria de niños autistas. Universidad Andrés Bello; 2024.

14. Contreras Veliz J. Intervenciones en salud que promueven el desarrollo integral en la infancia indígena: Revisión integrativa. Pontificia Universidad Católica de Chile; 2023.

15. Toronto CE, Remington R, editors. A Step-by-Step Guide to Conducting an Integrative Review [Internet]. Cham: Springer International Publishing; 2020. Available from: http://link.springer.com/10.1007/978-3-030-37504-1

16. Ganong LH. Integrative reviews of nursing research. Res Nurs Health [Internet]. 1987 Feb 19;10(1):1–11. Available from: https://onlinelibrary.wiley.com/doi/10.1002/nur.4770100103

17. Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs [Internet]. 2005 Dec 2;52(5):546–53. Available from: https://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.2005.03621.x

18. Kruk J. Good scientific practice and ethical principles in scientific research and higher education. Cent Eur J Sport Sci Med [Internet]. 2013;1(1):25–9. Available from: https://bibliotekanauki.pl/articles/1054624.pdf

19. Coronel-Carvajal C. Los objetivos de la investigación. Rev Arch Médico Camagüey [Internet]. 2023;27:e9591. Available from: http://scielo.sld.cu/pdf/amc/v27/1025-0255-amc-27-e9591.pdf

20. Richardson W, Wilson M, Nishikawa J, Hayward R. The well-built clinical question: A key to evidence-based decisions. ACP J Club [Internet]. 1995;123:12. Available from: https://pubmed.ncbi.nlm.nih.gov/7582737/

21. Stern C, Jordan Z, McArthur A. Developing the Review Question and Inclusion Criteria. AJN, Am J Nurs [Internet]. 2014 Apr;114(4):53–6. Available from: https://journals.lww.com/00000446-201404000-00030

22. Salas-Medina P. La pregunta clínica. Barcelona: FUOC; 2019. 20 p.

23. Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak [Internet]. 2007 Dec 15;7(1):16. Available from: https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/1472-6947-7-16

24. Centre for Reviews and Dissemination. Systematic reviews. CRD’s guidance for undertaking reviews in health care. New York: University of York; 2009. 294 p.

25. Petticrew M, Roberts H. Systematic Reviews in the Social Sciences [Internet]. Blackwell Publishing Ltd; 2006. 354 p. Available from: https://onlinelibrary.wiley.com/doi/book/10.1002/9780470754887

26. The ADAPTE Collaboration. Guideline adaptation: A resource toolkit [Internet]. Version 2. 2009. 95 p. Available from: https://g-i-n.net/wp-content/uploads/2021/05/ADAPTE-Resource-toolkit-V2.1-March-2010-updated-disclaimer.pdf

27. Dawes M, Pluye P, Shea L, Grad R, Greenberg A, Nie JY. The identification of clinically important elements within medical journal abstracts: Patient Population–Problem, Exposure– Intervention, Comparison, Outcome, Duration and Results (PECODR). Inform Prim Care. 2007;15:9–16.

28. de Oloveira Cruz Latorraca C, Rodrigues M, Leite Pacheco R, Cabrera Martimbianco A, Riera R. Busca em bases de dados eletrônicas da área da saúde: por onde começar. Diagn Trat. 2019;24(2):59–63.

29. Kestenbaum B. Population, Exposure, and Outcome. In: Epidemiology and Biostatistics [Internet]. Cham: Springer International Publishing; 2019. p. 5–6. Available from: http://link.springer.com/10.1007/978-3-319-97433-0_2

30. Schlosser R, O’Neil-Pirozzi T. Problem Formulation in Evidence-based Practice and Systematic Reviews. Contemp Issues Commun Sci Disord. 2006;33:5–10.

31. Wildridge V, Bell L. How CLIP became ECLIPSE: a mnemonic to assist in searching for health policy/management information. Heal Inf Libr J [Internet]. 2002 Jun 26;19(2):113–5. Available from: https://onlinelibrary.wiley.com/doi/10.1046/j.1471-1842.2002.00378.x

32. Booth A. Clear and present questions: formulating questions for evidence based practice. Libr Hi Tech [Internet]. 2006;24(3):355–68. Available from: https://web.archive.org/web/20170812112250id_/http://testbed.ischool.drexel.edu/I515/Booth.pdf

33. Cooke A, Smith D, Booth A. Beyond PICO: The SIPDER tool for qualitative evidence synthesis. Qual Health Res [Internet]. 2012 Oct 24;22(10):1435–43. Available from: http://journals.sagepub.com/doi/10.1177/1049732312452938

34. La literatura gris. Form Univ [Internet]. 2011;4(6):1. Available from: https://www.scielo.cl/pdf/formuniv/v4n6/art01.pdf

35. De Luca Canto G. Revisões Sistemáticas da Literatura: Guia Prático. Curitiba: Editora Brazil Publishing; 2020. 191 p.

36. Pereira MG, Galvão TF. Etapas de busca e seleção de artigos em revisões sistemáticas da literatura. Epidemiol e Serviços Saúde [Internet]. 2014 Jun;23(2):369–71. Available from: http://scielo.iec.pa.gov.br/scielo.php?script=sci_arttext&pid=S1679-49742014000200019&lng=en&nrm=iso&tlng=en

37. Pursio K, Kankkunen P, Sanner‐Stiehr E, Kvist T. Professional autonomy in nursing: An integrative review. J Nurs Manag [Internet]. 2021 Sep 26;29(6):1565–77. Available from: https://onlinelibrary.wiley.com/doi/10.1111/jonm.13282

38. Silva BB da, Lima MH de M, Saidel MGB. Mental health nursing care for people with diabetes mellitus: An integrative review. Rev Lat Am Enfermagem [Internet]. 2023 Dec;31:e4073. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-11692023000100612&tlng=en

39. Hannes K, Lockwood C. Pragmatism as the philosophical foundation for the Joanna Briggs meta‐aggregative approach to qualitative evidence synthesis. J Adv Nurs [Internet]. 2011 Jul 6;67(7):1632–42. Available from: https://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.2011.05636.x

40. Saeed Khan K, Bueno-Cavanillas A, Zamora J. Revisiones sistemáticas en cinco pasos: IV. Cómo sintetizar los resultados. Med Fam Semer [Internet]. 2022 Nov;48(8):101785. Available from: https://linkinghub.elsevier.com/retrieve/pii/S1138359322000776

41. Real Academia Española (RAE). dle.rae.es. 2023. Flujograma. Available from: https://dle.rae.es/flujograma

42. Real Academia Española (RAE). dle.rae.es. 2023. Calidad. Available from: https://dle.rae.es/calidad

43. Programa de Habilidades en Lectura Crítica Español (CASPe). redcaspe.org. 2022. Instrumentos para la lectura crítica. Available from: https://redcaspe.org/materiales/

44. Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study. J Clin Epidemiol [Internet]. 2019 Jul;111:49-59.e1. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0895435618300829

45. Joanna Briggs Institute (JBI). jbi.global. 2024. Critical appraisal tools. Available from: https://jbi.global/critical-appraisal-tools

46. O’cathain A, Murphy E, Nicholl J. The Quality of Mixed Methods Studies in Health Services Research. J Health Serv Res Policy [Internet]. 2008 Apr 21;13(2):92–8. Available from: http://journals.sagepub.com/doi/10.1258/jhsrp.2007.007074

47. Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Med [Internet]. 2010 Dec 24;8(1):18. Available from: http://bmcmedicine.biomedcentral.com/articles/10.1186/1741-7015-8-18

48. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Heal Care [Internet]. 2007 Sep 16;19(6):349–57. Available from: https://academic.oup.com/intqhc/article-lookup/doi/10.1093/intqhc/mzm042

49. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol [Internet]. 2008 Apr;61(4):344–9. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0895435607004362

50. O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for Reporting Qualitative Research. Acad Med [Internet]. 2014 Sep;89(9):1245–51. Available from: http://journals.lww.com/00001888-201409000-00021

51. Des Jarlais DC, Lyles C, Crepaz N. Improving the Reporting Quality of Nonrandomized Evaluations of Behavioral and Public Health Interventions: The TREND Statement. Am J Public Health [Internet]. 2004 Mar;94(3):361–6. Available from: https://ajph.aphapublications.org/doi/full/10.2105/AJPH.94.3.361

52. Valencia-Contrera MA, Orellana Yáñez AE. Fenómeno techo de cristal en enfermería: revisión integrativa. Rev Cuid [Internet]. 2022;13(1):e2261. Available from: https://revistas.udes.edu.co/cuidarte/article/view/2261

53. Pavinati G, Vinícius de Lima L, Paiano M, Jaques AE, Tavares Magnabosco G. Contextos de vulnerabilidade de adolescentes que (con)vivem com HIV: uma revisão integrativa. Rev Cuid [Internet]. 2023 Jun 30;14(2):e2803. Available from: https://revistas.udes.edu.co/cuidarte/article/view/2803

54. Prates P, Correa Júnior A, Teles A, Paraizo-Horvath C, Sonobe H. Enfermagem em crises convulsivas pediátricas e contribuições oncológicas: revisão integrativa. Rev Recien [Internet]. 2024;14(42):132–47. Available from: https://doi.org/10.24276/rrecien2024.14.42.132904

55. Lloyd R, Volleath M, Mertens D. Writing the Literature Review [Internet]. Fort Hays State University; 2023. 100 p. Available from: https://scholars.fhsu.edu/all_oer/8/

56. Chigbu UE, Atiku SO, Du Plessis CC. The Science of Literature Reviews: Searching, Identifying, Selecting, and Synthesising. Publications [Internet]. 2023 Jan 6;11(1):2. Available from: https://www.mdpi.com/2304-6775/11/1/2

57. Powell JT, Koelemay MJW. Systematic Reviews of the Literature Are Not Always Either Useful Or the Best Way To Add To Science. EJVES Vasc Forum [Internet]. 2022;54:2–6. Available from: https://linkinghub.elsevier.com/retrieve/pii/S2666688X21000605

FINANCING

None.

CONFLICT OF INTEREST

None.

AUTHORSHIP CONTRIBUTION

Conceptualization: Miguel Valencia-Contrera.

Data curation: Miguel Valencia-Contrera, Naldy Febré, Daniella Cancino Jiménez, Flérida Rivera-Rojas, Jenifer Villa-Velasquez, Solange Vallejos Vergara, Alba Lozano-Romero, Rodrigo-Alejandro Ardiles-Irarrazabal, José Castro-Bastidas, María Quintana-Zavala, Maria Lucia do Carmo Cruz Robazzi, Vivian Vilchez-Barboza, Sandra Valenzuela-Suazo.

Formal Analysis: Miguel Valencia-Contrera, Naldy Febré, Daniella Cancino Jiménez, Flérida Rivera-Rojas, Jenifer Villa-Velasquez, Solange Vallejos Vergara, Alba Lozano-Romero, Rodrigo-Alejandro Ardiles-Irarrazabal, José Castro-Bastidas, María Quintana-Zavala, Maria Lucia do Carmo Cruz Robazzi, Vivian Vilchez-Barboza, Sandra Valenzuela-Suazo.

Research: Miguel Valencia-Contrera, Naldy Febré, Daniella Cancino Jiménez, Flérida Rivera-Rojas, Jenifer Villa-Velasquez, Solange Vallejos Vergara, Alba Lozano-Romero, Rodrigo-Alejandro Ardiles-Irarrazabal, José Castro-Bastidas, María Quintana-Zavala, Maria Lucia do Carmo Cruz Robazzi, Vivian Vilchez-Barboza, Sandra Valenzuela-Suazo.

Methodology: Miguel Valencia-Contrera, Naldy Febré.

Project management: Miguel Valencia-Contrera.

Resources: Miguel Valencia-Contrera, Naldy Febré, Daniella Cancino Jiménez, Flérida Rivera-Rojas, Jenifer Villa-Velasquez, Solange Vallejos Vergara, Alba Lozano-Romero, Rodrigo-Alejandro Ardiles-Irarrazabal, José Castro-Bastidas, María Quintana-Zavala, Maria Lucia do Carmo Cruz Robazzi, Vivian Vilchez-Barboza, Sandra Valenzuela-Suazo.

Software: Miguel Valencia-Contrera.

Supervision: Miguel Valencia-Contrera.

Validation: Miguel Valencia-Contrera, Naldy Febré, Daniella Cancino Jiménez, Flérida Rivera-Rojas, Jenifer Villa-Velasquez, Solange Vallejos Vergara, Alba Lozano-Romero, Rodrigo-Alejandro Ardiles-Irarrazabal, José Castro-Bastidas, María Quintana-Zavala, Maria Lucia do Carmo Cruz Robazzi, Vivian Vilchez-Barboza, Sandra Valenzuela-Suazo.

Visualization: Miguel Valencia-Contrera, Naldy Febré, Daniella Cancino Jiménez, Flérida Rivera-Rojas, Jenifer Villa-Velasquez, Solange Vallejos Vergara, Alba Lozano-Romero, Rodrigo-Alejandro Ardiles-Irarrazabal, José Castro-Bastidas, María Quintana-Zavala, Maria Lucia do Carmo Cruz Robazzi, Vivian Vilchez-Barboza, Sandra Valenzuela-Suazo.

Original drafting: Miguel Valencia-Contrera, Naldy Febré, Daniella Cancino Jiménez, Flérida Rivera-Rojas, Jenifer Villa-Velasquez, Solange Vallejos Vergara, Alba Lozano-Romero, Rodrigo-Alejandro Ardiles-Irarrazabal, José Castro-Bastidas, María Quintana-Zavala, Maria Lucia do Carmo Cruz Robazzi, Vivian Vilchez-Barboza, Sandra Valenzuela-Suazo.

Writing-revision and editing: Miguel Valencia-Contrera, Naldy Febré, Daniella Cancino Jiménez, Flérida Rivera-Rojas, Jenifer Villa-Velasquez, Solange Vallejos Vergara, Alba Lozano-Romero, Rodrigo-Alejandro Ardiles-Irarrazabal, José Castro-Bastidas, María Quintana-Zavala, Maria Lucia do Carmo Cruz Robazzi, Vivian Vilchez-Barboza, Sandra Valenzuela-Suazo.