Page 1 of 16
European Journal of Applied Sciences – Vol. 11, No. 1
Publication Date: January 25, 2023
DOI:10.14738/aivp.111.13891. Mangia, C. M. F., Toledo, M. D., Rossi, R., Nakano, E. Y., Carneluti, A., Kopelman, B. I., Carvalho, W. B., & Andrade, M. C. (2023).
Performance of Brazilian Pediatric Risk of Severity Model for Illness (Br PRISM) Compared to Pediatric Index of Mortality and
Pediatric Risk of Mortality 2. European Journal of Applied Sciences, Vol - 11(1). 287-302.
Services for Science and Education – United Kingdom
Performance of Brazilian Pediatric Risk of Severity Model for
Illness (Br PRISM) Compared to Pediatric Index of Mortality and
Pediatric Risk of Mortality 2
Mangia, C. M. F. MD, MSc, MBA, PhD.
Pediatric Critical Care Division, Escola Paulista de Medicina,
Universidade Federal de São Paulo, Brazil
Toledo, M. D., MD.
Pediatric Critical Care Division, Escola Paulista de Medicina,
Universidade Federal de São Paulo, Brazil
Rossi, R., MD.
Pediatric Critical Care Division, Escola Paulista de Medicina,
Universidade Federal de São Paulo, Brazil
Nakano, E. Y.,PhD.
Statistics Department, Universidade de Brasília, Brazil
Carneluti, A., MD.
Faculdade de Medicina, FMABC, Brazil
Kopelman, B. I., PhD.
Pediatrics Department Escola Paulista de Medicina,
Universidade Federal de São Paulo, Brazil
Carvalho, W. B., PhD.
Pediatric Critical Care Division, Universidade de São Paulo
Andrade, M. C., MD, MSc, PhD.
Pediatric Nephrology Division, Escola Paulista de Medicina,
Universidade Federal de São Paulo, Brazil
ABSTRACT
Introduction: The best prognosis score with which to evaluate high-risk patients
upon admission into pediatric intensive care is not well established in resource- limited settings. The objective of study was to formulate a risk-of-illness severity
model for pediatric mortality to be applied upon PICU admission in resource- limited settings. Methods: Our study was designed to develop an illness severity
index and a prognostic model for critically ill children. A prospective, observational
multicenter pilot study, performed between February 1995 and October 1999,
evaluated the variables, methodology and statistical techniques for the
development of a model. A single-center prospective cohort study, performed
Page 2 of 16
288
European Journal of Applied Sciences (EJAS) Vol. 11, Issue 1, January-2023
Services for Science and Education – United Kingdom
between November 1999 and October 2004, collected information from
consecutive admissions into the Pediatric Intensive Care Unit (PICU) at a high- complexity university, teaching, and reference hospital in São Paulo, Brazil. Results:
In the pilot study, 1,459 patients (a PICU mortality rate of 16%) were included, and
in the second study, 1,033 patients (a PICU mortality rate of 13.9% and a hospital
mortality rate of 6.9% after PICU discharge) were included. We used multivariable
regression to determine two probabilistic models; the first addressed survival and
the overall probability of death (hospital plus PICU deaths), and the second was
conditional (i.e., PICU death). An illness severity index stratified these probabilities
into three risk strata: low-, medium- and high-risk patients. In the final step, the
new death probabilities were estimated using a Bayesian adjustment. Conclusions:
The model estimates three probabilities (survival, death in the PICU and death in
the hospital after PICU discharge) stratified into three risk categories. To the best
of our knowledge, this is the first study using a Bayesian adjustment to determine a
prognosis and illness severity, and it should enable us to make therapeutic
adjustments and provide appropriate counseling for high-risk patients in resource- limited settings.
INTRODUCTION
Prognostication efforts are important steps towards understanding the effects of diseases,
medical interventions, and healthcare policies as determinants of outcomes. Mortality risk
models enable the evaluation of the healthcare system, management capacity and quality of
care and facilitate evidence-based decision-making and better resource allocation [1,2,3].
The outcome in intensive care depends on several factors associated with the patient in the first
24 hours after admission and the disease course during the intensive care stay. Severity scores
are usually comprised of two parts: a severity score, which is a number (in general, a high score
reflects a more severe condition), and a probability model, which is an equation that expresses
the probability of death in the hospital or intensive care unit (ICU) [3,4,5].
No consensus about the classification of score systems to be used in the ICU has been reached;
they could be used once or repeatedly over time. There are numerous examples of score
systems, but the main systems are scores based on abnormalities in the physiological variables
measured in the first 24 hours (APACHE, PRISM, PIM) or organ-specific scoring, in which the
main prognostic factors are the number and duration of multiple-organ failures (SOFA, PELOD).
The Brazilian healthcare system is a predominantly public enterprise with universal access for
all citizens. Over the past few years, as part of the Millennium Development Goals for the
reduction of child mortality, new pediatric intensive care units (PICUs) like those in other areas
of the world [6] have been introduced.In 1998, the Brazilian Ministry of Health suggested using
the Pediatric Risk of Mortality (PRISM) [7] score to assess the severity of illnesses and mortality
risks and evaluate PICU performance. Studies have reported that this score is not suited for
critically ill children in resource-limited settings [8], and it is well recognized that performance
scores are variable because the case mix, therapy and selection of patients admitted into the
PICU differ over time. Indeed, PRISM has been outdated for more than 10 years and,
consequently, is obsolete [1,3,4,5].
Page 3 of 16
289
Mangia, C. M. F., Toledo, M. D., Rossi, R., Nakano, E. Y., Carneluti, A., Kopelman, B. I., Carvalho, W. B., & Andrade, M. C. (2023). Performance of
Brazilian Pediatric Risk of Severity Model for Illness (Br PRISM) Compared to Pediatric Index of Mortality and Pediatric Risk of Mortality 2. European
Journal of Applied Sciences, Vol - 11(1). 287-302.
URL: http://dx.doi.org/10.14738/aivp.111.13891
Our main objective was to formulate a risk-of-illness severity model for pediatric mortality to
be applied upon PICU admission in resource-limited settings [3].
METHODS
The Brazilian Pediatric Risk of Severity Model for Illness (Br PRISM) study was developed in
two steps. The first study was a prospective, multicenter, 2 teaching hospitals and 1 private
hospital), cohort study performed between February 1995 and October 1999 and included
1,450 patients. This study evaluated the variables, methodology, and viability of performing a
multicenter study in Brazil and the statistical techniques required for developing a scoring
system and probability model. The second, a validation study, was a single-center, prospective,
observational cohort study performed between November 1999 and October 2004 and
included 1,100 consecutive patients admitted into the Hospital São Paulo from Universidade
Federal de São Paulo, Brazil. Hospital São Paulo, a resource for five million inhabitants, is a high- complexity hospital affiliated with the university teaching medical school. The hospital has 700
beds (80 pediatric) and receives 530,000 emergencies, 32,264 admissions and 163,305
surgeries per year. The pediatric intensive care unit (PICU) has 8 beds and admits medical and
surgical patients between the ages of 0 and 19 years. In the study period, we had 16 pediatric
intensive care residents each year. The medical staff included 2 physicians during the day, 1
physician at night and a total staff of 18 pediatric intensivists (including weekends). In addition,
the nurse-to-patient ratio was 1 nurse to each of 3 beds and one physiotherapist to 8 beds.
Sample Selection
All the consecutive admissions of patients under the age of 19 were analyzed, except the
following: a) patients with a PICU stay of less than 24 hours; b) patients admitted while
receiving continuous cardiopulmonary resuscitation without stable signs for at least 2 hours;
c) brain-dead patients admitted for organ donation. For those patients with multiple PICU
admissions during the same hospital stay, only the data from the first admission were analyzed.
Re-admissions were analyzed if they occurred more than 30 days after PICU discharge. To
determine the outcome, the patients were followed up until they were discharged from the
hospital. Any patients remaining in the hospital after October 31, 2004, were excluded from the
study because their status could not be assessed. The study was approved by the institutional
ethics committee, and parental consent was obtained in all cases.
Variable Selection:
The variables were selected based on our past experience with first study, clinical judgment,
interviews with the intensivists and score review and considered a wide range of citations on
the literature, such as the Pediatric Risk of Mortality II [7] and III [9], the Acute Physiology, Age,
and Chronic Health Evaluation III [10], the Simplified Acute Physiology Score II [11], Mortality
Probability Models II [12], the Pediatric Index of Mortality [13] and the Acute Physiology, Age,
Chronic Health Evaluation II [14]. All the data were collected by the main investigator.
The physiological variables that were eligible for analysis are as follows: the systolic blood
pressure, the heart rate, the respiratory rate, the axillary temperature, any pupillary reactions,
the coma status, diuresis, arterial gasometry, the PaO2/FiO2 ratio, glucose, potassium, sodium,
creatinine, urea, hemoglobin, hematocrit, platelet count, the white blood cell count, and the
prothrombin and activated partial thromboplastin times. The other, non-physiological,
variables are as follows: age, age group, gender, the in-hospital location before PICU admission,
Page 4 of 16
290
European Journal of Applied Sciences (EJAS) Vol. 11, Issue 1, January-2023
Services for Science and Education – United Kingdom
any comorbidities, the clinical or surgical status, the diagnosis by system and etiology (during
the first 24 hours), the use of vasoactive drugs, the use of external oxygen or mechanical
ventilation, the length of stay (LOS) before and after the PICU, the PICU LOS, the total length of
the hospital stay and any outcome data (the vital status in the PICU and upon hospital
discharge) [13,14,15,16,17,18].
The physiological variables were collected upon admission, and, if the laboratory had missed
any biochemical data, we recorded the worst value achieved in the first 24 hours according to
the strict definitions of the previously established variables (Appendix 1). When these values
were age-dependent, we used the range limits of the normal physiological values by age group.
We developed a comprehensive instruction manual, which described all the procedures that
led to the data collection and definition. This manual was based on the evidence in the literature
and included a full description of the study and strict definitions of the variables, their codes,
and, when applicable, their units and normal ranges according to the age group. The age group
was based on the recommendations of the Ministry of Health, which established the following
risk-specific age groups for Brazilian children: less than 12 months, between 12 and 59 months,
between 60 and 119 months, between 120 and 179 months and between 180 and 228 months
[19].
The data were collected in a clinical report form (CRF), the variables were codified, and the
internal quality of the data was checked before keyboarding into the ACCESS® database that
was specially created for this study. The program checked for any out-of-range data using a
logical error system and compiled a report regarding any inconsistent data for each patient.
The quality control of the database included double-keyboarding by two trained and
independent physicians. The first and second sets of keyboarding were compared to the CRF to
determine the reliability of the data from the first and second procedures. The reliability of the
data was compared to that of the CRF and the medical record. To determine the diagnostic
category after PICU admission, we developed a list of the ten major categories of clinical
diseases and nine major categories of surgical interventions. For each major category, we
developed a list of 124 etiology classes according to the age group and the epidemiology of the
pediatric diseases [9,11,12]. The same procedure was used for the categorization of the
comorbidities [15].
Statistical Analysis
The demographic data were represented as absolute numbers and percentages, and the
continuous variables were represented as medians and interquartile ranges. A p-value of less
than 0.05 was significant.
We used multivariable regression to determine two probabilistic models; the first addressed
the probability of death in the PICU, and the second was conditional (i.e., the probability of
death in the hospital after the PICU stay) [20,21,22].
We eliminated variables from the models by backward deletion. These two models produced a
of probabilities for each patient. The first element focuses on hospital survival, the second
focuses on death in the hospital after PICU discharge, and the third probability focuses on the
death probability during the PICU stay. Based on these three probabilities, we created a
severity index that stratifies patients from the worst (PICU death) to best (hospital survival)
Page 5 of 16
291
Mangia, C. M. F., Toledo, M. D., Rossi, R., Nakano, E. Y., Carneluti, A., Kopelman, B. I., Carvalho, W. B., & Andrade, M. C. (2023). Performance of
Brazilian Pediatric Risk of Severity Model for Illness (Br PRISM) Compared to Pediatric Index of Mortality and Pediatric Risk of Mortality 2. European
Journal of Applied Sciences, Vol - 11(1). 287-302.
URL: http://dx.doi.org/10.14738/aivp.111.13891
outcomes according to a previously published method [23,24]. The a priori probabilities of
model were: 10% (death in the PICU), 5% (death in the hospital after PICU stay) and 85%
(hospital survival) [23,24]. As a final step, the Bayesian method was applied to estimate the
new adjustment of probabilities (a posteriori) using the severity index. Statistical analysis was
performed using SPSS (version 11.0) and Excel 2000.
RESULTS
The variables were collected from 1,100 patients, 67 patients were excluded (37 patients were
discharged within 24 hours after PICU admission, 2 patients were still hospitalized at the end
of the study, 17 patients died within the first 24 hours after PICU admission, and 11 patients
were admitted into the PICU for organ donation after brain death). Following the exclusions,
1,033 patients were included in the development of the model. The patients’ characteristics are
summarized in Table 1. Comorbidities were present in 73.9% of the patients. The main
comorbidities were as follows: congenital cardiac disease (21.2%), chronic neurological
disease (10.6%), chronic renal disease (7.6%) and chronic pulmonary disease (6.2%).
Table 2 presents the logarithm of the first regression analysis, and Table 3 presents the log of
the second regression analysis.
The severity index (SI) was calculated using the following equation: SI = (21⁄2 +1)Pr(U) -
(21⁄2)Pr(H), where Pr(U) is the probability of dying in the PICU, and Pr(H) is probability of dying
in the hospital after the PICU stay (table 4). The cutoff value was £ 0.15 for the SI of the
survivors, 0.16 to 0.30 for hospital mortality after the PICU stay, and 3 0.30 for PICU mortality.
Next, we re-adjusted the probabilities using 3 severity classes based on the cutoff points of the
index (high, medium, or low probability of death). The index demonstrated good differentiation
among the 3 severity classes (p<0.001). The area under the ROC curve (AUC) for the survivors
(index £ 0.15) was good (0.821; 95% CI, 0,789 - 0,854). The cutoff point was 0.1564 (sensitivity,
0.738; 1-specificity, 0.265). The Hosmer-Lemeshow goodness-of-fit chi-squared value for the
survivors was 22.154 with 8 degrees of freedom (p=0.005).
The AUC for death in the PICU (index 3 0.30) was good (0.746; 95% CI, 0,676 - 0,817). The
cutoff point was 0.3058 (sensitivity, 0.674; 1-specificity, 0.324). The Hosmer-Lemeshow
goodness-of-fit chi-squared value for the deaths in the PICU was 9.300 with 8 degrees of
freedom (p=0.318). The a posteriori probabilities for each diagnostic category (hospital
survival, patient death in the hospital after the PICU stay and patient death during the PICU
stay) and the risk strata are presented in Table 5.
Br PRISM Compared to PIM and PIM 2
We compared the performance of Br PRISM to two scores with free access in the literature. The
Pediatric Index of Mortality (PIM; versions 1 and 2) met this criterion. The PIM and PIM 2 scores
were collected for 387 patients. The standardized mortality rate (SMR) for the PIM score was
2.464 (95% CI, 1.413 – 3.515), and the odds ratio was 0.56 (95% CI, 0.33 – 0.95); the SMR for
the PIM 2 score was 2.526 (95% CI, 1.366 – 3.687), and the odds ratio was 0.94 (95% CI, 0.57 –
1.57). The area under the ROC curve was 0.882 (95% CI, 0.846 - 0.913) for Br PRISM, 0.736
(95% CI, 0.689 - 0.7790) for PIM and 0.720 (95% CI, 0.672 - 0.764) for PIM 2. The pairwise
comparison of the ROC curves for Br PRISM vs. PIM and PIM2 showed a difference between the
Page 6 of 16
292
European Journal of Applied Sciences (EJAS) Vol. 11, Issue 1, January-2023
Services for Science and Education – United Kingdom
area under ROC curve (0.146; 95% CI, 0.054 - 0.238; p = 0.002). The areas of Br PRISM vs. PIM
2 were different (0.163; 95%, CI 0.066 - 0.260; p = 0.001). The areas of PIM vs. PIM 2 were also
different (0.017; 95% CI 0.044 -0.077; p = 0.591).
DISCUSSION
We undertook this study to develop an illness severity index and a prognostic risk model for
critically ill children that will be useful and relevant to middle-income environments. This
model is based on variables that are easily collected [25] at the bedside and includes well- defined [26] variables that are selected a priori. The investigators collecting the data were
blinded to the study objectives, and continuous, rigorous monitoring was done to eliminate the
possibility of missing [26,27] information to guarantee a high-quality database [28].
The laboratory variables were collected upon admission; however, when the sample was lost
or when technical problems arose during the processing of the samples by the laboratory, the
worst value during the first 24 hours was recorded [9]. We adopted this criterion because, in
our practice, the first sample was susceptible to loss (breakdown of the sample bottle, for
example), or there was a lack of the reagents with which to process the sample immediately or
during sampling [16].
We defined any admission that occurred after thirty days after PICU discharge as a new
admission. This admission would most likely be the result of a new clinical indication and,
therefore, unlikely to be due to an inappropriately early discharge [29]. Additionally, in our
medical practice, mortality in the PICU is an inadequate measure with which to evaluate the
outcomes of a critical disease. The inclusion of the hospital mortality after PICU discharge adds
a new element to the prediction and improves our knowledge regarding outcomes outside the
PICU, which may have a direct bearing on PICU care or the early discharge of unstable patients
[29].
By analyzing the regression models, we found that the overall survival depended more strongly
on the physiological variables. However, the biochemical abnormalities in the conditional
model were determinants of a major risk for dying in the hospital after PICU discharge.
The model supplied a vector of probabilities with three components (survival, PICU death and
hospital death after PICU stay). However, simultaneous interpretations of these data were
deemed to be too complex to explain to families and healthcare providers. Therefore, the
severity index simplifies the information because only one probability is necessary to explain
the gravity of each case [30,31].
The model described in this study don ́t need revised by new validation because the inclusion
of new patients in the database and the modification of its initial information the model will be
auto adjusted. The attraction of the Bayesian model is that it is a dynamic model and superior
to the information provided by the previous risk scores.
Comparing Br PRISM to the PIM and PIM 2 scores showed that the PIM and PIM 2 scores
overestimated the mortality in the high- and very high-risk bands and underestimated the
mortality in mild- and low-risk bands. The ROC curve analysis demonstrated that both had low
sensitivity and specificity in our population [32]. These observations could be explained by the
Page 7 of 16
293
Mangia, C. M. F., Toledo, M. D., Rossi, R., Nakano, E. Y., Carneluti, A., Kopelman, B. I., Carvalho, W. B., & Andrade, M. C. (2023). Performance of
Brazilian Pediatric Risk of Severity Model for Illness (Br PRISM) Compared to Pediatric Index of Mortality and Pediatric Risk of Mortality 2. European
Journal of Applied Sciences, Vol - 11(1). 287-302.
URL: http://dx.doi.org/10.14738/aivp.111.13891
fact that the PIM and PIM 2 scores have limitations in resource-limited settings. The
applicability of scores such as PIM and PIM 2 in our country is difficulted because of flaws in
these scores to assessment of the prognostics after PICU stay (like as death in the hospital stay
or hospital survival) and by differences of quality of care (including human, technological and
economic factors) rendered by PICU in resource limited settings compared to PICU of high- income settings where these scores were developed.
Our study has several limitations, including the fact that the final model was based on a single- center study. However, this option gave us control over data entry into the CRF, which ensured
data integrity (loss of sample and laboratory errors) and immediate mistake correction. For
example, in the pilot multicenter study, we observed that several variables with units different
from those that were standardized by the study were included in the database [20].
Another limitation relates to patient admission into the PICU. In resource-limited settings,
owing to the high demand for intensive-care beds, patients who should have been admitted into
the PICU earlier had stayed in other hospital settings, and, therefore, specific PICU treatments
were delayed [8].In these setting, patients requiring mechanical ventilation and patients after
high-risk surgery are priorities. This situation results in the baseline condition upon PICU
admission being worse than in high-income settings [32,33].
Parents also have difficulties on early recognition of the severity of diseases. Additionally, due
to socioeconomic reasons, a lack of transport or being transferred from other hospitals, some
patients may present late to emergency care [34]. This situation was represented in another
study as lead-time bias or the possibility that the patients may have had a higher-than- predicted mortality, which may generate some degree of error during scoring. However, this
situation is impossible to control [8]. In addition, the current model refers to a specific
population and will need to be used and validated in a new cohort similar to the reference
population before it can be used in a large-scale setting [16,35,36].
This model is the first pediatric model developed for resource-limited settings such as Brazil.
Brazil does not have a prognostic and severity model with which to examine the effect of
disease severity on patient admission into the PICU, after PICU stay and after Hospital stay. This
information would be useful when analyzing the costs (especially PICU and infirmary care
costs) to the Unified Health System (SUS) and the relative benefits to society. We believe that
the use of this model could improve the training of PICU teams by aiding in the development of
the skills necessary to discriminate between the severity categories of diseases, establish early
treatment strategies and minimize costs, mortality and sequelae.
Contributors
Mangia CMF wrote the methodology of this study, drafted study, carried out the cohort study,
all data collection, data typewrite, development of all steps of the database software, performed
the descriptive and inferential statistical analysis and wrote the manuscript. Nakano EY
performed the inferential and Bayesian statistical analysis. Carneluti A wrote the manuscript
and contributed with ideas and critical revision. Carvalho WB, Kopelman BI, Rossi R, Toledo
MD and Andrade MC contributed with ideas and critical revision, development of all steps of
this study. The paper was revised and approved by all contributors.
Page 8 of 16
294
European Journal of Applied Sciences (EJAS) Vol. 11, Issue 1, January-2023
Services for Science and Education – United Kingdom
Acknowledgements
We would kindly like to thank Pereira CAB, PhD (Bayesian analysis). Ms. Gucoff C for her
valuable work to develop the five versions of the database software and its updates. In addition,
we would kindly like to thank de Carvalho WB for his indispensable permission for the
accomplishment of this study in the PICU and to colleagues of all centers of pilot study.
Funding: None.
Conflict of Interest Statement: The authors declare that they have no conflict of interests.
References
1. Wasson JH, Sox HC, Neff RK et al. Clinical prediction rules: Applications and methodological standards. N
Engl J Med 1985; 313:793-798.
2. Logan RL, Scott PJ. Uncertainty in clinical practice: implications for quality and costs of health care. Lancet
1996; 347:595-598.
3. Mangia, CMF; Carvalho WB and Kopelman, BI. Brazilian Pediatric Prognostic Model for Critical Illness. In: 5
th World Congress on Pediatric Critical Care, June 24-28, 2007, Geneve. Ped Crit Care 2007; 8: A194.
4. Mangia, CMF, Kopelman, BI. Performance of Pediatric Index of Mortality (PIM) and PIM 2 in a Pediatric
Intensive Care Unit in a Middle-Income Country. In: 5th World Congress on Pediatric Critical Care,
2007.Pediatric Critical Care Medicine. Baltimore: Lippincott Willians and Wilkins, 2007. v. 8. p. A71-.
5. Mangia, CMF; Nakano, EY; PEREIRA, Carlos Alberto de Bragança. Brazilian Pediatric Prognostic Model for
Critical Illness. In: EBEB X - 10o Encontro Brasileiro de Estatística Bayesiana, 2010, Rio de Janeiro. EBEB X -
10o Encontro Brasileiro de Estatística Bayesiana, 2010.
6. WHO-UNICEF. Declaração Mundial Sobre a Sobrevivência, a Proteção e o Desenvolvimento das Crianças nos
Anos 90.http://www.unicef.org/brazil/summit2.htm.
7. Pollack MM, Ruttimann UE, Getson PR: Pediatric Risk of Mortality (PRISM) score. Crit Care Med 1988;
16:1110-1116.
8. Wells M, Riera-Fanego JF, Luyt DK et al. Poor discriminatory performance of the Pediatric Risk of Mortality
(PRISM) score in a South African intensive care unit. Crit Care Med 1996; 24(9): 1507-1513.
9. Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated pediatric risk of mortality score. Crit Care Med
1996; 24:743-752.
10. Knaus WA, Wagner DP, Draper EA et al. The APACHE III prognostic system: risk prediction of hospital
mortality for critically ill hospitalized adults. Chest 1991; 100:1619-1636.
11. Le Gall JR, Lemeshow S and Saulnier F. A new simplified acute physiology score (SAPS II) based on a
European/North American multicenter study. JAMA 1993; 270:2957-63.
12. Lemeshow S, Teres D, Klar J et al. Mortality probability models (MPM II) based on an international cohort of
intensive care unit patients. JAMA 1993; 270:2478-2486.
13. Shann F, Pearson G, Slater A et al. Paediatric index of mortality (PIM): a mortality prediction model for
children in intensive care. Intensive Care Med 1997; 23:201-207.
14. Knaus WA, Draper EA, Wagner DP et al. APACHE II: a severity of disease classification system. Crit Care Med
1985; 13:918-829.
Page 9 of 16
295
Mangia, C. M. F., Toledo, M. D., Rossi, R., Nakano, E. Y., Carneluti, A., Kopelman, B. I., Carvalho, W. B., & Andrade, M. C. (2023). Performance of
Brazilian Pediatric Risk of Severity Model for Illness (Br PRISM) Compared to Pediatric Index of Mortality and Pediatric Risk of Mortality 2. European
Journal of Applied Sciences, Vol - 11(1). 287-302.
URL: http://dx.doi.org/10.14738/aivp.111.13891
15. Iezzoni LI, Heeren T, Foley SM et al. Chronic conditions and risk of in-hospital death. HSR 1994:29(4):435-
460.
16. Pollack MM, Patel KM, Ruttiman U et al. Frequency of variable measurement in 16 pediatric intensive care
unit: Influence on accuracy and potential for bias in severity of illness assessment. Crit Care Med 1996;
24:74-77.
17. Pitte TD, Thiévent B, Wenzel RP et al. Importance of pre-existing co-morbidities for prognosis of septicemia
in critically ill patients. Intensive Care Med 1993; 9:265-272.
18. Knaus WA, Wagner DP, Zimmerman JE and Draper EA. Variations in mortality and length of stay in intensive
care units. Ann Intern Med 1993; 118(10):753-761.
19. Brazilian Ministry of Healthy – DATASUS [database on the Internet].[cited 1999 Oct 1]. Available from:
http://tabnet.datasus.gov.br/cgi/ deftohtm.exe? sih/cnv/miuf .def.
20. Kollef MH, Schuster DP: Predicting intensive care unit outcomes with scoring systems. Crit Care Clin 1994;
10:1-18.
21. Concato J, Feinstein AR and Holferd TR. The risk of determining risk with multivariable models. Ann Intern
Med 1993;118: 201-10.
22. Lemeshow S, Le Gall JR. Modeling the severity of illness of ICU patients. A system update. JAMA 1994;
272:1049-1055.
23. Barlow BE and Pereira CA de B. Medical diagnosis using influence diagrams. Networks 1990;20(1):565-577.
24. Pereira CA de B and Pericchi P. Analysis of diagnosability. J Royall Statistical Society C (Applied Statistics)
1990; 39(2):189-204.
25. Marcin JP and Pollack MM. Review of the methodologies and applications of scoring systems in neonatal and
pediatric intensive care. Ped Crit Care Med 2000; 1:20-7.
26. Van Keulen JG, Gemke RJBJ and Polderman KH. Effect of training and strict guidelines on the reliability of risk
adjustment systems in pediatric intensive care. Intensive Care Med 2005; 31:1229-1234.
27. Curtis JR, Rubenfeld GD and Hudson LD. Training pulmonary and critical care physicians in outcome
research. Should we take the challenge? Am J Resp Crit Care Med 1998; 157:1012-1015.
28. Marcin JP, Pollack MM, Patel KM et al. Decision support issues using physiology-based score. Intensive Care
Med 1998; 24(12): 1299 -1304.
29. Daly K, Beale R and Chang RWS. Reduction in mortality after inappropriate early discharge from intensive
care unit: logistic regression triage model. BMJ 2001; 322:1274-1276.
30. Lucas P. Bayesian analysis, pattern analysis, and data mining in health care. Curr Opin Crit Care 2004;
10:399-403.
31. Marcin JP, Pollack MM, Patel KM et al. Combining physician`s subjective and physiology-based objective
mortality risk predictions. Crit Care Med 2000; 28:2984-2990.
32. Thukral A, Lodha R, Irshad M and Arora NK. Performance of Pediatric Risk of Mortality (PRISM), Pediatric
Index of Mortality (PIM) and PIM 2 in a pediatric intensive care unit in a developing country. Pediatr Crit
Care Med 2006;7(4): 356-361.
Page 10 of 16
296
European Journal of Applied Sciences (EJAS) Vol. 11, Issue 1, January-2023
Services for Science and Education – United Kingdom
33. Van Keulen JG, Polderman KH, Gemke RJBJ. Reliability of PRISM and PIM scores in paediatric intensive care.
Arch Dis Child 2005; 90:211-214.
34. Higgins TL, McGee WT, Steingrub JS et al. Early indicators of prolonged intensive care unit stay: Impact of
illness and pre-intensive care unit lengh of stay. Crit Care Med 2003; 31:45-51.
35. Marik PE, Varon J. Severity scoring and oucome assessment. Crit Care Clin 1999; 15:633-46.
36. Le Gall JR. The use of severity scores in the intensive care unit. Intensive Care Med 2005; 31(12):1618-1623.
Page 11 of 16
297
Mangia, C. M. F., Toledo, M. D., Rossi, R., Nakano, E. Y., Carneluti, A., Kopelman, B. I., Carvalho, W. B., & Andrade, M. C. (2023). Performance of
Brazilian Pediatric Risk of Severity Model for Illness (Br PRISM) Compared to Pediatric Index of Mortality and Pediatric Risk of Mortality 2. European
Journal of Applied Sciences, Vol - 11(1). 287-302.
URL: http://dx.doi.org/10.14738/aivp.111.13891
Table1– Demographic characteristics of the patients.
N Frequency (%)
Number of patients 1033 100
Gender
Female 469 45.4
Male 564 54.6
Age group
< 12 months 447 43.3
12[―60 months 331 32.0
60[—120 months 150 14.5
120[— 180 months 88 8.5
180|—]228 months 17 1.6
Intra-hospital location before PICU admission
Emergency room 292 28.3
Ward 243 23.5
Intermediate care unit 106 10.26
Operating room 392 37.94
Major Categories of disease
Clinic 641 62.1
Surgical 392 37.9
Main Clinics Grupo Disease
Sepsis 190 18.4
Cardiovascular 102 9.9
Respiratory 221 21.4
LOS*, days (median, Q1 –Q3)†
LOS before PICU‡ 2.00 0 – 8
LOS PICU 5.00 2 – 9
LOS after PICU 9.00 3 –23
LOS Hospital 23.00 11 – 47
Outcome
Hospital Survival 818 79.2
Mortality
PICU mortality 144 13.9
Hospital mortality after PICU 71 6.9
Total mortality 215 20.8
*LOS: length of stay; † Q1, Q3: lower and upper interquartile range, respectively; ‡PICU: Pediatric intensive care
unit
Page 12 of 16
298
European Journal of Applied Sciences (EJAS) Vol. 11, Issue 1, January-2023
Services for Science and Education – United Kingdom
Table 2 – Variables for regression 1, the probability of death in PICU
with their estimated coefficients, standard error (SE), Wald statistic,
adjusted odds ratio, and 95% confidence intervals for the adjusted odds ratio.
B S.E. Wald df Sig. Exp(B) 95,0% C.I. for
EXP(B)
Lower Upper
Medical disease* 1,057 ,269 15,429 1 ,000 2,877 1,698 4,874
Comorbidities† 1,179 ,251 22,141 1 ,000 3,250 1,989 5,311
Hypotension‡ ,569 ,279 4,150 1 ,042 1,766 1,022 3,053
Pupil reaction‡ 2,006 ,500 16,096 1 ,000 7,431 2,789 19,795
Metabolic Coma § ,774 ,359 4,647 1 ,031 2,169 1,073 4,384
Hypoxemia || ,541 ,266 4,120 1 ,042 1,717 1,019 2,894
Mechanical ventilation¶ 1,014 ,286 12,593 1 ,000 2,757 1,575 4,828
Coagulopathy** ,850 ,240 12,489 1 ,000 2,339 1,460 3,747
Hyperglycemia †† ,789 ,245 10,399 1 ,001 2,201 1,363 3,555
Hyponatremia 1,047 ,291 12,913 1 ,000 2,848 1,609 5,040
Vasoactive drugs 1,059 ,240 19,531 1 ,000 2,884 1,803 4,613
Constant -5,603 ,436 165,364 1 ,000 ,004
β: coefficient; SE: standard error; Wald: statistic Wald; Sig: p value; CI 95%: confidence interval of 95%.
*Medical diseases: non-surgical patients; †comorbidities: congenital cardiophaties, oncologic diseases, chronic
kidney failure, chronic liver failure, genetic syndromes, AIDS; ‡systolic blood pressure (SBP): between 0 to 119
months £ 70mmHg and between 120 months to 180 months £ 90 mmHg. ‡ Pupil reaction: both fixed, miosis
bilateral and few reactions, pinpoint and no reactive, one fixed and one reactive.; §Glasgow metabolic: Glasgow
coma score for metabolic disease £ 10; ||Arterial oxygen saturation (Sat O2) < 90%; **APTT: 1.5 fold up to
reference value or > 52.5 sec; ††Glucose 3 150mg/dL; ‡‡Sodium £ 130mEq/L; Use of Mechanical ventilation¶ and
§§Vasoactive drugs.
†ATTP: Activated partial thromboplastin time
Page 13 of 16
299
Mangia, C. M. F., Toledo, M. D., Rossi, R., Nakano, E. Y., Carneluti, A., Kopelman, B. I., Carvalho, W. B., & Andrade, M. C. (2023). Performance of
Brazilian Pediatric Risk of Severity Model for Illness (Br PRISM) Compared to Pediatric Index of Mortality and Pediatric Risk of Mortality 2. European
Journal of Applied Sciences, Vol - 11(1). 287-302.
URL: http://dx.doi.org/10.14738/aivp.111.13891
Table 3–Conditional regression probability probability of death in the hospital
after PICU stay and their estimated coefficients, standard error (SE), Wald statistic,
adjusted odds ratio, and 95% confidence intervals for the adjusted odds ratio [EXP(B)].
B S.E. Wald Sig. Exp(B) 95,0% C.I. for EXP(B)
Lower Upper
Age group < 12 months* 1,203 ,305 15,537 ,000 3,330 1,831 6,056
Comorbidities† ,648 ,298 4,743 ,029 1,912 1,067 3,426
LOS before PICU‡ 1,660 ,324 26,305 ,000 5,258 2,788 9,914
Hyperthermia § -,876 ,350 6,255 ,012 ,417 ,210 ,827
Oliguria|| ,873 ,366 5,689 ,017 2,395 1,168 4,907
Metabolic Coma¶ 1,699 ,413 16,964 ,000 5,469 2,437 12,277
Coagulopathy /APTT ** ,715 ,302 5,610 ,018 2,045 1,131 3,697
Hypercalemia†† 1,701 ,433 15,403 ,000 5,477 2,343 12,804
Constant -4,206 ,370 129,219 ,000 ,015
Table 4 – Distribution of patients according to
severity index stratum and the area under ROC curve
Stratum Classification based in the Severity index and AUC
Diagnosis category Survivors
I* ≤ 0.1564
Low Risk
Hospital death
0.1564< I* <0.3058
Middle Risk
PICU death
I* ≥ 0.3058
High Risk
Total (n)
Survivors 604 146 68 818
Hospital death 34 14 23 71
PICU death 23 24 97 144
Total ( n) 661 184 188 1033
*Index (I) = (21⁄2 +1) Pr(U) - (21⁄2)Pr(H)
Table 5- Classification of severity according to its a posteriori probability.
Index Stratum Classification
Low Risk Middle Risk High Risk
Diagnosis category a priori The patient's probabilities after index (a posteriori)
Survival 85% 94% 85% 46%
Hospital death 5% 3.5% 5.6% 10.5%
PICU death 10% 2.4% 9.4% 43%
Interpretation:
a) If the patient is classified in the low index stratum, the vital status probabilities for each diagnosis category
(Survival; Hospital_death and PICU_death) are improved from 85%, 5% and 10% ( a priori) to 94%; 3.5% and
2.4% (posteriori), respectively;
b) If the patient is classified in the middle index stratum, the vital status of probabilities for each diagnosis
category (Survival; Hospital_death and PICU_death) are improved from 85%, 5% and 10% ( a priori) to 85%;
5.6% and 9.4% (posteriori), respectively;
Page 14 of 16
300
European Journal of Applied Sciences (EJAS) Vol. 11, Issue 1, January-2023
Services for Science and Education – United Kingdom
c) If the patient is classified in the high index stratum, the vital status of probabilities for each diagnosis category
(Survival; Hospital_death and PICU_death) are improved from 85%, 5% and 10% ( a priori) to 46%; 11% and
43% (posteriori), respectively;
Figure 1 - The area under the ROC curve (AUC) for survivors (BrPRISM index £ 0.15)
Figure 2 - The AUC for death in the PICU (BrPRISM index 3 0.30)
Page 15 of 16
301
Mangia, C. M. F., Toledo, M. D., Rossi, R., Nakano, E. Y., Carneluti, A., Kopelman, B. I., Carvalho, W. B., & Andrade, M. C. (2023). Performance of
Brazilian Pediatric Risk of Severity Model for Illness (Br PRISM) Compared to Pediatric Index of Mortality and Pediatric Risk of Mortality 2. European
Journal of Applied Sciences, Vol - 11(1). 287-302.
URL: http://dx.doi.org/10.14738/aivp.111.13891
Figure 3 - Comparison between ROC curves Br PRISM, PIM and PIM 2
APPENDIX 1
The values considered worst value in the model developing.
Variable Values
Age Group < 12 months
Medical No surgical patients
Comorbidity congenital cardiopathy, cancer, chronic kidney failure, chronic
liver failure, genetic syndrome, acquired immunodeficiency.
Pupil reactions Worst situation: both fixed, miosis bilateral and few reactions,
pinpoint and no reactive, one fixed and one reactive.
Glasgow coma scale Glasgow < 10 in the metabolic coma
Mechanical ventilation In the first 24 hr.
Vasoactive drugs In the first 24 hr.
Systolic blood pressure (mmHg)
< 12 months
12 to 59 months
60 to 119 months
120 to 179 months
> 180 months
£ 70 mmHg
£ 70 mmHg
£ 70 mmHg
£ 90 mmHg
£ 90 mmHg
Temperature (To. C) > 38 o
C
Diuresis (ml/kg/hr) < 1.0 ml/kg/hr.
Saturation O2 < 90%
Coagulation (sec) APTT (sec) 1.5 times up to reference value or > 52.5 sec.
Sodium (meq/l) £ 130
Potasium (meq/l) £3.0
Glucose (mg/dL) 3 150
LOS before PICU 3 8 days
The cutoff points were based in the literature review and univariate analysis of data
Page 16 of 16
302
European Journal of Applied Sciences (EJAS) Vol. 11, Issue 1, January-2023
Services for Science and Education – United Kingdom
APPENDIX 2
Sample Calculation of Score
Consider a child who is admitted to intensive care with the following data: age 16 months ( age
group < 12 months= no=0), severe sepsis (medical = yes=1), hasn ́t comorbities (no=0) ,
receiving vasoactive drugs (yes = 1), is ventilated immediately after admission (mechanical
ventilation = yes = 1), has a systolic blood pressure (SBP) of 63 mmHg ( SBP < 70mmHg
considering age group = yes=1 temperature of 38o.C ( Temp >38 o.C = yes=1), diuresis of 0.8
ml/kg/hr (diuresis £ 1.0ml/kg/hr = yes=1), ATTP of 105 sec. ( ATTP 3 52.5 sec = yes=1),
potassium of 2.0 mEq/L ( K < 3 mEq/L = yes=1), sodium of 129 mEq/L ( Na £ 130 mEq/L = yes
= 1), SatO2 of 89% ( Saturation arterial O2 < 90% = yes=1),normal pupils (no = 0) and glasgow
coma score > 10 ( no=0), waited in the emergency room for a bed in the ICU for 6 days (LOS
before PICU > 8 days = no= 0).
Using the coefficients in Table 2 and 3 we have final model derived from the first regression
and conditional regression, the Br PRISM logit for:
First Regression is
= (1.057*1) + (1.179*0) + (0.569*1) + (2.006*0) + (0.774*0) + (0.541*1) + (1.014*1) +
(0.850*1) +(0.789*1) + (1.047*1) + (1.059*1+ (- 5.603) = 1.323
Conditional Regression is
= (1.203*0) + (0.648*1) + (1.660*0) + (0.876*1) + (0.873*1) + (1.699*0) + (0.713*1) +
(1.701*1) + (-4.206) = -1.793
The logit should be converted to the predicted probability of death.
The predicted probability of death first regression)
= elogit/(1+elogit) =e 1.323/ (1+e 1.323) =0.78968
The predicted probability of death in the PICU (conditional regression)
= elogit/(1+elogit) = e -1.793/ (1+e -1.793) =0.142705
After this step, the probabilities should be used for calculating de index of severity-of- illness
= (21⁄2 +1) Pr(U) - (21⁄2) Pr(H)= [(21/2 +1) * 0.78968]-[(21⁄2) *0.142705= 0.65 (high risk index 3
0.3058 (table 4).
In the table 5, the high-risk patient is in the column 3 the a priori probability of death in PICU
was 10% and after adjustment the probability of death in PICU was estimated in 43% (a
posteriori probability). A priori probability for hospital death after PICU stay was 5% after
adjustment the probability is 10.5%. A priori probability for survival after hospital stay was
85% and a posteriori 46%