Visualization Techniques in Healthcare Applications: A Narrative Review

Affiliations.

  • 1 Health Informatics, King Saud Bin Abdulaziz University for Health Sciences, Riyadh, SAU.
  • 2 Researcher, King Abdullah International Medical Research Center, Riyadh, SAU.
  • 3 Researcher, King Saud Bin Abdulaziz University for Health Sciences, Riyadh, SAU.
  • 4 Health Sciences, King Abdullah International Medical Research Center, Riyadh, SAU.
  • PMID: 36514654
  • PMCID: PMC9741729
  • DOI: 10.7759/cureus.31355

Nowadays, healthcare management systems are adopting various techniques that facilitate the achievement of the goals of evidence-based medical practice. This review explores different visualization techniques and their importance in healthcare contexts. We performed a thorough search on databases such as the SLD portal, PubMed, and Google Scholar to obtain relevant studies. We selected recent articles published between 2018 and 2021 on visualization techniques in healthcare. The field of healthcare generates massive volumes of data that require visualization techniques to make them easily comprehensible and to guide their efficient presentation. Visualization in healthcare involves the effective presentation of information through graphics, images, and videos. Big data systems handle a massive amount of information and require visualization techniques to present it in a comprehensible manner. The significance of visualization techniques in healthcare is not confined to healthcare practitioners and healthcare management but encompasses all the stakeholders; patients can benefit from the visualization of his/her data for a better understanding of their condition. In short, visualization techniques have demonstrated their benefits in the healthcare sector and can be extended to the payer and the patient. They have also had a positive impact on the quality of the healthcare provided as well as patient safety.

Keywords: big data; graphical presentation; interactive visualization; visualization; visualization techniques.

Copyright © 2022, Abudiyab et al.

Publication types

Graph-Representation of Patient Data: a Systematic Literature Review

  • Systems-Level Quality Improvement
  • Open access
  • Published: 12 March 2020
  • Volume 44 , article number  86 , ( 2020 )

Cite this article

You have full access to this open access article

  • Jens Schrodt   ORCID: orcid.org/0000-0002-9768-4781 1 ,
  • Aleksei Dudchenko   ORCID: orcid.org/0000-0002-4717-2307 1 , 2 ,
  • Petra Knaup-Gregori 1 &
  • Matthias Ganzinger   ORCID: orcid.org/0000-0002-2716-5425 1  

11k Accesses

14 Citations

8 Altmetric

Explore all metrics

Graph theory is a well-established theory with many methods used in mathematics to study graph structures. In the field of medicine, electronic health records (EHR) are commonly used to store and analyze patient data. Consequently, it seems straight-forward to perform research on modeling EHR data as graphs. This systematic literature review aims to investigate the frontiers of the current research in the field of graphs representing and processing patient data. We want to show, which areas of research in this context need further investigation. The databases MEDLINE, Web of Science, IEEE Xplore and ACM digital library were queried by using the search terms health record , graph and related terms. Based on the “Preferred Reporting Items for Systematic Reviews and Meta-Analysis” (PRISMA) statement guidelines the articles were screened and evaluated using full-text analysis. Eleven out of 383 articles found in systematic literature review were finally included for analysis in this literature review . Most of them use graphs to represent temporal relations, often representing the connection among laboratory data points. Only two papers report that the graph data were further processed by comparing the patient graphs using similarity measurements. Graphs representing individual patients are hardly used in research context, only eleven papers considered such kind of graphs in their investigations. The potential of graph theoretical algorithms, which are already well established, could help increasing this research field, but currently there are too few papers to estimate how this area of research will develop. Altogether, the use of such patient graphs could be a promising technique to develop decision support systems for diagnosis, medication or therapy of patients using similarity measurements or different kinds of analysis.

Similar content being viewed by others

use graphical presentation in health sciences data

The role of artificial intelligence in healthcare: a structured literature review

Silvana Secinaro, Davide Calandra, … Paolo Biancone

use graphical presentation in health sciences data

Artificial Intelligence to Automate Network Meta-Analyses: Four Case Studies to Evaluate the Potential Application of Large Language Models

Tim Reason, Emma Benbow, … Bill Malcolm

use graphical presentation in health sciences data

Big Data Analytics in Healthcare

Avoid common mistakes on your manuscript.

Introduction

Today, electronic health records (EHR) are the predominant way of documenting health care activities. While the establishment of EHR started decades ago, there are still lively research activities associated with it. Querying PubMed for the term “electronic health record” shows an increasing number of publications year by year until now (cf. Figure 1 ).

figure 1

Development of papers per year using the keyword „electronic health record” until 2019

From an information technology perspective, the way how EHR data are persisted has also experienced some development over the years. One of the first approaches that was broadly applied was storing EHR data in relational databases [ 1 ]. Today, most hospital information systems are probably based on relational database management software and are accessed with the corresponding Structured Query Language (SQL) [ 2 , 3 ].

However, the development towards NoSQL database management systems also influenced EHR systems [ 4 , 5 ]. For instance, document oriented database management systems such as CouchDB [ 6 ] or MongoDB [ 7 , 8 ], no longer require the organization of data in tables. Instead, data are stored as documents written in data formats such as the JavaScript Object Notation (JSON) [ 9 ] or Extensible Markup Language (XML) [ 10 ]. Another type of NoSQL databases are graph databases [ 11 , 12 ]. In contrast to document databases, data are stored as properties in structures consisting of nodes and edges. Graph databases have many advantages in comparison to relational databases, for instance, graph databases are much easier to scale, are faster especially at highly connected data, and have a higher level of availability than common relational databases [ 13 ].

Graph databases are already in use in various social networks like Facebook and in other Internet companies like Amazon or Google [ 14 ]. In social networks, graph databases are very useful because of their property to store the relationships between different members of the social network directly and intuitively (for the user). This direct storage decreases computation time and makes it possible to create queries, which can access these relationships directly. To use these stored relationships of graph databases is also much easier than using the relational data model and its SQL statements for the same purpose. In the relational data model, complex join statements would be necessary to get the same effect and this complicates the creation of queries and increases their computation time compared to graph database queries [ 14 ]. And there are also many more fields of applications for graph databases apart from social networks e.g. biomolecular pathways [ 15 ], for integration of heterogeneous biological data [ 16 ] and for representing disease networks [ 17 ].

Graphs in context of graph theory are very clearly defined as a set of nodes connected by edges, which represent a relation between the connected nodes [ 18 ], this definition is used for the expression graph in the following chapters. Figure 2 shows the schematic representation of a graph. The dots are the vertices or nodes, the connections between the nodes represent the edges. Graph theory is a well-established area of mathematics that also covers methods to compare graphs. Apart from graph databases, these methods make the usage of graphs in medical context very interesting, for example to model patient data of EHR systems. With approaches like this, diagnoses, therapies and medications could be suggested on the basis of previous patients and the experiences made at treatment of these patients. Such a system could also be part of a decision support system for physicians in clinical context. For example, graphs are used for spatial description of cerebral anatomy [ 19 ] or for clustering of patients and for making a diagnosis [ 20 ]. Other approaches are closer related to EHR. Such projects focus for example on visualizing collaborative electronic health record usage with heart failure [ 11 ], modeling disease graphs [ 21 ] or to predicting knowledge graphs of unknown adverse drug reactions [ 22 ].

figure 2

Schematic representation of a directed graph. The dots are called nodes, the connections between the nodes are called edges. The edges are directed, this is shown by the arrow, which points the edge in a direction.

The aim of this literature review is to investigate the frontiers of the current research in the field of graphs representing and processing patient data. We want to show, which areas of research require further investigation. Before planning research projects in this area we would like to get an overview of already established applications of graphs for individual patients on which, for example, similarity comparisons were performed or the temporal relationships in patient data were used for research projects. To compare patients, it seems to be necessary that patient data are represented by individual graphs or at least sub-graphs. The resulting main questions for the review were:

Which kinds of graph-based representations or graph database models of patient data are appropriate established for individual patients?

How is the patient data technologically processed after using the graph theoretical framework (e.g. by using graph databases or temporal modeling)?

Our systematic literature review is based on the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) approach [ 23 ].

Search strategy

We used the keywords health records and graph with the synonyms medical record, patient record and all plural forms of the keywords for our database search in the databases MEDLINE, Web of Science, IEEE Xplore and ACM digital library. The fields, which were investigated with the search terms, were title and abstract. The keywords were included in the query syntax of each investigated database. The specific database queries are shown in Suppl. Fig.  1 .

Inclusion criteria

The investigated papers were screened against the following inclusion criteria. The first main criterion was the usage of the term graph in the sense of graph theory. This means that the graphs should contain nodes and edges, which is one main definition criterion for graphs of graph theory. Many papers used this word in another context, e.g. as some graphs used the term graph as a synonym for illustration and were therefore excluded. Articles were also excluded if they do not use graphs representing individual patients but for example for a set of patients. Further, only articles written in English or German were included. The database search was done at 20.03.2018 and therefore only articles published and indexed until this date were considered in the review.

Selection of articles

The articles retrieved from the database queries were screened by four reviewers according to the inclusion criteria based on their title and abstract. If there is no abstract, the full text of the article was used. Initially, all four reviewers tested the inclusion criteria on the same sample of ten articles independently. The results of this test review were discussed afterwards to reach a consensus understanding of the inclusion criteria.

To reduce the workload for the reviewers, the total number of articles was split into two halves, which were assigned to two teams of two reviewers. The members of each team assessed the articles assigned with their partners’ results blinded to ensure that each article received two independent votes. Reviewers marked each article as “included” or “excluded”. For excluded articles, a reason for the exclusion was documented. Articles, which were included by both reviewers, were selected for full-text investigation. Those articles, that were included by one reviewer and excluded by the other reviewer, were assessed by a third reviewer to reach a final decision. The third reviewer decided about inclusion or exclusion of the respective article.

Data extraction and synthesis

The articles included in the steps before were analyzed in full-text. Some articles still had to be excluded in this phase, because the fulfillment of inclusion criteria, which was recognized in the screening phase, could not be seconded by full-text analysis. To support full-text analysis, the computer-assisted qualitative data analysis software (CAQDAS) MAXQDA was used [ 24 , 25 ]. In MAXQDA we established a coding system, which was initially created using one article as basis. In a coding system, all central keywords of all investigated and included articles were collected as a hierarchical structure. Each keyword can be assigned to multiple articles and each article can be assigned to multiple keywords. The coding system was iteratively developed by investigating the further articles. Therefore, the papers were loaded in MAXQDA as PDF files to tag the information expressed by the codes in the coding system. Afterwards, cross article occurrences of the different codings were analyzed and main statements were extracted: the kinds of graphs used in the papers, the kinds of data sources, the node and edge contents as well as the processing methods used in the papers.

Overall results

By database search, we found 201 hits in MEDLINE, 107 hits in Web of Science, 58 hits in IEEE Xplore and 92 hits in ACM digital library. After eliminating duplicates, the total number of articles was 383. After assessing the inclusion criteria, the reviewers agreed on including 320 of 383 abstracts (84%). For 63 abstracts, they had contradicting opinions, which made the decision of a third reviewer necessary. Finally, 42 abstracts were included by the agreement of the first two reviewers; six further abstracts were added by the third reviewer checking the conflicting articles in abstract (and full-text if necessary). So, in total 48 articles (12.5%) were included, 335 (87.5%) were excluded. The main reasons for exclusion were

Papers did not use the term graph in a graph theoretical manner

Papers used graph theory, but the graphs did not represent individual patients (Supplementary Table 3 ).

After the step of full-text analysis of the 48 articles eleven of these articles finally remained (2.9%) [ 19 , 20 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ] for analysis in MAXQDA by using a coding scheme.

Coding scheme

The coding scheme used in MAXQDA is shown in Suppl. Fig.  3 . The main categories after analyzing the eleven remaining articles were a) data source b) overall purpose / function c) graph properties d) investigated disease e) technical processing of graph. At first, the methods of representing and saving data in graphs and graph databases were investigated and this investigation can be aligned to the codings data source and graph properties . Theses codings define how the graphs are generated and what they contain as nodes and edges. The second main question that we investigated was the processing methods of these graphs and their contents. Thus, we investigated the processing methods of these graphs and their contents by using the codings overall purpose / function and especially technical processing of graph. This second step helps us to understand how currently existing investigations handle the processing of patient graphs and what kinds of goals these investigations would like to reach.

Graph properties

In Table 1 , all different kinds of node contents from the eleven articles included are shown. Six of the papers used laboratory data represented in nodes, five of the papers used medications and diagnosis. Functional nodes were used four times, whereas anatomic nodes and patient problems were used two times each. Procedures, vital signs and patient nodes (a node to identify the patient used in this graph) were used only one time in the papers.

In contrast to Table 1 , Table 2 shows the content of the edges used in the eleven papers. In two papers, the edges represent causal relations, so the nodes are connected in a causal context. In one paper, the edges represent anatomic-functional relations, whereas in two papers spatial relations were represented by the edges. In detail, the edges show the spatial relations of brain areas. Taxonomical relations and status and date are two more edge contents used, each in one paper. The edge content most often used by the included articles are temporal relations , which were used in six different papers.

Graph types

Table 3 shows all types of graphs used in the articles to represent electronic health records of a patient. Most of the remaining articles use a representation of electronic health records in a graph representing an individual patient in a temporal manner (temporal event data mining) [ 20 , 27 , 28 , 30 , 32 , 33 ]. In contrast to that, causal networking represents the causal context of patient data and was used in two papers for representing patient data [ 29 , 30 ]. Heterogeneous data mining was used by one paper and describes the representation of very different kinds of data of the patient in one graph [ 30 ] whereas database / data structural approaches were used in two papers. These papers demonstrate possible methods of saving patient data in a graph database or in a graph like structure [ 26 , 34 ]. Two further papers use the graphs for structural representation of tissue areas in the brain [ 19 , 31 ].

Data sources

We also investigated the different data sources for patient data used in the included articles as shown in Table 4 . Electronic health records are the biggest part of data sources used in the articles (62.5%). Some articles also use image-based information (12.5%) or data from a healthcare information system (12.5%). One article uses SNOMED CT clinical findings and another one uses a research database (each 6.25%).

Processing of Graphs

Table 5 shows the number of papers, which used the shown kinds of processing methods for patient graphs. Only one paper uses the model for prognosis issues [ 29 ]. Five papers investigate the storage of patient graphs in some kind, e.g. in graph databases. In only two papers, the authors are interested in similarity comparisons of the created patient graphs, whereas in nine papers the plain presentation of patient data in a graph plays a central role.

Goals and Content of the Articles

The research goals described in the different articles differ very much in detail, but an application often mentioned in the articles was personalized medicine, which was named by 6 of the 11 investigated papers. The other goals were quality improvement, information gaining, predictive modeling, disease diagnosis, patient segmentation (each used in 2 papers), population management, data mining, data warehouse and disease pattern (each used in one paper).

To reach these goals the papers follow very different strategies. Atif et al. used image-based information of brains to create a graph-based cerebral description of brain anatomy. This spatial graph is created manually for every patient and afterwards patients could be compared using these graphs [ 19 ]. In contrast, Campbell et al. used the SNOMED CT concept model in a graph database architecture because of the ontology character and polyhierarchy of SNOMED CT, which makes it difficult to implement electronic health records in relational databases [ 26 ]. The created graphs save SNOMED CT data in a specifically created graph format. Risk prediction is the main goal of Chen et al., so the authors developed a graph-based, semi-supervised learning algorithm to reach this goal [ 27 ]. By modelling the clinical evolution of an individual patient with kidney failure Esteban et al. wanted to develop the basis for a future clinical decision support system. This graph-based model contains thousands of events like laboratory results, ordered tests and diagnoses [ 28 ] and represents a patient in a graph. Hanzlicek et al. described MUDR EHR, a multimedia distributed health record for decision support. This electronic health record contains multiple medical concepts, which should help describing a patient in a structured way, apart from free text records [ 34 ]. Kaur et al. described a model, which combines different data stores of patient data. In this architecture the user creates his request at the interface and the architecture below translates this request into a query to get the data from the most suitable data store for this request [ 29 ]. The resulting graph of this paper helps to get the right information from the data stores. Liu et al. used longitudinal patient data to create so called temporal graphs. These graphs were clustered in different phenotypes, so that using these phenotypes helps improving diagnosis performance [ 20 ]. The resulting graphs represent a patient and his medical events in temporal context. The focus of Müller et al. was the lack of clinical context in other approaches. The authors solve this problem by creating a graph-grammar approach to design and implement a graph-oriented patient model, which allows the representation of the clinical context [ 30 ]. Puentes et al. also used (similar to Atif et al.) graphs to gain information out of image-based brain information to model spatial relationships of brain anatomical singularities of individual patients. This approach is especially used for spatial modelling of cerebral tumors [ 31 ]. Zhang et al. [ 32 ] created a convolutional neural network on heterogeneous attributes of a patient (e.g. diagnoses, procedures and medications) using a graph, which gains its data from electronic health records [ 32 ]. Zhang et al. [ 33 ] created a unified graph representation of the electronic health records of an individual patient in a temporal manner. Using this graph, in the second step a modified algorithm was used to create a temporal profile of each patient. This approach was used for risk prediction [ 33 ].

General Findings

Our literature review shows that there are many articles published in context of medical records and graphs, but only a small number of authors investigated graphs in the sense of graph theory or used graphs to represent individual patient data. The initial database query produced a sample size of almost 400 papers. Many of these papers had to be excluded because they used the term graph in a different context than for graph theory. But if the graph mentioned by the paper could be related to graph theory, the second most exclusion criterion often arises: Most of the papers, which use graphs, do not use these graphs to represent data of individual patients. Surprisingly, the exclusion resulted in only eleven papers remaining. From a theoretical point of view, the modeling of patient data as a graph could result in some advantages in analysis of this patient data because of the well-established tools and methods contents of graph theory. The usage of such already established algorithms could facilitate the development of new methods enormously. Research question 1 of this literature review corresponds to possible patient representations in graphs that were used in literature before. Table 1 and Table 2 show the different possibilities of using nodes and edges to represent patient data used in the investigated articles. Especially laboratory data as nodes and temporal relationships between nodes were most commonly used in the investigated papers. The graph type used most often was temporal event data mining, followed by causal networking, structure representation, database structural approaches and heterogeneous data mining. The focus on temporal relationships shows that most data in this field of research was investigated in context of temporal relationships but the low number of papers included in this literature review also shows that there is much more potential for further analysis. With Question 2 we wanted to get an overview of all different processing mechanisms used to investigate the patient graphs. The low number of papers found through the literature review made it very difficult to identify real tendencies, but the main results are shown in Table 5 – only two of the investigated papers do really process the patient data after setting it to a graph. In contrast to that, nine papers use the graphs only to represent the patient data and five papers also use the graphs to store the data in a specific form. This very low number of papers using graphs for representing individual patients and the even lower number of papers processing theses graphs raise different questions for further investigation: Is it reasonable to represent patient data in graphs and process them or is there any reason why this has not been done very often so far? What is the best the way to proof plausibility of such systems?

Limitations

In this study only the four databases MEDLINE, Web of Science, IEEE Xplore and ACM digital library were used, thus there might be some papers indexed in other databases, which were not found by our review. Apart from the databases, there also might be papers connected to this topic, which would have been captured by using another query. Also publication bias plays a role in literature reviews, especially in this context. The low number of papers using graph theory for representing patient data could also be caused by a high rate of unsuccessful papers in this field of research.

Our review shows the current state of research in context of patient graphs. The concentration of many of the eleven papers on the recent past might indicate, that this is a rather young research area, which could expand in the next few years, but currently the total number of papers connected to the research field is too low to make a clear statement. Altogether representing a patient in a graph is a very promising technique, which is already used in very different medical areas as shown by the content of the included papers. These different areas (brain tumors, kidney failures, patients in general and so on) show that there is much potential for further studies in this field of research. The possibilities with such systems are very broad and open new opportunities, for example in clinical context. We could imagine a system that helps analyzing patient graphs for finding differential diagnosis, the right medication, or even to get therapy proposals based on experiences made in previous patient cases.

Friedman, C., Hripcsak, G., Johnson, S. B. et al. A generalized relational Schema for an integrated clinical patient database. Proc Annu Symp Comput Appl med care: 335–339, 1990

Codd, E. F., A relational model of data for large shared data banks. Commun. ACM 13(6):377–387, 1970. https://doi.org/10.1145/362384.362685 .

Article   Google Scholar  

ISO/IEC Information technology—Database languages—SQL—Part 1: Framework (SQL/Framework)(ISO/IEC 9075–1:2016), 2016

Klein, J., Gorton, I., Ernst, N. et al., Performance evaluation of NoSQL databases. In: Singhal, R., Chahal, D. (Eds), Proceedings of the 1 st workshop on performance analysis of big data systems - PABS ‘15. New York: ACM Press, 2015, 5–10.

Google Scholar  

Ercan, M. Z., Lane, M., Evaluation of NoSQL databases for EHR systems. In: 25 th Australasian conference on information systems, 2014

Rascovsky, S. J., Delgado, J. A., Sanz, A. et al., Informatics in radiology: Use of CouchDB for document-based storage of DICOM objects. Radiographics 32(3):913–927, 2012. https://doi.org/10.1148/rg.323115049 .

Article   PubMed   Google Scholar  

Luo, L., Li, L., Hu, J., Wang, X., Hou, B., Zhang, T., and Zhao, L. P., A hybrid solution for extracting structured medical information from unstructured data in medical records via a double-reading/entry system. BMC Med Inform Decis Mak 16(1):114–114, 2016. https://doi.org/10.1186/s12911-016-0357-5 .

Article   PubMed   PubMed Central   Google Scholar  

Xu, W., Zhou, Z., Zhou, H., et al. MongoDB improves big data analysis performance on electric health record system. In: Ma S, Jia L, Li X et al. (eds) Life system modeling and simulation, vol 461. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 350–357, 2014

Bray, T., The javascript object notation (json) data interchange format, 2017

Bray, T., Paoli, J., Sperberg-McQueen, C. M., et al. Extensible markup language (XML) 1.0, 2008

Soulakis, N. D., Carson, M. B., Lee, Y. J. et al., Visualizing collaborative electronic health record usage for hospitalized patients with heart failure. J Am Med Inform Assoc 22(2):299–311, 2015. https://doi.org/10.1093/jamia/ocu017 .

Zhou, X., Han, H., Chankai, I., et al. Approaches to text mining for clinical medical records. In: Haddad HM (ed) the 2006 ACM symposium, p 235, 2006

Nayak, A., Poriya, A., and Poojary, D., Type of NOSQL databases and its comparison with relational databases. International Journal of Applied Information Systems (IJAIS) 5(4):16–19, 2013.

Moniruzzaman, A. B. M., Hossain, S. A., NoSQL database: New era of databases for big data analytics -classification, characteristics and comparison. International Journal of Database Theory and Application 6(4), 2013

Fabregat, A., Korninger, F., Viteri, G., Sidiropoulos, K., Marin-Garcia, P., Ping, P., Wu, G., Stein, L., D'Eustachio, P., and Hermjakob, H., Reactome graph database: Efficient access to complex pathway data. PLoS Comput Biol 14(1):e1005968, 2018. https://doi.org/10.1371/journal.pcbi.1005968 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Yoon, B.-H., Kim, S.-K., and Kim, S.-Y., Use of graph database for the integration of heterogeneous biological data. Genomics Inform 15(1):19–27, 2017. https://doi.org/10.5808/GI.2017.15.1.19 .

Lysenko, A., Roznovăţ, I. A., Saqi, M., Mazein, A., Rawlings, C. J., and Auffray, C., Representing and querying disease networks using graph databases. BioData Min 9(1):23–19, 2016. https://doi.org/10.1186/s13040-016-0102-8 .

Bollobás, B., Modern graph theory, 5. Print. Graduate texts in mathematics, vol 184. Springer, New York, NY, 2010

Atif, J., Hudelot, C., Nempont, O., et al. GRAFIP: A FRAMEWORK FOR THE REPRESENTATION OF HEALTHY AND PATHOLOGICAL CEREBRAL INFORMATION. In: 2007 4 th IEEE international symposium on biomedical imaging: From Nano to macro, pp 205–208, 2007

Liu, C., Wang, F., Hu, J., et al. Temporal Phenotyping from longitudinal electronic health records. In: Cao L, Zhang C, Joachims T et al. (eds) the 21th ACM SIGKDD international conference, pp 705–714, 2015

Yousefi, A., Mastouri, N., Sartipi, K., Scenario-oriented information extraction from electronic health records. In: 22 nd IEEE International Symposium on Computer-Based Medical Systems, 2009: CBMS 2009 ; date: 2–5 [i.e. 3–4] August 2009, Albuquerque, New Mexico, USA. IEEE, Piscataway, NJ, pp 1–5, 2009

Bean, D. M., Wu, H., Iqbal, E. et al., Knowledge graph prediction of unknown adverse drug reactions and validation in electronic health records. Sci Rep 7(1):1–11. https://doi.org/10.1038/s41598-017-16674-x .

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and PRISMA Group, Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. BMJ 339:b2535, 2009. https://doi.org/10.1136/bmj.b2535 .

VERBI Software GmbH (2018) MAXQDA: Software for qualitative and mixed methods research. VERBI Software GmbH

Kuckartz U (2014) Qualitative text analysis: A guide to methods, practice & using software. Sage, Los Angeles, London, New Delhi, Singapore, Washington, DC

Campbell, W. S., Pedersen, J., McClay, J. C. et al., An alternative database approach for management of SNOMED CT and improved patient data queries. J Biomed Inform 57:350–357, 2015. https://doi.org/10.1016/j.jbi.2015.08.016 .

Chen, L., Li, X., Sheng, Q. Z. et al., Mining health examination records—A graph-based approach. IEEE Trans. Knowl. Data Eng. 28(9):2423–2437, 2016. https://doi.org/10.1109/TKDE.2016.2561278 .

Esteban, C., Schmidt, D., Krompass, D., et al. Predicting sequences of clinical events by using a personalized temporal latent embedding model. In: 2015 international conference on healthcare informatics (ICHI), pp 130–139, 2015

Kaur, K., and Rani, R., Managing data in healthcare information systems: Many models, one solution. Computer 48(3):52–59, 2015. https://doi.org/10.1109/MC.2015.77 .

Müller, R., Thews, O., Rohrbach, C., Sergl, M., and Pommerening, K., A graph-grammar approach to represent causal, temporal and other contexts in an oncological patient record. Methods Inf Med 35(2):127–141, 1996.

Puentes, J., Batrancourt, B., Atif, J., Angelini, E., Lecornu, L., Zemirline, A., Bloch, I., Coatrieux, G., and Roux, C., Integrated multimedia electronic patient record and graph-based image information for cerebral tumors. Comput Biol Med 38(4):425–437, 2008. https://doi.org/10.1016/j.compbiomed.2008.01.009 .

Zhang, J., Gong, J., Barnes, L., HCNN: Heterogeneous convolutional neural networks for comorbid risk prediction with electronic health records. In: 2017 IEEE/ACM international conference on connected health: Applications, systems and engineering technologies (CHASE), pp 214–221, 2017

Zhang, S., Liu, L., Li, H., et al. MTPGraph: A data-driven approach to predict medical risk based on temporal profile graph. In: 2016 IEEE Trustcom/BigDataSE/I​SPA, pp 1174–1181, 2016

Hanzlicek, P., Spidlen, J., and Nagy, M., Universal electronic health record MUDR. Stud Health Technol Inform 105:190–201, 2004.

PubMed   Google Scholar  

Download references

Acknowledgements

The authors would like to thank Liwen Zhu for helping to screen the articles of this literature review.

Open Access funding provided by Projekt DEAL. This study was funded within the systems medicine project “Clinically-applicable, omics-based assessment of survival, side effects, and targets in multiple myeloma” (CLIOMMICS) by the German Federal Ministry of Education and Research (BMBF, grant id: 01ZX1609A) as part of the e:Med initiative.

Author information

Authors and affiliations.

Institute for Medical Biometry and Informatics, Heidelberg University, Im Neuenheimer Feld 130.3, 69120, Heidelberg, Germany

Jens Schrodt, Aleksei Dudchenko, Petra Knaup-Gregori & Matthias Ganzinger

School of Translational Information Technologies, ITMO University, Kronverksky Pr. 49, 197101, Saint-Petersburg, Russia

Aleksei Dudchenko

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Matthias Ganzinger .

Ethics declarations

Conflict of interest.

All authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Systems-Level Quality Improvement

Electronic supplementary material

(PDF 82 kb)

(PDF 150 kb)

(PDF 89 kb)

(PDF 90.4 kb)

(PDF 1.19 mb)

(PDF 720 kb)

(PDF 1.03 mb)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Schrodt, J., Dudchenko, A., Knaup-Gregori, P. et al. Graph-Representation of Patient Data: a Systematic Literature Review. J Med Syst 44 , 86 (2020). https://doi.org/10.1007/s10916-020-1538-4

Download citation

Received : 03 April 2019

Accepted : 07 February 2020

Published : 12 March 2020

DOI : https://doi.org/10.1007/s10916-020-1538-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Graph theory
  • Systematic literature review
  • Electronic health record
  • Temporal patient graph

Advertisement

  • Find a journal
  • Publish with us
  • Track your research
  • Correspondence
  • Open access
  • Published: 11 April 2008

Graphical presentation of diagnostic information

  • Penny F Whiting 1 ,
  • Jonathan AC Sterne 1 ,
  • Marie E Westwood 2 ,
  • Lucas M Bachmann 3 ,
  • Roger Harbord 1 ,
  • Matthias Egger 4 &
  • Jonathan J Deeks 5  

BMC Medical Research Methodology volume  8 , Article number:  20 ( 2008 ) Cite this article

21k Accesses

101 Citations

6 Altmetric

Metrics details

Graphical displays of results allow researchers to summarise and communicate the key findings of their study. Diagnostic information should be presented in an easily interpretable way, which conveys both test characteristics (diagnostic accuracy) and the potential for use in clinical practice (predictive value).

We discuss the types of graphical display commonly encountered in primary diagnostic accuracy studies and systematic reviews of such studies, and systematically review the use of graphical displays in recent diagnostic primary studies and systematic reviews.

We identified 57 primary studies and 49 systematic reviews. Fifty-six percent of primary studies and 53% of systematic reviews used graphical displays to present results. Dot-plot or box-and- whisker plots were the most commonly used graph in primary studies and were included in 22 (39%) studies. ROC plots were the most common type of plot included in systematic reviews and were included in 22 (45%) reviews. One primary study and five systematic reviews included a probability-modifying plot.

Graphical displays are currently underused in primary diagnostic accuracy studies and systematic reviews of such studies. Diagnostic accuracy studies need to include multiple types of graphic in order to provide both a detailed overview of the results (diagnostic accuracy) and to communicate information that can be used to inform clinical practice (predictive value). Work is required to improve graphical displays, to better communicate the utility of a test in clinical practice and the implications of test results for individual patients.

Peer Review reports

Readers of a research report evaluating a diagnostic test may wish to assess the test's characteristics (diagnostic accuracy) or evaluate the impact that its use has on diagnostic decisions (predictive value) for individual patients. Graphical displays of results of test accuracy studies allow researchers to summarise and communicate the key findings of their study. We discuss the types of graphical display commonly encountered in primary diagnostic accuracy studies and systematic reviews of such studies, and systematically review the use of graphical displays in recent diagnostic systematic reviews and primary studies. Table 1 defines the various measures of diagnostic accuracy used.

Types of graphical display

Primary studies.

Figure 1 illustrates four types of graphical display commonly used to present data on diagnostic accuracy for primary diagnostic accuracy studies. We used data from a study of the biochemical tumour marker CA-19-9 antigen to diagnose pancreatic cancer to construct these graphs [ 1 ].

figure 1

Example graphical displays for primary study data . a. Dot plot. b. Box-and-whisker plot. c. ROC Plot. d. Flow diagram.

Dot plots (Figure 1a ) and Box-and-whisker plots (Figure 1b )

Dot plots are used for test results that take many values, and display the distribution of results in patients with and without the target condition. Box and whisker plots summarise these distributions: the central box covers the interquartile range with the median indicated by the line within the box. The whiskers extend either to the minimum and maximum values or to the most extreme values within 1.5 interquartile ranges of the quartiles, in which case more extreme values are plotted individually [ 2 ]. Sometimes an indication of the threshold used to define a positive test result is included, for example by adding a horizontal line or shading at the relevant point. Such plots can be used to clearly summarise a large volume of data, but are only able to display differences in the distribution of test values between patients with and without the target condition; they do not directly display the diagnostic performance of the test.

Although the CA-19-9 antigen test to diagnose pancreatic cancer (used to construct Figure 1 ) is an example of continuous data, it is also possible to construct similar graphs for categorical test results providing that the number of categories is reasonably large. Alternatively, for smaller numbers of categories, similar information can be conveyed using paired bar charts/histograms. Paired histograms show the distribution of test results in patients with the target condition above the x-axis and the distribution in patients without the target condition below the x-axis. These types of graphical display are less commonly used. It is not possible to construct any of these graphs for truly dichotomous test results. However, truly dichotomous tests rarely occur in practice. Examples of dichotomous tests include dipstick tests that change colour if the target condition is said to be present (although these are based on an underlying implicit threshold) or the presence/absence of certain clinical symptoms.

Receiver operating characteristic (ROC) plot (Figure 1c )

ROC plots show values of sensitivity and specificity at all of the possible thresholds that could be used to define a positive test result [ 3 ]. Typically, sensitivity (true positive rate) is plotted against 1-specificity (false positive rate): each point represents a different threshold in the same group of patients. Stepped lines are used for continuous test results while sloping lines are used for ordered categories. ROC curves may be derived directly from the observed sensitivity and specificity corresponding to different test thresholds, or by fitting curves based on parametric [ 4 ], semi-parametric [ 5 , 6 ], or non-parametric methods [ 7 ]. The area under the ROC curve (AUC) is a summary of diagnostic performance, and takes values between 0.5 and 1. The more accurate the test, the more closely the curve approaches the top left hand corner of the graph (AUC = 1). A test that provides no diagnostic information (AUC = 0.5) will produce a straight line from the bottom left to the top right. ROC curves may be restricted to a range of sensitivities or specificities of clinical interest.

ROC plots show how estimated sensitivity and specificity vary according to the threshold chosen, and can be used to identify suitable thresholds for clinical practice if the points on the curve are labelled with the corresponding threshold as in Figure 1c , which shows for example that the sensitivity and specificity corresponding to a threshold of 39.3 are 74% and 90%, respectively. Confidence intervals can be added to indicate the uncertainty in estimates of test performance at each point. ROC plots also allow comparison of the performance of several tests independently of choice of threshold, by plotting data sets for multiple tests in the same ROC space. However, they are thought to be difficult to interpret as they describe the characteristics of the test in a way which does not relate directly to its usefulness in clinical practice; research has shown that ROC plots are generally poorly understood by clinicians [ 8 ].

Flow charts (Figure 1d )

These depict the flow of patients through the study: for example how many patients were eligible, how many entered the study, how many of these had the target condition, and the numbers testing positive and negative. Such charts require categorisation of test results, for example as "positive" and "negative". Although flow charts do not directly present diagnostic accuracy data, addition of percentages to the test result boxes (as in Figure 1d ) can be used to report test sensitivity (68/90 = 76%) and specificity (46/51 = 90%). Charts that first separate individuals according to test result before classification by disease status may similarly be used to depict positive and negative predictive values. The STARD (standards for reporting of diagnostic accuracy) statement, an initiative to improve the reporting of diagnostic test accuracy studies similar to the CONSORT statement for clinical trials, recommends the inclusion of a flow diagram in all reports of primary diagnostic accuracy studies [ 9 ]. This should illustrate the design of the study and provide information on the numbers of participants at each stage of the study as well as the results of the study. The example flow chart in Figure 1d is not a full STARD flow diagram as we do not have data on numbers of withdrawals or uninterpretable results from this study. It does, however, show the design (diagnostic case-control) and results of the study.

Systematic reviews

Figure 2 illustrates two graphical displays commonly used to present data on diagnostic accuracy in diagnostic systematic reviews. Data from a systematic review of dipstick tests for urinary nitrite and leukocyte esterase to diagnose urinary tract infections were used to construct these graphs [ 10 ].

figure 2

Example graphs for systematic review data . a. Paired forest plots of sensitivity and specificity for LE dipstick. b. ROC plot with SROC curves.

Forest plots (Figure 2a )

Forest plots are commonly used to display results of meta-analysis. They display results from the individual studies together with, optionally, a summary (pooled) estimate. Point estimates are shown as dots or squares (sometimes sized according to precision or sample size) and confidence intervals as horizontal lines [ 11 ]. The pooled estimate is displayed as a diamond whose centre represents the estimate and tips the confidence interval.

For diagnostic accuracy studies, measures of test performance (sensitivity, specificity, predictive values, likelihood ratios or diagnostic odds ratio) are plotted on the horizontal axis. Diagnostic test performance is often described by pairs of summary statistics (e.g. sensitivity and specificity; positive and negative likelihood ratios), and these are depicted side-by-side. Between-study heterogeneity can readily be assessed by visual examination. Results may be sorted by one of a pair of test performance measures, usually that which is most important to the clinical application of the test. A disadvantage of paired forest plots is that they do not directly display the inverse association between the two measures that commonly results from variations in threshold between studies.

ROC plots and summary ROC (SROC) curves (Figure 2b )

ROC plots can be used to present the results of diagnostic systematic reviews, but differ from those used in primary studies as each point typically represents a separate study or data set within a study (individual studies may contribute more than one point). A summary ROC (SROC) curve can be estimated using one of several methods [ 12 – 15 ] and quantifies test accuracy and the association between sensitivity and specificity based on differences between studies. As with forest plots, ROC plots provide an overview of the results of all included studies. However, unless there are very few studies, it is not feasible to display confidence intervals as the plot would become cluttered. Results for several tests can be displayed on the same plot, facilitating test comparisons. It is also possible to display pooled estimates of sensitivity and specificity together with associated confidence intervals or prediction regions. ROC plots may also be used to investigate possible explanations for differences in estimates of accuracy between studies, for example those arising from differences in study quality. Figure 3 shows results for a recent review that we conducted on the accuracy of magnetic resonance imaging (MRI) for the diagnosis of multiple sclerosis (MS) [ 16 ]. By using different symbols to illustrate studies that did (diagnostic cohort studies) and did not (other study designs) include an appropriate patient spectrum we were able to show that studies that included an inappropriate patient spectrum grossly overestimated both sensitivity and specificity.

figure 3

Sensitivity plotted against specificity, separately for cohort studies and for studies of other designs for MRI for diagnosis of multiple sclerosis.

Other plots

Various other graphical methods have been developed to display the results of systematic reviews and meta-analyses [ 17 , 18 ]. Although not generally developed specifically for diagnostic test reviews these can be adapted to display the results of such reviews. Funnel plots [ 19 ] and Galbraith plots [ 20 ] are often used to assess evidence for publication bias or small study effects in systematic reviews of the effects of medical interventions assessed in randomized controlled trials. However, their application to systematic reviews of diagnostic test accuracy studies is problematic [ 20 ]. Diagnostic odds ratios are typically far from 1, and it has been shown that, for data of this type, sampling variation can lead to artefactual associations between log odds ratios and their standard errors [ 21 ]. It is therefore recommended that the effective sample size funnel plot be used in reviews of test accuracy studies [ 20 ].

Predictive value

A number of graphical displays aim to put results of diagnostic test evaluations into clinical context, based either on primary studies or systematic reviews. Two graphical displays commonly used for this purpose are the likelihood ratio nomogram (Figure 4a ) and the probability-modifying plot (Figure 4b ). Each allows the reader to estimate the post-test probability of the target condition in an individual patient, based on a selected pre-test probability. To use the likelihood ratio nomogram, the reader needs an estimate of the likelihood ratios for the test. He then draws a line through the appropriate likelihood ratio on the central axis, intersecting the selected pre-test probability, to derive the post-test probability of disease. The probability-modifying plot depicts separate curves for positive and negative test results. The reader draws a vertical line from the selected pre-test probability to the appropriate likelihood ratio line and then reads the post-test probability off the vertical scale. Both graph types are based on a single estimate of test accuracy (likelihood ratio), although it is possible to plot separate curves on the probability-modifying plot or lines on the nomogram to depict confidence intervals around the estimated likelihood ratios. Each assumes constant likelihood ratios across the range of pre-test probabilities. However, this assumption may be violated in practice [ 22 ], because populations in which the test is used may have different spectrums of disease to those in which estimates of test accuracy were derived.

figure 4

Example graphs for interpreting diagnostic study result . a. Likelihood ratio nomogram. b. Probability modifying plot.

Use of graphical displays in the literature

We systematically reviewed how graphical displays are currently incorporated in studies of test performance. We included primary diagnostic accuracy studies published in 2004, identified by hand searching 12 journals (Table 2 ), and diagnostic systematic reviews published in 2003, identified from DARE (Database of Abstracts of Reviews of Effects) [ 23 ]. Searches were conducted in 2005 and so these years were the most complete available years for searching (there is a delay in adding studies to DARE). Diagnostic accuracy studies were studies that provided data on the sensitivity and specificity of a diagnostic test and that focused on diagnostic (whether the patient had the condition of interest) rather than prognostic (disease severity/risk prediction) questions. Journals were selected to provide a mixture of the major general medical and specialty journals. We particularly aimed to select journals that clinicians read. We extracted data on the different graphical displays used to summarise information about test performance, defined as any graphical method of summarising data on diagnostic accuracy or the predictive value of a test (Table 1 ).

We located 56 primary studies and 49 systematic reviews (Web Appendix). Fifty-seven percent of primary studies and 53% of systematic reviews used graphical displays to present results. In publications using graphics, the number of graphs per publication ranged from 1 to 51 (median 2, IQR 1 to 3 for primary studies and median 4, IQR 2 to 7 for systematic reviews). Table 3 summarises the categories of tests evaluated in the primary studies and systematic reviews. None of the tests evaluated in any of the primary studies were truly dichotomous: they all gave continuous or categorical results. Three of the eight systematic reviews that assessed clinical examination looked at whether a variety of signs or symptoms were present or absent: these can be considered as truly dichotomous tests. All other reviews evaluated continuous or categorical tests.

Dot-plots or box-and-whisker plots were the most commonly used graphic and were included in 22 (39%) studies. Generally the plots showed individual test results separately for patients with and without the target condition, with four including an indication of the threshold used to define a positive test result. Three studies included both a dot plot and a box-and-whisker plot on the same figure. Other variations included separate plots for different patient subgroups, different symbols to indicate different stages of disease, or separate plots for different tests. The majority of studies using these types of plots were of laboratory tests. An ROC curve was displayed in 15 (26%) studies. All of these plotted full ROC curves; only two provided any indication of the thresholds corresponding to one or more of the points. Thirteen studies included separate ROC curves for different tests, either on the same plot (10 studies) or on separate plots (3 studies). Five studies included separate ROC plots for different patient subgroups. Although all the primary studies were published in 2004, after the publication of the STARD guidelines, only one included a STARD flow diagram.

ROC plots were included in 22 (45%) reviews. Twenty showed individual study estimates of sensitivity and specificity, 14 fitted SROC curves, and two displayed a summary point. One study, which did not fit an SROC curve, added a box and whisker plot to each axis to show the distributions of sensitivity and specificity. One study plotted only summary estimates of sensitivity and specificity in ROC space, with no SROC curves. Some reviews included separate plots for different tests, for different patient subgroups, or for different thresholds used to define a positive test result.

Ten reviews (20%) used forest plots to display individual study results. One study provided a plot of diagnostic odds ratios, while all others displayed paired plots of sensitivity and specificity (8 reviews), positive and negative likelihood ratios (3 reviews), or positive and negative predictive values (1 review). Several studies displayed more than one set of forest plots, including plots for more than one summary measure, for different stages of diagnosis, different test thresholds or for different tests. One study included a forest plot of summary data only, showing how pooled estimates of positive and negative likelihood ratios varied for different patient subgroups.

None of the studies included a likelihood ratio nomogram. One primary study and five systematic reviews included a probability-modifying plot.

Research in the area of cognitive psychology suggests that sensitivity and specificity are generally poorly understood by doctors [ 8 , 24 ] and are often confused with predictive values [ 8 , 25 , 26 ]. Doctors tend to overestimate the impact of a positive test result on the probability of disease [ 27 , 28 ] and this overestimation increases with decreasing pre-test probabilities of disease [ 29 ]. This research suggests that the most informative measures for doctors may be estimates of the post-test probability of disease (predictive value), which can be presented as a range corresponding to different pre-test probabilities. However, graphical displays that facilitate the derivation of post-test probabilities, such as likelihood ratio nomograms, are usually based on summary estimates of test characteristics (positive and negative likelihood ratios) without allowing for the precision of the estimate, or its applicability to a given population. Use of summary estimates in this way is questionable in the context of reviews of diagnostic accuracy studies, which typically find substantial between-study heterogeneity [ 30 ]. It is particularly problematic if the summary estimate is the only information conveyed in a graphic and the graphic is taken as the key message of the paper.

The inclusion of some form of graphical presentation of test accuracy data has a number of advantages compared to not using such displays. It allows fuller reporting of results, for example (S)ROC plots can display results for multiple thresholds whereas reporting test accuracy results in a text or table generally requires the selection of one or more thresholds. In addition, (S)ROC plots depict the trade-off between sensitivity and specificity at different thresholds. Use of such displays also have the advantage of presenting all of the results of a primary study or systematic review without the need for selected analyses, which may be biased depending on the analyses selected. The inclusion of graphical displays, such as SROC plots or forest plots, in systematic reviews of test accuracy studies allows a visual assessment of heterogeneity between studies by showing the results from each individual study included in the review. There is also a suggestion that graphical displays may be easier to interpret than text or tabular summaries of the same data.

Diagnostic accuracy studies will usually need to include more than one graphic in order both to provide a detailed description of results (diagnostic accuracy) and to communicate appropriate summary measures that can be used to inform clinical practice (predictive value); the more detailed graphic provides context for the interpretation of summary measures. Further work is required to improve on existing graphical displays. The starting point for this should be further evaluation of the types of graphical display most helpful to assessing the utility of a test in clinical practice and the implications of test results for individual patients.

We hope that this paper will contribute to an increase in the use and quality of graphical displays in diagnostic accuracy studies and systematic reviews of these studies. To achieve this, journal guidelines and the STARD statement need to encourage the use of graphs in reports of test accuracy. Currently, journal guidelines say very little about this issue. A brief review of the instructions for authors from a selection of leading medical journals ( Annals of Internal Medicine , BMJ , Clinical Chemistry , JAMA , Lancet , New England Journal of Medicine ) found that these only provide formatting guidelines rather than discussing when and what type of graphical displays should be used, although all except the New England Journal of Medicine recommend that the STARD guidelines be followed and include references to the STARD flow diagram. STARD itself does not comment on how graphical displays should be used to convey results of test accuracy studies other than to recommend the inclusion of a flow diagram and to provide an illustration of a dot-plot as a suggestion for how individual study results may be displayed. Guidelines on the type of graphical displays that should be included in reports of test accuracy studies could be considered when STARD is next updated, and should be considered by journals in their instructions for authors.

Our review suggests that graphical displays are currently underused in primary diagnostic accuracy studies and systematic reviews of such studies. Graphical displays of diagnostic accuracy data should provide an easily interpretable and accurate representation of study results, conveying both diagnostic accuracy and predictive value. This is not usually possible in a single graphic: the type of information presented in the most commonly used graphs does not directly allow clinicians to assess the implications of test results for an individual patient.

Web Appendix: Studies included in the review

A. primary studies.

1. Arvanitakis M, Delhaye M, De Maertelaere V, Bali M, Winant C, Coppens E, et al. Computed tomography and magnetic resonance imaging in the assessment of acute pancreatitis. Gastroenterology 2004;126:715–723.

2. Baldas V, Tommasini A, Santon D, Not T, Gerarduzzi T, Clarich G, et al. Testing for Anti-Human Transglutaminase Antibodies in Saliva Is Not Useful for Diagnosis of Celiac Disease. Clin Chem 2004;50:216–219.

3. Banks E, Reeves G, Beral V, Bull D, Crossley B, Simmonds M, et al. Influence of personal characteristics of individual women on sensitivity and specificity of mammography in the Million Women Study: cohort study. BMJ 2004;329:477.

4. Baschat AA, Guclu S, Kush ML, Gembruch U, Weiner CP, Harman CR. Venous Doppler in the prediction of acid-base status of growth-restricted fetuses with elevated placental blood flow resistance. Am J Obstet Gynecol 2004;191:277–284.

5. Biel SS, Nitsche A, Kurth A, Siegert W, Ozel M, Gelderblom HR. Detection of Human Polyomaviruses in Urine from Bone Marrow Transplant Patients: Comparison of Electron Microscopy with PCR. Clin Chem 2004;50:306–312.

6. Bluemke DA, Gatsonis CA, Chen MH, DeAngelis GA, DeBruhl N, Harms S, et al. Magnetic Resonance Imaging of the Breast Prior to Biopsy. JAMA 2004;292:2735–2742.

7. Brugge WR, Lewandrowski K, Lee-Lewandrowski E, Centeno BA, Szydlo T, Regan S, et al. Diagnosis of pancreatic cystic neoplasms: a report of the cooperative pancreatic cyst study. Gastroenterology 2004;126:1330–1336.

8. Bulterys M, Jamieson DJ, O'Sullivan MJ, Cohen MH, Maupin R, Nesheim S, et al. Rapid HIV-1 Testing During Labor: A Multicenter Study. JAMA 2004;292:219–223.

9. Carnevale V, Dionisi S, Nofroni I, Romagnoli E, Paglia F, De Geronimo S, et al. Potential Clinical Utility of a New IRMA for Parathyroid Hormone in Postmenopausal Patients with Primary Hyperparathyroidism. Clin Chem 2004;50:626–631.

10. Chye SM, Lin SR, Chen YL, Chung LY, Yen CM. Immuno-PCR for Detection of Antigen to Angiostrongylus cantonensis Circulating Fifth-Stage Worms. Clin Chem 2004;50:51–57.

11. Cotton PB, Durkalski VL, Pineau BC, Palesch YY, Mauldin PD, Hoffman B, et al. Computed Tomographic Colonography (Virtual Colonoscopy): A Multicenter Comparison With Standard Colonoscopy for Detection of Colorectal Neoplasia. JAMA 2004;291:1713–1719.

12. DeWitt J, Devereaux B, Chriswell M, McGreevy K, Howard T, Imperiale TF, et al. Comparison of Endoscopic Ultrasonography and Multidetector Computed Tomography for Detecting and Staging Pancreatic Cancer. Ann Intern Med 2004;141:753–763.

13. Esteban A, Fernandez-Segoviano P, Frutos-Vivar F, Aramburu JA, Najera L, Ferguson ND, et al. Comparison of Clinical Criteria for the Acute Respiratory Distress Syndrome with Autopsy Findings. Ann Intern Med 2004;141:440–445.

14. Foxman EF, Jarolim P. Use of the Fetal Fibronectin Test in Decisions to Admit to Hospital for Preterm Labor. Clin Chem 2004;50:663–665.

15. Gibot S, Cravoisy A, Levy B, Bene MC, Faure G, Bollaert PE. Soluble Triggering Receptor Expressed on Myeloid Cells and the Diagnosis of Pneumonia. NEJM 2004;350:451–458.

16. Greenough A, Thomas M, Dimitriou G, Williams O, Johnson A, Limb E, et al. Prediction of outcome from the chest radiograph appearance on day 7 of very prematurely born infants. Eur J Pediatr 2004;163:14–18.

17. Grenache DG, Hankins K, Parvin CA, Gronowski AM. Cervicovaginal Interleukin-6, Tumor Necrosis Factor-, and Interleukin-2 Receptor as Markers of Preterm Delivery. Clin Chem 2004;50:1839–1842.

18. Hammerer-Lercher A, Ludwig W, Falkensammer G, Muller S, Neubauer E, Puschendorf B, et al. Natriuretic Peptides as Markers of Mild Forms of Left Ventricular Dysfunction: Effects of Assays on Diagnostic Performance of Markers. Clin Chem 2004;50:1174–1183.

19. Hattori H, Kujiraoka T, Egashira T, Saito E, Fujioka T, Takahashi S, et al. Association of Coronary Heart Disease with Pre-[beta]-HDL Concentrations in Japanese Men. Clin Chem 2004;50:589–595.

20. Herget-Rosenthal S, Poppen D, Husing J, Marggraf G, Pietruck F, Jakob HG, et al. Prognostic Value of Tubular Proteinuria and Enzymuria in Nonoliguric Acute Tubular Necrosis. Clin Chem 2004;50:552–558.

21. Hetzel M, Hetzel J, Arslandemir C, Nussle K, Schirrmeister H. Reliability of symptoms to determine use of bone scans to identify bone metastases in lung cancer: prospective study. BMJ 2004;328:1051–1052.

22. Hift RJ, Davidson BP, van der Hooft C, Meissner DM, Meissner PN. Plasma Fluorescence Scanning and Fecal Porphyrin Analysis for the Diagnosis of Variegate Porphyria: Precise Determination of Sensitivity and Specificity with Detection of Protoporphyrinogen Oxidase Mutations as a Reference Standard. Clin Chem 2004;50:915–923.

23. Hong KM, Najjar H, Hawley M, Press RD. Quantitative Real-Time PCR with Automated Sample Preparation for Diagnosis and Monitoring of Cytomegalovirus Infection in Bone Marrow Transplant Patients. Clin Chem 2004;50:846–856.

24. Imperiale TF, Ransohoff DF, Itzkowitz SH, Turnbull BA, Ross ME, the Colorectal Cancer Study Group. Fecal DNA versus Fecal Occult Blood for Colorectal-Cancer Screening in an Average-Risk Population. NEJM 2004;351:2704–2714.

25. Jung K, Reiche J, Boehme A, Stephan C, Loening SA, Schnorr D, et al. Analysis of Subforms of Free Prostate-Specific Antigen in Serum by Two-Dimensional Gel Electrophoresis: Potential to Improve Diagnosis of Prostate Cancer. Clin Chem 2004;50:2292–2301.

26. Kageyama S, Isono T, Iwaki H, Wakabayashi Y, Okada Y, Kontani K, et al. Identification by Proteomic Analysis of Calreticulin as a Marker for Bladder Cancer and Evaluation of the Diagnostic Accuracy of Its Detection in Urine. Clin Chem 2004;50:857–866.

27. Kiesslich R, Burg J, Vieth M, Gnaendiger J, Enders M, Delaney P, et al. Confocal laser endoscopy for diagnosing intraepithelial neoplasias and colorectal cancer in vivo. Gastroenterology 2004;127:706–713.

28. Kramer H, van Putten JWG, Post WJ, van Dullemen HM, Bongaerts AHH, Pruim J, et al. Oesophageal endoscopic ultrasound with fine needle aspiration improves and simplifies the staging of lung cancer. Thorax 2004;59:596–601.

29. Kriege M, Brekelmans CTM, Boetes C, Besnard PE, Zonderland HM, Obdeijn IM, et al. Efficacy of MRI and Mammography for Breast-Cancer Screening in Women with a Familial or Genetic Predisposition. NEJM 2004;351:427–437.

30. Lacey JM, Minutti CZ, Magera MJ, Tauscher AL, Casetta B, McCann M, et al. Improved Specificity of Newborn Screening for Congenital Adrenal Hyperplasia by Second-Tier Steroid Profiling Using Tandem Mass Spectrometry. Clin Chem 2004;50:621–625.

31. Lennon PV, Wingerchuk DM, Kryzer TJ, Pittock SJ, Lucchinetti CF, Fujihara K, et al. A serum autoantibody marker of neuromyelitis optica: distinction from multiple sclerosis. Lancet 2004;364:2106–2112.

32. Leung Sf, Tam JS, Chan ATC, Zee B, Chan LYS, Huang DP, et al. Improved Accuracy of Detection of Nasopharyngeal Carcinoma by Combined Application of Circulating Epstein-Barr Virus DNA and Anti-Epstein-Barr Viral Capsid Antigen IgA Antibody. Clin Chem 2004;50:339–345.

33. Leung GM, Rainer TH, Lau FL, Wong IOL, Tong A, Wong TW, et al. A Clinical Prediction Rule for Diagnosing Severe Acute Respiratory Syndrome in the Emergency Department. Ann Intern Med 2004;141:333–342.

35. Liebeschuetz S, Bamber S, Ewer K, Deeks J, Pathan AA, Lalvani A. Diagnosis of tuberculosis in South African children with a T-cell-based assay: a prospective cohort study. Lancet 2004;364:2196–2203.

36. Llorente MJ, Sebastián M, Fernández-Aceñero MJ, Prieto G, Villanueva S. IgA Antibodies against Tissue Transglutaminase in the Diagnosis of Celiac Disease: Concordance with Intestinal Biopsy in Children and Adults. Clin Chem 2004;50:451–453.

36. McLean RG, Carolan M, Bui C, Arvela O, Ford JC, Chew M, et al. Comparison of new clinical and scintigraphic algorithms for the diagnosis of pulmonary embolism. Br J Radiol 2004;77:372–376.

37. Miglioretti DL, Rutter CM, Geller BM, Cutter G, Barlow WE, Rosenberg R, et al. Effect of Breast Augmentation on the Accuracy of Mammography and Cancer Characteristics. JAMA 2004;291:442–450.

38. Mikolajczyk SD, Catalona WJ, Evans CL, Linton HJ, Millar LS, Marker KM, et al. Proenzyme Forms of Prostate-Specific Antigen in Serum Improve the Detection of Prostate Cancer. Clin Chem 2004;50:1017–1025.

39. Minguez M, Herreros B, Sanchiz V, Hernandez V, Almela P, AnonAnon R, et al. Predictive value of the balloon expulsion test for excluding the diagnosis of pelvic floor dyssynergia in constipation. Gastroenterology 2004;126:57–62.

40. Palomaki GE, Neveux LM, Knight GJ, Haddow JE, Pandian R. Maternal Serum Invasive Trophoblast Antigen (Hyperglycosylated hCG) as a Screening Marker for Down Syndrome during the Second Trimester. Clin Chem 2004;50:1804–1808.

41. Palomaki GE, Knight GJ, Roberson MM, Cunningham GC, Lee JE, Strom CM, et al. Invasive Trophoblast Antigen (Hyperglycosylated Human Chorionic Gonadotropin) in Second-Trimester Maternal Urine as a Marker for Down Syndrome: Preliminary Results of an Observational Study on Fresh Samples. Clin Chem 2004;50:182–189.

42. Papadopoulos MC, Abel PM, Agranoff D, Stich A, Tarelli E, Bell PBA, et al. A novel and accurate diagnostic test for human African trypanosomiasis. Lancet 2004;363:1358–1363.

43. Parsi MA, Shen B, Achkar JP, Remzi FF, Goldblum JR, Boone J, et al. Fecal lactoferrin for diagnosis of symptomatic patients with ileal pouch-anal anastomosis. Gastroenterology 2004;126:1280–1286.

44. Raad I, Hanna HA, Alakech B, Chatzinikolaou I, Johnson MM, Tarrand J. Differential Time to Positivity: A Useful Method for Diagnosing Catheter-Related Bloodstream Infections. Ann Intern Med 2004;140:18–25.

45. Rathbun SW, Whitsett TL, Raskob GE. Negative D-dimer Result To Exclude Recurrent Deep Venous Thrombosis: A Management Trial. Ann Intern Med 2004;141:839–845.

46. Rietveld RP, Riet Gt, Bindels PJE, Sloos JH, van Weert HCPM. Predicting bacterial cause in infectious conjunctivitis: cohort study on informativeness of combinations of signs and symptoms. BMJ 2004;329:206–210.

47. Rosenberg WMC, Voelker M, Thiel R, Becka M, Burt A, Schuppan D, et al. Serum markers detect the presence of liver fibrosis: A cohort study. Gastroenterology 2004;127:1704–1713.

Schwertz E, Kahlenberg F, Sack U, Richter T, Stern M, Conrad K, et al. Serologic Assay Based on Gliadin-Related Nonapeptides as a Highly Sensitive and Specific Diagnostic Aid in Celiac Disease. Clin Chem 2004;50:2370–2375.

49. van Gelder RE, Nio CY, Florie J, Bartelsman JF, Snel P, de Jager SW, et al. Computed tomographic colonography compared with colonoscopy in patients at increased risk for colorectal cancer. Gastroenterology 2004;127:41–48.

50. Van Meensel B, Hiele M, Hoffman I, Vermeire S, Rutgeerts P, Geboes K, et al. Diagnostic Accuracy of Ten Second-Generation (Human) Tissue Transglutaminase Antibody Assays in Celiac Disease. Clin Chem 2004;50:2125–2135.

51. Vasbinder GB, Nelemans PJ, Kessels AGH, Kroon AA, Maki JH, Leiner T, et al. Accuracy of Computed Tomographic Angiography and Magnetic Resonance Angiography for Diagnosing Renal Artery Stenosis. Ann Intern Med 2004;141:674–682.

52. Vlahou A, Giannopoulos A, Gregory BW, Manousakas T, Kondylis FI, Wilson LL, et al. Protein Profiling in Urine for the Diagnosis of Bladder Cancer. Clin Chem 2004;50:1438–1441.

53. Warner E, Plewes DB, Hill KA, Causer PA, Zubovits JT, Jong RA, et al. Surveillance of BRCA1 and BRCA2 Mutation Carriers With Magnetic Resonance Imaging, Ultrasound, Mammography, and Clinical Breast Examination. JAMA 2004;292:1317–1325.

54. Wasmuth J-C, Grün B, Terjung B, Homrighausen, Spengler A, Spengler U. ROC Analysis Comparison of Three Assays for the Detection of Antibodies against Double-Stranded DNA in Serum for the Diagnosis of Systemic Lupus Erythematosus. Clin Chem 2004;50:2169–2171.

55. Wildi SM, Judson MA, Fraig M, Fickling WE, Schmulewitz N, Varadarajulu S, et al. Is endosonography guided fine needle aspiration (EUS-FNA) for sarcoidosis as good as we think? Thorax 2004;59:794–799.

56. Zehentner BK, Persing DH, Deme A, Toure P, Hawes SE, Brooks L, et al. Mammaglobin as a Novel Breast Cancer Biomarker: Multigene Reverse Transcription-PCR Assay and Sandwich ELISA. Clin Chem 2004;50:2069–2076.

b. Systematic reviews

1. Arbyn M, Schenck U, Ellison E, Hanselaar A. Metaanalysis of the accuracy of rapid prescreening relative to full screening of pap smears. Cancer Cytopathology 2003;99(1):9–16.

2. Austin MP, Lumley J. Antenatal screening for postnatal depression: a systematic review. Acta Psychiatr Scand 2003;107:10–17.

3. Babu AN, Kymes SM, Fryer SM. Eponyms and the diagnosis of aortic regurgitation: what says the evidence. Ann Intern Med 2003;138:736–742.

4. Bachmann LM, Kolb E, Koller MT, Steurer J, ter RG. Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review. BMJ 2003;326:417–419.

5. Bastian LA, Smith CM, Nanda K. Is this woman perimenopausal? JAMA 2003;289:895–902.

6. Bipat S, Glas AS, van d, V, Zwinderman AH, Bossuyt PM. Computed tomography and magnetic resonance imaging in staging of uterine cervical carcinoma: a systematic review. Gynecol Oncol 2003;91:59–66.

7. Boustani, M., Peterson, B., Hanson, L., Harris, R., and Lohr, K. N. Screening for dementia. 2003.

8. Cardarelli R, Lumicao TG. B-type natriuretic peptide: a review of its diagnostic, prognostic, and therapeutic monitoring value in heart failure for primary care physicians. J Am Board Fam Pract 2003;16:327–333.

9. Carnero-Pardo C. Systematic review of the value of positron emission tomography in the diagnosis of Alzheimer's disease. Rev Neurol 2003;37:860–870.

10. Chunilal SD, Eikelboom JW, Attia J, Miniati M, Panju AA, Simel DL. Does this patient have pulmonary embolism? JAMA 2003;290:2849–2858.

11. Dinnes J, Loveman E, McIntyre L, Waugh N. The effectiveness of diagnostic tests for the assessment of shoulder pain due to soft tissue disorders: a systematic review. Health Technol Assess 2003;7:1–178.

12. Farquhar C, Ekeroma A, Furness S, Arroll B. A systematic review of transvaginal ultrasonography, sonohysterography and hysteroscopy for the investigation of abnormal uterine bleeding in premenopausal women. Acta Obstet Gynecol Scand 2003;82:493–504.

13. Framarin, A. First-trimester prenatal screening for Down syndrome and other aneuploidies. Montreal, PQ, Canada. Agence d'Evaluation des Technologies et des Modes d'Intervention en Sante (AETMIS) 2003: 81.

14. Gilbert DL, Sethuraman G, Kotagal U, Buncher CR. Meta-analysis of EEG test performance shows wide variation among studies. Neurology 2003;60:564–570.

15. Glas AS, Roos D, Deutekom M, Zwinderman AH, Bossuyt PM, Kurth KH. Tumor markers in the diagnosis of primary bladder cancer: a systematic review. J Urol 2003;169:1975–1982.

16. Goerres GW, Mosna-Firlejczyk K, Steurer J, von Schulthess GK. Assessment of clinical utility of F-18-FDG PET in patients with head and neck cancer: a probability analysis. Eur J Nucl Med Mol Imaging 2003;30:562–571.

17. Goto M, Noguchi Y, Koyama H, Hira K, Shimbo T, Fukui T. Diagnostic value of adenosine deaminase in tuberculous pleural effusion: a meta-analysis. Ann Clin Biochem 2003;40:374–381.

18. Gould MK, Kuschner WG, Rydzak CE, Maclean CC, Demas AN, Chan JK, et al. Test performance of positron emission tomography and computed tomography for mediastinal staging in patients with non-small-cell lung cancer. Ann Intern Med 2003;139:879–892.

19. Grady, D., McDonald, K., Bischoff, K., Cabou, A., Chaput, L., Hoerster, K., Shahpar, C., Walsh, J., Sorrough, G., and Won, G. Results of systematic review of research on diagnosis and treatment of coronary heart disease in women. Rockville, MD, USA, Agency for Healthcare Research and Quality. 2003; 268

20. Gupta S, Bent S, Kohlwes J. Test characteristics of alpha-fetoprotein for detecting hepatocellular carcinoma in patients with hepatitis C. Ann Intern Med 2003;139:46–50.

21. Heffner JE, Highland K, Brown LK. A meta-analysis derivation of continuous likelihood ratios for diagnosing pleural fluid exudates. Am J Respir Crit Care Med 2003;167:1591–1599.

22. Hollingworth W, Nathens AB, Kanne JP, Crandall ML, Crummy TA, Wang MC, et al. The diagnostic accuracy of computed tomography angiography for traumatic or atherosclerotic lesions of the carotid and vertebral arteries: a systematic review. Eur J Radiol 2003;48:88–102.

23. Honest H, Bachmann LM, Coomarasamy A, Gupta JK, Kleijnen J, Khan KS. Accuracy of cervical transvaginal sonography in predicting preterm birth: a systematic review. Ultrasound Obstet Gynecol 2003;22:305–322.

24. Ioannidis JP, Lau J. F-18-FDG PET for the diagnosis and grading of soft-tissue sarcoma: a meta-analysis. J Nucl Med 2003;44:717–724.

25. Jackson JL, O'Malley PG, Kroenke K. Evaluation of acute knee pain in primary care. Ann Intern Med 2003;139:575–588.

26. Johnston R, V, Burrows E, Raulli A. Assessment of diagnostic tests to inform policy decisions-visual electrodiagnosis. Int J Technol Assess Health Care 2003;19:373–383.

27. Liberman M, Sampalis F, Mulder DS, Sampalis JS. Breast cancer diagnosis by scintimammography: a meta-analysis and review of the literature. Breast Cancer Res Treat 2003;80:115–126.

28. Makrydimas G, Sotiriadis A, Ioannidis JP. Screening performance of first-trimester nuchal translucency for major cardiac defects: a meta-analysis. Am J Obstet Gynecol 2003;189:1330–1335.

29. Neumayer L, Kennedy A. Imaging in appendicitis: a review with special emphasis on the treatment of women. Obstet Gynecol 2003;102:1404–1409.

30. Olaniyan OB. Validity of colposcopy in the diagnosis of early cervical neoplasia: a review. Afr J Reprod Health 2002;6:59–69.

31. Pai M, Flores LL, Pai N, Hubbard A, Riley LW, Colford JM. Diagnostic accuracy of nucleic acid amplification tests for tuberculous meningitis: a systematic review and meta-analysis. Lancet Infect Dis 2003;3:633–643.

32. Pasternack I, I, Malmivaara A, Tervahartiala P, Forsberg H, Vehmas T. Magnetic resonance imaging findings in respect to carpal tunnel syndrome. Scand J Work Environ Health 2003;29:189–196.

33. Pastor-Gomez J, Pulido-Rivas P, De Sola RG. Review of the literature on the value of magnetoencephalography in epilepsy. Rev Neurol 2003;37:951–996.

34. Patton LL. The effectiveness of community-based visual screening and utility of adjunctive diagnostic aids in the early detection of oral cancer. Eur J Cancer 2003;39:708–723.

35. Pirozzo S, Papinczak T, Glasziou P. Whispered voice test for screening for hearing impairment in adults and children: systematic review. BMJ 2003;327:967–970.

36. Rietveld RP, Van Weert HC, ter RG, Bindels PJ. Diagnostic impact of signs and symptoms in acute infectious conjunctivitis: systematic literature search. BMJ 2003;327:789.

37. Riley RD, Burchill SA, Abrams KR, Heney D, Lambert PC, Jones DR, et al. A systematic review and evaluation of tumour markers in paediatric oncology: Ewing's sarcoma and neuroblastoma. Health Technol Assess 2003;7:1–162.

38. Romagnuolo J, Bardou M, Rahme E, Joseph L, Reinhold C, Barkun AN. Magnetic resonance cholangiopancreatography: a meta-analysis of test performance in suspected biliary disease. Ann Intern Med 2003;139:547–557.

39. Rosado B, Menzies S, Harbauer A, Pehamberger H, Wolff K, Binder M, et al. Accuracy of computer diagnosis of melanoma: a quantitative meta-analysis. Arch Dermatol 2003;139:361–367.

40. Rothman R, Owens T, Simel DL. Does this child have acute otitis media? JAMA 2003;290:1633–1640.

41. Scholten RJ, Opstelten W, van der Plas CG, Bijl D, Deville WL. Accuracy of physical diagnostic tests for assessing ruptures of the anterior cruciate ligament: a meta-analysis. J Fam Pract 2003;52:689–694.

42. Sotiriadis A, Makrydimas G, Ioannidis JP. Diagnostic performance of intracardiac echogenic foci for Down syndrome: a meta-analysis. Obstet Gynecol 2003;101:1009–1016.

43. Takata GS, Chan LS, Morphew T, Mangione-Smith R, Morton SC. Evidence assessment of the accuracy of methods of diagnosing middle ear effusion in children with otitis media with effusion. Pediatrics 2003;112:1379–1387.

44. Toloza EM, Harpole L, McCrory DC. Noninvasive staging of non-small cell lung cancer: a review of the current evidence. Chest 2003;123:137S-146S.

45. Toloza EM, Harpole L, Detterbeck F, McCrory DC. Invasive staging of non-small cell lung cancer: a review of the current evidence. Chest 2003;123:157S-166S.

46. Trowbridge RL, Rutkowski NK, Shojania KG. Does this patient have acute cholecystitis? JAMA 2003;289:80–86.

47. van Gelder JM. Computed tomographic angiography for detecting cerebral aneurysms: Implications of aneurysm size distribution for the sensitivity, specificity, and likelihood ratios. Neurosurgery 2003;53:597–605.

48. Watson LC, Pignone MP. Screening accuracy for late-life depression in primary care: a systematic review. J Fam Pract 2003;52:956–964.

49. Watson EJ, Templeton A, Russell I, Paavonen J, Mardh PA, Stary A. The accuracy and efficacy of screening tests for Chlamydia trachomatis: a systematic review. J Med Microbiol 2002;51:1021–1031.

Wieand S, Gail MH, James BR, James KL: A Family of Nonparametric Statistics for Comparing Diagnostic Markers with Paired Or Unpaired Data. Biometrika. 1989, 76: 585-592. 10.1093/biomet/76.3.585.

Article   Google Scholar  

Armitage P, Berry G, Matthews JNS: Statistical Methods in Medical Research. 2002, Oxford, Blackwell Science Ltd, Fourth

Book   Google Scholar  

Altman DG, Bland JM: Statistics Notes: Diagnostic tests 3: receiver operating characteristic plots. BMJ. 1994, 309: 188-

Article   CAS   PubMed   PubMed Central   Google Scholar  

Zweig MH, Campbell G: Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem. 1993, 39: 561-577.

CAS   PubMed   Google Scholar  

Cai T: Semi-parametric ROC regression analysis with placement values. Biostatistics. 2004, 5: 45-60. 10.1093/biostatistics/5.1.45.

Article   PubMed   Google Scholar  

Wan S, Zhang B: Smooth semiparametric receiver operating characteristic curves for continuous diagnostic tests. Stat Med. 2007, 26: 2565-2586. 10.1002/sim.2726.

Zou KH, Hall WJ, Shapiro DE: Smooth non-parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests. Stat Med. 1997, 16: 2143-2156. 10.1002/(SICI)1097-0258(19971015)16:19<2143::AID-SIM655>3.0.CO;2-3.

Article   CAS   PubMed   Google Scholar  

Reid MC, Lane DA, Feinstein AR: Academic calculations versus clinical judgments: practicing physicians' use of quantitative measures of test accuracy. Am J Med. 1998, 104: 374-380. 10.1016/S0002-9343(98)00054-0.

Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HC: Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003, 326: 41-44. 10.1136/bmj.326.7379.41.

Article   PubMed   PubMed Central   Google Scholar  

Whiting P, Westwood M, Bojke L, Palmer S, Richardson G, Cooper J, Watt I, Glanville J, Sculpher M, Kleijnen J: Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model. Health Technol Assess. 2006, 10: iii-xiii, 1.

Lewis S, Clarke M: Forest plots: trying to see the wood and the trees. BMJ. 2001, 322: 1479-1480. 10.1136/bmj.322.7300.1479.

Moses LE, Shapiro D, Littenberg B: Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993, 12: 1293-1316.

Rutter CM, Gatsonis CA: A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001, 20: 2865-2884. 10.1002/sim.942.

Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH: Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005, 58: 982-990. 10.1016/j.jclinepi.2005.02.022.

Macaskill P: Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis. J Clin Epidemiol. 2004, 57: 925-932. 10.1016/j.jclinepi.2003.12.019.

Whiting P, Harbord R, Main C, Deeks JJ, Filippini G, Egger M, Sterne JA: Accuracy of magnetic resonance imaging for the diagnosis of multiple sclerosis: systematic review. BMJ. 2006, 332: 875-884. 10.1136/bmj.38771.583796.7C.

Baujat B, Mahe C, Pignon JP, Hill C: A graphical method for exploring heterogeneity in meta-analyses: application to a meta-analysis of 65 trials. Stat Med. 2002, 21: 2641-2652. 10.1002/sim.1221.

Galbraith RF: A note on graphical presentation of estimated odds ratios from several clinical trials. Stat Med. 1988, 7: 889-894. 10.1002/sim.4780070807.

Light RJ, Pillemer DB: Summing up: the science of reviewing research. 1984, Cambridge, MA, Harvard University Press

Google Scholar  

Deeks JJ, Macaskill P, Irwig L: The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005, 58: 882-893. 10.1016/j.jclinepi.2005.01.016.

Sterne JA, Gavaghan D, Egger M: Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol. 2000, 53: 1119-1129. 10.1016/S0895-4356(00)00242-0.

Whiting P, Rutjes AWS, Reitsma JB, Glas AS, Bossuyt PMM, Kleijnen J: Sources of variation and bias in studies of diagnostic accuracy - A systematic review. Ann Intern Med. 2004, 140: 189-202.

Centre for Reviews and Dissemination databases. 2008, [ http://www.crd.york.ac.uk/crdweb/ ]

Young JM, Glasziou P, Ward JE: General practitioners' self ratings of skills in evidence based medicine: validation study. BMJ. 2002, 324: 950-951. 10.1136/bmj.324.7343.950.

Casscells W, Schoenberger A, Graboys TB: Interpretation by physicians of clinical laboratory results. NEJM. 1978, 299: 999-1001.

Eddy DM: Probabilistic reasoning in clinical medicine: Probems and opportunities. Judgment under uncertainty: Heuristics and biases. Edited by: Kahneman D, Slovic P and Tversky A. 1982, Cambridge, Cambridge University Press

Steurer J, Fischer JE, Bachmann LM, Koller M, ter Riet G: Communicating accuracy of tests to general practitioners: a controlled study. BMJ. 2002, 324: 824-826. 10.1136/bmj.324.7341.824.

Lyman GH, Balducci L: Overestimation of test effects in clinical judgment. J Cancer Educ. 1993, 8: 297-307.

Lyman GH, Balducci L: The effect of changing disease risk on clinical reasoning. J Gen Intern Med. 1994, 9: 488-495. 10.1007/BF02599218.

Lijmer JG, Bossuyt PM, Heisterkamp SH: Exploring sources of heterogeneity in systematic reviews of diagnostic tests. Stat Med. 2002, 21: 1525-1537. 10.1002/sim.1185.

Altman DG, Bland JM: Statistics Notes: Diagnostic tests 1: sensitivity and specificity. BMJ. 1994, 308: 1552-

Deeks JJ, Altman DG: Diagnostic tests 4: likelihood ratios. BMJ. 2004, 329: 168-169. 10.1136/bmj.329.7458.168.

Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM: The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003, 56: 1129-1135. 10.1016/S0895-4356(03)00177-X.

Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P: Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol. 2004, 159: 882-890. 10.1093/aje/kwh101.

Altman DG, Bland JM: Statistics Notes: Diagnostic tests 2: predictive values. BMJ. 1994, 309: 102-

Hattori H, Kujiraoka T, Egashira T, Saito E, Fujioka T, Takahashi S, Ito M, Cooper JA, Stepanova IP, Nanjee MN, Miller NE: Association of Coronary Heart Disease with Pre-{beta}-HDL Concentrations in Japanese Men. Clin Chem. 2004, 50: 589-595. 10.1373/clinchem.2003.029207.

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/8/20/prepub

Download references

Acknowledgements

This work was supported by the MRC Health Services Research Collaboration. Jonathan Deeks is funded by a Senior Research Fellowship in Evidence Synthesis from the Department of Health.

Author information

Authors and affiliations.

Department of Social Medicine, Canynge Hall, Whiteladies Road, Bristol, BS8 2PR, UK

Penny F Whiting, Jonathan AC Sterne & Roger Harbord

Centre for Reviews and Dissemination, University of York, YO10 5DD, UK

Marie E Westwood

Horten Centre, Zürich, Switzerland

Lucas M Bachmann

Department of Social and Preventive Medicine, University of Bern, Switzerland

Matthias Egger

Medical Statistics Group/Diagnostic Research Group, Department of Public Health and Epidemiology, University of Birmingham, B15 2TT, UK

Jonathan J Deeks

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Penny F Whiting .

Additional information

Competing interests.

The author(s) declare that they have no competing interests.

Authors' contributions

All authors contributed to the design of the study and read and approved the final manuscript. PFW and MEW identified relevant studies and extracted data from included studies. PFW carried out the analysis and drafted the manuscript with help from JD and RH.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, authors’ original file for figure 3, authors’ original file for figure 4, authors’ original file for figure 5, authors’ original file for figure 6, authors’ original file for figure 7, authors’ original file for figure 8, authors’ original file for figure 9, rights and permissions.

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Whiting, P.F., Sterne, J.A., Westwood, M.E. et al. Graphical presentation of diagnostic information. BMC Med Res Methodol 8 , 20 (2008). https://doi.org/10.1186/1471-2288-8-20

Download citation

Received : 24 October 2007

Accepted : 11 April 2008

Published : 11 April 2008

DOI : https://doi.org/10.1186/1471-2288-8-20

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Celiac Disease
  • Receiver Operating Characteristic Curve
  • Down Syndrome
  • Graphical Display
  • Diagnostic Odds Ratio

BMC Medical Research Methodology

ISSN: 1471-2288

use graphical presentation in health sciences data

IMAGES

  1. What Are The Graphical Methods Of Data Presentation

    use graphical presentation in health sciences data

  2. 4 Steps to Make Attractive Medical Data Chart in PowerPoint

    use graphical presentation in health sciences data

  3. Health Data Research Ppt Powerpoint Presentation Portfolio Mockup

    use graphical presentation in health sciences data

  4. Graphical representation of data mohit verma

    use graphical presentation in health sciences data

  5. Various info graphics to showcase Emdeon's research data in the medical

    use graphical presentation in health sciences data

  6. 7 Best Ways How Data Visualization Techniques are Shaping the Health

    use graphical presentation in health sciences data

VIDEO

  1. RM4

  2. Data Visualization #short

  3. Presentation of Data (lec. and SGT)

  4. Data Presentations: Graphs and Diagrams

  5. Lecture 4: data presentation

  6. Powerful Data Design presentation

COMMENTS

  1. Interactive Visualization Applications in Population Health and Health

    This translates into a powerful technique for using big health care data, allowing a deep exploratory dive without an a priori hypothesis to identify data-driven trends and patterns. In this review, although we observed various applications of interactive visualization, we found limited evidence of its use in global health.

  2. Principles of Effective Data Visualization

    Graph, plot, and chart often refer to the display of data, data summaries, and models, while image suggests a picture. Figure is a general term but is commonly used to refer to visual elements, such as plots, in a scientific work.

  3. Graphical Use in Public Health Research Introduction

    A graph of different kinds can be applied in many ways.Among the essential stages of public health research, one of the most important is the identification of data with which the researcher is...

  4. Visualization Techniques in Healthcare Applications: A Narrative Review

    Visualization in healthcare involves the effective presentation of information through graphics, images, and videos. Big data systems handle a massive amount of information and require visualization techniques to present it in a comprehensible manner.

  5. Visualization Techniques in Healthcare Applications: A ...

    Visualization in healthcare involves the effective presentation of information through graphics, images, and videos. Big data systems handle a massive amount of information and require visualization techniques to present it in a comprehensible manner.

  6. Introduction to Data Analysis and Graphical Presentation in

    Introduction to Data Analysis and Graphical Presentation in Biostatistics with R Authors: Thomas W. MacFarland 0; Thomas W. MacFarland ... Readers will be equipped to run analyses and make graphical presentations based on the sample dataset and their own data. ... Statistics for Life Sciences, Medicine, Health Sciences, Statistics and Computing ...

  7. Data Presentation

    Encourage the eye to compare different pieces of data. Reveal the data at several levels of detail, from a broad overview to the fine structure. Serve a clear purpose: description, exploration, tabulation, or decoration. Be closely integrated with the statistical and verbal descriptions of the data set. From E. R. Tufte.

  8. Data Presentation: Use of Tables and Graphics

    Data Presentation: Use of Tables and Graphics Anand J. Thakur Chapter First Online: 08 June 2022 387 Accesses Abstract In an era of evidence based medicine, data is an important factor and needs to be presented in a way that is quickly understood by the audience. People are generally weak at understanding data.

  9. Introduction to Data Analysis and Graphical Presentation in

    Through real-world datasets, this book shows the reader how to work with material in biostatistics using the open source software R. These include tools that are critical to dealing with missing data, which is a pressing scientific issue for those engaged in biostatistics. Readers will be equipped to run analyses and make graphical presentations based on the sample dataset and their own data.

  10. The impact of the format of graphical presentation on health-related

    The impact of the format of graphical presentation on health-related knowledge and treatment choices - ScienceDirect Abstract Introduction Section snippets References (30) Cited by (302) Patient Education and Counseling Volume 73, Issue 3, December 2008, Pages 448-455

  11. Effects of graphic presentation on understanding medical risks and

    Abstract. Abstract: The purpose of this study was to identify the method of presentation (text, bar graphs, or pictographs) that best enhances the understanding of medical risk information among Japanese adults. We also investigated the effect of graphic format on understanding by differences in numeracy. This is the first study conducted in Japan to clarify understanding the risks and ...

  12. Graph-Representation of Patient Data: a Systematic ...

    Graph theory is a well-established theory with many methods used in mathematics to study graph structures. In the field of medicine, electronic health records (EHR) are commonly used to store and analyze patient data. Consequently, it seems straight-forward to perform research on modeling EHR data as graphs. This systematic literature review aims to investigate the frontiers of the current ...

  13. Graphical presentation of diagnostic information

    Graphical displays of results allow researchers to summarise and communicate the key findings of their study. Diagnostic information should be presented in an easily interpretable way, which conveys both test characteristics (diagnostic accuracy) and the potential for use in clinical practice (predictive value). We discuss the types of graphical display commonly encountered in primary ...

  14. The impact of the format of graphical presentation on health-related

    Methods Participants were randomized to receive numerical information about the risks and benefits of a hypothetical medical treatment in one of six graph formats. Each described the benefits of taking one of two drugs, as well as the risks of experiencing side effects.

  15. The impact of the format of graphical presentation on health-related

    Numeracy and literacy may influence comprehension and compliance with screen designs or information flows; use of pictograms over tabular presentation of on-screen data may improve comprehension ...

  16. Graphical displays of patient-reported outcomes (PRO) for use in

    One approach is to use existing principles to optimize PRO data graphical presentation. As many healthcare agencies have a mandate to clearly communicate quantitative information to a number of stakeholders, several initiatives have developed guidance on communicating data in general, which can also inform efforts to develop best practices for ...

  17. Introduction to Data Analysis and Graphical Presentation in

    The purpose of this lesson is to provide context for the science of biostatistics and to highlight a few of the major contributors. Emphasis is given to the role of data analysis for the various ...

  18. The impact of the format of graphical presentation on health-related

    The impact of the format of graphical presentation on health-related knowledge and treatment choices. ... Search life-sciences literature (42,842,218 articles, preprints and more) Search ... and the limited processing of your personal data in order to function.

  19. Radar plots: a useful way for presenting multivariate health care data

    The purpose of this paper is to introduce radar graphing as a useful technique for the presentation of multivariate health care data. Most graphs in popular media present limited amounts of information. The bar chart and the trend graph are examples of graphical forms that usually present limited data.

  20. Presenting Data

    Effective data presentation depends upon articulating the purpose for communicating, understanding audiences and context, and developing storylines to be communicated, taking into account the need to present data ethically and in a manner easily understood.

  21. The impact of the format of graphical presentation on health-related

    DOI: 10.1016/j.pec.2008.07.023 Corpus ID: 43214606; The impact of the format of graphical presentation on health-related knowledge and treatment choices. @article{Hawley2008TheIO, title={The impact of the format of graphical presentation on health-related knowledge and treatment choices.}, author={Sarah T. Hawley and Brian J. Zikmund‐Fisher and Peter A Ubel and Aleksandra Jancovic and Todd ...