An Electronic Screening System for Oral Health Examination and Collection of Critical Data in a Nonclinical Setting: Validation Trial

Joseph R. Greenberg, DMD; Stephen Sinclair, MD; Cheryl A. Janssen BHS, CNP; Mimi O’Brien, RDH; Kate Flanagan, BS; and Michael A. Posner, PhD

The US Surgeon General's 2000 Report on Oral Health in America called the lack of access to dental care among socio-economically challenged citizens a "national crisis."¹ More recent studies document the extent of the social and financial costs that result from poor access, including lost school time, reduced quality of life, and the need for seeking expensive care through hospital emergency departments.^2-5

Screening of individuals has been defined as the examination or testing of persons to separate those who are well from those who have an undiagnosed disease or who may be at risk, with repeated examination used to predict the risk for progression within those individuals. In 2003the World Health Organization endorsed school-based dental screening as an essential element in oral health promotion.⁶ This was theorized to facilitate early detection and timely intervention of oral disease. Twelve US states have passed resolutions requiring dental screening for school children before school entry, but the requirements demonstrate wide variability.⁷ Furthermore, the policies vary significantly with regard to the qualifications of the healthcare professionals who can conduct the screenings, ranging from a dentist to a non-dental trained nurse, while none specify the conditions or devices that must be used during the examination. Patel discusses the myriad of differences between these laws and studies that have confirmed their inefficiency and ineffectiveness.⁸ She states that because these kinds of studies show that such programs do not improve the oral health of a community, one must question why school dental screenings are mandated. Patel suggests that this may be a case of government officials failing to heed the advice of scientific evidence, "which could eventually raise issues regarding public health malpractice." Moreover, the outcome of the data collected indicates widespread failure in the evaluation. As exemplified in Pennsylvania, often it is left up to each school district to decide how to notify the parents and then how to collect and store data, with the result being that data is never compiled or analyzed.⁹

A British study revealed that parents often mistake the screening visit as appropriate treatment time for their child's dental health.¹⁰ Screenings that examine only for tooth decay miss a wide variety of other important oral and overall health signs and symptoms that may impact growth and development such as diet/nutrition/obesity, social issues (trauma, malocclusion), and the impact of community health interventions (fluorosis). Visual inspection for tooth decay alone certainly does not represent a complete oral health check-up.

Sheiham and co-authors have opined that the major reason why most approaches to improving and reducing inequalities in oral health are relatively ineffective is because there are no sound, evidence-based, sustained, interlinked, multitargeted studies directed at the proximal causes of dental caries, periodontal diseases, and oral or oropharyngeal cancer.¹¹ The development of effective protocols, however, must be founded on well-defined population-based assessments and progression analyses of the individuals, which have traditionally been done through screenings.

Effective, Complete Oral Health Screening

The first task in understanding the progressive course of any chronic, infectious disease is to define the staging that is clinically detected by participating examiners. Despite the dental community's more than 100 years of study of dental caries, a general lack of agreement remains on how to define and measure this condition. Recently, an international team of collaborators developed and published useful, easy to use, and clearly defined criteria for clinical visual caries detection, namely the International Caries Detection and Assessment System (ICDAS).¹² The system has proven reliable for the staging of clinically detected dental caries on coronal tooth surfaces, even when used by inexperienced dental examiners.¹³ The American Dental Association has suggested a simplification of the ICDAS seven stages of caries disease, termed the Caries Classification System (CCS).¹³ This system features the presentation of pictures to which the examiner compares the examined individual's teeth when staging the disease.

What has been needed is an efficient and easy-to-use electronic device that allows the screening examiner to enter his or her observations during the screening of each child. The electronic oral health screening system (EOHSS) described here uses images based on the ICDAS/CCS system for visual caries grading and additional images for the detection and staging of eight other oral conditions, all with touchscreen visual prompts. It is designed for use in field conditions operated by a dentist, dental hygienist, or other suitably trained individual given the expected time and visual constraints imposed by fieldwork. The purpose of this study was to assess the ease and efficiency of use of this tablet-based screening program in a nonclinical setting and secondarily to compare two examination techniques.

Methods

Interactive Recording Program

The interactive program developed by the authors is operated on an iPad^® that communicates in a HIPPA-compliant manner with an Internet-stationed database server. Touchscreen technology is used. A unique HIPPA-compliant identifier is assigned for each patient. The program contains demographic information, consent provided by parents/caregivers, and a dental and medical health history that is scanned from the returned paper consent form and uploaded prior to the clinical examination. This data is communicated to the central database and compared to ensure proper identification of the patient and to identify potential prior examinations.

For recording of the examination results, the findings are compared with images that define the grading for various conditions:

Diagrams on which the examiner identifies with a touch the deciduous or permanent teeth present in the child's mouth (Figure 1).

Pictures and descriptors of enamel surface changes/caries severity based on the ICDAS and CCS systems (Figure 2). In this diagram, the examiner "taps" the tooth in question, then the image representing the level of caries. After teeth that have surface changes/caries are identified, the remaining teeth are acknowledged as sound.

Images that define three stages of the severity of dental fluorosis using the Dean's Index¹⁴ (Figure 3) to allow the fluorosis to be graded.

Images that define three stages of periodontal inflammation: normal, localized, and generalized.

Images that define three stages of the patient's oral hygiene status: good, moderately poor, and very poor.

Images of restorations (eg, composite, silver amalgam, crown) and sealants that are identified for any involved teeth; the involved teeth are also identified in a similar fashion.

Images that define four categories of orthodontic malocclusion. The examiner is required to identify one of the categories, but the malocclusion is not graded.

Presence of dental trauma, identified by tapping the involved tooth and then the type of traumatic appearance (eg, teeth that are cracked, broken, or darkened).

Presence or absence of temporomandibular joint (TMJ) dysfunction.

Presence, absence, or uncertainty of early childhood caries based on history.

Recommendations for follow-up care (routine, urgent, emergent).

Once the children present for the screening are identified, the examination portion may be conducted in an environment without Internet access, and the data can be uploaded later when the program has Internet connectivity again.

Clinical Methods

A clinical trial of the EOHSS was recently completed at the Kids Smiles Children's Dental Health Center (kidssmiles.org) in southwest Philadelphia, Pennsylvania. The trial was approved and supported by the Solutions IRB (solutionsirb.com). Children appearing for dental examination and/or treatment at the KSCDHC were invited to participate in the trial. Upon their acceptance and parental completion of consent, an experienced public health dental hygienist (PHDH) performed the oral screening in the reception area in a chair placed under ambient lighting, examining with a tongue blade and flashlight. The PHDH had screened more than 10,000 children in various field settings (nursery schools, daycare centers, public schools, etc), and no attempt was made to alter the PHDH's usual technique except for the presence of the research associate recorder who prompted the examiner through all the parameters with visual screens as needed.

The child was next seen by a staff dentist in the dental operatory for examination and recording of the same criteria using routine equipment (dental lighting, air syringe drying, etc), but excluding x-ray examination. The same recorder (research associate) performed all the data entry for the dentist as well. All three participants completed a prior calibration exercise. However, the dentist did not have access to the results of the prior examination. The same research associate recorder prompted the dentist through the data screens of the EOHSS. The same dentist did all the examinations for the children entered in this study.

The objective of this study was not to vary the usual technique and protocol of either the screener or the dentist, except to "calibrate" the evaluations according to the EOHSS grading images and to add the verbal prompting and recording by the research associate. This ensured thorough collection of all the data categories in real-time.

A weighted Kappa coefficient using squared weights was employed to measure the level of agreement between the PHDH and dentist. The squared weight, which is used with ordinal scales like lesion rating, assigns inconsistent ratings that are far from each other with a higher weight in determining discordance. These calculations were done using RStudio version 1.0.153 and R version 3.4.1 (RStudio, rstudio.com).^15,16 A previous study with 11 dentists experienced in epidemiological surveys examining school children for dental caries also used Kappa statistics to assess interexaminer reliability.¹⁷

Results

A total of 89 children were consented to undergo this screening process. Once started, there were no refusals to complete the screening/examination process, which on average required 2 to 3 minutes per patient for screener or dentist, including data entry. In defining the teeth present, the screener and dentist both identified the same deciduous and permanent teeth present in 63 children. In 10 children the screener and dentist identified different subsets of the same teeth, while in 10 children the screener missed one or more teeth identified by the dentist, and in six children the dentist missed one or more teeth identified by the screener. However, the differences in the tooth identification did not affect the agreement in caries detection rates described below.

Identification of Surface Changes/Caries

The overall results of agreement between screener and dentist are presented in Table 1. Among the 89 mouths examined, 50 (56%) had no demineralization or caries identified by either examiner, while three mouths (3.3%) had mild lesions (in eight teeth), four mouths (4.5%) had moderate lesions (in eight teeth), and one mouth had extensive lesions (three teeth) identified by both examiners. The dentist identified mild lesions (38 teeth) in 17 mouths (19%) and moderate lesions (24 teeth) in 12 mouths (13.5%) that were not detected by the screener. No mouths with worse caries were identified by the dentist but missed by the screener. Twelve mouths had mild lesions (16 teeth) identified by the screener but not by the dentist and one mouth with moderate lesions. Otherwise, only sporadic single mouths had disagreement of one or two caries level assigned by the two screeners.

Table 2 presents weighted Kappa statistical analysis, demonstrating moderate agreement (K = 0.531) between the examiners with regard to caries levels for all teeth, with the lower back teeth showing the greatest agreement (K = 0.601) and the upper back and upper front teeth demonstrating less agreement (K = 0.446 and 0.468, respectively). Among the lower front teeth neither examiner identified any carious lesions.

Identification of Restorations and Sealants

Twenty-seven mouths had no restorations identified by either examiner, while both examiners identified 34 mouths with sealants, 17 with composites, 30 with silver amalgams, and four with crowns. Thirteen mouths had sealants identified by the dentist but missed by the screener, six had composites, and two had silver amalgams, while 11 mouths were identified with sealants by the screener but not by the dentist, and three with composites. Otherwise, only sporadic mouths had restorations with differences in the identified restorations by the two examiners.

Identification of Periodontal Inflammation, Oral Hygiene Status, Orthodontic Status, Fluorosis, and TMJ Status

In 87 mouths periodontal inflammation was judged as normal by both the screener and dentist,while in two mouths the dentist identified localized inflammation that was judged normal by the screener. In 84 mouths, the oral hygiene status, compared with the grading photographs, was judged normal by both, with four mouths judged mild by the screener but not by the dentist, and one mouth judged mild by the dentist but not by the screener. Eighty-three mouths were judged to have a normal orthodontic configuration, while in six mouths there was varying identification of crowded, open, or cross-bite malocclusions compared with the grading images. No fluorosis or abnormal TMJ status was recorded.

Election of Referral Status

Both screener and dentist recommended a routine examination follow-up for 80 of the children; in one case both examiners recommended urgent referral and in two cases emergent referral. In four cases the screener recommended urgent referral while the dentist recommended routine, and in one case each the dentist recommended urgent or emergent referral while the screener recommended routine.

Discussion

A total of 89 children were examined twice for eight visual parameters of oral health using prompts with grading comparison photographs of an electronic program on an iPad. The oral health parameters were surface demineralization/caries, restorations and sealants, incisor trauma, fluorosis, periodontal inflammation, personal oral hygiene, orthodontic malocclusion, and TMJ dysfunction. The first examination was performed by an experienced PHDH using a flashlight and tongue blade in an ambient-lighted dental reception area; the second examination was done shortly thereafter by a dentist in a dental operatory with all standard equipment and supplies available (excluding x-rays). The same research associate recorded both examinations. The dentist was blinded to the findings of the dental hygienist.

The results show specificity for identifying mouths with no level of caries was high at greater than 90%, but the low sensitivity in this study for detecting mild lesions (41%), moderate lesions (37.5%), and extensive lesions (50%) may be attributed to the small percentage overall incidence in this population of children that demonstrated such caries lesions. However, it may also be attributed to the examination modalities used by the screener (ambient lighting combined with a flashlight and tongue blade). Certainly, variations in visual perception, lighting conditions, tooth position, and patient cooperation may also skew examinations of the child sitting in the screening chair to view mild carious surface changes, which appeared to significantly influence the weighted Kappa scores calculated using the squared weight, which is commonly used with ordinal scales like lesion rating and assigns inconsistent ratings that are far from each other in determining discordance for various stages. Certainly, further studies are required to define whether the accuracy of the examiner/screener in the field may be improved by the use of additional equipment such as a mirror and headlamp. In addition, the existing EOHSS platform could be enhanced to interface with camera probes and other devices for photographic capture of dental structural changes and soft-tissue oral lesions to further improve screening detection and more precisely define the changes occurring on follow-up examinations.

Current oral disease examination systems reportedly are inconsistent across all populations, and most are incapable of recording or saving electronic data.^8,9,18 Another notable tool is the basic screening survey (BSS) developed by the Association of State and Territorial Dental Directors.¹⁹ This tool is for oral health surveillance, not research, and is limited in scope. The EOHSS described here is richer in parameters of data collection and is designed to provide more robust data for clinical research evaluations.

The EOHSS is designed for rapid data entry of what have been recognized as critical elements of oral disease when used by an examiner/screener and recorder team. Use of the system requires that the examiner (and separate recorder if one is used) become familiar with the grading images of the program; additionally, the authors recommend the examiner pursue online training in ICDAS. The relatively short screening examination completion time reported in the present study (2 to 3 minutes total for examination and recording) coupled with the richness of data collected make this, the authors believe, a potentially attractive solution for oral health epidemiologic research. The program requires all data fields to be completed to finalize the patient session. While the data did include a referral status that was judged by each examiner, it is possible that a consensus for such referral could be automatically inserted by the program based on the accumulated results of screenings.

Further studies are planned to test the effect of several variables on the accuracy of the data collected by the examiner/screener in the field such as comparing those examinations conducted with tongue blade and flashlight with examinations conducted with headlamp and disposable mirrors. Portable magnification (not used in this study) might improve the accuracy of surface changes/caries detection as well. If screening is to be mandated to define an individual child's progression within any of the disease processes identified, the ultimate goal is to define those stages at which professional dental intervention is warranted and to ensure that the recommended follow-up is completed successfully.

Conclusion

This study provides confirmation of the value of an EOHSS with visual images presented for comparisons and touchscreen data entry for efficient collection of oral health information. The accuracy comparisons of the screener's findings to the dentist's findings are favorable, especially considering the noted differences in observer conditions, including ambient-lighted room, tongue blade, and flashlight versus dental chair, chair light, and dental instruments, but the authors recognize the limitations of the few children with mild to moderate disease that may have led to the reduced sensitivity and weighted Kappa scores for detection and reduction. Further studies are needed to confirm that a team of trained dental hygienists using this visual-prompted program and a trained recorder can collect valid, accurate data on eight significant oral health parameters in 2 to 3 minutes per patient, and that this data can be stored for future study and analysis.

Acknowledgment

The authors wish to thank Dean Amid Ismail (Kornberg/Temple School of Dentistry) and Dr. Robert Weyant (Univ. of Pittsburgh School of Dentistry) for their critiques and reviews of this program, and Drs. Theodore Croll (Doylestown, PA) and Michael Rosenbaum (Philadelphia, PA) for their help collecting images for the program.

About the Authors

Joseph R. Greenberg, DMD
Clinical Professor of Restorative Dentistry, Kornberg School of Dentistry, Temple University, Philadelphia, Pennsylvania; Private Practice, Villanova, Pennsylvania

Stephen Sinclair, MD
Sinclair Retina Associates, Media, Pennsylvania

Cheryl A. Janssen, BHS, CNP
President/CEO, Kids Smiles Nonprofit Dental Health Centers for Children, Philadelphia, Pennsylvania

Kevin Krick, DMD
Associate Clinical Director, Kids Smiles Nonprofit Dental Health Centers for Children, Philadelphia, Pennsylvania

Mimi O'Brien, RDH
Public Health Dental Hygienist, Kids Smiles Nonprofit Dental Health Centers for Children, Philadelphia, Pennsylvania

Kate Flanagan, BS
Research Associate, Kids Smiles Nonprofit Dental Health Centers for Children, Philadelphia, Pennsylvania

Michael A. Posner, PhD
Associate Professor, Mathematics and Statistics, Villanova University,
Villanova, Pennsylvania

References

1. US Dept of Health and Human Services. Oral Health in America: A Report of the Surgeon General. Rockville, MD: US Dept of Health and Human Services, National Institute of Dental and Craniofacial Research, National Institutes of Health; 2000.

2. American Dental Association. Breaking down barriers to oral care for all Americans: the role of workforce. J Calif Dent Assoc. 2011;39(7):491-502.

3. A Costly Dental Destination. The Pew Center on the States. February 2012. https://www.pewtrusts.org/~/media/assets/2012/01/16/a-costly-dental-destination.pdf. Accessed June 9, 2017.

4. Edelstein BL. About CDHP. The Children's Dental Health Project website. https://www.cdhp.org/board/edelsteinbd. Accessed June 9, 2017.

5. Casamassimo PS, Thikkurissy S, Edelstein BL, Maiorini E. Beyond the dmft: the human and economic cost of early childhood caries. J Am Dent Assoc. 2009;140(6):650-657.

6. World Health Organization. Oral Health Promotion: An Essential Element of a Health-Promoting School. WHO Information Series on School Health, Document 11. Geneva: WHO; 2003.

7. Association of State and Territorial Dental Directors. State Laws on Dental "Screening" for School-Aged Children. Emerging Issues in Oral Health. October 2008. https://www.aapd.org/assets/1/7/CDHP_StateLawsSchoolEntDentalScreening1008.pdf. Accessed June 9, 2017.

8. Patel S. Mandated dental screenings: what state governments consider. Part 1 of 2. Pa Dent J (Harrisb). 2013;80(1):20-25.

9. Patel S. Mandated dental screenings: what state governments consider. Part 2 of 2. Pa Dent J (Harrisb). 2013;80(3):35-38.

10. Preston ST, Davies GM, Craven R. An investigation of parents' attitudes to dental health and school dental screening. Community Dent Health. 2001;18(2):105-109.

11. Sheiham A, Williams DM, Weyant RJ, et al. Billions with oral disease: a global health crisis-a call to action. J Am Dent Assoc. 2015;146(12):861-864. Comment in: Calls to action. J Am Dent Assoc 2016.

12. Ismail AI, Sohn W, Tellez M, et al. The International Caries Detection and Assessment System (IDCAS): an integrated system for measuring dental caries. Community Dent Oral Epidemiol. 2007;35(3):170-178.

13. Thompson VP, Schenkel AB, Penugonda B, et al. A pilot study of dentists' assessment of caries detection and staging systems applied to early caries: PEARL Network findings. Gen Dent. 2016;64(3):20-27.

14. Dean HT. Classification of mottled enamel diagnosis. J Am Dent Assoc. 1934;21:1421-1426.

15. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement. 1960;20(1):37-46.

16. Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of kappa and weighted kappa. Psychological Bulletin. 1969;72(5):323-327.

17. Assaf AV, Tagliaferro EP, Meneghim Mde C, et al. A new approach for interexaminer reliability data analysis on dental caries calibration. J Appl Oral Sci. 2007;15(6):480-485.

18. Nelson S, Mandelaris J, Ferretti G, et al. School screening and parental reminders in increasing dental care for children in need: a retrospective cohort study. J Public Health Dent. 2012;72(1):45-52.

19. Association of State and Territorial Dental Directors. The Basic Screening Survey: A Tool for Oral Health Surveillance Not Research. February 2011, updated June 2015. www.astdd.org/docs/bss-surveillance-not-research-june-2015.pdf. Accessed June 9, 2017.