Cost-effectiveness of School-Based Follow-up Eye Examinations (2024)

Importance Many preschool visual screening programs incorporate school-based comprehensive examinations, but the follow-up rates and cost-effectiveness of this approach are not well studied.

Objective To determine the follow-up rates and cost-effectiveness of referral to community-based eye care professionals vs to a mobile eye examination unit (mobile follow-up) among preschool children with failed visual screening results.

Design, Setting, and Participants This retrospective cohort cost-effectiveness study with decision analytic modeling and probabilistic sensitivity analysis included 3429 children in 37 public preschools in San Francisco, California, who underwent visual chart screening during the 2009-2012 academic years and 1524 children in the same schools who underwent autorefraction screening during the 2012-2013 academic year. One hundred seventy-five children who underwent visual chart screening were referred for community-based comprehensive eye examinations; 204 who underwent autorefractive screening were referred for preschool-based mobile follow-up. Data were collected from October 1, 2009, to May 29, 2013, and analyzed from June 30, 2013, to January 16, 2016.

Main Outcomes and Measures Cost-effectiveness of community-based vs mobile follow-up standardized for referral method.

Results Of the 175 children referred for community-based follow-up (91 boys [52.0%]; 84 girls [48.0%]; mean [SD] age, 3.8 [0.7] years), 104 attended (59.4%). Of 204 children referred for mobile follow-up (89 boys [43.6%]; 115 girls [56.4%]; mean [SD] age, 4.1 [0.6] years), 112 attended (54.9%). Costs per case detected were $664 and $776, respectively. In univariate analysis, mobile follow-up was equally cost-effective if it increased the follow-up rate to 73% or if its costs were reduced by at least 27%. In multivariate analysis with Monte Carlo simulation, community-based follow-up was more cost-effective than mobile follow-up in 88% of simulated cases and had typical savings of $112 (95% CI, −$77 to $368) per case detected.

Conclusions and Relevance Community-based eye care professionals may provide more cost-effective care than a mobile eye examination unit visiting the preschool among children with failed preschool-based visual screening.

Visual screening at 3 to 5 years of age facilitates early detection and prevention of amblyopia and is widely recommended.^1-3 In the busy setting of a primary care pediatrics office, visual screening is frequently not conducted in these younger children.⁴ Accordingly, schools and nonprofit institutions have attempted to fill this gap by offering visual screening to preschool students.

Screening programs typically consist of 1 or more visual tests, followed by referral to an eye care professional as necessary. Achieving high follow-up rates in patients referred from preschool-based screening remains challenging. Follow-up rates in preschool-based screening programs are approximately 50%.^5-9 One method of increasing adherence to follow-up may be to bring comprehensive follow-up eye examinations into the schools. Although this intervention has been gaining popularity,^10-15 only a few studies have assessed its effectiveness, and these have contradictory results.^14,15 Given the costs of arranging preschool-based follow-up, whether this follow-up method is more cost-effective than referral to community-based eye care professionals remains unclear. We herein compared the follow-up rates and cost-effectiveness of community-based follow-up and preschool-based follow-up in a mobile examination unit (hereinafter referred to as mobile follow-up) for a range of rates and costs.

We conducted a retrospective analysis to determine the costs and effects of 2 visual screening programs and performed decision analytic modeling to compare the cost-effectiveness of referral to community-based follow-up vs mobile follow-up. The perspective is that of a third-party payer such as a charitable foundation or an insurance provider. The study received approval from the institutional review board of the University of California, San Francisco, and was conducted in accord with the Health Insurance Portability and Accountability Act. Obtaining informed consent for this retrospective study after the screening and examinations was not feasible, and the data were deidentified. Written informed consent for examination and future use of information for research was obtained for patients who underwent mobile follow-up.

The first program was implemented by Prevent Blindness Northern California (PBNC), a 501(c)(3) tax-exempt organization, in 134 preschools in San Francisco during the 2009-2012 academic years. Thirty-seven of these preschools subsequently performed autorefraction screening during the 2012-2013 academic year. Children screened in the first program underwent testing with a visual acuity chart and corneal light reflex test by a hired nurse screener. Children who met referral criteria were directed to find independent follow-up with a community-based eye care professional. Referral reason, basic information about visual screening, and PBNC’s contact information were provided to the teachers of children with referral for follow-up to be passed on to parents. Because the screeners did not have the individual families’ contact information, follow-up was encouraged with telephone calls to the school nurse and confirmed when eye care professionals returned a postage-paid, self-addressed envelope to PBNC. This mailer confirmed follow-up but included only minimal clinical information; of note, information on refractive error was not included.

In the 2012-2013 academic year, PBNC implemented a new program in 168 preschools, 37 of which had been included in the previous 2009-2012 screening. All of these preschools were public or Head Start preschools. This second visual screening program used a combination of autorefraction (Retinomax; Righton), cover-uncover testing, and corneal light testing performed by hired lay screeners. Children who were absent, were already known to have glasses, or could not reliably complete the autorefraction screening with a confidence score of greater than 8 as required elsewhere were excluded from analysis.^16,17 Children who met any of the referral criteria (eTable 1 in the Supplement) were provided comprehensive eye examinations at their preschool at a later date in a mobile unit operated by PBNC. These examinations were provided at no cost to the patient or the family. To receive this examination, children were required to return a permission slip and have a guardian attend their eye examination. Demographics, screening test results, and complete examination results of those undergoing follow-up were recorded at the time of service delivery. Study data were collected and managed using REDCap electronic data capture tools hosted at University of California, San Francisco, from October 1, 2009, to May 29, 2013.¹⁸ Positive cases were defined by refixation on cover-uncover testing or American Association for Pediatric Ophthalmology and Strabismus criteria for amblyogenic refractive errors (eTable 1 in the Supplement).¹⁹

We limited our effectiveness analysis to the 37 preschools that performed both screening programs. Our final effectiveness measure is detection of a child meeting the case definition of amblyopia. This measure is directly calculated for the 2012-2013 program of autorefraction with mobile follow-up. To standardize for the referral program, we created a decision model using the screening results and positive predictive value of the 2012-2013 autorefraction program with the follow-up results of the community-based program (Figure 1). This process allowed us to model a comparator program of autorefraction-based screening with community-based follow-up.

Screening costs were calculated from supplies, travel, labor, and indirect costs in 2012 US dollars. Supply costs were abstracted from program expense reports. Durable electronic supplies were amortized for 5 years at a discount rate of 3% and a future resale value of 25% of the purchase price. Mileage costs were determined by round-trip distance from PBNC headquarters to a school multiplied by the reimbursem*nt rate of $0.56 per mile determined by the Internal Revenue Service. Staff time as a percentage of full-time employment was determined by the report of the program director. Hourly wage data were extracted from the May 2012 Bureau of Labor and Statistics report occupation profiles using “healthcare support workers, all others” for vision screeners and “first-line office supervisors” for program directors.²⁰ Wage data were considered to account for 70% of employer expenses, and an additional 30% was added to cover benefits for full-time employees based on national compensation survey data for private industry employees.²¹ Costs were allocated to individual screening, preschools, or the programming year. Costs that were distributed across the entire programming year were allocated on the basis of the proportion of students who attended the 37 included preschools of all children served in a year. Indirect costs for the program were allocated from PBNC’s total annual indirect costs on the basis of the program’s proportional direct costs.

Costs for mobile follow-up were similarly calculated from supplies, travel, labor, and indirect costs. Durable supplies were amortized for a 5- to 10-year useful life with a discount rate of 3% and future resale value of 25% based on Internal Revenue Service depreciation timelines. The duration of a complete eye examination performed by the mobile optometrist was directly observed for 17 examinations at 3 different preschools and multiplied by the Bureau of Labor and Statistics hourly reimbursem*nt without additional compensation for benefits. Mileage, staff labor costs, and indirect costs were calculated with the same methods used as for screening costs. Costs were allocated to individually examined children, preschools, and programming year and divided proportionally.

Follow-up costs for community-based eye care professionals were estimated across a range of reimbursem*nt options, including public insurance, private insurance, and cash payment options.²² These costs were combined with the cost of mailers and staff time for telephone calls to encourage follow-up and enter examination results to determine the total cost for community-based follow-up.

Data were analyzed from June 30, 2013, to January 16, 2016. Cost-effectiveness was defined as the total screening and examination costs required per case of amblyopia detected. We conducted a threshold analysis of cost per case detected across varying mobile follow-up rates. Screening costs were held constant across this variation. Total follow-up costs were held constant for fixed costs (eg, mobile examination price), thereby becoming less expensive per child at higher follow-up rates. Staff time costs were varied linearly with follow-up rates. Indirect costs were recalculated as a fraction of proportional direct costs and added to determine total costs. Referral rate, positive predictive value, and community follow-up rates and costs were held constant.

We performed multiple univariate sensitivity analyses and multivariate analyses. For univariate analyses, we individually varied the effectiveness measures of referral rate, positive predictive value, community-based follow-up rate, and mobile follow-up rate across their 95% CIs. Screening and mobile follow-up costs were individually varied by 50%. The cost of community-based follow-up was varied from baseline to the most extreme values reimbursed by public insurance or regional health maintenance organizations. For multivariate analyses we performed a Monte Carlo simulation of 10 000 scenarios with all inputs simultaneously varied. The results were ranked by relative cost-effectiveness, and the central 9500 observations were used to generate a 95% CI. For this analysis, cost variables were randomly varied across the range used in univariate analysis, assuming an underlying triangular probability distribution centered at the baseline value, and the effectiveness variables were randomly varied, assuming an underlying binomial distribution. Although referral rate, positive predictive value, and screening costs were varied across simulations, they were assumed to be equal for mobile or community-based follow-up in any individual simulation. A small administrative cost was accrued for referred children independently of whether they attended follow-up owing to the time invested in encouraging follow-up for all referrals. Given that this cost was less than 2% of total program costs, we held this input constant in the univariate and multivariate analyses.

During the 2009-2012 academic years, 3429 children underwent screening with visual charts at the 37 included San Francisco preschools, and 175 children (91 boys [52.0%]; 84 girls [48.0%]; mean [SD] age, 3.8 [0.7] years) were referred to community-based practices for a follow-up examination (referral rate, 5.1%; 95% CI, 0.8%-9.4%). Eye-care professionals sent back cards confirming completion of a follow-up eye examination for 104 referred children (follow-up rate, 59.4%; 95% CI, 50.5%-68.4%). We found no significant difference in screening visual acuity of those who did vs those who did not adhere to follow-up care with logMAR visual acuities of 0.39 vs 0.39 OD and 0.40 vs 0.39 OS (Snellen equivalent, 20/50 for all) (P = .41 and 0.72, respectively, Wilcoxon rank sum test).

During the 2012-2013 academic year, 1524 children underwent autorefraction screening at the same 37 San Francisco preschools, and 204 children (89 boys [43.6%]; 115 girls [56.4%]; mean [SD] age, 4.1 [0.6] years) were referred for the mobile follow-up (referral rate, 13.4%; 95% CI, 8.0%-18.7%). One hundred ninety-five of the 204 referred children (95.6%) were referred for failure of autorefraction criteria. Of those referred, 112 (follow-up rate, 54.9%; 95% CI, 46.0%-63.8%) attended the mobile examination, of whom 56 (50.0%; 95% CI, 37.3%-62.7%) met the case definition for amblyopia. A mean of 41 students from each school underwent screening and a mean of 5.5 underwent the mobile follow-up. We found no significant difference in follow-up rate between the follow-up methods (P = .41, Fisher exact test) (Table).

We simulated an autorefraction-based program with community-based follow-up by combining the 2012-2013 screening data with the 2009-2012 follow-up data. At the 59.4% follow-up rate with referral to community-based eye care professionals, 121 of the 204 children referred in the 2012-2013 autorefraction screening program would have been referred for mobile follow-up, of whom 61 children (50.4%) would be expected to have met the case definition for amblyopia (Figure 1).

Screening and mobile follow-up costs consisted of supply, travel, and staff time. Total direct screening costs were $8.38 per child screened, which raised to a comprehensive cost of $13.03 per child when proportional indirect costs were included (eTable 2 in the Supplement). Total direct mobile follow-up costs were $141.55 per examination, which rose to $206.64 per examination when indirect costs were included (eTable 3 in the Supplement). Costs for encouraging follow-up of referrals were considered separately from examination costs, with 5% of screener staff time devoted to encouraging follow-up on referrals ($2.31 per referred child in the study population).

The cost of community-based follow-up was derived for public insurance, private insurance, and cash payments (eTable 4 in the Supplement). The baseline examination cost was defined as the median health maintenance organization reimbursem*nt value of $161 for the examination with refraction. An additional $1.94 was included for all referrals to account for staff time dedicated to a 5-minute telephone call to encourage follow-up and $3.89 for each examination to account for 10 minutes of staff time to enter examination results from the mailer.

Under baseline conditions, referral to community-based eye care professionals led to more children receiving a diagnosis at a lower overall cost, with the cost per case detected of $776 for the mobile follow-up and $664 for community-based follow-up. We performed a threshold analysis to determine what mobile follow-up rate would be equally cost-effective with community-based follow-up. Holding community-based follow-up values at baseline, the mobile follow-up would become equally cost-effective at a follow-up rate of 73% and incrementally more cost-effective at greater follow-up rates. At 100% follow-up, the mobile referral would have a mean cost of $562 per case detected and an incremental cost of $406 per additional case compared with community-based referral (Figure 2).

We compared community-based follow-up with mobile follow-up using multiple univariate analyses of all major input variables (Table). Changes in the follow-up rate did not change estimations of cost-effectiveness. When we compared the lowest modeled community follow-up rate of 50.5% with the highest modeled mobile follow-up rate of 63.8%, community-based follow-up cost was $2.60 less per child with amblyopia detected. In contrast, changes in follow-up cost may alter which program is more cost-effective. An increase in community-based follow-up costs by more than 34% or a decrease in mobile follow-up costs by more than 27% would cause mobile follow-up to have a lower cost per case detected. The variables of screening cost, referral rate, and positive predictive value should vary equivalently regardless of follow-up method and do not change the finding that community follow-up detects more children with amblyopia at a lower cost (Figure 3).

Finally, we performed a multivariate Monte Carlo simulation. We found that the mean savings per case detected with community-based rather than mobile follow-up was $112. In the 10 000 modeled Monte Carlo simulations, community-based follow-up was more cost-effective than mobile follow-up in 88% of simulated scenarios and had both lower costs and more children treated in 59% of scenarios. In contrast, mobile follow-up was more cost-effective in the remaining 12% of modeled cases and had a 2% probability of lower costs and higher treatment rates. The central 95% of Monte Carlo cost-effectiveness simulations ranged from an increased cost of $77 per case detected to a savings of $368 per case detected with community-based compared with mobile follow-up.

We herein determined the relative cost-effectiveness of referral to community-based eye care professionals vs a mobile follow-up eye examination in children with failed preschool visual screening. First, we found no significant difference between follow-up rates in children referred to community-based eye care professionals compared with preschool-based free mobile follow-up. Although few studies have reported on this topic, our finding is consistent with a secondary analysis of treatment rates.¹⁴ Our follow-up rate among the community-based referral population is toward the higher end found in the literature.^5-7,9 This rate may be owing to the program target of populations with public health insurance and follow-up with adherence calls. The failure to obtain higher follow-up rates with free preschool-based mobile examinations in our study may be owing to requirements for timely permission slip return, guardian attendance during the examination, and limited scheduling options. The most common reason for lack of adherence with the mobile follow-up was failure to schedule an appointment, which accounted for 79 of the 92 referred children who did not attend a follow-up examination (86%). Lack of follow-up generally resulted from failure to return signed examination consent forms. An additional 13 children (14%) of children missed their scheduled examination date owing to absenteeism or lack of an available guardian. In community-based and mobile follow-up programs, written information on the failed screening and potential risks for amblyopia was provided to the teachers to pass on to the child’s parents.

We found that referral to community-based eye care professionals detected more children with amblyopia at a lower cost than referral to a mobile follow-up under baseline conditions. In our threshold analysis, we found that if mobile follow-up were to increase to 73% from the observed follow-up rate of 54.9% (95% CI, 46%-63.8%), the mobile follow-up may become the more cost-effective option. A higher follow-up rate may be achievable in programs where screening and examinations occur in the same day, as has been done elsewhere,^10,14 although this schedule would usually preclude parental attendance. Our model was most sensitive to changes in follow-up costs. Under scenarios within the sensitivity analysis of a 27% or greater cost reduction with the mobile examination or 34% or greater increase of costs of community follow-up, the mobile follow-up would be the more cost-effective option. In the multivariate Monte Carlo simulation, the probability of variations substantial enough to change cost-effectiveness preference was 12%, consistent with a strong trend favoring community-based follow-up.

Our study has several limitations. The greatest limitation is likely our assumption that follow-up is independent of referral method. Visual charts may detect children who are more symptomatic than those detected by autorefraction screening and therefore who may be associated with higher follow-up rates. Accordingly, our assumption that community-based follow-up rates would remain equal in an autorefraction screening program could overestimate follow-up rate and tend to overestimate the cost-effectiveness of an autorefraction screening program with community follow-up. However, we did not find any difference in screening visual acuity between those children who did and did not attend follow-up care in our population, arguing that baseline visual acuity did not significantly affect follow-up rates in our population. Our study was retrospective, and little demographic data were obtained about children screened in the 2009-2012 academic years, so we cannot control for many potential confounders such as race or income. We reduce the effect of these possible confounders by matching children on the basis of schools where such demographic factors are unlikely to have changed significantly during our 4-year study period. The population studied was that of a diverse, urban public preschool, where most of the students were Hispanic or Asian American. The perspective of our study was that of a third-party payer interested in maximizing health outcomes, and we did not incorporate parental costs for travel or missed work. The mobile follow-up has relatively less flexibility for scheduling but greater geographic proximity compared with community-based follow-up, and the relative weights of these conveniences to a child’s health care professional are not clear. Our recorded follow-up rates may underestimate actual follow-up rates for both programs. In the community-based follow-up program, not all eye care professionals may have returned the mailers. In the mobile follow-up program, some children may have attended community-based examinations rather than the mobile follow-up and were not recorded. We limit our analysis to cost-effectiveness with the outcome of detection of amblyogenic risk factors in children. We did not perform a full cost-benefit analysis given the uncertainty surrounding adherence, rates of progression to amblyopia, and wide variations in the value assigned for monocular and bilateral visual impairment.^23-26

Our results suggest that community-based eye care professionals may provide more cost-effective eye care than a mobile eye care unit visiting the preschool among children with failed preschool-based visual screening. The substantial variations in cost estimates allowed within the sensitivity limits of these analyses suggest a 12% possibility of increased cost-effectiveness with mobile follow-up. In addition, programmatic changes to increase the follow-up rate have the potential to substantially improve cost-effectiveness of mobile follow-up.

Corresponding Author: Alejandra G. de Alba Campomanes, MD, MPH, Department of Ophthalmology, University of California, San Francisco, 10 Koret Way, San Francisco, CA 94143 (dealbaa@vision.ucsf.edu).

Submitted for Publication: November 25, 2015; final revision received February 18, 2016; accepted February 21, 2016.

Published Online: April 14, 2016. doi:10.1001/jamaophthalmol.2016.0619.

Author Contributions: Dr Lowry had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Both authors.

Acquisition, analysis, or interpretation of data: Lowry.

Drafting of the manuscript: Lowry.

Critical revision of the manuscript for important intellectual content: Both authors.

Statistical analysis: Lowry.

Obtained funding: de Alba Campomanes.

Administrative, technical, or material support: Both authors.

Study supervision: de Alba Campomanes.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

Previous Presentation: This paper was presented as a poster at American Academy of Ophthalmology Conference; October 19, 2014; Chicago, Illinois.