Author: Tremblay, Tim; Smith, James; Porter, Alice; Weathers, Robert
Date published: April 1, 2011
The annual return-to-work rate for people with disabilities who are Social Security Disability Insurance (SSDI) beneficiaries is approximately one half of one percent (Social Security Administration, 2009, Tables 1 & 50). That is, only one half of one percent of SSDI disabled workers each year has had their benefits terminated due to work above the Substantial Gainful Activity (SGA) threshold defined by the Social Security Administration (SSA). Longitudinal analyses have found that, within 10 years following initial SSDI award, less than 4 percent of beneficiaries have had their benefits terminated after finding work, and more than a quarter of those individuals have had their benefits reinstated by the end of the 10-year period (Liu & Stapleton, 2010). These low return-to-work rates for SSDI beneficiaries have been observed against the backdrop of a long-term decline, occurring since the early 1980s, in employment rates for people with disabilities in general relative to those of people without disabilities, as measured by multiple, longstanding, national surveys (Weathers & Wittenburg, 2009). In an attempt to change these patterns, SSA has created a variety of work incentive programs intended to encourage SSDI beneficiaries to increase their employment participation and earnings, but none of those programs has addressed the "all or nothing" benefit provisions of SSDI. Under those provisions, beneficiaries lose their entire monthly benefit check if and when their earnings rise above SSA's Substantial Gainful Activity (SGA) threshold, following a nine-month Trial Work Period (TWP) defined under SSA rules. (The SGA income threshold for most beneficiaries in 2008, the last year of the timeframe of analysis, was $940 per month.) The sudden income drop from receiving a full benefit one month to receiving nothing the next has been called the SSDI "cash cliff'. For decades, SSDI beneficiaries and disability advocates have pointed to the "cash cliff' as a powerful disincentive to work, and cited it as a key factor behind the extremely low return-to-work rate among SSDI beneficiaries (Johnson-Lamarche & Baird, 1997; Status of Veterans' Employment, 2010). A majority of SSDI beneficiaries do not engage in employment and, among those who do, a statistically significant number have been observed to "park" their earnings below the SGA threshold (Schimmel, Stapleton, & Song, 2010). Many beneficiaries and advocates have argued that replacing the cash cliff with a graduated benefit offset, similar to that applied to the earnings of Supplemental Security Income (SSI) recipients, would incentivize employment, and encourage SSDI beneficiaries to increase their earnings and gradually reduce their dependence on cash benefits.
As part of the Ticket to Work and Work Incentives Improvement Act of 1999, Congress mandated that SSA conduct a demonstration to test whether replacing the cash cliff with a gradual reduction in benefits would increase employment levels and earnings for SSDI beneficiaries. In response to that mandate, SSA contracted with four states in 2005 to carry out small-scale pilots, as a first step in preparing for SSA's Benefit Offset National Demonstration (BOND). (For further information on the BOND, please see http://www.ssa.gov/disabilityresearch/offsetnational.htm.) The pilots tested a graduated benefit offset which would reduce a beneficiary's SSDI benefit check by $1 for every $2 earned above the SGA threshold, for earnings which otherwise would have resulted in a complete termination of the individual's cash benefit. The purpose of this study was to evaluate employment and earnings outcomes from one of those pilots, the Vermont SSDI Benefit Offset Demonstration.
This study evaluated employment and earnings outcomes for the Vermont SSDI Benefit Offset Demonstration Pilot within a random-assignment, experimental design. After each eligible person provided informed consent for participation in the Pilot, he or she was randomly assigned to one of two study groups: treatment or control. Both groups received benefits counseling (i.e., specialized counseling on public benefits eligibility, benefits management, and the interaction between employment earnings and benefits eligibility) at enrollment and had equivalent access to standard employment services, including additional ongoing benefits counseling, Vocational Rehabilitation (VR) services, and supported employment services, as needed throughout the study period. The intervention tested was the application of alternative SSDI rules, which provided a benefit offset provision for earnings at or above the SGA threshold, for treatment group members. A summarized comparison of study conditions for the two groups is provided in Table 1.
Eligibility for participation in the Vermont Pilot was limited by both regulatory and technical constraints on administration of the Pilot. To be eligible for the Pilot, at the time of enrollment a person with a disability must have been a Vermont resident, aged 1 8 years or older, an SSDI-only Social Security beneficiary (i.e., with no concurrent SSI eligibility), eligible for SSDI on the basis of their own work record (i.e., not receiving any childhood-disability or disabled-widow benefits), and they must have been less than 72 months beyond their ninth and final Trial Work Period (TWP) month, as defined under SSA rules.
Outreach and recruitment into the Vermont Offset Pilot focused on people with disabilities with a demonstrated interest in employment, primarily among individuals who had a history of contacts with the Vermont Division of Vocational Rehabilitation. Over the course of a roughly 14-month enrollment period from August 24, 2005 to October 31, 2006, 582 people with disabilities who met the above eligibility criteria provided informed consent and attempted to enroll in the Pilot. Five of those individuals (all initially assigned to the treatment group) were later found ineligible by SSA, leaving 577 valid enrollees in the Pilot. Approximately 95% of those individuals had some history of involvement with VR services, and more than 90% had been served by VR in the 3 years prior to enrollment. Of the remaining 577 eligible enrollees, 284 were randomly assigned to the treatment group and 293 were randomly assigned to the control group. Following random assignment, five individuals (1 control and 4 treatment group members) withdrew consent for participation and for data collection prior to the final outcomes analysis, resulting in a voluntary attrition rate of just less than 1%. This left a maximum sample size of 280 treatment group members and 292 control group members (572 total). For the purpose of this analysis, we have labeled the group of those 280 treatment and 292 control individuals as our "full sample".
Full sample. Baseline characteristics for our full sample of participants, at or just prior to the date of enrollment, are compared in Table 2 below. The random assignment process was successful in creating treatment and control groups that generally did not differ significantly from each other in measured demographics, with an exception for individuals with less than a high school education (10% for control versus 5% for treatment), though the difference was relatively small. Beyond demographics, there was a borderline-significant advantage for the control group at baseline in terms of participation in the state's Medicaid Buy-In program, a healthcare-related employment-support program, (34% for control versus 27% for treatment) and in terms of early baseline employment rates (up to 41% for control versus 34% for treatment). The latter difference was controlled for in this study's regression analyses. Following initiation of the Pilot, eleven enrollees (7 treatment group members and 4 control group members) from the full sample died prior to the end of the timeframe of analysis.
Subgroups. Subgroup analyses were conducted along several baseline variables. One was whether participants had ever been enrolled in the state's Medicaid Buy-In program, prior to their date of enrollment into the Pilot. The purpose of this analysis was to examine effect sizes of the pilot among a sample of enrollees who had received healthcare coverage designed specifically as a work incentive and employment support for people with disabilities. Ver mont's Medicaid Buy-In program provides healthcare coverage for people with disabilities who are working (current evidence of work is an eligibility requirement for the Buy-In) who would otherwise qualify for Medicaid coverage if it were not for their earnings from work. A second subgroup variable was whether participants had completed their TWP prior to their date of enrollment into the Pilot. The TWP variable was of particular interest in the analysis of earnings above SGA, since the TWP-completed group consisted of individuals for whom earnings over SGA had the potential to immediately reduce or eliminate their benefit check in that month, depending on which study group to which they were assigned. A third subgroup variable compared early enrollees into the project to later enrollees. Anecdotally, reports from Pilot staff suggested that early enrollees in the Vermont Pilot may have been more workready and/or motivated to utilize an offset provision than later enrollees.
The four-state Benefit Offset Pilots were designed to test the impact of changing the rules applied to benefit check reductions when an SSDI beneficiary works at a level where benefit payments would normally be suspended. The Social Security Administration conducted the Pilots under a ground-rule of "do no harm", and no benefit currently available to beneficiaries under standard SSDI rules was taken away from Pilot participants. This meant that all Pilot participants continued to have available to them all existing work incentives that allowed earnings above the SGA level without affecting benefit payments (e.g., Trial Work Period, Cessation Month, and Grace Period). The standard SSDI rules and test rules, and their respective impacts on benefit payments, are summarized in Table 3. Similarly, no other public benefits (federal, state, or local) currently available and due to SSDI beneficiaries were taken away as a result of participation in the Pilot. No participants were disadvantaged, then, in terms of eligibility for public benefits as a result of the Pilot. Upon providing written, informed consent, all participants equally received a 50%, randomized chance of gaining the option to utilize the test SSDI rules, which offered additional SSDI benefits for individuals who work above the SGA level when they would otherwise lose their benefits. By program rules, the impact of the Pilots could only mean that individuals might become eligible for additional public benefits as a result of their voluntary participation, and not for less. On a larger scale, the Pilots allowed SSA to test alternative program rules that might advantage SSDI beneficiaries generally, if found to be beneficial in a rigorous evaluation and later used in the design of new standard rules for the SSDI program. (As an example of research approved by department or agency heads for the evaluation of public benefit or service programs, the Vermont Agency of Human Services Institutional Review Board (IRB) determined this study to be exempt from IRB review under Title 45 of the Code of Federal Regulations, Section 46.101(b)(5).)
Outcome measures for employment and earnings for this analysis were derived from quarterly wage records of the state's Unemployment Insurance (UI) program, as of July, 2009. Although UI wage data does not include all earnings (primarily absent are earnings from self-employment and out-of-state work), it covers a large majority of wage earnings in Vermont and was used in this study as an economic indicator variable for group comparisons. All dollar values were adjusted for inflation using the Consumer Price Index for Urban consumers (CPI-U), with Calendar Quarter 3 of 2005 as the 100% reference value.
We derived 3 dependent variables from quarterly UI wage data: SGA rate, average earnings, and employment rate. Average earnings were calculated across time and across individuals from the sum of reported UI earnings for each individual for each quarter. To obtain an SGA measure, for each quarter where UI earnings equaled or exceeded the standard monthly SGA level multiplied by 3, the quarter was coded as 1, and as 0 otherwise. Similarly, to obtain an employment measure, for each quarter where UI earnings exceeded $0, the quarter was coded as 1 , and as 0 otherwise. Averages of SGA quarters and employment quarters provided SGA rate and employment rate as a percent of quarters.
Data Analysis Methods
Prior to analysis, calendar dates associated with each earnings record were converted on a person-by-person basis to time-quarters relative to the individual's date of enrollment in the Benefit Offset Pilot. The timeframe of analysis for the evaluation was from the fourth quarter prior to the quarter of enrollment through the eighth quarter following the quarter of enrollment, for each individual. Additionally, for the subgroup of early enrollees into the project (Calendar Year 2005 enrollees), for whom further additional post-enrollment data was available, we examined outcomes from four quarters prior to the quarter of enrollment through twelve quarters following the quarter of enrollment.
For the Vermont analyses, in order to test the statistical significance of outcome changes for the treatment group, we used differences-in-differences linear regressions, comparing the before/after changes for the treatment group to the before/after changes for the control group. In doing so, we used differences-indifferences regression models described by Bertrand, Duflo, & Mullainathan (2004) and summarized by Rose (2005).
For each analysis group, we first compared the year prior to enrollment to the year immediately following enrollment, and then compared the year prior to enrollment to the second year following enrollment. In each comparison, we collapsed the time-series of our dependent variable into two observations for each individual: one before enrollment and one after enrollment. We did this by averaging the quarterly outcomes for each individual across the four quarters prior to the quarter of enrollment and across the four quarters following the quarter of enrollment. (The quarter of enrollment itself represents a mix of the intervention and non-intervention conditions, and was therefore dropped from the analysis.) For each regression analysis, there were 3 independent variables, each coded 1 or 0: group (treatment = 1 ; control = 0), time relative to enrollment (after = 1; before = 0), and the interaction (product) of group multiplied by time. We used the one-tailed probability (p) of the group-by-time interaction coefficient as the significance test of the treatment effect, and used ? <= 0.05 as our standard for statistical significance. In visual presentations of our outcomes, we used an adjusted treatment-group mean, defined as the pre-enrollment treatment mean plus the pre/post difference in means for the control group. The difference between the post-enrollment observed mean for the treatment group and the pre-enrollment adjusted mean for the treatment group was equal to the effect size, in the unit of analysis (percentage points or dollars). Data for beneficiaries who died prior to the end of the analysis timeframe were excluded from these analyses, in order to maintain a constant sample size (treatment N = 273; control N = 288) over time for the differences-indifferences comparisons of annual means.
For the full sample of enrollees, 22% of the treatment group utilized the benefit offset by 1/1/2009, which was approximately 2 years post-enrollment for the majority of participants. Among early enrollees, for whom we were able to observe 3 years post-enrollment, the offset utilization rate was 41%.
Of the three dependent variables derived from UI wage reports (SGA rate, average earnings, and employment rate), SGA rate appeared to be the most sensitive measure of offset treatment effects, associated with the greatest statistical significance observed and the widest findings across samples.
For the full sample of enrollees, there was a significant effect of the offset intervention on SGA rate in the first year following the quarter of enrollment, with a modest effect size of 7 percentage points, representing 35% of the post-enrollment SGA rate for the treatment group. Those results are displayed in Figure 1 below. There were only borderline-significant effects on average earnings or employment rate in the first year following enrollment, and there were no significant effects across any measure during the second year post-enrollment. In an alternative-method reliability check on the magnitude of our full-sample findings, a separate set of analyses, designed by the Social Security Administration and reported in Porter et al. (2009), showed significant effect sizes on SGA rate in individual quarters up to 6 percentage points, with outcomes varying by quarter, relative to the quarter of enrollment. (Those SSA-designed analyses differed from the ones described in this study, in that they included outcomes for individuals who died within the timeframe of the evaluation, and they used different statistical models.)
The baseline Medicaid Buy-In subgroup (Figure 2) showed a similar pattern of results to those of our full sample, but with almost twice the effect size of the full sample in the first year following enrollment, or 13.7 percentage points, representing 51% of the post-enrollment SGA rate for the treatment group. As with the full sample, there was no significant effect on SGA rate in the second year following enrollment, and no significant effects during the first or second years post-enrollment for average earnings or employment rate.
For the baseline-TWP-completed subgroup (Figure 3), there was a significant positive treatment effect on SGA rate in the first year following the quarter of enrollment, with a relatively large effect size of 1 7.0 percentage points, representing 46% of the post-enrollment SGA rate for the treatment group. There was no significant effect on SGA rate in the second year following enrollment, and no significant effects during the first or second year postenrollment for average earnings or employment rate. While not statistically significant at the annual level, for average earnings and employment rate, there was some evidence of a reversal trend toward negative treatment effects in the second post-enrollment year. That is, in the second year post-enrollment, the offset treatment may have been weakly associated with an actual drop in average earnings and employment.
There was a dramatic difference in employment-related outcomes between early enrollees into the Vermont Pilot and later enrollees (Figure 4). For calendar-year 2005 enrollees, who enrolled in the first two calendar quarters of the Pilot enrollment period, there were large, statistically significant treatment effects on SGA rate across not only the first and second years post-enrollment, but also the third year post-enrollment. The effects on SGA rate were 20.6 percentage points in the first year post-enrollment, 16.5 percentage points in the second year, and 20.5 percentage points in the third year (representing 55%, 48%, and 60% of the post-enrollment SGA rate for the treatment group, respectively).
For early enrollees, there was also a borderline-significant effect on average earnings in the first year following enrollment, with an effect size of $823 in additional quarterly earnings (36% of postenrollment mean earnings for the treatment group), and a fully significant effect in the third year following enrollment, with an effect size of $1,042 in additional quarterly earnings (47% of post-enrollment mean earnings for the treatment group). There were no significant effects on employment rate during the first, second, or third years post-enrollment.
There were several important limitations to this Pilot as a study of potential offset impacts. First, beneficiaries who were more than 72 months beyond the end of their Trial Work Period were excluded from enrollment, which may have eliminated the most persistent earners among SSDI beneficiaries. Second, beneficiaries knew that the Pilot was time-limited, and therefore may have been unwilling to commit to higher-paying career paths knowing that the offset would end within a few years. Third, implementation of the offset treatment was hampered by technical issues that resulted in repeated errors in application of the offset, including overpayments, underpayments and inappropriate cessation of benefits. Beneficiaries may have avoided earning more due to fears of repeated errors which would endanger critical state and federal benefits on which they depend. Despite those limitations, this study provided important evidence regarding likely impacts of an SSDI benefit offset.
Participant outcomes from the pilot demonstrated that providing an offset can have statistically significant impacts on the earnings of beneficiaries, and relatively large and enduring impacts for certain subsets of beneficiaries.
The finding for the Medicaid Buy-In subgroup of an SGA effect size nearly double that of the full sample suggests that enrollees who were exposed to the healthcare safety net of the Medicaid BuyIn, in conjunction with benefits counseling, may have been more prepared to utilize a benefit offset.
The fact that the baseline-Trial- Work-Period-completed subgroup, which might be expected to be the sample most sensitive to offset effects, showed a reversal pattern, with large positive effects on SGA rate in the first post-enrollment year, but negative effects on average earnings and employment rate in the second post-enrollment year, suggests that there was a problem with the intervention itself. The reversal pattern may have resulted from an increased cumulative error rate for offset checks which built up in the second post-enrollment year, due to technical problems with implementation of the offset. The reversed outcomes for this subgroup may also have been largely responsible for the elimination of positive treatment effects in the second post-enrollment year for the full sample.
Findings for early enrollees into the Pilot were consistent with anecdotal reports from the Vermont Pilot's benefits counselors that early enrollees tended to be more work-ready or more work-motivated than later enrollees, due to pent-up demand for an offset provision in the state. For this subgroup of enrollees, it may have been that greater work readiness and/or motivation overwhelmed whatever barriers were created by problems with implementation of the offset treatment. In examining the baseline characteristics of this subgroup, we found that more of the early enrollees into the Pilot had completed their Trial Work Period (TWP) prior to enrollment than in the full sample, and the early-enrollees subgroup had higher average earnings and a higher SGA rate in the two quarters immediately prior to enrollment into the Pilot. Enrollment of less workready or less motivated individuals later in the Pilot may have suppressed treatment effects for the full sample of enrollees.
Overall, outcome findings from this Pilot demonstrated that an SSDI offset can have a significant, large, and enduring effect on the SGA rate of certain beneficiaries, but that the effect may be limited to a subset of individuals who are more able and/or more motivated to work than the average SSDI beneficiary. They also suggest that the impact of an offset may be enhanced when paired with healthcare protection for workers with disabilities, similar to that provided by Medicaid Buy-In programs. The findings from the pilot also suggest the converse, that current SSDI provisions may have the unintended effect of suppressing beneficiary earnings for this subset. As such, this study provides modest support for advocates of policy change that would remove the "cash cliff' built into the SSDI program and replace it with a benefit offset.
This program pilot and evaluation were funded by Social Security Administration contract number SS00-05-60009.
Bertrand, M., Duflo, E., & Mullainathan, S., (2004). How much should we trust differences-in-differences estimates? Quarterly Journal of Economics, 119(1), 249-275.
Porter, A., Smith, J., Payette, A., Tremblay, T., & Burt, P. (2009). SSDI $l-for-2 benefit offset pilot demonstration: Vermont pilot final report. Contractor's report to the Social Security Administration (Contract Number SSOO-05-60009). Waterbury, VT: Vermont Division of Vocational Rehabilitation. Report posted at http ://www.socialsecuritv. gov/ disabilityresearch/offsetpilot.htm. and report with appendices posted at http ://www. vermontworks2 . org/Vt 1 for2FinalReportDownloadPage.htm.
Johnson-Lamarche, H., and Baird, P. (1997). The barriers to employment faced by persons with disabilities: Problems and solutions. Report to the Governor and the 1997 Vermont General Assembly, January 1997. Waterbury, VT: Vermont Department of Aging and Disabilities.
Liu, S., and Stapleton, D. (2010). How many SSDI beneficiaries leave the rolls for work? More than you might think (Disability Policy Research Brief, Number 10-01). Washington, DC: Center for Studying Disability Policy, April 2010. Document No. PP 10-29.
Rose, Shanna (2005). Econometric approaches to causal inference: Difference-in-differences and instrumental variables. Lecture presentation for Graduate Methods Master Class, Department of Government, Harvard University, February 25, 2005. Web posting, March, 2008, at http://www.peo ple.fas.harvard.edu/~rtmoore/GMMC/SRoseGMMC.ppt. This file is currently available at http .-//www, vermontworks2.org/SRoseGMMC.ppt.
Schimmel, J., Stapleton, D., and Song, J. (2010). How common is "parking" among Social Security Disability Insurance (SSDI) beneficiaries? Evidence from the 1999 change in the level of substantia] gainful activity (SGA) (Working Paper Number WP 2009-220). Ann Arbor, MI: University of Michigan Retirement Research Center, June, 2010.
Social Security Administration, (2009). Annual statistical report on the Social Security Disability Insurance Program, 2008 (SSA Publication No. 13-11826). Washington, DC: Social Security Administration, Released July 2009.
Status of Veterans' Employment: Statement of Richard Daley, Associate Legislation Director, Paralyzed Veterans of America. Hearing of the House Committee on Veterans' Affairs, Subcommittee on Economic Opportunity, 111th Congress. April 15, 2010. Retrieved from http://veterans.house.gov/hearings/Testimonv_Print.aspx?newsid=56 3&Name= Richard Daley
Weathers, R., and Wittenburg, D. (2009). Employment (Chapter 4). In A. Houtenville, D. Stapleton, R. Weathers, and R. Burkahuser (eds.), Counting working-age people with disabilities: What current data tell us and options for improvement (pp. 101-144). Kalamazoo, MI: WJ. Upjohn Institute for Employment Research.
Vermont Division of Vocational Rehabilitation
Vermont Division of Vocational Rehabilitation
Vermont Division of Vocational Rehabilitation
Social Security Administration
James Smith, Vermont Division of Vocational Rehabilitation, 103 South Main Street, Weeks IA Building, Waterbury, VT 056712303.