Assessing Outcomes through Data Collection and Establishing Benchmarking Goals

• Metro South/West Regional Workforce Investment Board, MA
• Seattle/King County Workforce Development Council, WA
• Illinois Workforce Investment Board and Illinois Department of Employment Security, IL
• Maryland State Workforce Investment Board

Overview

The Maryland Workforce Investment Board adopted a system wide measure for self-sufficiency as one of nine core measures to evaluate its performance. The "Self-Sufficiency Rate" measured the proportion of customers served and placed with earnings at 150 percent of the poverty level or higher.

top ^

History

In 2000, the Maryland WIB identified a need for a set of measures to evaluate the success of its state’s workforce system beyond that required under the Workforce Investment Act (WIA). In order to analyze the system’s successes and shortcomings, the Board established a System Measures Workgroup to assess how and where the system was working, and where there were gaps. The Workgroup was charged with reviewing the results of an existing 36-measure System Measure Survey Report and to winnow those down to a manageable set of performance measures that would address the Board’s goals. Over a period of about six months the workgroup gathered extensive input from stakeholders, and ultimately developed nine key performance measures.

The workgroup concentrated its efforts on system-wide measures that were feasible and able to guide the system to desired outcomes. To maximize feasibility, the workgroup redefined measures to take advantage of available data (e.g., drop-out rate, credential rate, etc.) and identified ways to produce the data via the Maryland Integrated Data System.

The workgroup also proposed the implementation of new data collection initiatives (e.g. job openings by occupation, industry and region) and added two new measures (Student Outcome Achievement Rate and a measure of Board Effectiveness). The workgroup issued a report that presented the measures within the framework of the WIB’s goals whileoutlining next steps and assignments for each measure. You might want to add the four goals in the attachments

The Self-Sufficiency Rate was defined as: "The number of participants served and placed by the workforce development system with earnings above 150% of the poverty level divided by the total number of participants who were employed following exit from the workforce development system during the reporting period."

This measure was in response to the Board’s explicit interest in knowing how well their core constituencies were being served The Board identified low-income individuals as a core constituency and, they asked the workgroup to develop a measure to determine if these customers were experiencing a growth in income. The workgroup developed the Self-Sufficiency Rate as their mechanism to make this assessment and to explore how the income of a customers change over time and as an indicator of how well low-income customers were being served by the system.

top ^

Partners and Funding

The Illinois WIB’s adoption of the Self-Sufficiency Standard and incorporation of it into benchmarks evolved from the building of relationships among many local and state partners. The Chicago Workforce Board (CWB) initially adopted the Self-Sufficiency Standard as its self-sufficiency benchmark. Also in Chicago, a community-based organization, Women Employed (WE), initially created the Self-Sufficiency Budget Calculator with the Mayor's Office of Workforce Development. The Executive Director of Women Employed serves on both the CWB and the Illinois WIB, which greatly facilitated the partnership.

Following these activities, 12 workforce investment boards around the state requested a comparable calculator. In response, the Illinois Department of Employment Security (IDES) adapted the calculator for use by any workforce investment board statewide. IDES also funded the development of corollary materials that could be used by customers and counselors.

Following these activities , the workforce investment boards around the state requested a comparable calculator. In response, the Illinois Department of Employment Security (IDES) adapted the calculator for use by any workforce investment board statewide. IDES also funded the development of corollary materials that could be used by customers and counselors.

Challenges

One challenge in including a system measure of self-sufficiency that sets a higher wage standard was the fear that the standard would not be met. The private sector WIB leadership kept stressing the business imperative of understanding trends and being able to clearly identify areas where improvement was needed, as well as areas of success. These leaders argued that it would be far better to discover areas where services were falling short and to correct them, than it would be to remain ignorant of possible shortcomings and thus continue with ineffective practices.

Unfortunately, in 2003, with a change in State Administration, new priorities were established for the workforce development system, and the System Measures (including the self-sufficiency rate) were discontinued.

top ^

Keys to Successful Implementation

• Strong leadership from private sector board chair.
• Extensive gathering of input from stakeholders, including employers, which created solid buy-in for all the measures that were developed, including self-sufficiency.
• Creation of measures that could be addressed through existing data collection.

top ^

Model Materials

• EXECUTIVE SUMMARY, System Measures Workgroup, Report to the Executive Committee Governor’s Workforce Investment Board, State of Maryland, September, 2000 [PDF, 196KB]

top ^