On Nov. 2-4, the US Dept. of Education (USDOE) Office of Career, Technical, and Adult Education (OCTAE) held its annual Data Quality Institute (DQI) in Washington, DC. The theme of this year’s DQI was “Identifying and Addressing Equity Gaps in CTE.”
Lauren Hall, associate public policy director for the National Alliance for Partnerships in Equity (NAPE), attended the DQI to learn more about how states are determining and collecting information and results from their career and technical education (CTE) programs – funded under the latest Perkins (V) Act. Here’s a recap of the event:
The three-day event kicked off with a half-day “Perkins 101” session targeted for new state department of education directors, CTE staff, teachers, and other school administrators. The session gave the basics on Perkins funding for CTE, data collection around which students are engaging in CTE programs at the high school and postsecondary level, and state accountability in using the funds and demonstrating student/participant progress in state CTE programs (through what the USDOE calls “quality” or “performance indicators.”)
Participants heard from USDOE Office of Career, Technical, & Adult Education (OCTAE) Deputy Assistant Secretary Luke Rhine throughout the event. (See the event agenda for more details.) USDOE’s OCTAE is concerned with adult education, civics education for adult learners, CTE programs, and community colleges operating as the “central bridge” between students and workforce programs, according to Rhine. (Rhine also pointed out that the USDOE was thrilled with the upcoming July 2023 improvements to the Pell Grant/FAFSA, due to the FAFSA Simplification Act; NAPE agrees!
On day two of the DQI, participants learned strategies to strengthen Perkins-related data collections systems and analysis. They also learned about programs aimed at ensuring equitable outcomes for students with respect to the Perkins V program quality indicators (essentially, the state data indicating, or “proving” to the government that students are not falling behind in gov-funded state CTE programs).
Throughout the event, participants heard from OCTAE program specialists and regional Perkins coordinators who have been tasked with helping the states handle enrollment and performance data (through what is called the consolidated annual report, or CAR, which states must fill out and submit to USDOE each year). They also aid in administering the state programs by providing states with technical assistance w/ CTE; ensure fiscal responsibility by reviewing and approving state CTE plans; and assist with local applications and states’ comprehensive local needs assessments (CLNAs), which collect and display student performance data and must include a description of how a state’s CTE program offerings are sufficient in size, scope, and quality to meet the needs of all students served by the eligible Perkins grant recipient (the state).
This CLNA must include an analysis of the government-deemed “special populations” served by Perkins, including those: with disabilities, from low-income families, those preparing for “nontraditional” fields (for their gender, e.g., women in manufacturing), single parents and pregnant women, out-of-workforce individuals, homeless students, youth who are in (or have aged out of) the foster care system, young people with a parent who is a member of the armed forces and is on active duty, migrant students, and individuals with other barriers to educational achievement (including those with limited English proficiency).
OCTA Education Program Specialist Jose Figueroa and OCTAE Chief of Program Administration & Accountability Adam Flynn-Tabloff spoke on how the states have leeway in creating their own programs by choosing which government-accepted quality (performance) indicators they want to track (through their consolidated annual reports). States can also make adjustments to their state-determined performance levels from year to year (as long as they are generally performing well). The performance indicators the government has outlined track the outcome/success of students who take CTE classes and are coded; for instance, 1S1 is the code for a student obtaining a four-year (high school) degree; 5S1 is a student attaining a recognized postsecondary (after college) CTE credential; 5S3 is a student participating in (paid) work-based learning (WBL), etc. The more students meeting these goals (e.g., graduating, enrolling in work-based learning), the better. In general, the CAR outlines: CTE program enrollment, student/participant performance, state CTE financial status reports, and state improvement plans (if relevant).
Many states could be clearer in how they enter and compile CTE data year-to-year. When it comes to collecting CAR and other data, event speakers pointed out that states should generally know: Who participates in analyzing CTE data? What sources are used? What calculations employed? Is a timeline followed? How is quality ensured? Finally, what documentation exists – is there written guidance to teach new folks how info is collected and reported, consistently? Data dictionaries can be used to describe the individual data elements that ultimately will go into CAR.
States are put on improvement plans (created/monitored by the USDOE) if they fail to show at least 90% improvement in their selected CTE performance indicators year-to-year. States can submit their CARs and other state plans through the Perkins web portal at perkins.ed.gov. (States can also get additional info here on the process involved in improvement plans and monitoring visits, if necessary.)
Many states are just beginning to identify and have conversations around performance gaps in CTE. Unfortunately, so far most research looks at access, as opposed to performance, or excludes the demographic data we would like to see to ascertain equity. Despite this, states are beginning to employ common equity strategies: Develop disaggregated data dashboards; deploy targeted communications campaigns to help students/families understand that CTE equates to high wage jobs potential; and braid state and federal funding resources (using Perkins and state resources given by governor).
Day three allowed participants to dive deeply into state-level CTE enrollment data through the online Perkins Collaborative Resource Network, particularly the ability to see “Data Explorer” and “State Profiles” and “National Performance Profile” pages, and to compare state data to identify potential gaps in CTE program access. Website visitors can also use the resource to select which indicators you want to look at – either secondary or postsecondary – and see which states did or did not meet their state indicators (by at least 90%, the USDOE-set standard).
Towards the middle of day three, hosts opened the room up to a lively discussion on state opportunities and challenges around increasing diversity, equity, and the quality of state data collection/reporting. Below are some of the major concerns / points brought up:
- Data systems must promote the thoughtful use of info to leverage systemwide change. States should: conduct an annual data system audit, apply an “improvement” over “compliance” mindset, contextualize performance in terms of students, offer annual training to data administrators, and create institutional “memory” around how data is collected. All this frees states up to make meaningful programmatic changes (if needed) to tell their state “CTE and equity” story, as opposed to only having a “technical understanding” of the data. (It is best to automate these processes as much as possible, and operationalize them consistently across your state.)
- It is difficult to compare state-to-state performance data in an “apples-to-apples” manner, since states can choose to collect data for different performance indicators.
- Some states “contract out” Perkins V data collection/record-keeping efforts, and others hire data scientists, specifically. Regardless, it’s important to be able to tell a story with the data. Rhine emphasized: “Our ability to collectively advocate starts with our ability to tell a story from state to state; our ability to produce reliable data and validate it. As a field, this storytelling is something we will have to spend time on, so what we value is measured.”
- Equity and access challenges occur even before CTE classes/programs begin for students. Some are held back from joining CTE programs due to prior “disciplinary” action (which is often disproportionately levied at students of color). Others are discouraged from applying for a program due to transportation challenges (no car, no public transportation to a class). And still others are saddled with pre-requisite, co-requisite or testing/scoring requirements that may or may not be necessary for joining CTE programs.
- MA was cited as a good example of state commitment to equity. The state built aggregated data dashboards to define disparities in enrollment between students of color, white students and lower- and higher-income students. Based on what they found (and to become more equitable), they are now requiring schools to remove disciplinary history as a requirement to be in CTE programs, and to advertise CTE programs in many different languages.
- States should look into “accelerating” students into CTE, and DE state was mentioned as a good example of how funding can be used to do so: Students who didn’t get “high enough” SAT grades in DE can take dual enrollment math or English classes and use Perkins funds to support this, in order to move on to CTE classes.
- When considering if Perkins funds can be used, a state should be able to answer “yes” to the following questions: Is the cost necessary and reasonable? Is the cost related to the grant? Perkins funds must be used for CTE; there should be a “reasonable tangential tie-in” to CTE programs/services.
- Some data is harder to get then others, and states have to be creative. States must decide whether to use survey data (e.g. pre-graduation student surveys) or administrative school-generated data (that they’ve been collecting). States can also pull wage data, labor/unemployment insurance (UI) data, census data, etc. Some states (like WI) are collecting data through student apps, where students can also clock in and out of work-based learning (WBL) activities, get teacher sign-offs, see total hours participation in CTE, etc.
- What qualifies as WBL? School based enterprises, employer supervised intensive projects, entrepreneurial experiences, service learning projects, internships, clinicals, and apprenticeships! Some states (e.g., IL) provide tax-based incentives (e.g., credits) for businesses to participate in WBL. In other states, employers get a “matching” amount (e.g., ND) to invest in student internship programs and/or apprenticeships.
- It’s very difficult to get to 100% of the data you want, and the USDOE stated it understands this. States should try to solve for percentage gains (improvement due to CTE classes over time). Even if a state only knows where 85% of students are going after a CTE program (into good paying jobs), this would be considered a “good” amount of data.
- Some states said that the “1P1” performance indicator is the most difficult for them to collect data around because there are often delays in the state getting and giving administrators data. (1P1 is postsecondary job placement, and Ed Dept. said states can pull UI and starting wage data to ascertain this).
- The government is encouraging states to incorporate other government grant program funding like WIOA (the Workforce Innovation & Opportunity Act), particularly to help youth w/ disabilities transfer into the workforce or higher ed. Specifically, WIOA has pre-employment transitional vocational rehabilitation services, which can and should serve as “wraparound” services when using Perkins funds. States can partner w/ regional disability programs or their Social Security office and use this grant funding in both secondary and postsecondary environments. (The federal government is working on creating public-facing materials to clarify the interrelated nature of these types of federal grants and to encourage states to apply to related grants when they apply to Perkins.)
- Unfortunately, the USDOE doesn’t have much data on CTE concentrators (generally, students who take multiple CTE classes), as this data collection was just recently required under Perkins V. Definitions of CTE concentrators can vary by state, with varying numbers of CTE hours required.