To investigate the viability of the STEM internship model, the internal and external evaluators conducted a study using both descriptive and comparative methods.
Descriptive Methods
The descriptive methods involved collecting data from various sources; such as, course grades, intern reflections/exit interviews, mentor teacher/university liaison feedback, and a series of focus groups and interviews. In total, 59 STEM MAT program affiliates participated in either focus groups or interviews. Table 1 provides the total number of individuals affiliated with the program, how many participated in the study, and the method used to collect their information.
Table 1. Response rates of MAT program affiliates invited to participate in the studyProgram affiliates | Total | Study participants | Data Collection Method | |
Number | Percent | |||
UMBC Executive Committee Members | 6 | 6 | 100.0 | Interviews |
BCPS Executive Committee Members | 6 | 5 | 83.3 | Interviews |
Interns | 56 | 17 | 30.4 | Focus Group |
Lead Teachers | 40 | 6 | 6.7 | Focus Group |
Academy Principals | 8 | 5 | 62.5 | Interview |
Site Liaisons | 9 | 6 | 66.7 | Focus Group |
UMBC Liaisons | 9 | 9 | 100.0 | Focus Group |
Instructors | 18 | 5 | 27.8 | Focus Group |
Total | 152 | 59 | 38.8 |
|
Individualized semi-structured focus group and interview protocols were designed to address specific topics relative to participants' role in the STEM MAT program. In total, 12 interviews and 7 focus groups were conducted. Given the relatively limited amount of data and individualized data collection protocols, no standardized coding mechanism was employed for the qualitative data analysis. Instead, two researchers reviewed and discussed the 19 focus group and interview transcripts to summarize salient findings.
Comparative Methods
A post hoc comparison of program entry and exit Praxis scores was conducted to evaluate how well the preparation received through the STEM MAT model compares with that received through the existing or traditional preservice teacher preparation program at UMBC. The study sample consisted of 56 STEM MAT interns and 24 comparison interns from the existing or traditional UMBC MAT program. Relevant intern data is provided in Table 2, below.
Table 2. STEM and Traditional MAT Intern CharacteristicsCharacteristics |
STEM MAT | Traditional MAT |
||
Number | Percent | Number | Percent | |
All Interns | 56 | 100.0 | 24 | 100.0 |
Certification |
|
|
|
|
Elementary | 37 | 66.1 | 16 | 66.7 |
Secondary | 19 | 33.9 | 8 | 33.3 |
Cohort Status |
|
|
|
|
Cohort 1 (2005-06) | 24 | 44.6 | 12 | 50.0 |
Cohort 2 (2006-07) | 20 | 35.7 | 12 | 50.0 |
Cohort 3 (2007-08) | 11 | 19.6 | 0 | 0.0 |
For the purpose of this study, the Praxis I and II exams were used as a standardized measure of baseline knowledge and program influence. Scores from the Praxis I, taken by both intern types upon entry, serve as a baseline assessment of pre-professional skills acquired prior to program participation. Scores on the Praxis II pedagogy test provides a standard measure of program effect on interns' pedagogical understanding. Lastly, scores on the Praxis II content test provide a comparison of the different methods employed by the MAT programs to ensure interns acquired sufficient STEM content knowledge. Scores have been averaged for each Praxis exam and then tested for significant differences between group averages. Significant differences in student performance were determined by an independent t-test of group mean scores.