MISEP Teachers' Science Achievement
Table 1 shows the results of the middle school teachers' pre and final mean scores on the content test. There was a highly significant increase in the teachers' scores after, compared to before, they completed the project. A Rasch measure of 0 indicates a respondent answered all questions of this test incorrectly, while a measure of 100 indicates a respondent answered all questions correctly. Teachers' Rasch mean scores were calculated using 53 items. Reliability for the middle school teacher science test is 0.69.
Year | n | Mean | SD |
t-value | df | p-value |
Pre (2005) | 21 | 43.96 | 5.76 | -12.80 | 20 | < 0.001 |
Final (2007) | 21 | 58.24 | 4.75 |
|
|
|
Science Achievement Middle School Students - The Discovery Inquiry Test
(DIT)
The DIT was constructed using public-released items from the 1990
and 1992 National Assessment of Educational Progress (NAEP) science tests. The test
measures a student's ability to analyze and interpret data, extrapolate from one situation
to another, and utilize conceptual understanding. The test includes 29 items across four
science content areas (Kahle, Meece, & Scantlebury, 2000). It has been used with
over 20,000 middle school students. In this case, the DIT had a Cronbach alpha value
of 0.66. Rasch scores were computed on student scores. Table 2 shows the students' mean
scores for the first cohort of middle school science teachers.
Table 2. Comparison of DIT Rasch Mean Scores by Baseline (Pre), Post Year 1, and Post Year 2 for Cohort A
Group | n | Mean | SD |
F-value | df | p-value |
Year 2005 (Pre) | 555 | 54.93 | 12.10 | 12.19 | 2, 1142 | < 0.001 |
Year 2006 (Post1) | 327 | 56.47 | 10.62 |
|
|
|
Year 2007 (Post2) | 263 | 59.14 * | 10.75 |
|
|
|
Multiple regression analyses were conducted to examine whether gender, student group, and/or Student Questionnaire subscale mean scores predicted achievement test scores for 5th, 6th, 7th, or 8th grade students of MISEP teachers. These results (standardized coefficient estimates) are shown in Table 3.
Table 3. Multiple Regression Standard Coefficients for the DIT by Grade Level: Cohort A, Post Year 2, Spring 2007
| 5th | 6th | 7th | 8th |
| n = 33 | n =75 | n = 73 | n = 73 |
Female | 0.01 | 0.17 | 0.05 | -0.21 |
White | -0.04 | 0.31 ** | 0.43 *** | 0.06 |
What I do in class | -0.20 | 0.06 | 0.26 | 0.10 |
What my teacher does in class | 0.07 | 0.30 * | 0.02 | -0.11 |
What my friends do | -0.04 | -0.16 | -0.29 | 0.00 |
What parents/adults do | 0.19 | 0.16 | -0.18 | -0.15 |
My attitude towards science | 0.75 *** | -0.11 | 0.28 * | 0.23 |
Adj R2 | 0.50 | 0.30 | 0.19 | 0.07 |
|
|
|
|
|
* p < 0.05, ** p <
0.01, *** p < 0.001 |
The combination of the variables explained approximately 50% of the test variance for 5th graders, 30% for 6th graders, 19% for 7th graders, and 7% for 8th graders. For students in the 8th grade, none of the demographic variables (gender and student group) nor behavioral and attitudinal variables were statistically significant predictors of test scores. For 5th and 7th graders, their subscale score "Attitudes towards science" was a significant predictor of achievement. This finding is supported by other research (Kahle, Meece, & Scantlebury, 2000) and is particularly evident among middle school students.
MCEP Teacher Chemistry AchievementIn 2005, Cohort A MCEP teachers completed the high school chemistry teacher test, composed of 62 items with a reliability of 0.85. Table 4 shows the pre/post Rasch mean scores for MCEP chemistry test. There was a highly significant increase in the teachers' chemistry content knowledge after, compared to before, they completed the Penn STI project.
Table 4. Rasch Mean Scores of Cohort A MCEP Teachers' Content Tests: Pre (Spring 2005) and Final (Fall 2007)Year | n | Mean | SD |
t-value | df | p-value |
Pre (2005) | 17 | 45.16 | 6.17 | -7.67 | 16 | < 0.001 |
Final (2007) | 17 | 56.55 | 7.07 |
|
|
|
Using t-tests, comparisons were made on Rasch mean scores for the achievement test by gender and by student group for Cohort A high school chemistry students, as shown in Table 5. Comparisons were made between boys and girls as well as between White and Non-White students. Significant differences were found by gender, favoring boys. There were no significant differences between White and Non-White students.
Table 5. Comparisons of HSCCCT Rasch Mean Scores by Gender and by Student Group: Cohort A, Post Year 2, Spring 2007Group | n | Mean | SD |
t-value | df | p-value |
Female | 76 | 40.94 | 10.21 | -2.30 | 152.00 | 0.023 |
Male | 78 | 45.07 | 12.00 |
|
|
|
Non-White | 64 | 42.72 | 14.88 | -0.40 | 89.63 | 0.689 |
White | 91 | 43.54 | 8.16 |
|
|
|
Multiple regression analyses were conducted to examine whether gender, student group, or Student Questionnaire Rasch mean subscale scores predicted achievement scores for 10th and 11th grade students of Cohort A high school chemistry teachers. Regression results (standardized coefficient estimates) are displayed in Table 6.
Table 6. Multiple Regression Standard Coefficients for the HSCCCT by Grade Level: Cohort A, Post Year 2, Spring 2007
| 10th | 11th |
Female | 0.23 * | 0.16 |
White | -0.01 | 0.15 |
What I do in class | -0.17 | 0.06 |
What my teacher does in class | -0.07 | 0.04 |
What my friends do | 0.22 | -0.07 |
What parents/adults do | -0.39 ** | -0.18 * |
My attitude towards science | 0.46 ** | 0.68 *** |
Adj R2 | 0.31 | 0.42 |
* p < 0.05, ** p < 0.01, ***
p < 0.001 |