PennSTI
Comparisons Before and Three Years After Teacher Initial Participation in PennSTI
Differences were found between pre and final subscale means on the teaching ("What I do in class") and learning ("What my students do") subscales for teachers participating in the PennSTI (Table 3). These analyses used raw mean scores. Science teachers reported slightly more use of inquiry by their students, while chemistry teachers significantly increased their own ("What I do in class") use of inquiry.
Table 3. Comparison of Subscale Mean Scores for Penn Science and Chemistry Teachers: Pre (Spring 2005) and Final (Fall 2007).
Course |
Subscale |
Time |
n |
Mean |
SD |
t-value |
df |
p-value |
n2 |
Science |
What I do in class |
Pre |
16 |
3.51 |
0.39 |
-1.20 |
15 |
0.250 |
0.09 |
|
Final |
16 |
3.65 |
0.38 |
|
|
|
|
|
What my students do |
Pre |
15 |
3.38 |
0.40 |
-1.86 |
14 |
0.084 |
0.20 |
|
|
Final |
15 |
3.65 |
0.32 |
|
|
|
|
|
Chemistry |
What I do in class |
Pre |
15 |
3.07 |
0.28 |
-3.55 |
14 |
0.003 |
0.47 |
|
Final |
15 |
3.43 |
0.34 |
|
|
|
|
|
What my students do |
Pre |
15 |
3.10 |
0.46 |
-2.04 |
14 |
0.060 |
0.23 |
|
|
Final |
15 |
3.40 |
0.38 |
|
|
|
|
1 = "Almost Never" and 5 = "Very Often"
During the spring of 2005, 2006, and 2007, the student questionnaire was administered to science or chemistry students in one typical class of each participating teacher. Although the student groups changed, they were taught by the same teacher. Item Response Theory (IRT)-Rasch Model-was used to analyze the student data. For the teaching and learning subscales that request frequency ratings, a Rasch measure of 0 indicates that the respondent selected "Almost Never" for all subscale items. A Rasch measure of 100 indicates that the respondent choose "Very Often" for all subscale items.
Science students reported more use of standards-based instruction on both the teaching ("What my teacher does") and learning ("What I do") subscales after teacher participation (Table 4). Likewise, chemistry students of PennSTI teachers reported increased frequency of inquiry learning after teacher participation (Table 5).
Table 4. Comparison of Rasch Mean Subscale Scores of Penn Science Students: Pre, Post Year 1, and Post Year 2.
Subscale |
Time |
n |
Rasch Mean |
SD |
F-value |
df |
p-value |
n2 |
What I do in class |
Pre |
555 |
46.68 * |
4.83 |
18.30 |
2, 1323 |
< 0.001 |
0.03 |
Post Yr 1 |
413 |
48.41 |
4.93 |
|
|
|
|
|
Post Yr 2 |
358 |
48.10 |
4.36 |
|
|
|
|
|
What my teacher does in class |
Pre |
552 |
60.50 * |
12.31 |
7.20 |
2, 1316 |
0.001 |
0.01 |
Post Yr 1 |
410 |
62.89 |
13.10 |
|
|
|
|
|
Post Yr 2 |
357 |
63.29 |
10.98 |
|
|
|
|
* Mean scores for Post Year 1 and Post Year 2 are significantly higher than Pre mean scores.
0 = respondent chose "Almost Never" for all subscale items.
100 = respondent chose "Very Often" for all subscale items.
Table 5. Comparison of Rasch Mean Subscale Scores of Penn Chemistry Students: Pre and Post Year 2.*
Subscale |
Time |
n |
Rasch Mean |
SD |
t-value |
df |
p-value |
n2 |
What I do in class |
Pre |
362 |
46.70 |
5.87 |
-2.89 |
610 |
0.004 |
0.01 |
Post Yr 2 |
250 |
48.08 |
5.80 |
|
|
|
|
|
What my teacher does in class |
Pre |
362 |
57.16 |
12.19 |
-1.56 |
576 |
0.120 |
0.004 |
Post Yr 2 |
249 |
58.60 |
10.62 |
|
|
|
|
* The questionnaires were not administered Post Year 1.
0 = respondent chose "Almost Never" for all subscale items.
100 = respondent chose "Very Often" for all subscale items.
MSPinNYC
Comparisons Before and One Year After Teacher Participation in the MSPinNYC Summer 2006 Institutes
Item Response Theory (IRT)-Rasch Model-was used in order to provide the project with rigorous and appropriate analyses. Comparisons were made between Rasch mean scores before the institute and one year after participating as a TRT teacher in an institute (Table 6). Significant differences were found (in favor of more frequent use of inquiry and other standards-based instruction) on both the teaching and learning subscales one year after teacher participation.
Table 6. Comparison of Rasch Mean Scores for Teaching and Learning Subscales for MSPinNYC TRT Teachers: Spring 2006 and Spring 2007.
Subscale |
Time |
n |
Rasch Mean |
SD |
t-value |
df |
p-value |
n2 |
What I do |
Spring2006 (Pre) |
17 |
57.77 |
8.81 |
-2.04 |
16 |
0.058* |
0.21 |
Spring2007 (Post) |
17 |
61.13 |
7.95 |
|
|
|
|
|
What my students do |
Spring2006 (Pre) |
17 |
49.45 |
7.21 |
-2.66 |
16 |
0.017 |
0.31 |
Spring2007 (Post) |
17 |
53.31 |
6.23 |
|
|
|
|
* Using alpha = 0.1.
0 = respondent chose "Almost Never" for all subscale items.
100 = respondent chose "Very Often" for all subscale items.
Students in TRT teachers' classes completed the student questionnairepre and post teacher participation in MSPinNYC. Four different institutes were offered in 2006 and the findings for the learning subscales are displayed in Table 7 by specific institute/class.
Table 7. Comparison of Rasch Mean Scores for Learning Subscale for Students of MSPinNYC TRT Teachers: Spring 2006 and Spring 2007.
Subscale |
Time |
n |
Rasch Mean |
SD |
t-value |
df |
p-value |
n2 |
Mathematics A |
Spring2006 (Pre) |
59 |
46.38 |
2.27 |
-2.36 |
130 |
0.020 |
0.04 |
Spring2007 (Post) |
73 |
47.50 |
3.01 |
|
|
|
|
|
Mathematics B |
Spring2006 (Pre) |
37 |
46.32 |
2.39 |
-3.13 |
91 |
0.002 |
0.10 |
Spring2007 (Post) |
56 |
47.99 |
2.59 |
|
|
|
|
|
Living Environment |
Spring2006 (Pre) |
99 |
45.77 |
2.64 |
1.72 |
182 |
0.088* |
0.02 |
Spring2007 (Post) |
85 |
45.01 |
3.35 |
|
|
|
|
|
Chemistry |
Spring2006 (Pre) |
68 |
44.97 |
3.17 |
-2.19 |
138 |
0.030 |
0.03 |
Spring2007 (Post) |
73 |
46.12 |
3.00 |
|
|
|
|
* Using alpha = 0.1.
0 = respondent chose "Almost Never" for all subscale items.
100 = respondent chose "Very Often" for all subscale items.
Mathematics A and B and chemistry students indicated significantly more use of problem solving and other active learning strategies on the learning subscale after their teacher participated in an MSPinNYC institute. However, for students in Living Environment classes the reverse was true.
Comparison between Participant and Non-Participant Teachers in MSPinNYC Spring 2007
Comparisons also were made between the responses of the two groups of teachers and those of their students. No statistically significant differences were found for any of the subscale comparisons on the responses of teachers who participated in the Summer 2006 institutes (TRT team members) and non-participant Hub school colleagues. However, significant differences were found in comparing responses of the two groups of students (Table 8).
Table 8. Comparison of Rasch Mean Scores for the Learning Subscale for Students of MSPinNYC Participant and Non-Participant Teachers: Spring 2007.
Subscale |
Time |
n |
Rasch Mean |
SD |
t-value |
df |
p-value |
n2 |
Mathematics A |
Participants |
73 |
47.50 |
3.01 |
1.82 |
179 |
0.071* |
0.02 |
Non-Participants |
110 |
46.64 |
3.17 |
|
|
|
|
|
Mathematics B |
Participants |
56 |
47.99 |
2.59 |
2.94 |
97 |
0.004 |
0.08 |
Non-Participants |
43 |
46.34 |
2.97 |
|
|
|
|
|
Living Environment |
Participants |
86 |
45.09 |
3.40 |
-1.65 |
322 |
0.101 |
0.01 |
Non-Participants |
238 |
45.72 |
2.96 |
|
|
|
|
|
Chemistry |
Participants |
98 |
46.68 |
3.03 |
2.31 |
121 |
0.023 |
0.04 |
Non-Participants |
25 |
45.05 |
3.56 |
|
|
|
|
* Using alpha = 0.1.
0 = respondent chose "Almost Never" for all subscale items.
100 = respondent chose "Very Often" for all subscale items.