Performance Data PBL vs. Traditional

When making changes to our instruction, educators need to know if these changes are leading them in the right direction.  Sometimes this data is hard to come by. But, responsible educators need to look for any signs they can face going the wrong way too fast.

The 2013-14 school year was my initial implementation of project based learning in the classroom.  I have lots of anecdotal and qualitative observations of student achievement that tell me I am going the right way with my curriculum redesign.  It is still important, though, to have some quantitative data to back up my decision to stay the course with PBL.

Following the completion of term 3, I administered a test to 51 students in the project based setting and 29 students in the traditional setting.  The test measured mastery of 7 key physics objectives taught during term 3.  The test was given 3 weeks into term 4.  Students were given no prior warning to the term.  This means there was no chance to prepare for the test by any student.  The test was intended to measure retention of knowledge rather than studying that could have been done prior.  The table below shows percentages of students who proved mastery of each objective.  The table also indicates the percent of students who were proficient or advanced in a particular objective.

	Objective 1	Objective 2	Objective 3	Objective 4	Objective 5	Objective 6	Objective 7
Traditional% Mastery	93	31	69	20	97	66	79
Traditional% Proficient or Advanced	86	17	45	0	66	34	38
Project Based % Mastery	96	76	78	39	90	67	92
Project Based % Proficient or Advanced	57	53	53	2	76	35	61

The table indicates that in most areas, the project based students showed slightly higher levels of mastery and proficiency.  It is a bit disconcerting to realize the low level of mastery for objective 4 (Newton’s 2nd Law of Motion).  

It’s tough to gain a full picture of the gains of the project based students by taking a written test.  The goal of the curriculum was to help students apply knowledge.  So, perhaps in the future I could design a more authentic form of assessment and administer it to traditional and nontraditional settings.  It is encouraging to see that students in the project based setting did not show significant gaps when compared to students in the traditional setting in terms of content acquisition or test taking skills.

The following data represents testing done at the end of term 4 without prior notice So again, no time to study in preparation.  Also, no formulas given for any calculations.  The assessment included a total of 13 objectives from both the first term of the course and objectives from the final term of the course (some objectives were omitted because they did not lend themselves to this form of testing).

Objective	1	2	3	4	5	6	7	8	9	10	11	12	13
Traditional% Mastery	92	26	67	21	80	62	67	63	67	79	37	88	83
Traditional% Proficient or Advanced	71	13	46	0	67	33	29	42	38	62	8	55	62
Project Based % Mastery	94	61	69	59	84	80	86	88	82	92	51	73	73
Project Based % Proficient or Advanced	84	33	33	16	47	51	53	67	59	75	20	59	39

In the first 12 objectives tested, one can see that the project based students showed similar or greater mastery than those students in the traditional setting.  The one objective where we see the traditional course scores are significantly higher is the last unit of the year.  This can be attributed to the fact that in the project based classroom, we ran into a time crunch at the end of the year.  Because of this we were not able to delve into the depth of a full project.  In the project based course, we investigate each topic in more depth which requires more time, so we had less time to deal with the topics in our final unit when compared to the traditional setting.  Now that I have had the opportunity to teach the course all the way through, I believe that this issue can be resolved in the future with a keener eye to pacing.

At the end of this school year, I administered the same final exam to my 2013-14 honors physics students that was administered to the 2014-15 honors students.  The table that follows reports the mean score for both schools years.  I feel that it is not entirely valid to make a direct comparison between the two scores because of the many variables that surrounded the testing.  In the 2013-14 school year, this exam was 20% of the student’s grade and we reviewed concepts in class for two days prior to the exam.  In addition students were assigned an exam review packet to help prepare.  In the 2014-15 school year, this assessment was not a part of the student grade.  For this year’s students, they completed a portfolio rather than a paper and pencil final exam.  In addition, there was no time in class focused on a review.  This time was dedicated to work on the student portfolio.

Having said this, I think the results are quite telling about the learning going on in the project based setting.  I took the curve from the 2013-14 exam and applied it to the test taken by the honors physics student in the 2014-15 school year.  The fact that there is only a 6% difference in the scores shocks me. These results show that the know the material without having to study.  The knowledge has been learned not simply memorized.  The PBL students were given no time in class to review or forewarning about this assessment as opposed to the traditional students who were given dedicated time to study and review materials.  Just as important though, the students took this opportunity seriously as a way to defend the project based learning experience.  They knew that the results would be used as evidence to support the use of PBL in the classroom.  Their excellent performance on the assessment show that the students take pride in their work.  I can help but feel the results are a validation of the PBL method I have used this year.

Results

Year	Mean Score
2013-14 (traditional)	92%
2014-15 (Project based)	86%