Stillwater MN


Research Study - Seven Years Later

Stillwater MN 7 Years into laptop study Comparison of 1:1 and 3:1 Programs

Purpose & Objectives
Research conducted by the Center for Applied Research and Educational Improvement, University of Minnesota.
This research project was a yearlong study completed fall of 2009 of two very similar junior high schools in the same district. Both were very identical in many respects other than one was in it’s 5th year of a highly effective 1:1 program and the other in it’s 5th year of a highly effective 3:1 program.
Both schools serve similar demographics, have identical staffing, teachers who received identical technology training at the start of the two programs, use the same hardware and software and receive similar levels of support from instructional and technical staff.
This research sought to measure and describe differences between these two program across a number of measures and study groups: students, teachers and parents. It examined several years of standardized test data, instructional skills by teachers, frequency of higher order thinking and a number of other elements as follows:
Teaching: instructional behaviors (preparation and teaching), content of the curriculum, facilitation of students’ higher order thinking, training and continuing professional development

Student Outcomes: student-teacher and student-student communication, opinions about the use of laptops, on-task classroom behavior and student engagement, student achievement

Parents: how parents assist their children with technology-based schoolwork, frequency and desirability of accessing students’ homework assignments and grades, changes in parent-child conversations as a result of greater access to technology
The school district has used and intends to use the findings from this study to inform their decision making regarding the future of these programs and considerations that could merit their expansion to other locations in the district.

Perspective/Theoretical Framework
This research used a combination of three standardized and widely used observation protocols for many classroom observations: a rubric developed by Newmann, Secada, & Wehlage, (1995) to rate the level of authentic instruction observed. Second included included sections from the Inside the Classroom: Observation and Analytic Profile, (Horizon Research Inc., 2000). The third observation protocol included three items from the 11-item Integration Observation
Tool section of the Observation Protocol for Technology Integration in the Classroom (OPTIC), (Northwest Regional Education Laboratory, 2004)
Survey collection was modified to best fit the goals of this study and was in based in part on the survey tool used by Silvernail, D.L. and Lane, D.M.M. (February 2004) in The Impact of Maine’s One-to-One Laptop Program on Middle School Teachers and Students from the Maine Education Policy Research Institute, University of Southern Maine.

Research Methods
The study participants were as follows: students and teachers at the 3:1 and the 1:1, parents of students at the 3:1 and the 1:1, and 10th grade students at the high school.

In October 2007 there were 1,016 students enrolled at the 1:1 and 1,084 enrolled at the 3:1. Overall, the student demographics were very similar. The 3:1 had a slightly higher proportion of students enrolled in 9th grade than the1:1 (38% versus 34%). The 1:1 had a slightly higher proportion of students who were of minority race/ethnicity than the 3:1 (10% versus 4%).
The University of Minnesota Center for Applied Research in Educational Improvement (CAREI) developed the content of the teacher survey, student survey, and parent survey in collaboration with district and school staff. Many of the items for the teacher survey and the student survey were drawn directly from or adapted from surveys developed by the Maine Learning Technology Initiative. The items on all three surveys were intentionally written to reflect the context of both the cart model at the 3:1 and the one-to-one model at the1:1. For example, teachers and students were asked to answer some questions based specifically on their behavior when students had access to computers or laptops in the classroom. This specification was necessary because the purpose of the study was to compare teaching and learning in the presence of computers or laptops and these tools were not available every day in classrooms at the3:1. The survey items were also written to reflect the reality that students at the 3:1 may use computers at school in the computer lab, so the surveys used the phrase “computer or laptop” in many areas.
Finally, the items were worded to include both desktop computers and laptops to reflect the fact that students may use either a laptop or a desktop computer at home. The content of each survey was identical for respondents at the 3:1 and respondents at the 1:1.

District staff created an online version of each survey. Technology coordinators at each school added links for the surveys to the online school planners so students, parents, and teachers could access the surveys easily. The district provided CAREI with a spreadsheet of response data for each online survey. Tables were prepared for each survey that showed how the respondents from each school answered each item. A Pearson Chi-Square statistic was calculated for each survey item to determine the probability that any observed differences in how respondents from each school answered the item were due to chance factors rather than differences in respondents’ behavior or perception. In a few cases an independent samples t-test was used in place of a Pearson Chi-Square test when the response scale for the survey item was consistent with the characteristics recommended for use of the t-test statistic.

Rate of Completion for Student Survey. A total of 856 students completed the student survey; 17 students were excluded from the analysis because they did not indicate which school they attended. Of the remaining 839 students, 218 were enrolled at the 3:1 and 621 were enrolled at the1:1. The response rate for 3:1 students was 20.1% in contrast to the response rate of 60.1% for 1:1 students.
Ideally these proportions would be similar to the proportions in the student population as a whole so that the characteristics of the survey respondents mirror the characteristics of students enrolled in each school. At each school, 7th grade students were more likely to complete the survey than their older peers. At the 3:1, 7th grade students made up 53% of the survey respondents but only 30% of the student population. Likewise, at the 1:1 school, 7th grade students comprised 43% of the survey respondents but only 32% of the student population. A higher proportion of 8th grade students completed the survey at the 1:1 than at the 3:1; for the 1:1, the proportion of 8th grade students who completed the survey was similar to the proportion of 8th grade students in the student population. For both schools, the proportion of male and female students who completed the survey was similar to the proportion of male and female students enrolled in the school.
Rate of Completion for Teacher Survey. A total of 68 teachers completed the teacher survey for a response rate of 59%. Twenty-eight of the survey respondents worked at the 3:1 and 40 worked at the 1:1. These numbers represent 47% and 71% of the teaching staff at 3:1 and 1:1, respectively. The 1:1 survey results were more likely to represent all 1:1 teachers because over two-thirds of the teachers completed a survey. Despite the lower rate of completion by 3:1 teachers, survey data indicate that the 3:1 respondent group was not confined to teachers who were high frequency users of the laptop carts. 3:1 teachers were asked to respond to a survey question about how often their students used laptops in their classroom this year. The number of days varied widely among the teachers who completed the survey. Almost one-fourth (23%) of the teachers said students had used the laptops for more than 60 days. In contrast, 19% said their students had used laptops less than 10 days. The wide range in how often students had access to laptops in the classroom implies that the survey results for 3:1 teachers represent teachers who vary widely in their use of the laptop carts.
Rate of Completion for Parent Survey. Overall, 505 parents completed the survey. Eleven of them did not indicate which school their child attended so their surveys are not included in this report. A higher proportion of parents of 1:1 students completed the survey 31% (313) than parents of 3:1 students, 17% (181).
Teacher Interviews

A semi-structured interview protocol was developed by researchers in consultation with district staff. The purpose of the teacher interviews was to gather information on how the laptop initiative might have affected the following: how teachers prepare for and deliver instruction, the content of the curriculum, and teachers’ capacity to facilitate students’ higher-order thinking.
The interview protocol asked teachers to reflect on how much they thought having laptops influenced their instruction in four specific areas: student-centered instruction, active exploration of information, critical thinking, and student collaboration. Each of the first three areas is part of the National Educational Technology Standards for Teachers 2000 (International Society for Technology Education, 2000); the fourth area of student collaboration appears in the 2008 version of the standards. In contrast to the classroom observation protocol, which was designed to measure qualities of the instruction on a particular day, the interview was intended to collect information on how teachers’ practice may have changed since the start of the laptop initiative.

Researchers asked the technology coordinator and principal at each school to compile a list of 6- 8 teachers whom they considered to be strong in terms of their integration of laptops into instruction. Then, the researchers contacted each teacher and invited them to participate in a telephone interview. During the interview, the researchers made written notes of teachers’ responses to each question. Researchers reviewed the notes from each teacher interview to identify themes in the teachers’ responses for each item. Results for 3:1 teachers and 1:1 teachers were compared to determine if any differences existed between the teachers’ experiences at each school.

In May 2008, each school identified 8 teachers for inclusion in the interviews. Twelve of the 16 agreed to participate in an interview; 7 were teachers at 3:1 and 5 were teachers at 1:1.
One notable difference was that the group of 3:1 teachers included a family and consumer sciences teacher and a world languages teacher, whereas the group of 1:1 teachers was comprised of teachers in the core subjects.
Observations of classroom instruction were guided by a protocol comprised of 3 existing tools. The first component was a rubric developed by Newmann, Secada, & Wehlage, (1995) to rate the level of authentic instruction observed. The rubric included four dimensions, as follows:

Higher Order Thinking – Instruction involves students in manipulating information and ideas by synthesizing, generalizing, explaining, hypothesizing, or arriving at conclusions
that produce new meanings and understandings for them.

Deep Knowledge – Instruction addresses central ideas of a discipline or topic with enough thoroughness to explore connections and relationships and to produce relatively complex understandings.

Substantive Conversation - Students engage in extended conversational exchanges with the teacher and/or their peers about subject matter in a way that builds an improved and shared understanding of ideas or topics. 1) The talk is about subject matter in the
discipline and includes higher order thinking. 2) The conversation involves sharing of ideas—participants explain themselves or ask questions in complete sentences, and then they respond directly to comments of previous speakers. 3) The dialogue builds coherently on participants’ ideas to promote improved collective understanding of a
theme or topic.

Connections to the World Beyond the Classroom – Students make connections between substantive knowledge and either public problems or personal experience.

Each dimension was rated on a scale from 1 to 5, with 5 being the highest. Qualitative descriptions for each rating level on each dimension appear in the Appendix.

The second component of the observation protocol included sections from the Inside the Classroom: Observation and Analytic Profile, (Horizon Research Inc., 2000). The sections measured the following aspects of the lesson:

The proportion of students that were physically engaged in the lesson at specific points in the class period; The number of minutes during the lesson spent on instructional learning activities; The number of minutes during the lesson spent on housekeeping or other non-
instructional activities; and The proportion of instructional time spent in whole class work, pairs/small group work, and individual work.
Physical engagement was defined as “eyes/hands/feet appropriately tracking instructional activity—including teacher led, whole groups, small group or individual work.”

Finally, the observation protocol included three items from the 11-item Integration Observation Tool section of the Observation Protocol for Technology Integration in the Classroom (OPTIC), (Northwest Regional Education Laboratory, 2004). The items measured the following aspects of instruction: The degree to which students exhibited collaboration in group activities using technology; the extent to which students are focused on the intended curricular objectives when using technology and; the extent to which the technology use represents learning activities that could not otherwise be easily done.

Each item was rated on a scale from 1 to 5, with 5 being the highest.

Prior to the visits, the principal and technology coordinator at each school compiled a list of teachers in each subject area who they considered to be the most skilled at integrating technology into instruction. Researchers worked with the coordinators to schedule the observation dates at each school to avoid overlap with events such as testing and field trips. In addition, observations at SJHS were scheduled for dates on which the teachers would have a cart of laptops in their classroom. Prior to each scheduled visit, the technology coordinators informed all of the teachers in their building that researchers might visit their classroom to observe instruction on the given day; however, neither the technology coordinator nor the teachers knew which period the researchers would visit any given classroom. Researchers entered the observation data into a spreadsheet and calculated frequencies and percentages for each item on the protocol, by school. Statistical analysis for each item was conducted using the Mann-Whitney test to determine if differences between the schools were statistically significant.
Standardized Achievement Tests

Students’ scores on the Northwest Education Association’s Measures of Academic Progress (MAP) tests in reading and mathematics were used to explore how students’ levels of access to laptops may be related to their levels of learning. The tests are administered by the schools and the data was obtained from the district. The MAP is a computer-adaptive assessment tool that is designed to align with each state’s standards. Since the test is computer-adaptive, there is no “ceiling” score on how high a student can score because the test is designed to continually present more challenging items as the student answers items correctly, therefore providing a more accurate assessment of the student’s actual level of achievement. The MAP uses Rasch Unit (RIT) scale scores as its unit of measurement. The name for the unit of measurement is based on the Rasch model for dichotomous data in item response theory which is the theoretical model of measurement on which computer-adaptive testing is based. This score can be interpreted as a number that indicates a student’s instructional level, independent of grade level. In addition, an individual student’s scores can be compared over time to determine their rate of growth in reading or mathematics achievement.
Focus Group Protocol

In February 2008 CAREI conducted focus groups with six groups of 10th grade students at The High School. Three of the groups included students who had attended the 3:1 and three groups included students who had attended the 1:1. The purpose of the focus groups was to collect information from students who, as junior high students, had experienced three years of the laptop initiative and now had had one semester of experience at the senior high school, which had a notably lower student computer or laptop ratio than either junior high school. Researchers developed a focus group protocol in collaboration with district staff. The protocol contained a set of 11 open-ended questions about how laptops or desktop computers were used in junior and senior high and how that use may have affected the following: students’ interest in a subject, students’ learning in a subject, student communication with teachers and peers, and students’ preparation for future careers.
Researchers and district staff also developed a brief questionnaire for students to complete at the end of the focus group. The questionnaire contained fixed response questions that were more appropriate to a survey format than a focus group format. The questions concerned students’ perceptions about how access to laptops may have affected their grades.

To generate a list of students to invite to the focus groups, district staff first sorted all of the 10th grade students into low, medium, and high achievement level groups based on their scores on the 2007 MCA-II tests in reading and mathematics. Then, within each achievement level group, staff randomly selected 10 students who had previously attended SJHS and 10 students who had previously attended the 1:1. The result was six groups of students, sorted by achievement level and junior high school. District staff told the researchers which sessions corresponded to each achievement level group only after the sessions were completed.

Prior to the focus group date the district mailed a letter to each student’s parents to explain the purpose of the focus groups and inform them that their child would be invited to participate. Parents who did not want their child to participate were asked to contact the researchers. Seven parents requested that their child be excluded and district staff randomly selected a replacement for each student. In total, 60 students were invited to participate in the focus groups: 30 former 3:1 students and 30 former 1:1 students.

The focus groups took place in a classroom or conference room at the high school during the school day. High school staff gave each student a written reminder about the focus groups the day before their scheduled session. At the start of each session the researchers explained the purpose and procedures for the focus group and how students were selected. Then, they asked students to review a written assent form and sign their name if they were willing to participate. At the end of each focus group the researchers gave students the brief written questionnaire to complete. Students did not need to write their name on the questionnaire.

Overall, 23 students out of the 60 that were invited participated in a focus group. Twenty-two students did not show up for their scheduled session and one student declined to participate after the researchers explained the purpose and procedures of the group. Some of the students who did not appear for their session were absent or told high school staff that they could not miss class because they had an exam. All 23 students who participated in the focus groups filled out the questionnaire at the end of the session.

Researchers made an audio recording of each focus group session and took written notes during the session. After each session, researchers typed up the written notes and reviewed the recording to capture any additional relevant information. When the six sessions were completed, researchers examined the notes for all six focus groups to identify themes in students’ responses. In addition to identifying themes across the six groups, researchers also analyzed the notes to determine if there were patterns in students’ responses based on achievement level and the junior high school they attended. Researchers also compiled students’ responses to the brief questionnaire.

Results or Expectations
While the results from this study are quite extensive, for the purposes of this application please see the following summary and discussion:
The overall goal of the study was to collect information about how the laptop initiative may have affected teaching and learning. Several different data collection methods were used to gather data from students and teachers at 3:1 and 1:1, and parents of students at 3:1 and 1:1. The study results detailed numerous ways in which the laptop initiative may have had a positive influence on teaching and learning. The results of the study should not be used, however, as evidence of a causal relationship between the use of computers and laptops and changes in teaching and learning. Many other factors in addition to, or in place of, the use of computers and laptops may have affected the outcomes described in this report. To estimate the potential influence of other factors would have required the inclusion of a comparison school in the study design that was similar to both 3:1 and 1:1, but did not have laptops available to students. This type of design was not feasible within the scope of this study. Nonetheless, the study provides a detailed portrait of teaching and learning in a laptop environment.

When interpreting the study results it is important to keep in mind that data from the surveys and interviews were based on the self-report of students, teachers, and parents. As with any self-report measure in any study, people may have, intentionally or not, provided the response options that they thought would make them or their school look best. To reduce this possibility, teachers, students, and parents in this study completed the surveys anonymously. The evaluation design also included researcher observations of classroom instruction with computers or laptops to serve, in part, as a check against the possible bias of self-report. Although the scope of the information gathered through the classroom observations was not as broad as the scope of the student and teacher surveys or the teacher interviews, a qualitative comparison of the results among these data sources did not reveal any substantive differences that would suggest bias.

Teaching
The results indicated that the laptop initiative contributed to enhancements in the quality of teaching at both 3:1 and 1:1. Through surveys and interviews teachers provided many examples of how their instruction and preparation for instruction had changed in ways that benefited students. Data from researchers’ observations of classroom instruction showed that the quality of instruction was high at both 3:1 and 1:1.

Facilitation of Students’ Higher Order Thinking. Information from multiple sources indicated that the laptop initiative expanded teachers’ capacity to facilitate students’ higher order thinking. On the teacher survey, over three-fourths of the teachers (84%) said access to a computer or laptop contributed some or a lot to students demonstrating more higher-order thinking. In interviews with the researchers, teachers readily described how their instruction had changed to incorporate more learning activities that helped develop students’ critical thinking skills. When the researchers observed instruction that incorporated the laptops they found that the level of higher order thinking was high in both schools. They also found strong use of disciplinary knowledge in the instruction. Students and teachers were addressing the central ideas of a discipline or topic thoroughly and exploring connections and relationships to produce deep understanding.
Instruction. The laptop initiative also supported changes in other areas of teacher practice. Ninety-four percent of teachers said they were better able to access diverse teaching materials and resources when students had access to computers or laptops. Further, 90% of the 1:1 teachers and 81% of the 3:1 teachers agreed with the statement, “When I know that my students will have access to computers or laptops I am better able to individualize my curriculum to fit student needs.” Ninety-one percent of teachers agreed that a laptop helps them to access more up-to-date information for their students, 90% of teachers agreed that through the use of computers or laptops students are able to explore topics in greater depth, and 90% of teachers agreed that having computers or laptops in the classroom helped them create materials that better meet district goals.

The survey results were consistent with data from teacher interviews. During interviews with the researchers, most of the teachers offered examples of how their instruction had changed in the following areas: instruction became more student-centered; students had more opportunity to actively explore information; and instructional content was more up-to-date and interesting for students.

The teachers were less likely to remark on change in how often students collaborated in their learning. For most of the teachers, the laptop initiative seemed to facilitate changes in practice that were consistent with their existing beliefs about effective instruction. The teachers did describe, however, how students’ access to laptops had made it easier to implement instructional strategies they had long valued, such as reducing the amount of time they spent lecturing and instead, asking students to find information and synthesize it to form conclusions.

Data from the teacher survey also demonstrated how the laptop initiative influenced teachers’ instruction and preparation for instruction. Teachers reported that, in the classroom, students were most likely to use computers or laptops to do the following: manage and analyze information; research information using the Internet; and work on short-term assignments/worksheets.

Teachers reported using their laptops most often to do the following: communicate with colleagues, manage student information, communicate with parents and students, and develop instructional materials. Some teachers also acknowledged challenges that can occur when students have access to computers or laptops. Although two-thirds of the teachers agreed with the statement “When we are using computers or laptops there is less classroom management that needs to take place,” one-third disagreed with the statement. When asked if using computers orlaptops in the classroom increased their workload, the teachers’ responses were split nearly half and half between agree (51%) and disagree (49%).

From a more holistic perspective, some interesting ideas emerged about how the introduction of laptops might change instructional practice and teacher beliefs about effective instruction. One surprise to CAREI researchers was that, as one teacher said, even teachers who have a student- centered teaching philosophy can be helped by laptops. There are things laptops facilitate— things even the best teacher may have long wanted to do but were not possible before laptops.

Teachers for the most part believe that laptops increase students’ interest and engagement in learning activities. It is important to remember that the teachers who were invited for interviews and whose classrooms were observed were intentionally selected because they were strong users of technology. A look at teachers as a whole may balance that somewhat with concern about how laptops can be distracting. Nonetheless, in the interviews teachers explained how the use of laptops makes instruction more interesting to students. They have more choices, both in terms of where they look for information and in how they present their work. Teachers can more easily make instruction relevant to students’ lives and current events because they can access up to date information more easily on their own laptops and can send students out via their laptops to get current information. Neither teachers nor students are limited any longer to what’s in the textbook or school library.

Content of the Curriculum. Results from the teacher survey suggested a positive relationship between the laptop initiative and the content of the curriculum. Eighty-four percent of the teachers agreed that they were better able to meet their curriculum goals when students were using computers or laptops. In observations of instruction with laptops at both schools, researchers found that a high proportion of the students were focused on the intended curriculum objectives when using the laptops. In addition, the technology use in the lesson represented learning activities that could not otherwise be easily done. These results provide evidence that the laptops are being used to enhance the curriculum rather than serving as an add-on to the standard curriculum.

Students
The study results also offered evidence of how the use of computers and laptops can benefit student learning. Students and teachers at both schools reported advantages for students that were associated with the laptop initiative.

On-Task Classroom Behavior and Student Engagement. Information from several sources suggested that student engagement was higher when students had access to computers or laptops. On the teacher survey, 90% of the teachers said students show greater engagement in the task when they are using computers or laptops. When researchers observed instruction, they found that students’ physical engagement in the lesson was high at both 15 and 30 minutes into the class period. The survey results also showed that teachers believe students explore topics in greater depth when they have access to computers or laptops.
Another indicator of students’ engagement in learning is how often students use a laptop to learn things beyond what teachers have assigned in class. Fifty-five percent of the 3:1 students and 60% of the 1:1 students said they use the laptop to learn about things not assigned in class. When asked to comment on this use, the majority of the students at both schools described how they used their laptops to pursue their curiosities once their interests were piqued about a topic. This suggests that students’ access to technology could spark or, at minimum, support the development of life-long learning practices and perspectives.

Many students associated positive effects with their use of computers or laptops. For example, 93% of the 3:1 students and 92% of the 1:1 students agreed when asked if they were more likely to revise/edit their work when it was done on a computer or laptop. Ninety percent of the students at both schools agreed that having access to a computer or laptop helped them be better organized.

Opinions about the Use of Laptops. Students’ opinions about the laptops, as measured by their responses on the student survey, were very positive. Many students associated positive effects with their use of computers or laptops. For example, 93% of the 3:1 students and 92% of the 1:1 students agreed when asked if they were more likely to revise/edit their work when it was done on a computer or laptop. Ninety percent of the students at both schools agreed that having access to a computer or laptop helped them be better organized. Only a small percentage of students at each school indicated that they wanted to use laptops in school less than they do now. Similarly, only a small proportion said that they wanted to use a computer or laptop for homework less than they do now.

Communication Between Teacher-Student and Student-Student. Student access to a computer or laptop was more likely to increase their frequency of communication with teachers when students were at home than when students were at school. On the student survey about one-third indicated that they communicated more with their teachers when they had access to a computer or laptop at home. The effect on students’ communication with their peers was similar. About half of the students indicated they communicated more with other students about school projects and assignments when they were at home. When asked about communication with their peers while at school, over one-third reported a higher frequency of communication when they had access to a computer or laptop.

Achievement. With one exception, statistical analyses of the growth in student achievement in reading and mathematics during junior high revealed no statistically significant differences between students at 3:1 and students at 1:1. 3:1 students who began 7th grade in fall 2006 had a higher rate of growth in reading achievement from fall 2006 to fall 2007 than their peers at 1:1. The estimated average score on the reading test for 3:1 students was 219.8 in fall 2006 and 224.4 in fall 2007. In contrast, the estimated average score on the reading test for 1:1 students was 222.1 in fall 2006 and 225.1 in fall 2007. The results suggest that neither the one-to-one model nor the cart model of laptop access detract from students’ performance on standardized assessment measures.
Parents
Results from the parent survey illustrated how parents and their children have benefited from the initiative. One component of the laptop initiative was providing parents and students with the means to access students’ homework assignments/class calendar and grades online and at least three-quarters of the parents surveyed indicated that they used this service. Eighty-five percent of the 3:1 parents and 58% of the 1:1 parents said it was very important for them to have online access to their child’s assignments/class calendar and grades.

Over three-fourths of the parents at each school reported that they had worked on schoolwork with their child using a computer. Most often the parents reported working with their child to edit a paper or research information. This suggests that parents are not discouraged from helping their child with schoolwork, even though it may require knowledge of computers and software. It appears that although there could be some new skills or insights that parents may need to help their student(s) with homework, parents are not opposed to this challenge.

Sixty-one percent of the 3:1 parents and 46% of the 1:1 parents said they spend more time talking with their child about school work now in comparison to previous years when parents could not check students’ assignments/class calendar online. When asked how the availability of online grades affected their communication with their child, 66% of the 3:1 parents and 53% of the 1:1 parents indicated that they now spend more time than before talking with their child about grades. In both cases, the proportion of 3:1 parents who said they now talked with their student more than before was significantly higher than the proportion of 1:1 parents.

Comparisons Between the 1:1 and 3:1 Student-to-Laptop Ratios
Given the different student-to-laptop ratios at 3:1 and 1:1, a secondary focus of the study was to explore potential differences in how the laptop initiative, among other factors, may have affected teaching, learning, and parents at each school. Statistically significant differences occurred in several areas. Teachers at 3:1 were more likely to report that access to a computer or laptop in the classroom contributes a lot to students’ interest in class. 3:1 students were more likely to say that using a computer or laptop, at school or at home, makes schoolwork more enjoyable. As noted earlier, 3:1 students were also more likely than 1:1 students to indicate that they’d like to use laptops in school more and use a computer or laptop for homework more. Forty-four percent of 3:1 students said they do not have enough access time on a computer or laptop at home to work on their homework. A higher proportion of 1:1 teachers agreed with the statement “there is more open communication between students and teachers in classrooms when computers or laptops are in use.” 1:1 teachers were also more likely to agree that students’ access to computers or laptops made it easier for them to individualize the curriculum to fit student needs and more able to cover more material in class. Notable differences occurred as well on the parent survey results. 3:1 parents checked their child’s assignments and grades online significantly more often than 1:1 parents and a higher proportion of 3:1 parents said it was very important for them to have online access to assignments and grades.
Overall, there were relatively few statistically significant differences between the survey results for 3:1 teachers and 1:1 teachers. Given 1:1 students’ relatively unlimited access to a laptop, it was anticipated that 1:1 students would be more likely to use a laptop for writing first drafts of papers or for editing papers. Or, that 1:1 teachers would be more likely than 3:1 teachers to report that their instruction had become more student-centered since the laptop initiative began, or that they had increased the amount of critical thinking and collaborative work students were required to do in their courses. The results did not match these expectations, however.

Data from teacher interviews and classroom observations also did not reveal clear distinctions in teaching and learning between the one-to-one model and the cart model of laptop access. In many areas on the survey, the differences that did occur between the schools were differences in teachers’ level of agreement with a statement rather than differences between agreement and disagreement with a statement. For example, Table 54 (in the full paper) shows how teachers from each school responded to a question about how the use of computers or laptops affects the quality of their students’ work. The most notable difference in how 3:1 and 1:1 teachers responded was the difference in the proportion of teachers that selected “somewhat agree” or “agree” for their answer, rather than the difference in the proportion of teachers that disagreed or agreed, at some level, with the statement.
One complication in attempting to identify clear differences between the one-to-one model and the cart model is that, in some cases, the differences between the schools were in the opposite direction of what would be expected given 1:1 students’ relatively unlimited access to a laptop computer. For example, 3:1 teachers were significantly more likely to report that students published content to the web as part of their ongoing work and significantly more likely to report that students used computers or laptops to create culminating projects to show what they have learned. 3:1 teachers were also significantly more likely to indicate that computer or laptop access in the classroom contributed a lot to students’ interest in class. And, 3:1 students were more significantly likely than 1:1 students to strongly agree with the statement “I am more likely to revise/edit my work when it is done on a computer or laptop.”

Interpreting the school to school comparisons is also complicated by the low survey completion rates, and, in particular, the low completion rates for 3:1 students and parents, relative to the 1:1 completion rates for these surveys. Because 3:1 students don’t take school laptops home, those students and parents who completed the survey may have been more likely to be those that have easy computer and Internet access at home. This difference, in turn, may have made their experiences in the laptop initiative different from the group of 3:1 students and parents as a whole.

On the other hand, potential differences between the one-to-one model and the cart model regarding teaching and learning may have been minimized by the large proportion of 3:1 students who had computer and Internet access at home. Well over three-fourths of the 3:1 students (88%) reported that they have access to a working computer or laptop at home and 91% said that computer or laptop was connected to the Internet. In addition, although the cart model didn’t give 3:1 students access to a laptop beyond the classroom, the 3:1 student-computer ratio in the cart model at 3:1 may not have been sufficiently different from the 1:1 student-computer ratio at 1:1 to produce statistically significant differences in teaching and learning as measured in this study.

As mentioned earlier, the results from interviews and classroom observations with a sample of teachers at each school who were considered strong technology integrators also did not show any differences in teaching between the schools. A closer examination of the combined interview data from both schools, however, revealed some interesting ideas about how the introduction of laptops might change instructional practice and teachers’ beliefs about effective instruction. For example, several teachers suggested that even if a teacher’s beliefs about instruction supported student-centered instruction, student access to laptops could still affect that teacher’s practice by making it easier to put students at the center of learning activities. Teachers also talked about the role shift that occurred in the classroom when teachers are less knowledgeable about a particular software program or laptop function than the students: “It has also brought the students to the teachers. For example, this morning we had a meeting about using a probe, bringing some science equipment into the math classrooms. I said ‘No, I don’t have training on this equipment’ and someone said ‘Go to the students.’” As students step up and help their teachers or other students with laptops and other technology, students and teachers may become more comfortable with students functioning as teachers and this may carry over to other areas of the course. Students may gain confidence in their level of knowledge about laptops and other technology and find satisfaction in taking more initiative for their own learning beyond the area of technology.

Several teachers discussed how powerful it could be for teachers who were reluctant to integrate laptops into their instruction to see their students working on laptops and notice how engaged and creative they were. As noted in research on effective professional development for teachers, seeing how a new tool or strategy affects their students in their classroom is a more persuasive argument for most teachers than hearing another adult tell them “This is a good strategy. You should do more of this.” Although many teachers described how laptops facilitated positive changes in their instruction, the laptop initiative seemed to have less effect on the content of the curriculum. Some teachers commented that they used the laptops less in courses that had more prescriptive content, such as an Advanced Placement course or courses that were closely linked to district or state tests.
The survey completion rates suggest caution in generalizing the results to all students and teachers at 3:1 and 1:1, and all parents of students at 3:1 and 1:1. As noted earlier,
20% of the students at 3:1 and 60% of the students at 1:1 completed the survey. A comparison for each school of the characteristics of students enrolled and students who completed the survey identified some differences between these groups. For both schools, 7th grade students were overrepresented and 9th grade students were underrepresented in the group of respondents. Eighth grade students were also underrepresented in the respondent group at 3:1. As a result, the survey findings may not fully reflect the experiences of all junior high students, especially 8th grade students at 3:1 and 9th grade students at either school. Differences between 1:1 students and 3:1 students in terms of how easily students could access a computer or laptop to complete the online survey may have contributed to the lower response rate at 3:1, and may have influenced the results towards 3:1 students who have more frequent access to a computers or a laptop relative to their peers. Because 1:1 students each have their own laptop they could complete the online survey at almost any time. In contrast, 3:1 students who wanted to complete the survey would have had to come into school early or stay after school to access the school’s computer lab, unless they had sufficient access to a computer or laptop at home that had Internet access at a speed conducive to taking an online survey.

The completion rate for the parent survey was low at both schools; 17% of 3:1 students’ parents and 31% of 1:1 students’ parents completed the survey. As a result, the study results may not fully portray the perspectives of all junior high parents. The survey respondents affiliated with either school may be more likely than their non-respondent peers to have easy access to a computer that is connected to the Internet and this may have influenced their experiences in the laptop initiative, and therefore their survey responses.

The completion rate for the teacher survey was the highest among the three surveys. Almost half of the 3:1 teachers (47%) and over two-thirds of the 1:1 teachers (71%) completed the survey. Nonetheless, caution is needed in generalizing the survey results to all teachers, especially at 3:1. However, the wide range of responses from 3:1 teachers concerning the number of days their students had used laptops in the classroom this year indicated that the group of respondents was not over-represented by teachers who were frequent users of the laptop carts. This suggests that despite a completion rate of less than 50%, the results for 3:1 teachers may represent a range of teacher experiences in the laptop initiative.

In sum, the study data offer numerous examples of positive changes that teachers and students have observed in teaching and learning when computers or laptops are made available to students. Parents and students have also benefited from online access to information about their students’ assignments and grades. The implications of the study data are less obvious, however, when comparisons are made between the one-to-one model and the cart model. Further study may be needed to clarify potential similarities and differences between the models regarding their influence on teaching and learning.

Educational and/or Scientific Importance
We feel this work adds to body of research around 1:1 computing for two primary reasons. First, although there are now more, there are still relatively few full year-long studies that have sought to look at 1:1 programs that have been in existence for five years or more. Second, while not purely empirical, it compares two near identical schools in a same district of which both schools started their programs at the same time, received identical training at the start of their program, received similar support (technical and instructional) since their inception, have a very similar demographic of students and staff, have identical models of computers, yet vary as one had a 1:1 ratio of students to computers while the other has a 3:1 ratio. In many respects if one were to study an environment hypothesizing that there is very little difference between 1:1 and 3:1 access, arguably this study's environment arguably might form that condition, with an inverse that might suggest any measurable difference between two such programs may have more significant implications than one might readily assume.
References
Horizon Research, Inc. (2000). Inside the Classroom: Observation and Analytic Profile, retrieved June 15, 2008 at http://www.horizon-research.com/insidetheclassroom/instruments/obs.php.

International Society for Technology Education. (2000). National Educational Technology Standards for Teachers 2000. Retrieved July 1, 2008 at http://www.iste.org/Content/NavigationMenu/NETS/ForTeachers/2000Standards/NETS_for_Teachers_2000.htm

Newmann, F. M. , Secada, W.G., and Wehlage, G.G. (1995). A Guide to Authentic Instruction and Assessment: Vision, Standards, and Scoring. Madison, WI: Wisconsin Center for Education Research, University of Wisconsin.

Observation Protocol for Technology Integration in the Classroom (OPTIC) developed in 2004 by the Northwest Regional Education Laboratory, retrieved June 15, 2008 at

http://www.netc.org/assessing/home/integration.php
Univeristy of MN gathered data.

Online surveys for students, teachers, parents
Individual interviews with teachers
Classroom observations
Standardized achievement tests
Focus group with sophomores

Access divers materals
Individualiz curriculum
Access up to date info
Create materals that meet district goals
Cover more material
Students explore topics more in depth
Less classroom magagement issues

More inered
Actively involved
Homework completion increases

Disruptive? 88% disagree
68% disagree: had to have backup lessons due to computer problems
51% agree that using computers or laptops in classroom increased my work load

On Task Behavior and Student engagement

Student engaement high at 15 to 30 minutes – no difference between 1:1 and 1:3

Students report:
More inovlced
Do more work
More interested
Dig into topics more deeply

Higher order thinking?

Both 1:1 and 1:3 laotps
Researchers noted that a big part of lessons involved higher order

In 1:1 teachers said:
Notes on computers, send/receive email, more open communication between students and teachers
In 3:1 teachers said:
Students publish content to the web as part of ongoing student work
Students create culminating projects to show what they have learned

Students in a 1;1 say:
Use a computer for writing drafts and final copy at school
More personal instrests included in school subjects
Spend more time in research related to school work and current news events More free to use Internet and explore their interests

1:1 agreed more open communication in classroom: they didn’t have to be the only expert in the classroom
1:1 able to individualize instruction more frequently
1:1 Cover more curriculum

Final overall info in both 1:1 and 3:1:
Students more engaged
Lessons required more higher order thinking skills
Test scores not diminished

In the 1:1 program students’ personal interests were often tied to the curriculum they were involved in at school.

Complete Stillwater Technology Report (pdf)


Evaluation Study of Laptops in NC High Schools


E-mail:
jktingen@ncsu.edu

Purpose & Objectives
In the fall of 2007, the North Carolina State Board of Education awarded a contract to a university-based research institute to evaluate a one-to-one (1:1) pilot learning initiative in a number of North Carolina high schools reaching approximately 2000 students and 200 school staff. Four high schools have been added to the evaluation since the first year of implementation, adding approximately 4000 new students to the study. This paper presents the report on findings from the second year of the project, including:
• Evaluation of progress toward implementation of 1:1 environment
• Summary of barriers, concerns, successes, and lessons learned
• Suggestions for improvement as schools head into the third and final year of the project.
The focus of this evaluation study is two-fold: examine the extent to which the twelve high schools implemented the 1:1 initiative strategies and achieved the 1:1 project objectives; and, inform local and state-decision-makers on future technology policy and funding. To do this, we followed seven Early College High Schools (ECHS) implementing 1:1 environment, seven matched comparison ECHS schools not implementing 1:1 environments, five traditional high schools implementing a 1:1 environment, and matched traditional comparison schools.
Research (Muir, Manchester, & Moulton, 2005; Penuel, 2006; Rockman, 2000) identifies important features that define a successful 1:1 learning environment in schools. These critical features consist of an adequate management and technical infrastructure (including connectivity, a wireless network, hardware and software resources, and school procedures/policies), professional development (PD), technology support personnel, and the use of the laptop by teachers and students leading to changes in instructional practices and student learning outcomes. This prior research informed our evaluation in terms of the data we collected.
A consortium of public and private enterprises provided the framework and logistics necessary for such a large-scale initiative. These organizations teamed up to provide funding for this initiative involving both Early College High Schools (ECHS) and traditional NC high schools. Funding was provided by partners as follows: North Carolina Department of Public Instruction provided state funding for infrastructure, professional development, and evaluation; SAS provided funding for teacher laptops; Golden Leaf Foundation provided funding for student laptops; and The Friday Institute for Educational Innovation supported the 1:1 Learning Collaborative, a resource-rich website and professional development program targeting 1:1 projects.
This school-based evaluation has three broad focus questions by year. In year one, are school leaders, teachers, students, and parents ready to utilize laptops in instruction, and what implementation issues impact their readiness? In year two, is classroom instruction changing? In year three, what are the achievement outcomes of the initiative, and is the environment sustainable?

Perspective/Theoretical Framework
The Joint Committee on Standards for Educational Evaluation (1994) defined program evaluation as “the systematic investigation of the worth or merit of a program” (p. 3). There are many different approaches to program evaluation. Determining which approach to use is dependent upon a number of factors such as the evaluation’s purpose and use, the evaluator’s philosophy and training, and the context of the educational program. Since the focus of this evaluation study is two-fold: 1) examining the extent to which the twelve high schools implemented the 1:1 initiative strategies and achieved the 1:1 project objectives; and, 2) to inform local and state-decision-makers on future technology policy and funding. In as much, this study examining a complex pilot 1:1 Learning Initiative could be categorized as both objective and management-oriented approaches (Fitzpatrick, Sanders, & Worthen, 2003).
Research Methods
A matched-group research design was selected for this evaluation study. The schools participating in the 1:1 pilot were selected prior to the involvement of the researchers and without consideration of any research design. During 2008-2009 school year, the NC 1:1 Learning Initiative involved twelve public high schools in North Carolina. The 1:1 schools are situated in regions across North Carolina, which has a richly diverse geographic and cultural landscape. The 1:1 pilot schools are made up of two distinct groups – the seven ECHS’s and five traditional high schools. Each traditional high school has approximately 1000 students and 100 staff. In contrast, the ECHS have less than 200 students and sometimes as few as six full-time educators on staff. A group of comparison schools, similar in type, size, student demographics, and student achievement on the prior year English and Algebra I End-of-Course tests were selected to provide comparative data from schools not implementing 1:1 programs. Many of the comparison schools had a significant amount of technology available for instructional purposes.
This study utilized a mixed methods process to perform evaluation in order to leverage the benefits of both methodologies (Creswell & Clark, 2007). Quantitative methods utilized matched comparison groups to compare outcomes for students, teachers, and other groups, in terms of end of grade tests, 21st century skills, graduation test scores, and dropout rates. Qualitative methodologies will primarily include focus groups/interviews with school staff, document analysis, analysis of archival data, and classroom observation. Once the research design was defined and goals for the project were identified, specific evaluation questions were developed to align to each of the project goals. Subsequently data sources were either identified or developed to answer each evaluation question
Standard, research-based, appropriate online surveys were administered to administrators, teachers, and students. These surveys were designed to address a number of variables at different levels. The online surveys were made available to the school personnel to administer over a two-week time period for both teacher and student surveys. Non-intrusive site observations occurred at each intervention school. These observations included brief visits to classrooms so that the research team could determine the level of technology usage occurring across study sites. Additionally, during the visits the evaluation project team conducted focus groups with staff. Also, school-level evaluation coordinator, often the technology facilitator, at each intervention school collected various artifacts (e.g., training schedules, technology lab schedules, equipment check-out logs, newsletters, etc.) related to implementation of the 1:1 learning initiative. Finally, the evaluation team examined archival data for all participating schools such as teacher rosters, aggregated survey data collected by NCDPI, and student records (EOC, grades, attendance, etc.) to investigate the effects of the model.

Results or Expectations
In the second year, these schools appear to have implemented the critical building blocks of an effective 1:1 computing environment and are seeing some major shifts in teachers’ and students’ attitudes about technology, increases in staff and students’ technology knowledge and skills, as well as changes in how technology is used in the classroom for teaching and learning. The research team is currently working a cumulative report and will be ready to present results on each of these areas, as well as student learning outcomes at the time of the conference.
The following is information from Year 1 of the study, which highlights results from the initial year of implementation of a 1:1 project at the pilot high schools. Teachers received professional development in important areas and had time to acclimate to their laptops before students received theirs; school infrastructures, policies, and staff were put into place; parents were informed about the 1:1 initiative and agreed to their responsibilities; students appear to have been relatively well prepared to receive and use their laptops; technology facilitators played an important role in helping teachers integrate these new technologies into the instructional life of the classroom; and students and teachers have begun using the computers in a wide variety of ways to support teaching and learning.
With this progress, many lessons have been learned that can inform future work at the 1:1 pilot schools and other schools that may implement 1:1 environments in the future. The largest overall lesson is that laying the groundwork for students to receive their laptops takes time. Preparing teachers to integrate the technology into instruction and preparing all the other essentials for a successful 1:1 initiative (e.g., getting the wireless infrastructure in place, developing the needed policies and preparing for their implementation and enforcement, engaging parents, hiring technology facilitators and technicians, and acquiring the necessary software and hardware to go with the laptops) needs to be done before a school is ready to distribute student laptops. Teachers and administrators agreed it would be best to plan to distribute student computers at the start of a school year, so the planning and preparation have to begin during the prior school year.
Other key lessons from the year one evaluation include:
• Ongoing professional development is imperative. Professional development needs to be continuous and directly meet the needs of teachers as they become more proficient at using the technology to enhance teaching and learning.
• Defining the appropriate balance between student safety, acceptable use, and access to web-based resources is difficult. While very complex, it is also important to find ways to meet student safety needs, set acceptable use requirements, and avoid viruses, spyware, and hacking, without overly limiting what teachers and students can access and do with the computers. Models of how to create the right balance are needed.
• Classroom management strategies and tools require further exploration. Teachers continue to look for guidance on classroom management in 1:1 classes, but have found that the technology solutions offered to them have limitations and, in some cases, did not function as claimed and were not worth using. Further attention needs to be directed to classroom management strategies and how they can be best supported with technological tools.
• Skilled Technology Facilitators play a significant role in the success of technology integration into classroom practices. The important role of onsite technology facilitators to help teachers and students use the technology to improve learning, established in prior research, was once again confirmed.
• Careful short- and long-term budget planning is important to the success and sustainability of the 1:1 initiative. Many resources are needed to support the use of the computers, ranging from displays to printers to specialized equipment for science experiments to content-specific software. Budgets need to be planned to include these resources and their immediate upkeep and support, as well as long-term costs of replacing hardware and supplies (e.g., expensive projector bulbs, ongoing software licenses, replacement of obsolete, damaged laptops).
• Attending to the details makes all the difference. Having ways to plug-in computers and charge batteries, make printer supplies available, establish email class lists for teachers, backup teacher and student machines, respond promptly to technical problems, and address the many other day-to-day needs of making the use of 1:1 laptops go smoothly in classrooms is essential for successful use of the technology to improve student learning.
• Broad-based engagement of key stakeholders can facilitate sustainability of the 1:1 initiative. It is important for schools to engage representatives from the school, district, college partners, business partners, and the community to help inform planning; guide decision-making; provide support to the students, teachers, staff and administrators; and support the sustainability of the 1:1 initiative.

Educational and/or Scientific Importance
The educational significance of this study is twofold, where the results from this evaluation can be used to determine important project strategies and outcomes for future 1:1 learning initiatives, and lessons learned about conducting evaluation in schools can be used to inform evaluation practices for future educational technology projects.
References
Creswell, J. W., & Clark, V. L. P. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage Publications.
Fitzpatrick, J., Sanders, J., & Worthen, B. (2004). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). New York: Addison, Wesley, Longman.
Joint Committee on Evaluation Standards for Educational Evaluation. (1994). The program evaluation standards (2nd ed.). Thousand Oaks, CA: Sage
Muir, M., Manchester, B., & Moulton, J. (2005). Special topic: Learning with laptops. Educational Leadership, 62(Summer)
Penuel, W. (2006). Implementation and effects of 1:1 computing initiatives: A research synthesis. Journal of Research on Technology in Education, 38(3), 329-348.
Rockman, S. (2000).A more complex picture: Laptop use and impact in the context of changing home and school access. Retrieved July 5, 2007, from download.microsoft.com/download/8/d/c/8dc3ebfe-6849-4534-a4b7-846a8c327874/Research3Summary.doc