VITAL: Early Childhood Mathematics Education | NSF Grant Application: Project Description

NSF Grant Application: Project Description

Download PDF (155 kb)
Submitted: September 2003

Video Interactions for Teaching and Learning (VITAL):
A Learning Environment for Courses in Early Mathematics Education

A consortium based at Columbia University, led jointly by the Columbia Center for New Media Teaching and Learning (CCNMTL) and Teachers College, and including William Paterson University as a partner, proposes the development of a professional resource for preparing and supporting teachers of early childhood mathematics (K-3).

GOALS AND OUTCOMES
The VITAL: Early Childhood Mathematics Education project's main goal is to develop and distribute a resource that will enhance undergraduate- and graduate-level programs in early childhood mathematics education. This in turn will address the national need for improved instruction in early childhood mathematics. The content and methodology of the proposed resource are based on a series of mathematics education courses taught by Prof. Herbert Ginsburg (Co-PI) at Teachers College, Columbia University, and by Prof. Rochelle Kaplan (Investigator) at William Paterson University. These courses employ learning activities using brief video clips (or "cases") to help students (prospective and practicing teachers) in undergraduate and graduate courses analyze the development of young children's mathematical thinking and learning, and critically examine early mathematics instruction. The proposal's specific goals are to enhance and expand this video-based model so that it will be useful for a broader audience--in particular, mathematics education professors and early childhood education professors who do not have background in the psychology of children's mathematical thinking and learning, and who have limited acquaintance with early mathematics education, but who nevertheless are responsible for the preparation and professional development of early childhood mathematics teachers. To make the model accessible and feasible for this broader audience of professors, and hence for their students, the consortium proposes to develop a learning environment, VITAL: Early Childhood Mathematics Education. This resource consists of a curriculum, a digital library, and videos, contained within an online community workspace. Already in prototype, VITAL: Early Childhood Mathematics Education will be completed by CCNMTL, a service enterprise at Columbia University directed by Dr. Frank Moretti (PI). CCNMTL has extensive experience in creating new tools for teaching and learning and in researching their implementation. By the end of the grant period, the resource will be ready to be distributed to teacher-education programs nationwide.

Learning Goals
The VITAL: Early Childhood Mathematics Educationproject's learning goals align with those of the National Council of Teachers of Mathematics, which released its Principles and Standards for School Mathematics in 2000. VITAL's goals are to train prospective teachers to:

Understand from a cognitive developmental psychology perspective how children learn and think about mathematics. The NCTM states: "Teachers of young students--including parents and other caregivers--need to be knowledgeable about the many ways students learn mathematics, and they need to have high expectations for what can be learned during these early years" (NCTM, 2000). This includes, as NCTM suggests, training teachers how to introduce children to formal mathematics principles by building on and extending their "intuitive and informal mathematics knowledge" (NCTM, 2000).
Assess children's mathematical knowledge and plan instructional activities accordingly. NCTM states: "Teachers need to determine what students already know and what they still have to learn. Information from a wide variety of classroom assessments . . . helps teachers plan meaningful tasks that offer support for students whose understandings are not yet complete and helps teachers challenge students who are ready to grapple with new problems and ideas" (NCTM, 2000). NCTM stresses that early childhood educators should be familiar with assessment techniques that are appropriate for young children, such as clinical interviews and observations, rather than group tests.
Develop an evidence-based understanding of effective and developmentally appropriate teaching methods and curricula. NCTM states: "Teachers also must decide what new tasks will challenge students and encourage them to construct strategies that are efficient and accurate and that can be generalized" (NCTM, 2000). This involves developing practical, personal, and disciplined theories of children's mathematics education.
Develop a basic understanding of key mathematical concepts. Effective mathematics education, even at the youngest age levels, requires teachers to understand key mathematical concepts. This involves knowing and appreciating the mathematics that children must learn. Early education deals with big mathematical ideas, like cardinal number or pattern, and teachers need to understand them.

Short-Term Outcomes
The VITAL: Early Childhood Mathematics Education consortium anticipates the following outcomes during the grant period:

Increased knowledge of developmental and cognitive psychology and its implications for mathematics education among prospective and practicing teachers of young children.
Increased accessibility to high-quality case-based video material for students of early childhood mathematics education.
Improved opportunities for teaching early childhood mathematics among education professors in a broad range of teacher training programs.
Increased current understanding of successful case-based learning activities and instructional interventions that foster and support undergraduate and graduate students' understanding and practice of early mathematics education.

Long-Term Outcomes
The VITAL: Early Childhood Mathematics Education consortium anticipates the following long-term outcomes from the project:

An increased number of qualified early childhood mathematics educators.
Improved mathematics learning opportunities for young children across the nation.
Project findings incorporated into research projects in other STM content areas.

ANTICIPATED PRODUCTS
The most important product will be an integrated online learning environment that enhances undergraduate and graduate students' preparation in early mathematics education. This resource will be based on the design research and prototype development of VITAL: Early Childhood Mathematics Education during the 2002-2003 academic year by CCNMTL in collaboration with Prof. Ginsburg. VITAL: Early Childhood Mathematics Education will consist of three distinct yet tightly integrated resources that provide the basis for both the undergraduate and graduate courses:

The VITAL: Early Childhood Mathematics Education curriculum plan includes the project's statement of philosophy, pedagogical rationale, and history; describes the sequence of presentations, classroom interactions, and homework assignments; and outlines strategies for the facilitation of these activities.
The VITAL: Early Childhood Mathematics Education digital library that contains all the primary source materials--video cases, expert commentary, and scholarly commentary in the form of monographs--that students will use during the activities specified in the curriculum plan.
The VITAL: Early Childhood Mathematics Education online community workspace, in which students complete assignments from the curriculum plan by editing online video segments, annotating their selections, and using these collected resources as a dataset to compose their multimedia essays. These essays then become accessible online to their instructor and classmates and thus form a foundation for critique and exchange.

This integrated learning environment has already been prototyped and was deployed in spring 2003 in Prof. Ginsburg's course The Development of Mathematical Thinking. Thirty-nine students from several departments at Teachers College, including pre-service and in-service teachers, participated in the course.

**The Existing VITAL: Early Childhood Mathematics Education Prototype: The Point of Departure for New Developments**
The present VITAL: Early Childhood Mathematics Education prototype's curriculum plan stipulates that students spend the first nine weeks of the course examining the developmental and psychological features of children's mathematical knowledge in the context of a series of discrete video-based "cases." During the second half of the semester, students focus on how certain aspects of mathematics instruction, including pedagogical theory and curriculum design, can and should be informed by an understanding of children's mathematical thinking and learning.

Throughout the course, the instructor uses video clips of children solving mathematical problems to illustrate important concepts. Each week, carefully selected video clips pertaining to the class topics are uploaded to the digital library (Appendix A, Figure 1). The digital library is available on the Web, permitting students to view and study the clips at their convenience. More importantly, they are able to do this in an interactive environment that permits them to excerpt and annotate the clips with the goal of creating personalized datasets (Appendix A, Figure 2). After building these datasets, the students are expected to write essays. The essay creation tool allows students to insert links in their essays to specific annotated video segments (Appendix A, Figure 3). This capacity to point to discrete cognitive behaviors in their essays enables students to communicate subtle ideas about the psychology of children's mathematical thinking. Students can then submit their essays online for peer review or critique by the teaching assistants and/or the professor. Students are also provided with a personal homepage that stores all the essays they have worked on (Appendix A, Figure 4).

For their final project, the students create, teach, and videotape their own math lesson or activity in an authentic school environment. A member of the Center's video production staff edits the tapes into smaller segments, digitizes these clips, and then uploads them to the digital library. In the online community workspace, the students then study and annotate the clips of their classroom performance and produce a multimedia essay about their teaching experience (Appendix A, Figure 5).

**The First Full Version of VITAL: Early Childhood Mathematics Education**
Based on an assessment of the prototype's performance in the spring 2003 semester, the CCNMTL design research team proposes a number of significant additions and enhancements to VITAL: Early Childhood Mathematics Education:

FROM PROTOTYPE TO ENTERPRISE SOLUTION
The prototype will be re-engineered to accommodate at least 1,000 classes simultaneously and permit customization by individual instructors. Instructors from around the country will be able to modify the online community workspace by, for example, adding and deleting student users, publishing new resources in the digital library, and creating new essay assignments and video lessons. They will be able to make available content that is familiar and appropriate to the local student population.

PEDAGOGICAL ENHANCEMENTS THROUGH INCREASED USER FUNCTIONALITY
Instructors will also be able to provide feedback to their students by annotating submitted essays directly within the online workspace. As with the video clips, students will be able to capture and annotate portions of these text materials and save them in their portfolios as assets. The video annotation tool will also be enhanced.

LIBRARY EXTENSION
The digital library will be expanded to accommodate text materials--e.g., articles, terms, and transcripts of expert commentaries--in addition to videos.

CURRICULUM REVISION
The curriculum plan will be revised to include a detailed course facilitation guide based on the assessment of the spring 2003 course to be completed in fall 2003. In addition to a syllabus describing all the course activities and assignments, each instructor will be provided with a PDF manual explaining VITAL's pedagogical strategies and the rationale behind them.

RATIONALE

Background: National Need and Opportunity
Recently, the learning and teaching of mathematics to young children has become an issue of widespread interest among early childhood educators and mathematics curriculum developers.

Educators and policymakers in the United States have come to recognize that American schools are underserving students in the area of mathematics, and that mathematics needs to be a central component of early childhood education. Children from China, Japan, and Korea outperform their American counterparts in mathematics achievement perhaps as early as kindergarten (Stevenson, Lee, & Stigler, 1986) and certainly by first grade (Stevenson et al., 1990) and then fourth grade (U.S. Department of Education, National Center for Education Statistics, 1997). Furthermore, there are disparities of mathematics achievement within the United States, corresponding to socioeconomic status (SES). Low-SES children--a group comprising a disproportionate number of African-Americans and Latinos (National Center for Children in Poverty, 1996)--show lower levels of academic achievement than do their middle- and upper-SES peers (Natriello, McDill, & Pallas, 1990). Schools in the United States have neglected mathematics in the early childhood years. Research indicates that preschool children arrive with a competent informal understanding of mathematics (Ginsburg, Klein, & Starkey, 1998) and are ready to learn complex mathematics (Greenes, 1999), and that a strong foundation in preschool education can promote learning in later years (Bowman, Donovan, & Burns, 2001). Many studies have indicated that high-quality education in the early grades can enhance later scholastic achievement (Bowman et al., 2001).
In response to these findings, states like Texas are expanding preschool programs, particularly for disadvantaged children. In the Abbott v. Burke rulings, New Jersey's State Supreme Court called for reforms in high-need urban districts, including mandatory full-day preschool. Georgia and New York have adopted a policy of universal preschool education. This approach is consistent with national efforts to implement the "No Child Left Behind" Act (2001) and put programs into place to ensure that all children have equitable high-quality and challenging mathematics opportunities (NCTM, 2000). This effort has created an immediate need to put large numbers of teachers and curricula in place at the early childhood level, often with little groundwork or time for preparation.

Also in response to the current need, the National Council of Teachers of Mathematics and the National Association for the Education of Young Children have collaborated to produce a joint position statement advocating increased attention to early childhood mathematics education (National Association for the Education of Young Children & National Council of Teachers of Mathematics, 2002). This statement emphasizes that teachers need to possess a psychological understanding of children's mathematical learning and thinking, use developmentally appropriate teaching methods based on that understanding, and employ sensitive assessment techniques for determining what children already know and what they need to learn.

Another factor that has affected early mathematics education is the curriculum reform movement, which the National Council of Teachers of Mathematics advocated in its Curriculum and Evaluation Standards for School Mathematics (NCTM, 1989) and supported even more strongly in its current Principles and Standards for School Mathematics (NCTM, 2000). These standards identify specific learning objectives for very young children and have revolutionized the concept of what mathematics education means and when and where it should begin.

As a result of these and other changes to educational policies and standards, many teachers and prospective teachers across the United States are now faced with a mandate to teach mathematics to children at ages 4 and 5 (preschool and kindergarten), and to do so according to research-supported standards. Most teachers and prospective teachers, however, are underprepared for the challenge and teacher education programs have not responded effectively to the task. At the college level, courses in teaching early childhood mathematics are rare. For many years, the early childhood community believed that formal mathematics education was not necessary or even desirable for young children. As a result, education students are required to take many reading and pedagogy courses, but usually only one "math methods" course. In sum, educators and teacher educators have had neither the time nor the resources to adequately prepare to teach mathematics to very young children.

The visionary reforms to mathematics education called for in recent government legislation, court rulings, and national standards will not be realized until early childhood educators and their teachers receive adequate and specialized training to develop their understanding of mathematical concepts and of young children's capacities to learn them.

Framework for Design and Development
The resource development proposed in this project is based on research in three major areas: the developmental and cognitive psychology of mathematical thinking, successful practices for adult learning (particularly as it relates to teacher education), and the principles of instructional design and interactive media.

THE PSYCHOLOGY OF MATHEMATICAL THINKING
The literature on the psychology of mathematical thinking underscores the importance of three themes: the "informal mathematics" that children possess on entrance to school; the ways in which children interpret and understand formal, school mathematics; and the ways in which effective mathematics education can help children to integrate informal understanding of mathematical concepts with the formal mathematics taught in school.

A large and consistent body of research (Ginsburg, Klein, & Starkey, 1998) shows that robust and surprisingly effective mathematical thinking develops in the years before school. Babies can perceive which of two sets has "more" (Antell & Keating, 1983) and may even understand something about addition and subtraction (Wynn, 1992). Young children learn basic principles of counting (Gelman & Gallistel, 1978), ideas about addition and subtraction (Brush, 1978), and calculational methods (Carpenter, Moser, & Romberg, 1982). Naturalistic observation shows that in their everyday play 4- and 5-year-olds exhibit a strong interest in pattern and shape, both in this country (Seo & Ginsburg, 2004) and in Taiwan (Ginsburg, Lin, Ness, & Seo, in press). In brief, children's minds already employ mathematical ideas and methods before the onset of formal education. Teachers cannot teach effectively without understanding this.

A second major theme is that children use their already existing informal knowledge of mathematics as a basis for assimilating school mathematics. Sometimes, this leads to accurate learning, as when they interpret addition in terms of combining sets and counting (Baroody & Dowker, 2003). Sometimes, it leads to a limited approach, as when children interpret the equals sign in terms of informal addition ("equals means get the answer") (Baroody & Ginsburg, 1983). In either case, whether accurate or inaccurate, children's learning of school mathematics is influenced to some degree by what they already know--their informal mathematics (Baroody, 1987). Teachers need to learn that children do not simply learn what is taught in school; instead they filter instruction through their mathematical minds.

A third major item is that effective mathematics education needs to help children integrate their informal (often incomplete and sometimes incorrect) understanding of mathematics with what needs to be learned in school (Resnick, 1989, 1992). For example, teachers need to understand that children begin by thinking that the equals sign means "get the answer" and that special manipulatives, like using the balance scale, may help children to overcome this initial impulse and to expand their interpretation of the equals sign to include notions of equivalence (Seo & Ginsburg, 2003).

In brief, teachers need to be informed by the psychology of mathematical thinking in order to learn what children bring to the learning situation, how they interpret what is taught, and how effective teaching can draw upon children's thinking.

RESEARCH ON ADULT LEARNING
Research on adult learning favors an instructional approach in which learning activities are rooted in real life situations (Bransford, 1999; Lave, 1996; Bruner, 1996). Thus, it is important for teachers to use cases from everyday life as the content of their curricular activities (CTGV, 1993; Shulman, 1992). The case-based format, already popular in legal, medical, and business education, makes it possible for students to learn in a context where theory and practice are genuinely and meaningfully intertwined (Williams, 1992). Increasingly, researchers are assessing the impact of video case-based learning environments in teacher education. Their research shows that students who study within such environments demonstrate an increased capacity for applying important theoretical concepts to their class work (Derry, 2001; Beck, King & Marshall, 2002). The use of video allows educators to capture various layers of information in their original format (images, sound, movement, etc.). And, unlike a narrative description, video presents these layers synchronously so that students can have an integrated perceptual experience similar to that of live observation (Kinzer & Risko, 1998). By engaging in iterative analysis of video cases, students become aware of the need for more advanced understandings of developmental psychology to make sense of what they see.

VITAL: Early Childhood Mathematics Education includes two categories of video cases that are essential for studying early childhood mathematics education. The first category provides students with access to specific episodes in the development of young children's mathematical thinking. Videotaped observations of children playing and clinical interviews of children solving math problems constitute cases that students can use to discuss developmental and cognitive psychology concepts. Such cases help students learn how to interpret child behavior and to develop personally relevant but evidence-based theories of children's thinking and learning.

The second category in VITAL: Early Childhood Mathematics Education provides students with cases of teaching practice, which introduce students to the complexity of classroom interactions. Video clips of mathematics lessons and the interactions between teachers and children provide students with opportunities to apply developmental and mathematical concepts in their analyses of instructional practice. The literature in this area indicates that students studying within a case-based learning environment develop an increased ability to apply relevant theoretical concepts about teaching and learning to their understanding of classroom practices (Derry, 2001; Beck, King & Marshall, 2002).

INSTRUCTIONAL DESIGN AND INTERACTIVE MEDIA
Although video, in and of itself, is effective at communicating the complexities of classroom case studies, digital technologies extend and enhance the way in which students interact with and learn from such cases. Studies that focused on video case-based instructional approaches have found that students learn more effectively when they are given extended time and multiple opportunities to analyze and interpret cases (Flake, 2002; Derry, 2001; Beck, King & Marshall, 2002; CTGV, 1997). Digital video technology, unlike analog videotape, allows students to instantly access specific points within the clip and review them as many times as needed, thus increasing their capacity for serious reflection. And since digital video can be made accessible via the Web, students are able to access the case studies at their convenience.

Observation of children's mathematical behavior or classroom activity should not be considered passive looking. It entails active analysis and conceptualization. In brief, being a good observer involves an objective attitude towards the facts and also a disposition to think carefully about what they mean. The VITAL: Early Childhood Mathematics Education method for teaching this type of skill is to engage students in a series of exercises in which they are asked to view and comment on video cases of children's behavior. Instead of focusing on superficial aspects of behavior such as "child counts" or "child works with manipulatives", they use video annotation tools to identify key aspects of mathematical learning and thinking that underlie the observed behavior.

In addition, new digital communication tools increase opportunities for students and instructors to read, discuss, and comment on each other's analyses. More research is needed to better understand how these tools affect learning. The investigators expect the project to contribute to the growing research base on case-based study and on technology designed to foster it.

In sum, case-based approaches in teacher education show that they provide significant opportunities for teachers to develop into reflective practitioners responsive to current understandings of how children think mathematically. VITAL: Early Childhood Mathematics Education offers a compelling, flexible, and focused context that encourages teachers to perform the following:

Analysis of mathematical thinking as part of the developmental process in early childhood with its links to language, play, and active learning (see Copley, 1999).
Exploration of the content and origins of young children's mathematical ideas in natural settings and real-world contexts (see Arnold et al, 2002; Copley, 1999; Smith, 2001).
In-depth study of the mathematical content that young children can master such as number relations, counting, use of symbols, spatial relations, and logical inference (see Copley, 1999; NCTM, 2000).
Detailed observation and analyses of children's interpretations of mathematical content (see Ginsburg, 1997; Kaplan, et al, 2000).
Examination of how deliberate and systematic teaching and assessment can contribute to the development of mathematical competence, sensible mathematical reasoning, and mastery of conventional concepts and procedures (see Copley, 1999; Fennema & Carpenter, 1996).

VITAL: Early Childhood Mathematics Education extends an education instructor's capacity to provide learning activities and tools for prospective and practicing teachers that shape and encourage analysis and critical thought.

DESIGN AND WORK PLAN (Please refer to the timeline in Appendix B)

Phase I: Design (Year 1: 6/04 - 11/04)
Based on the existing VITAL: Early Childhood Mathematics Education prototype, the completed evaluation of its use in Prof. Ginsburg's spring 2003 class, Development of Mathematical Thinking, and the educational research previously cited, the investigators will design curricular activities for the proposed learning environment. For each activity they will define learning purposes and outcomes, select primary source materials, and devise assessment strategies (7/04 - 9/04). The primary source materials, documented in a content specification, will include: video clips, journal articles, glossaries, lesson plans, and expert commentary. The functionality of the online community workspace will be modified to encourage the analysis and discussion of these materials. Finally, a technical specification will be produced that details the programming tasks entailed by the actual development of such a workspace (12/04).

Phase II: Development (Years 1-3: 12/04 - 8/06)
The development phase will consist of three concurrent activities: course preparation, content production, and software development. The Project Manager at CCNMTL will supervise these activities, including the following milestones: prototype (1/06), alpha version (4/06), beta version (6/06), and final product (9/06).

COURSE PREPARATION (11/04 - 8/05)
To prepare instructors for the challenge of teaching within the VITAL: Early Childhood Mathematics Education framework, Profs. Ginsburg and Kaplan will produce separate graduate- and undergraduate- level curriculum plans. Each plan will include a syllabus designating coverage of key areas of mathematical thinking, mathematics, pedagogy, and K-3 curriculum topics; a target list of video content (e.g., observations of geometric thinking during free play and of a teacher introducing notions of equivalence to second graders); a set of assessment activities (e.g., interview exercises); various types of homework activities and assignments (such as conducting focused analyses of specified video tapes); and general pedagogical strategies for delivering the course.

CONTENT PRODUCTION (11/04 - 5/06)
Several content production processes will be executed simultaneously. The most significant and labor intensive of these will be the creation of new video materials as prescribed by the content specification. Video specialists from CCNMTL will work with Profs. Ginsburg and Kaplan to videotape more than 150 clinical interviews and roughly 20 classroom observations. To ensure that all video footage is of the highest quality, the crew will adhere to professional standards of sound and lighting. Good sound is especially important for these videos: children's sometimes indistinct voices need to be made as clear as possible. As events are recorded, the resulting videos will be logged, captured, edited, exported, and encoded. The final video clips will be placed on Columbia's streaming QuickTime server and references to their location will be entered into the digital library database (11/04 - 5/06).

Meanwhile, a Production Assistant (new hire) at CCNMTL will digitize the texts listed in the content specification. The digitization process will involve several steps: scanning the original document, copy-editing the results, marking-up the text in XML, placing the file on the Web server, and entering a reference to its location into the digital library database (11/04 - 5/06).

As video and text materials are produced, the Rights Coordinator (new position) will ensure that the project is granted clearance to use and disseminate these materials within VITAL: Early Childhood Mathematics Education. Funds will be allocated to the Rights Coordinator to purchase content licenses as needed (11/04 - 12/05).

SOFTWARE DEVELOPMENT (1/05 - 5/06)
A team of software developers at CCNMTL will work from the technical specification to produce the community workspace and digital library. They will begin by constructing the VITAL database, which will eventually contain student assignments, user profile information, and assets from the digital library. Once the database is operational, they will write a series of queries for retrieving its information. The team will then program a set of classes that process, parse, and wrap this information for use in the workspace application.

The workspace application itself will be developed using an object-oriented programming methodology. This development will include the following general tasks: design of the object hierarchy, programming of object classes for the server application, creation of the Web-based client and its interface, and construction of the bridge between server and client applications.

While the programming team is building the interface for the online workspace, the Webmaster will design its graphic appearance. The Webmaster will also build the general project Website which will serve as a centralized access point for all of the VITAL components.

USABILITY TESTING (1/06 - 3/06)
After a year of intensive development, the formative evaluation team will begin to examine how users interact with the prototype. The team will arrange three usability tests over the course of three months. Each test will have at least ten subjects (pre-service teachers from the Teachers College community) and will attempt to answer the following questions: "How easy is it for users to accomplish basic tasks the first time they encounter the prototype? Once users have learned how to use the prototype, how quickly can they perform tasks? When users return to the prototype after a period of not using it, how easily do they reestablish proficiency? How many errors do users make, how severe are these errors, and how easily can they recover from the errors? How pleasant is it to use the prototype?" (Nielsen, 2003). The evaluation team will share the answers to these questions with the learning environment's designers. Based on this data, the designers will make changes to the original specification. The programmers will then work from the new specification to develop an alpha version of the software.

QUALITY ASSURANCE AND CONTENT REVIEW (6/06 - 8/08)
Once the beta version of the software has been completed, a team of CCNMTL student employees will begin to test the software for bugs and review the content for mistakes. All errors will be logged in a bug database and resolved by the development staff as they are discovered.

An external and formal review of both the software and curriculum will be conducted by Prof. Arthur Baroody of the University of Illinois. The review will ensure the accuracy of VITAL's content, the appropriateness of its pedagogy, and its general suitability for the intended audience.

Phase III: Implementation and Evaluation (Years 3-4: 9/06 - 11/08)
After it has been developed and reviewed, VITAL: Early Childhood Mathematics Education will be piloted in two mathematics education courses. In the fall 2006 semester, Prof. Ginsburg will teach a course on the development of mathematical thinking to 40 graduate students (pre-service and practicing teachers) at Teachers College and Prof. Kaplan will teach a similar course to 40 undergraduates (pre-service) at William Paterson University. The evaluation team will work with consultants from the Center for Children and Technology (CCT) to produce a formative evaluation of the pilot (9/06 - 3/07). The evaluation will determine whether the learning environment, as it has been developed up to that point, is successful at helping students achieve the learning goals described in the Goals and Outcomes section.

Based on the formative evaluation, the design team will modify features of the environment and revise both the curriculum plan and the functional specification to reflect these changes (1/07 - 3/07). The programmers will work from the new specification to produce an updated version of the environment for the field tests (4/07 - 1/08). At the same time, Prof. Ginsburg will work with the field test instructors to finalize arrangements at the field test sites (11/07 - 1/08). They will conduct a one-day orientation conference in which the instructors will learn about pedagogical and technical issues related to VITAL.

VITAL: Early Childhood Mathematics Education will be field tested in at least three courses simultaneously (1/08 - 5/08). The locations of the courses will be varied so that the field tests represent a national demographic. The field tests will be supervised by Prof. Ginsburg.

A summative evaluation of the field tests will be performed by researchers at CCT. For a detailed explanation and timeline of their evaluation activities, please refer to the Project Evaluation section. After receiving the evaluation results, the investigators will make final revisions to the learning environment. VITAL: Early Childhood Mathematics Education will then be examined one last time by the external reviewer, Prof. Baroody.

Phase V: Dissemination (Year 5: 12/08 - 5/09)
During the final six months of the grant, the investigators will attend conferences and workshops in order to raise awareness of VITAL among leaders in mathematics education. They will also work with technologists at Sapient Corporation to produce a scalable version of VITAL and make it available to educators throughout the country. See the Dissemination section for more details on these activities.

PROJECT EVALUATION (Please refer to the timeline in Appendix C)
The essential purpose of the evaluation is to determine whether VITAL: Early Childhood Mathematics Education helps students to apply developmental psychology theories of mathematical thinking to cases and through this develop effective teaching skills in early childhood mathematics. The central research question is whether VITAL: Early Childhood Mathematics Education, with its combination of synchronous and asynchronous components (with specific interest in the online community workspace) provides students with enhanced learning opportunities that extend their capacity for understanding and effective practice.

One aspect of the VITAL: Early Childhood Mathematics Education approach to be studied and assessed is related to the design, development, and use of digital video cases and their impact on students' understanding of content and methodological issues. Video allows viewers to examine interactions repeatedly and in slow motion, permitting close scrutiny of interactions. The project evaluation will determine whether digital video cases as used in the context of the VITAL: Early Childhood Mathematics Education online community workspace are an effective tool for improving early childhood mathematics instruction. The evaluation will also inform the VITAL: Early Childhood Mathematics Education implementation strategies for different settings. In order to support a broad-based dissemination, it is important to assess how the VITAL: Early Childhood Mathematics Education instructional strategies can be modified for different educational settings. This will allow the project investigators to adjust the VITAL: Early Childhood Mathematics Education curricular plans for different contexts.

The Center for Children and Technology (CCT), which is part of the nonprofit Educational Development Center, Inc., will supervise the formative evaluation and conduct the summative evaluation. CCT has provided external evaluations for a wide array of educational technology programs, undertaken formative evaluations of pilot projects, developed embedded evaluation procedures for ongoing internal project assessments, and tested evaluation methodologies and instruments, all using quantitative and qualitative methods designed specifically for each project's needs and goals.

During the first three years of the project, while the curriculum and the learning environment are being designed and produced, CCT will conduct formative research to determine whether the environment, as interpreted by users, reflects the goals of the project. During the last two years of the project, CCT will conduct a summative evaluation based on the field test of the learning environment.

The evaluation is based on a theory-driven model. Theory-based evaluation is "an approach to evaluation that requires surfacing the assumptions on which the program is based in considerable detail: what activities are being conducted, what effect each particular activity will have, what the program does next, what the expected response is, what happen next, and so on, to the expected outcomes. The evaluation then follows each step in the sequence to see whether the expected ministeps actually materialize... It seeks to find out whether the theories on which the program is based are realized in action" (Birckmayer & Hirschon Weiss, 2000).

Since this project involves innovation and the development of a new model for teaching, the purpose of the formative and summative aspects of the evaluation is to provide a theory of action of how education students use a digital learning environment to learn about mathematical thinking and early mathematics education.

The formative evaluation will assess the development of the project and its activities, focusing on three major levels--design, classroom, and cognitive--and the relations among them. These areas represent what Rogoff calls "critical aspects" involved in analyzing learning environments (Rogoff, B. in Collins, A.; Joseph, D. & Bielaczyc, K. In press).

Design level: CCT will focus on assessing the opportunities the system creates for students and faculty activities versus the constraints (if any) it imposes. The evaluators will gather information related to how students and faculty interact with the system, the nature of these interactions to support their teaching and learning purposes, and the relationship between these interactions and different features in the system.

Classroom level: CCT will focus on assessing effective teaching strategies in relationship to VITAL: Early Childhood Mathematics Education.

Cognitive level (focused on students' learning processes and outcomes): One desired outcome in the applications of the VITAL: Early Childhood Mathematics Education system is to help teachers develop and integrate psychological, methodological, and pedagogical understandings of mathematics education. CCT will examine whether and how students developed this integrated understanding throughout the implementation of the project.

The assessment approach is grounded in Hierarchical Schema Theory (Derry et al., 2001), which claims that students should develop during a course in at least four different ways.

The acquisition of new concepts, independent of how those concepts are activated and applied across contexts. Such acquisition is demonstrated if a student is able to define or use a concept correctly when expressly told to do. If a student develops only this level of performance, she has merely acquired unusable knowledge.
The course should increase students' general propensity to activate ideas derived from the learning sciences as frameworks for thinking about instructional situations.
The activated ideas should be the most relevant and important ones for analyzing particular situations.
Student teachers should integrate ideas to construct coherent theoretical interpretations of situations.

Years 1-2: During the first year, CCT will consult with project staff about conducting and analyzing ongoing formative development of components of VITAL in production. During the second year, CCT will work closely with the designers to make sure that the implementation of the environments allows for efficient use by students with different degrees of experience in the use of digital media. To that end, CCT will:

Administer a short survey to students and instructors in elementary school math courses to identify potential users with different degrees of familiarity with digital learning environments, and
Conduct think-aloud sessions with users who indicate different degrees of familiarity with digital media on the survey to test usability of the various components of the environment for different users.

Year 3: During the third year of the project, CCT will develop instruments to test the educational impact of the learning environment. CCT will try out different video-analysis assignments to see whether performance on them corresponds to other, more traditional documentation of student learning. To that end, CCT will ask a sample of students in two pilot-classes, one taught at Columbia University in New York and one at William Paterson College in New Jersey, each consisting of approximately 40 students, to:

Identify instances of mathematical reasoning in video segments;
Suggest the next question designed to elicit diagnostic assessment information based on watching a video segment of a child doing a math activity;
Discuss example lesson plans with an emphasis on content appropriateness and assessment opportunities; and
Discuss criteria for evaluating teaching materials.

Years 4-5: During the fourth and fifth years of the project, CCT will conduct a summative evaluation in five sites, using the instruments developed in the second year. CCT will compare learning outcomes of students in classes using VITAL: Early Childhood Mathematics Education with those of classes covering a similar curriculum, but without the use of the digital learning environment. CCT anticipates that the five sites will include between 150 and 200 students and that at least three of the sites will use VITAL: Early Childhood Mathematics Education.

The following chart represents the conceptual design of the summative evaluation:

Goals	Research Questions	Indicators	Data Sources
1. Understand from a cognitive developmental psychology perspective how children learn and think about mathematics.	1. Does guided examination of children's mathematical thinking portrayed in video segments facilitate a psychological understanding of children's mathematical cognition in pre-service education students?	The ability to analyze and describe mathematical thinking in children seen in video segments, using appropriate psychological terminology and theoretical concepts.	1.1. Content analysis of student essays about video segments not previously introduced or discussed in the course. 1.2. Interviews with high and low VITAL: Early Childhood Mathematics Education users about the relationship between reading theory and the learning environment to determine benefits and barriers of the learning environment.
2. Assess children's mathematical knowledge and plan instructional activities accordingly.	2. Are pre-service education students' effectively learning observation and clinical interview methods for assessment purposes through the use of video segments in the learning environment?	The ability to articulate appropriate clinical interview questions that permit diagnosis of the child's reasoning after viewing short video segments of a child's mathematical activity.	2. Pre-post comparison of students' on-line suggestions for the next diagnostic assessment question to be asked of a child in a test video and relate their choice to a theoretical rationale. Comparison of VITAL: Early Childhood Mathematics Education and regular classes.
3. Develop an evidence-based understanding of effective and developmentally appropriate teaching methods and curricula.	3.Does the video-based learning environment facilitate an evidence-based and developmentally differentiated understanding of teaching methods among pre-service education students?	Can pre-service education students evaluate curricular materials for teaching K-3 mathematics as a result of their experience with the video-based learning environment? The ability to articulate and apply criteria (based on theoretical notions, knowledge of mathematical concepts and appropriateness of curricular materials) to the evaluation of their own pedagogical practice. The ability to articulate appropriate evaluative criteria for judging a set of curricular materials for K-3 classrooms.	3.1 Content analysis of students' essays evaluating their own lesson plans for a particular topic and grade, with emphasis on the uses of diagnostic evidence to shape differentiated activities for children at different stages of mathematical thinking. Comparison of VITAL: Early Childhood Mathematics Education and regular classes. 3.2. Pre-post analysis of student ranking of a range of selected teaching materials for K-3 classrooms. Comparison of VITAL: Early Childhood Mathematics Education and control classes.
4. Develop a basic understanding of key mathematical concepts.	4. Is pre-service education students' knowledge of basic mathematical concepts enhanced by their experience with the video-based learning environment?	The ability to answer short tests questions of basic mathematical concept administered throughout the semester correctly.	4.1 Analysis of student responses to periodic quizzes about mathematical concepts. 4.2. Pre-post test of mathematical concepts.

Formative Data Analysis
CCT will participate in design meetings and consult with producers on how to conduct and analyze formative tests on materials still in production, from concept tests to usability tests. During the second year, CCT will conduct independent formative testing and report to the project team on an ongoing basis, both informally during curriculum and design meetings and formally through monthly reports.

During the fourth year CCT will conduct formative assessment to understand different levels of VITAL: Early Childhood Mathematics Education implementation. For example, through the use indicators to differentiate between full and partial implementation of the model (people who used it as intended and those who didn't) this particular assessment will identify and interpret possible problems or innovative strategies in the implementation of the project that may have an impact in students' outcomes. The purpose is to document these strategies and use them to revise the Instructor's Guide.

Summative Data Analysis
1.1. Criteria for the content analysis of student video essays will be developed in conjunction with project experts and consist of a list of specific indicators of theoretical comprehension, accurate observation and appropriate instructional strategies. Student work products will be coded by two independent judges and inter-rater reliability will be determined. CCT will compare the performance of students in classes using VITAL: Early Childhood Mathematics Education at the beginning and the end of each field trial semester.
1.2. CCT will track VITAL: Early Childhood Mathematics Education use among students electronically and code the interviews according to the types of benefits and limitations described by students to see whether there is a relationship between frequency of use and depth of theoretical understanding.
2.0. CCT will administer a video-based test at the beginning and end of the semester, in which students are asked to select among suggestions for a question they might ask a child at a particular moment in a video and select a rationale for their question from a list of theoretical concepts. The selection of questions and the selection of matching theoretical concepts will be scored separately. CCT will compare the relative gain in scores between VITAL: Early Childhood Mathematics Education and regular classes to determine whether VITAL: Early Childhood Mathematics Education students have gained substantially more ability to select good questions and relate them to theoretical concepts during the course.
3.1. CCT will code essays evaluating students' own lesson plans using a set of criteria developed by domain experts. CCT will compare the results for VITAL: Early Childhood Mathematics Education and control classes to determine whether students using the learning environment are better able to evaluate their own lessons.
3.2. CCT will ask experts to rank a set of teaching materials according to a set of criteria developed by project staff. CCT will then ask students to rank these materials and explain their ranking by selecting among the set of evaluative criteria articulated by the experts. CCT will ask them to evaluate similar materials at the beginning and the end of the semester. CCT will compare the results for VITAL: Early Childhood Mathematics Education users and students in regular classes to determine whether the use of VITAL: Early Childhood Mathematics Education helps students apply theoretical concepts to decision-making about teaching materials.
4.1. CCT will track student performance on periodic short test of mathematical concepts and compare results for high and low VITAL: Early Childhood Mathematics Education users to determine whether the use of the digital environment and its focus on clinical interview skills has any effect on the learning of mathematical concepts.
4.2. CCT will administer a pretest of basic mathematical concepts at the beginning of the semester and again at the end. CCT will compare results for the VITAL: Early Childhood Mathematics Education and regular classes to determine whether the emphasis on observation and clinical interview techniques has any impact on learning basic mathematical concepts.

External Content Review
Professor Arthur Baroody of the University of Illinois, who is a leading expert in the cognitive development research on early mathematics and in early mathematics education, will conduct the external and formal content revision of VITAL: Early Childhood Mathematics Education. Prof. Baroody will review both the software and the curriculum at two key points in the development and evaluation of VITAL: Early Childhood Mathematics Education: In Year 3, the first external revision will be conducted before the pilot testing to ensure accuracy of content and appropriateness of pedagogical strategies before introducing students to VITAL: Early Childhood Mathematics Education. In Year 5, the second external revision will be conducted after the recommendations for improvement that emerge from the field test have been made and VITAL: Early Childhood Mathematics Education is ready for dissemination.

DISSEMINATION
In order to maximize the impact and usability of VITAL: Early Childhood Mathematics Education at a critical point in the teacher professional continuum, the consortium will partner with a commercial technology services company for the dissemination phase. This company will provide both the supporting infrastructure and high-quality user-support services in conjunction with the core VITAL system and the VITAL: Early Childhood Mathematics Education consortium's teacher-education expertise to facilitate rapid adoption by as large a community as possible.

Sapient Corporation, a U.S-based business and technology-consulting company, will provide these services. Sapient brings a set of core competencies in software application planning, implementation, deployment, and support. Sapient is currently active in the higher education field, where it is partnering with other institutions on similar dissemination-phase activities. Examples include MIT's OpenCourseWare initiative, for which Sapient is providing much of the dissemination and support services; and Harvard Medical School's Medical Education Portal and Course Management Solution, which Sapient is deploying to the broader professional medical-education community worldwide.

The overall dissemination strategy for VITAL: Early Childhood Mathematics Education will have two main components:

Outreach
The consortium of CCNMTL, Teachers College, and William Paterson will lead the grassroots efforts to introduce VITAL: Early Childhood Mathematics Education to leaders in early mathematics education. These efforts would target appropriate professional organizations--e.g., the National Council of Teachers of Mathematics and the National Association for the Education of Young Children. The investigators have already disseminated information about the prototype through presentations at conferences and workshops including the Ed-Media Conference on Educational Multimedia, and the Education, Technology & Curriculum Summit. In addition, Profs. Ginsburg and Moretti will present VITAL: Early Childhood Mathematics Education at four (and perhaps more) national conferences in 2003-2004.

In Year 4 of the project, the investigators will begin to submit papers about VITAL: Early Childhood Mathematics Education and the evaluation results to conferences and journals, such as the National Council of Supervisors of Mathematics (NCSM) Annual Conference and the American Educational Research Association (AERA) Annual Meeting, with the expectation that these will be published in Year 5.

The consortium will investigate the possibility of submitting the resource to the National Science Digital Library and TE-MAT.

Deployment (Please refer to the diagram in Appendix D)
Sapient will lead the deployment activities, with support from the consortium personnel. The increasing trend towards use of video-based content as part of on-line distance learning can lead to accessibility challenges caused by network bandwidth constraints. To maximize the utilization of VITAL: Early Childhood Mathematics Education, the dissemination strategy should ultimately support two models for bringing the VITAL: Early Childhood Mathematics Education learning environment to potential user communities:

1. Application Service Provider (ASP) Model
The ASP model supports situations where individual or a small number of instructors wish to adopt the VITAL: Early Childhood Mathematics Education learning environment on a small scale, perhaps for a single course. The users (either instructors or students) would connect via any appropriate Internet access point to a centrally supported VITAL community Website where they would access their own course(s) containing curriculum plan(s), digital library(s) and online workspace(s) with appropriate security to control access. Instructors (or course administrators from the instructor's institution) would be able to set up and manage their own customized workspace(s) as described in the Anticipated Products section. However, using this approach, the support requirements at the instructor's institution would be limited to having reliable Internet access via a standard Web browser running on a standard PC. In this way, VITAL: Early Childhood Mathematics Education could be used without the need for a local installation process and at a lower cost to the instructor's institution. End-user support could be provided via e-mail or other appropriate methods.

The central VITAL: Early Childhood Mathematics Education community Website would also provide supplemental services to facilitate building a community of practice. These could include support for instructor collaboration and knowledge sharing events, access to expert advice and services, and access to additional tutorial and other resources. It is also envisaged that sharing of curriculum could also be accomplished via this type of community environment.

2. Turnkey or Installation "Behind the Firewall"
For schools where more widespread use of VITAL is anticipated, or where external network performance constraints exist, a local installation of the VITAL solution would be more appropriate. Here, the VITAL solution would be made available in a downloadable or installable form. Users would still be able to use the community Website described above but would request to receive a copy of the VITAL learning environment that can be hosted on a server within their local area network along with the required course modules.

For both of the approaches described above, it is anticipated that Sapient will provide services for core application installation, configuration, and appropriate customization along with ongoing support to users. Other components of deployment will include:

Training: To provide appropriate user training in areas including course facilitation, content development, and system operation and maintenance (for Turnkey installation only). User course facilitation training could be provided targeting instructors and students.
Support: To ensure continued use and evolution of the VITAL: Early Childhood Mathematics Education community, it would be proposed to provide a reasonable level of user support. Potential components of this include online support, telephone help desk support with associated defect management, and resolution processes.
Content development services: To support the continued creation of new cases by instructors within the VITAL: Early Childhood Mathematics Education community and by the VITAL: Early Childhood Mathematics Education consortium.