Research and Evaluation

Evaluation Plan

The project evaluation plan describes the process in which we will systematically collect and analyze information in order to determine whether and to what degree the objectives of the training program are being achieved. This project's evaluation plan consists of measuring the efficiency, effectiveness, and impact of the training program. It includes both process evaluation (quantitative and qualitative assessment conducted to provide data on the strengths and weaknesses of the components of the training program) and outcomes evaluation, the program's overall effectiveness.

Process Evaluation

The process evaluation of this project consists of both quantitative and qualitative assessments that provide complementary data on the strengths and weaknesses of the components of the training program. This type of evaluation involves obtaining ongoing regular feedback from the training program developers, faculty, and residents to refine and strengthen curriculum materials and delivery systems and to guide implementation.

For organization purposes, the process evaluation plan is divided into four areas: project implementation, project dissemination, curriculum development, and faculty development. The section below includes objectives, assessment measures, evaluation activities, and performance indicators by area. Assessment measures will include:

• Detailed description and documentation of the curriculum development process; including tools and educational scaffolding
• Didactic content module, materials, and activities evaluation forms used to assess content with respect to scope, clarity of presentation, accuracy quality and level of appropriateness
• Pre- and post- focus group feedback sessions, surveys, and/or interviews used to collect their opinions and perceptions about the appropriateness, relevance, and effectiveness of materials, activities, and resources.
• Web-resources surveys will focus on quality and value of the web resources as perceived by both residents and faculty. This survey will embrace five major issues: a) Content, b) Navigation features and interface, c) Interactivity, d) Usability and e) Value.

Outcomes Evaluation

Outcomes evaluation will focus on the residents’ performance and the effectiveness of the faculty development component. Assessment measures and performance indicators are divided into: training program's curriculum quality and effectiveness, curricular products, impact on residents learning, access, program completion, ethnicity of program’s participants, and program dissemination. Evaluation objectives and performance indicators are explained below.

Research Articles

Problem-solving

Hendricson WD, Andrieu SC, Chadwick DG et al. (2006). Educational strategies associated with development of problem-solving, critical thinking, and self-directed learning. Journal of Dental Education; 70(9):925-936.

Lohman MC and Finkelstein M (2002). Designing cases in problem-based learning to foster problem-solving skill. Eur J Dent Educ; 6:121-127.

Reflections

Reisman AB, Hansen H, Rastegar A (2006). The craft of writing: a physician-writer's workshop of resident physicians. Journal of General Internal Medicine; 21(10):1109-1111.

Strausee R, Modifi M, Sandler ES et al. (2003). Reflective learning in community-based dental education. Journal of Dental Education; 67(11):1234-1242.

Lifelong learning

Challis M (2000). AMEE Medical Education Guide No. 19: Personal learning plans. Medical Teacher; 22(3): 225-236.

Coles C (2004). Education and professional development. www.did.stu.mmu.ac.uk/carn/eventsdoc/coles.pdf

Davenport ES, Fry H, Pee B, Woodman T (2003). Learning throughout life: can a progress file help? British Dental Journal; 195(2):101-105.

Walker ADM, Carrotte PV, Dodd M, Ball G, Rennie JS (2003). Personal learning plans for general dental practitioners, a Scottish experience. Part I. British Dental Journal, 194(9):509-513.

Self-directed learning

Fisher M, King J, Tague G (2001). Development of a self-directed learning readiness scale for nursing education. Nurses Education Today; 21:516-525.

Hiemstra R (2005). Techniques, tools, and resources for the self-directed learner. http://www.distance.syr.edu/sdltools.html.

Hiemstra R (1994). Self-directed learning. In T. Jusen and T.N. Postlethwaite (Eds.), The International Encyclopedia of Education (second edition), Oxford: Pergamon Press.

Shokar GS, Shokar NK, Romero CM, Bulik RJ (2002). Self-directed learning: looking at outcomes with medical students. Family Medicine; 34(3):197-200.