Assessment Home »

2004 Scholarship of Assessment Grant

Final Report

Janet Gooch and Paula Cochran

Communication Disorders

September 28, 2004

Project Title: Authentic Assessment in Communication Disorders: A Comprehensive Exam Alternative

Contents

Abstract

Cover Memo

Clinical Process Summary (Overview)

Timeline for Implementation

Clinical Process Commentary #1 Student Guide

Clinical Process Commentary #1 Faculty Oral Presentation Evaluation Form

Clinical Process Commentary #1 Oral Presentation Peer Response Form

Clinical Process Commentary #1 Oral Presentation Self-Assessment and Cover Sheet

Clinical Process Commentary #2 Student Guide

Clinical Process Commentary #2 Faculty Evaluation Form

Clinical Process Writing Prompts (Fall, Spring, Summer)

Clinical Process Writing Prompts Directions to Students

Abstract

The purpose of this project is to re-evaluate the formative and summative assessment tools and procedures currently in use by the CMDS graduate program, and to update them with more effective alternatives. In keeping with new trends in assessment, new initiatives within the university community, as well as new guidelines from our national program accrediting agency, such re-evaluation and development is crucial at this time.

Current assessment at the graduate level is extensive, traditional, and somewhat effective. For example, all CMDS graduate students take a nationally standardized exam prior to graduation and Truman students have an extraordinarily high pass rate. Excellent as this may be, it provides little helpful information for curricular review or for counseling the occasional student who is struggling.

Through a review of our methods, methods of other programs, and new trends such as portfolios and case study presentations, our current comprehensive exam process will be modified. It is anticipated that this will result in presentations and products that will benefit not only graduate students, but also undergraduate students in the program. Dissemination will include all faculty and students in CMDS, Graduate Council, and possibly others. The outcomes of this project will address both university-wide priorities and discipline-directed issues. It will put CMDS in line with assessment goals being implemented across campus at both the undergraduate and graduate levels (e.g., formative and summative assessment, assessment that involves multiple faculty, both qualitative and quantitative data, student self-evaluation).

Cover Memo

Dear Vice President Gordon:

As stated in our grant proposal, the objectives of our project were:

1. To develop assessment tools and procedures that are compatible with ASHA Knowledge and Skills assessment and the recently approved Truman CMDS Graduate Program Mission, Objectives, and Outcomes.

2. To develop an assessment package for all students in the CMDS graduate program that includes some existing assessment tools and procedures, but also includes:

· newly developed assessment tools,

· directions for students,

· policies and procedures [to be approved by CMDS faculty],

· timeline for implementation,

· overview description of the assessment package that would be understandable to outside reviewers, and

· a revised graduate application for graduation

3. To develop the assessment package in a way that does not change the total number of hours required for a degree and does not discourage students from pursuing a thesis.

This final report includes the documents that were developed as products of the project to meet the objectives listed. We are currently discussing these new procedures and documents at each regularly scheduled CMDS faculty meeting and seeking feedback and approval for implementation. Once the faculty has approved, we will revise the graduate application for graduation, which is the only element listed above that is not completed. It is now anticipated that the new assessment package will be fully implemented as of Fall 2005.

Thank you for the opportunity to address this need in the CMDS graduate program.

We look forward to sharing our experiences with others at the October 25 colloquium on assessment sponsored by the Assessment Committee and the Center for Teaching and Learning.

Janet Gooch, Ph.D., Program Chair Paula S. Cochran, Ph.D.

Associate Professor Professor

Authentic Assessment in Communication Disorders: A Comprehensive Exam Alternative

Summer 2004 Assessment Grant Project

Clinical Process Commentary Summary

Janet Gooch and Paula Cochran

The Comprehensive Exam in Communication Disorders is currently associated with a non-thesis curricular option that also includes 6 semester hours of elective coursework. With the adoption of the Clinical Process Commentary, the exam will be eliminated for student entering the graduate program as of Summer 2004. The thesis/non-thesis options will continue, but will pertain to the 6 hours of credit only; students will choose 6 hours of thesis research, or 6 hours of elective coursework.

All graduate students will participate in the Clinical Process Commentary (CPC) activities. It is anticipated that undergraduate students and faculty will also be challenged and stimulated by the activities included in the CPC. They include 3 main components:

· Clinical Process Commentaries, #1 and #2

· Clinical Process Writing Prompts

· Portfolio of Key Assignments

A. Clinical Process Commentaries

The Clinical Process Commentaries (CPC) consist of a sequence of major projects that :

1. emphasize strong ties between the academic and clinical curricula,

2. provide opportunity for formative and summative evaluation, and

3. provide opportunity for students to demonstrate depth of knowledge and expertise through both oral and written presentation

The CPC will be required at 2 different times in the course of graduate study, usually the second semester (spring) and the last semester (fall) prior to internship. They should invite comparison on the part of students regarding issues such as:

· How were these two clinical experiences similar and different?

· What similarities/differences were there in the clients? In the challenges presented? In opportunities to address and achieve clinical competencies?

· How was course information/academic knowledge/research applied to the management of the second case?

· What evidence of growth is demonstrated by the clinician in the way the second client was managed compared to the first? (goals, record keeping, reporting, supervision needs, intervention techniques, estimations of progress?)

The expected/typical sequence would be:

1^st Fall Semester -à

Spring Semester -à

2^nd Fall Semester (may vary)

Single-subject research project involving client

Planned and implemented in CMDS 667, Research Methodology

Final paper includes response to key assignment prompt regarding how new knowledge/skills may be applied to future clinical challenges.

IRB paperwork will cover research project and CPC# 1.

Clinical Process Commentary #1, examines client and case management from clinical rather than research point of view. Makes use of reflections on the therapy process developed while seeing the client.

Includes analysis of videotaped session and clinician strengths and weaknesses.

Makes use of the same client as previous research study.

Requires self-assessment of presentation.

Clinical Process Commentary # 2, examines case management of a different client, chosen by the student. Includes analysis of video, comparison to first CPC, discussion of links to coursework/literature, and discussion of personal growth as a clinician.

Student must submit standardized IRB paperwork and schedule presentation with faculty (usually in conjunction with weekly faculty meetings.)

Presentation: Annual research forum, poster session, and final paper.

Assessment: Multiple faculty review of oral presentation and poster session. Peers invited to poster session. Performance reflected in course grade.

Presentation: Videotaped case study presentation during Friday Forum to faculty and peers. Includes PowerPoint and video clips.

Assessment: Self-assessment of presentation, faculty and peer feedback. PASS constitutes permission to proceed to CPC #2.

Presentation: Formal oral presentation to faculty accompanied by written Clinical Process Commentary paper.

Assessment: Multiple faculty review of oral presentation and final paper (summative assessment). PASS constitutes successful completion of comprehensive exam requirement.

B. Clinical Process Writing Prompts (CPW)

The purpose of this new component to our clinical curriculum and assessment activities is to explicitly encourage and develop the habit of self-reflection and self-assessment in student clinicians at the undergraduate and graduate levels. A series of writing prompts will be developed that all student clinicians will be expected to address weekly or bi-weekly. Responses will normally be submitted to clinical supervisors along with written lesson plans and discussed regularly in the Friday Forum.

It is intended that some writing prompts will encourage student clinicians to make connections between coursework and the management of their clients. Others will invite student clinicians to reflect on what they are doing and how they are doing it. A separate set of writing prompts for Fall (5), Spring (5), and Summer (3) use have been developed, taking into consideration the unique aspects of each part of the year for a typical student clinician.

Graduate students working on CPC#1 and CPC#2 will be encouraged to consult Clinical Process Prompts as well as self-reflections on key assignments that they have written in the past to help them recapture some of the thinking they were doing at the time they were working with their clients and relevant coursework. In addition, the Clinical Process Prompts will help clinicians document how their clinical knowledge, skills, and attitudes have changed for the purposes of the CPC and their portfolios.

At the undergraduate level, the CPW writing and discussions will encourage self-reflection in student clinicians. This is a common and valuable aspect of a senior capstone experience.

C. Portfolios of Key Course Assignments

Key course assignments in graduate classes will be required components of a graduate portfolio (currently students choose portfolio contents). These key assignments will be identified in advance, and will include a self-assessment component at the time they are submitted to various faculty instructors. The self-reflection component will normally be a written response to a standardized prompt, such as:

“How do you anticipate that you will apply the knowledge and skills you have acquired through this assignment in your future clinical practice?”

Whether or not the self-reflection component will be considered in evaluation of the project will be up to the faculty member, but students will be required to include it. The list of “key assignments” will be developed and approved by the faculty as a whole.

Key assignments under consideration include:

· Single-subject research project

· Language sample analysis

· Language disorders narrative/literacy project

· Phonology project/speech transcription

· Diagnostic report

· Computer class software development project

· Others . . .

Prior to graduation, students will be required to submit their portfolios, including a self-evaluation of their academic and clinical progress. They should make use of the reflective statements/papers that have been included in clinical practicum and key course assignments throughout their graduate study.

Communication Disorders

Truman State University

Clinical Process Commentary

Timeline for Implementation

Fall Semester 2004

Administer Comprehensive exam as usual

Presentation to faculty at faculty meeting(s)

Approval of general procedures and components by faculty

Orientation to CPC for new graduate students and Research Methodology component is implemented

Provide graduate students (old and new) with individual advising about who new procedures will apply to

Implement Clinical Process Prompts reflective writing exercises in CMDS 480/680, including increased faculty attendance at Friday Forum (make a schedule?)

Include reflective prompt in key graduate assignments for Fall 2004

Work on updating program and university documents, including:

Bulletin copy

Graduate Application for Graduation

CMDS web site

Clinic consent form revision

Course syllabi

Graduate portfolio directions

Orient new graduates (workshop?) for CPC #1 and portfolio

Check BT 200 schedule for next semester

Spring 2005

Administer Comprehensive exam as usual

First set of CPC#1 presentations in Friday Forum (4)

Faculty feedback forms

Peer response forms

Self-assessment forms

Meetings between presenters and advisors

Continue Clinical Process Prompts reflective writing exercises in CMDS 480/680

Students preparing for CPC#2 to be presented summer or fall (submit IRB’s)

New graduate students oriented to CPC#1 and CPC #2

Summer 2005

Possible presentation of first CPC#2’s and continue Prompts

Fall 2005

Graduate orientation includes complete packet of new forms, etc.

Research Methodology component is implemented

(administer comprehensive exam if needed)

New students getting ready for CPC#1

Presentations of CPC#2 – Start new faculty meeting schedule

Submission of first new portfolios

Spring 2006

Presentation of CPC#1 and CPC#2

Evaluate whether current CPC procedures are working well for students and faculty

Look data such as thesis, PRAXIS scores, 1-yr alumni surveys

List of stuff for P & J to create:

IRB template

Participation in Case Study consent form

Oral presentation feedback forms for faculty and peers (consider listing KASA)

· Oral presentation evaluation for CPC 2

Clinical Process Commentary

Student Guide for CPC #1

Truman State University

Communication Disorders

Overview

The purpose of this project is to provide an opportunity for your to demonstrate your development as a clinician and your grasp of the fundamentals of clinical practice. It also is a required component that you must do in partial fulfillment of the requirements for a master’s degree in communication disorders.

Successful completion of the CPC requires each student to develop an oral presentation Typically the presentation will occur during the second semester of graduate work.

Because you are publicly presenting clinical information about a specific client, the CPC will be treated much like a research project. You use of client information must be approved by the IRB. A standard IRB form and project explanation should be used (If you have had Research Methodology, this will be taken care of for you; otherwise obtain this from the CPC coordinator and submit to the IRB at least 6 weeks before your presentation is scheduled to occur).

****Schedule your CPC #1 presentation through the clinic director during the first 2 weeks of the semester – they will be scheduled on a first-come, first-serve basis, so to get your preferred time, plan accordingly.****

Your presentation will be:

· Created with PowerPoint slides

· Will include a brief video clip for illustration purposes (1-3 minutes) NOT the entire 15 minute tape that you will submit separately

· Formally presented to peers and faculty at a Friday Forum

· Approximately 15 minutes in length

· Followed by a brief question and answer period

You will submit the following within 2 weeks after your oral presentation:

· A 15 minute continuous and unedited videotape of the individual therapy session upon which your presentation was based

· The lesson plan and long range goals associated with this session, with identifying information removed

· Videotape of your presentation

· Written self-assessment of your presentation

· IRB approval letter

· Cover sheet

After your materials have been reviewed by the faculty, you will meet with your advisor to obtain the results. Your advisor will let you know if you have passed the CPC#1. You and your advisor will compare your self-assessment with feedback from the faculty about your presentation.

Evaluation Criteria

The evaluation criteria used to assess the CPC are based on KASA outcomes related to clinical knowledge and skills. The content of your presentation must demonstrate clear, consistent, and convincing evidence that you:

· Applied detailed knowledge of your client’s case history and individual characteristics.

· Managed client behavior, provided feedback, motivated maximum client effort

· Provided a therapy environment designed to optimize client success (considers safety, room arrangement, material selection, material management, conversational partners and/or family participation)

· Maintained professional and ethical behavior toward the client and family

· Reflected on the clinical process and personal goals for improvement

Additional criteria for evaluation of oral presentation: organization, use of PowerPoint and video clips, professional impresssion (dress, eye contact, manner), appropriate loudness and use of voice.

Contents of the CPC Oral Presentation

Draw on your previously written Clinical Process Prompts as well as examples from the videotaped segment to address the following points. Your presentation should be organized to include the following:

Therapy context

· Describe the client in a way that is brief but sufficient for the audience

· Describe any important environmental factors relevant to your therapy and the client’s participation

Planning and videotape analysis:

· Discuss your long range goals and why they were appropriate for this client

4. Discuss the goals for this particular session and how they fit into your long-term goals

5. Explain your rationale for choosing these particular activities to meet the goals of this session

6. Discuss your choice of materials and reasons for choosing them.

7. To what extent does this session reflect your ultimate communication goals for this client?

Self-Assessment: After your presentation

Based on your presentation and clinical experiences, complete the CPC # 1 Self-Assessment.

Getting Ready

Choosing the client

Normally, the first clinical process analysis will focus on your work with your research project client, continuing from a clinical point of view the single-subject research study that you did in CMDS 667 Research Methodology.

Choosing the therapy session

The session you choose should clearly show a logical step toward a long range goal. It should demonstrate your effective management of the client and the clinical setting, including materials, feedback, record keeping, etc.. Ideally, it will demonstrate how you have designed this activity to meet the specific needs of this client. You may plan in advance that this session will be one that you videotape with a CPC in mind, or it could be a session that was videotaped for other reasons.

Choosing the videotape segment

Ideally, you should have several videotape segments to choose from so that you can select one that fits all the requirements as well as possible, and shows your clinical skills. The segment should show your interaction with the client, although the camera need not always be on you. You need one continuous 15 minute, unedited segment to submit, from any point in the session as a whole. Select the segment that you think provides the best evidence of the evaluation criteria listed above.

Communication Disorders Faculty

Truman State University

CPC #1 ORAL Presentation Evaluation Form

Student: _______________ Reviewer: _____________ Date: _________

Directions:

Please indicate below your general impression of the student’s presentation, indicating which aspects of the presentation you considered to be weak or strong. Give an overall PASS/FAIL summary score. Shaded cells pertain to the student’s presentation skills, not to the clinical skills demonstrated or discussed.

Criteria	PASS	FAIL	Comments
Describes client and setting
Discusses goals for client
Explains rationale for materials and activities
Reflects on ultimate communication goals for client
Uses audio/visual materials effectively
Gives professional impression (eye contact, body language, dress)
Appropriate loudness and use of voice
Overall quality of this presentation

Global evaluation: (circle one) PASS FAIL

Communication Disorders

Truman State University

Clinical Process Presentation Peer Response Form

Your Name: _______________________________ Date: _______

Student presenter: __________________________

Purpose: The purpose of this exercise is to help student clinicians benefit from the clinical process presentations of their peers. The intention is to help the presenter’s peers apply information and thinking from the presentation to their own clinical experiences and progress as a student clinician. Your responses will not be shared with the student presenter, with the possible exception of the last question. Completion of this form is a required component of CMDS 480 and 681.

This presentation made me think about my own clinical skills.