Evaluation of EOT activities
From TeraGrid Wiki
TeraGrid EOT Evaluation Information
NSF and the annual review panel would like to see evaluations of the EOT programs and activities. The goal is to collect a common base of data to provide quarterly reports of impact, annual reports of impact, and longer-term longitudinal assessments of the impact across all of TeraGrid.
The objective is to conduct an evaluation across all TeraGrid EOT activities, and not to address activities at any particular RP or group of RPs.
Establishing Process and Instruments
We are working with Gypsy Abbott ( Gypsy's vita) as an external evaluator through a contract with the Krell Institute. Gypsy is helping to define the process and instruments to support a TeraGrid wide EOT evaluation. She has developed the following documents for review by the TeraGrid EOT Working Group. Each survey addresses multiple audiences through a branching process in survey monkey. Below are each set of surveys broken out by audience into PDFs so you can see which questions each audience will see. The actual surveys are all accessible on survey monkey.
Surveys are at http://www.surveymonkey.com
- Blueprint for evaluation - background information on how and why the three instruements were developed
- Formative evaluation instrument
- pre-event survey to be administered before an EOT event
- post-event survey to be administered right at the end of an event
- 6 month post survey to be administered 6 months after someone has attended an event
Please keep in mind that these are still drafts and that your feedback is critical to making them useful for a TeraGrid wide evaluation. No RP specific evaluation will be conducted and RP specific information will only be seen by Gypsy. The instruments build on the work done for the SC Education Program, and have been extended to include broader TeraGrid EOT events and impact. We recognize that work is still needed to cover the breadth of activities you are offering.
These instruments may be used to augment your own surveys in which you have other questions you may want to ask participants.
The critical aspect is having a common foundation of data collection to allow for a TeraGrid wide evaluation. Please feel free to send comments and suggestions at any time. You can directly reach Gypsy at: Gypsy Abbott <firstname.lastname@example.org>.
Thank you for your help, your advice, and your suggestions.
Please review these documents and send comments and suggestions to Gypsy, Scott and/or the EOT Working Group to improve the process.
All questions are included - we ask your help in identifying which questions that are appropriate for the various audiences, and whether there are changes needed to the questions. [Scott]
I also think that the information asked here could just as well be lumped with the post survey form. That would save everyone time and pain. For example, each question could have a "before" and "after" answer. Responding to surveys is not a pleasant exercise for most people. The only real use of a pre survey is to let the course giver know the expertise levels of the class. But this is more effectively done by talking about it with the class members. I always spend a few minutes talking with each student to elicit this information, and to get to know everyone. I would dump this survey.