Evaluation for Learning Group
Summary
The Evaluation for Learning group (EFL) is an evaluation, program planning, and social science research project specializing in K-12 education and based at the University of Pittsburgh’s Learning Research and Development Center. Directed by Professor W. E. Bickel, the EFL works with local and national institutions on the design, documentation, evaluation, and strategic assessment of a wide variety of educational programs and reform efforts. An overarching theme in the EFL work is using evaluative inquiry to support improved decision-making by educational leaders, practitioners, and policy shapers. The resulting products are intended to be useful to specific sponsoring decision makers and organizations, as well as to generate knowledge for the broader research and evaluation communities. Key personnel include Dr. Jennifer Iriti, Megan Walker, and Julie Meredith.
Major kinds of activities the EFL has engaged in include:
•External evaluation of educational programs or organizations
(including implementation and outcomes evaluation)
•Building capacity of educators to use data for decision-making
•Evaluability assessment
•Logic model development
•Integrating evaluative thinking and data use into foundation practices
•Educational literature reviews to support program development or refinement
•Strategic, retrospective reviews to capture programmatic or grant making lessons learned
•Scans of current policies, practices, and resources
These activities have been sponsored by a range of organizations including:
•Private and Semi-Public Foundations (e.g., Ball Foundation, Grable
Foundation, Heinz Endowments, Kellogg, Lilly Endowment, The
California Endowment; the National Science Foundation);
•Non-profit organizations (e.g., National Humanities Center,
National Writing Project, Rochelle Lee Fund, Western Pennsylvania
Writing Project).
•Public & Charter Schools (e.g., Pittsburgh Public Schools, City High Charter School, Fox Chapel Area School District, Propel Charter
Inc.)
A feature of EFL work has been the development from time to time of long-term partnerships with education or evaluation programs in private foundations. For example, the EFL had an eight year partnership (1990-1997) with the Education Program of the Lilly Endowment designed to “enhance the Endowment’s capacity to develop, coordinate, and manage” evaluative processes for the Education and Youth Development Divisions. For the past 14 years the EFL has partnered with the Heinz Endowments’ Education Program, a major regional grant maker in education. This partnership is centered on the development and management of an evaluation strategy to enhance the Endowments’ capacity to learn from its grant making in education, and to assist in the assessment of the overall impact of the Endowments’ investments in regional educational improvement.
The EFL has developed significant expertise and tools to support educational organizations in:
•Developing research-based and outcomes focused program
designs
•Monitoring implementation and making mid-course adjustments to
strengthen programs
•Infusing data use throughout their work to improve outcomes
attainment
•Supporting organizational capacity (knowledge, structures, and
culture) to capture and use data
•Learning from evaluative inquiry for both immediate and broader
audiences.
The EFL is committed to developing and disseminating knowledge from project work to local and national education practitioner groups, and the research and professional evaluation communities. Selected examples of EFL products for these various audiences follow:
People
Project Publications
- Bickel, W. E., Iriti, J. E., & Nelson, C. A. (2008). Integrating Evaluative Inquiry into the Work of Private Foundations: A Case Study of a Hybrid “Retainer Model.” Paper to be presented at the Annual Meeting of the American Evaluation Association, Denver, CO.
- Iriti, J., Bickel, W. E., Nelson, C. A. (2008). “Ball Foundation Partnership Evaluation: Understanding the Engagement Phase.” Pittsburgh, PA: EFL, Learning Research and Development Center, University of Pittsburgh. (Abstract-pdf)
- Bickel, W. E., Hendricks, A., Iriti, J., Nelson, C. A. (November, 2007) Barriers to Organizational Learning in Private Foundations” THINK TANK at the Annual Meeting of the American Evaluation Association, Baltimore, MD.
- Bickel, W. E., Iriti, J. E., & Nelson, C. A. (2007). Educational Leadership Initiative: Pilot Year Final External Evaluation Report. Pittsburgh, PA: EFL, Learning Research and Development Center, University of Pittsburgh. (Abstract-pdf)
- Bickel, W. E., Hendricks, A., Iriti, J., Nelson, C. A. (November, 2006) “Integrating Evaluation Into the Learning Processes of Private Foundations” Paper presentation at the Annual Meeting of the American Evaluation Association, Portland, Oregon.
- Nelson, C. A., Bickel, W. E., Iriti, J., with Murphy, N. (2006). “District-Wide Urban High School Reform: What does it look like? What does it take to do it right? Scan of National Trends, Best Practices, and Empirical Research.” Pittsburgh, PA: EFL, Learning Research and Development Center, University of Pittsburgh. (Abstract-pdf)
- Iriti, J. E., Bickel, W. E., & Nelson, C. A. (2005). Using recommendations in evaluation: A decision-making framework for evaluators. American Journal of Evaluation, 26 (4), 464-479.
- Bickel, W. E., Iriti, J. E., & Nelson, C. A. (2004). Heinz Endowments Education Program, Strategic Analysis of the Technology For Learning Grant-making Strand (Abstract-pdf)
- Kramer, M., & Bickel, W. E. (2004). In M. Braverman, J. K. Slater, N. Constantine (Eds). Foundations & Evaluation: Contexts & Practices for Effective Philanthropy. San Francisco: Jossey-Bass.
- Leviton, L., & Bickel, W. E. (2004). Integrating evaluation into foundation activity cycles, In: Editors M. Braverman, J. K. Slater, N. Constantine: Foundations & Evaluation: Contexts & Practices for Effective Philanthropy. San Francisco: Jossey-Bass.
- Nelson, C. A., Iriti, J. E. (2004). Evaluation as capacity building: Transforming tacit knowledge into decision-making tools. Paper presented at the Annual Meeting of the American Evaluation Association in Atlanta, Georgia.
- Bickel, W. E., Iriti, J. E., & Nelson, C. A. (2003). Foundation questions—evaluation responses: A resource for integrating evaluative thinking into the grant making cycle. Paper accepted for presentation at the Annual Meeting of the American Evaluation Association in Reno, Nevada.
- Bickel, W. E., Iriti, J. E., & Nelson, C. A. (2003). School Performance Network: An Assessment of the Status and Impact of the SPN in the Third Year of Its Development. Pittsburgh, PA: EFL, Learning Research and Development Center, University of Pittsburgh. (Abstract-pdf)
- Bickel, W. E., Millet, R., Nelson, C. A. (2002). “Challenges to the Role of Evaluation in Supporting Organizational Learning in Foundations.” Arlington, VA: Council on Foundations.
- Bickel, W. E., Post, J. E., Millett, R., & Nelson, C. A. (2002). Assessing organizational barriers to using evaluation in support of learning in foundations. Paper presented at the Annual Meeting of the American Evaluation Association in Crystal City, Virginia.
- Nelson, C. A., Post, J. E., & Bickel, W. E. (2002). Evaluating educational technology implementation: A two-part framework for assessing the institutionalization of technology in schools and classrooms. In T. Kellaghan, D. Stufflebeam, & L. Wingate (Eds.) (2002). International Handbook of Educational Evaluation. Kluwer Academic Publishers.
- Post, J. E., Nelson, C. A., & Bickel, W. E. (2002, April). Barriers and enablers to the effective implementation of the “teachers teaching teachers” staff development model: Lessons learned from a district-wide technology staff development initiative. Paper presented at the Annual Conference of the American Educational Research Association, New Orleans, LA.
- Nelson, C. A., Post, J. E., & Bickel, W. E. (2001). Institutionalization of technology in schools checklist. The Evaluation Center, Western Michigan University checklist project. Available at http://www.wmich.edu/~evalctr/checklists/view-download_page.htm (Abstract-html)
- Post, J. E., Nelson, C. A., & Bickel, W. E. (2001, November). Building evaluation capacity in foundations: The development of an on-line grantee reporting process. Paper presented at the Annual Meeting of the American Evaluation Association, St. Louis, MO.
- Bickel, W. E., Nelson, C. A., Post, J. (2000). "Briefing report: Heinz Endowments-PPS Technology Professional Development Program ," Pittsburgh, PA: EFL, Learning Research and Development Center, University of Pittsburgh. (Abstract-pdf)
- Nelson, C. A., Post, J. E., & Bickel, W. E. (2000, November). Does training achieve traction? Evaluating mechanisms for sustainability from teacher technology training. Paper presented at the Annual Meeting of the American Evaluation Association in Waikiki, Hawaii.
