Posted by: Jane Davidson
What are complex organizations doing right when it comes to evaluation, and where are they missing opportunities to apply their own best practices in other areas?
There is much that universities (and other organizations) do right, particularly in some of their intradisciplinary evaluation. But somehow when it comes to evaluating university courses and programs, it’s as though this is seen as a completely different sort of undertaking.
Saville and his research fellow, Acacia Cochise, kindly summarized the seminar, which we thought Genuine Evaluation readers might also enjoy.
Self evaluation in complex organizations including the University of Auckland
This breakfast seminar was well attended by University of Auckland colleagues and people external to the university. It was facilitated by Dr. Jane Davidson, and international evaluation practitioner and advisor to the New Zealand Qualifications Authority (NZQA) on institutional evaluation.
Some of the discussion centered on Dr. Davidson’s experience in working with NZQA to develop a collaborative, negotiated and grounded approach to institutional review and reporting.
This approach has been adopted across the NZ Tertiary Sector [i.e., higher education] with the exception of universities. It is based on a negotiated and evidence-based methodology for self-review complemented by external review (inspection).
The role of external reviewers is to work from the self-review to assure its quality and identify key development points. This extends beyond an indicators/metrics-based assessment of institutional performance and has significant elements of professional judgement and collegial consensus-building.
The requirement for an institution to conduct a self-review that is grounded in internal evidence is proving to stimulate change to organizational cultures. There is, that is to say, a wash-back effect shaping internal deliberative processes for critical self-reflection and legitimating diversity.
This provoked intensive discussion of the need for professional approaches to evaluative procedures in high-level educational institutions.
There are instances (e.g. thesis examination, peer review, personnel appraisal) where institutions rely for the identification of quality on professional and inter-subjective judgement, and what was proposed was an extension of these instances to matters such as course review and institutional performance assessment.
This would not assume that quality can be pre-specified and enshrined in indicators, but was a matter for exchange, identification and negotiation.
Complex institutions demand complex ways of knowing themselves. In contrast, there were expressions of concern that the quality of work at the university was increasingly subject to universal, generalized indicators and a ‘one-size-fits-all’ approach to evaluative method.