|
Resumen:
|
Over the past few years, higher education institutions have been submitted to various
changes largely conditioned by the demands of society. The change from a university for the
elite to a university for the masses, ...[+]
Over the past few years, higher education institutions have been submitted to various
changes largely conditioned by the demands of society. The change from a university for the
elite to a university for the masses, the increasing demand for the improvement in quality, the
globalization of knowledge, the competitiveness at national and international levels, the
development of internal improvement procedures, the changes in relationships with
administration and society in general; are some of the unavoidable factors of growing
importance in the field of higher education in Europe and around the world.
Accountability, to build trust between society and universities, and the demand for
transparency, for the design and implementation of plans for improvement and decision-making,
are key aspects for meeting society¿s needs. Nevertheless, it is a fact that universities offer less
openness than desired, although in recent years there has been a common effort to change this
situation.
Spanish universities have developed several initiatives to assess the quality of university
activities. In particular, one of the relevant challenges to ensure quality in the Spanish university
system is the assessment of the university teaching staff activity. The importance of the
assessment activities is ever increasing. Such activities can help ensure the quality of
educational processes affecting students, the university community and society in general. The
evaluation results will allow the design and implementation of plans for improvement for the
performance of university teaching staff at universities. It is therefore important to have
measurement tools to assess the university teaching staff activity in three dimensions: teaching,
research and management. However, commonly the assessment instrument used does not meet
the need for the transparency which society demands.
The aim of this research is to design methodological strategies for building composite
indicators to assess and measure multidimensional concepts related to higher education
management. To that end, a thorough review of the statistical techniques that can be applied in
the development of composite indicators is performed. Furthermore, the importance of the
indicators as a basic tool for suitable quality management and accountability in universities is
studied.
After reviewing a summary of the use of indicators in higher education, this study focuses on
the assessment of the academic activities of the Universitat Politècnica de València teaching
staff. First of all, an exploratory analysis of all indicators proposed by the university, for their
teaching staff assessment, is performed. To do this, statistical techniques of dimensionality
reduction, and classification techniques are applied. Subsequently, different methodologies for
constructing a composite indicator that measures teaching activity are proposed, according to
the normalization and weighting techniques applied. Lastly, the analytical characteristics of each of the methodologies proposed are studied, using uncertainty and global sensitivity
analysis techniques to evaluate the quality and robustness of the proposed models, according to
the goals set by the university. The uncertainty analysis is performed using a quasi-random
simulation process and applying a non-compensatory aggregation technique on the results,
known as the Copeland method. A nonparametric approach, known as the State Dependent
Parameter, based on the Kalman filter and smoothing techniques, is performed to apply global
sensitivity analysis. The uncertainty and sensitivity analysis are considered necessary techniques
to provide validity and methodological support for the composite indicators proposed.
Finally, the generalization of the composite indicator referring to teaching activity is studied,
and a composite indicator is designed to evaluate the academic activity of each teacher with
respect to the rest. This is based on the three main dimensions of university teachers: teaching,
research and university management. In order to do this, a non-parametric weighting and
aggregation technique called Benefit of the Doubt is applied. This technique measures the
relative effectiveness in the development of academic activities of each member of university
teaching staff, according to their academic profile.
The methodological techniques proposed in this study are applicable in any field of research,
and they provide a robust and transparent construction scheme of composite indicators.
[-]
|
