8.2.4Some concluding words

The proposed framework enables – and indeed encourages – the development of checklists to align available indicators with key assessment dimensions, thereby promoting structured and transparent decision-making (Robinson et al., 2024).

However, using the right indicators in a bibliometrically sound manner is not enough; they must be interpreted critically and with care, acknowledging the limitations posed by methodological weaknesses, data quality issues, and parameter selection.

Transparency and reproducibility must remain core principles of all evaluation methods. These can only be achieved if indicator use is standardized, well-documented, and paired with regular stakeholder engagement - including with researchers, institutions, and the broader community. Such engagement enhances both the meaningfulness and accuracy of the evaluation process, while also supporting the identification of acceptable margins of error and interpretative caveats.

To meet the complexity of today’s research landscape, bundles of valid and robust indicators should be selected, rather than constructed through the arbitrary combination of metrics into opaque composite indicators. The paper cautions against the use of such composites, as they often distort multidimensional realities, impose artificial linearity, and reduce both transparency and interpretability. These effects directly conflict with the core principles of responsible metric use.

Recent approaches such as “narrative bibliometrics” (Torres-Salinas et al., 2024) offer a promising alternative. By embedding bibliometric data within contextualized, narrative interpretations, this method can enrich our understanding of impact, particularly in relation to outputs that are less readily quantified. Yet, this approach also presents challenges: the shift from objective metrics to subjective narratives introduces interpretive variability, which may undermine the perceived neutrality typically associated with bibliometric evidence.

As Moed (2007) highlighted, the most effective assessments combine “advanced metrics” with “transparent peer review”. However, just as quantitative indicators metrics require clear criteria for validity and reliability, qualitative assessments also face limitations. Biases such as arbitrariness and fuzziness, critiqued by Bookstein (1997) in the context of quantitative analysis, are equally present in peer review and narrative evaluation.

Rethinking research assessment is a complex but necessary endeavour. Incorporating a diversity of research outputs, improving the appropriateness of metric use, and embedding evaluation practices within ethical, transparent, and participatory frameworks are all vital steps. Achieving this will require not just methodological innovation, but active collaboration among researchers, institutions, funders, and policymakers.