Onderstaande printversie van het indicatorenboek werd door uw browser gegenereerd, en zal niet steeds optimaal ogen. Via de ingebouwde printfunctie op de website van het Indicatorenboek (ronde knop rechts bovenaan) kan u een printvriendelijke PDF genereren met mooi ogende lay-out.

8.2.1Key framework dimensions for evaluative bibliometrics

In an influential contribution, Henk Moed (2017) introduced a visionary model of “evaluative informetrics,” emphasizing the practical application of bibliometric methods in research assessment. He refined this framework later, outlining the following four key questions that are essential to design evaluation practices:

  1. What is the unit of assessment (e.g., individual, institution, country)?
  2. Which aspect of the research process is under consideration (e.g., scholarly impact, social benefit, interdisciplinarity, collaboration)?
  3. What are the goals of the evaluation (e.g., resource allocation, performance improvement, strategic redirection)?
  4. What are the characteristics of the entities being assessed, including their developmental stage or systemic relevance (Moed, 2020, p. 4)?

Building on this foundation, we propose an expanded multidimensional framework by introducing two additional dimensions (see Daraio et al., 2024).

Table 1 gives an overview of the proposed dimensions.

Table 1. The six dimensions of our research evaluation framework

#DimensionDefinitionWarnings (or Pitfalls)
1Aggregation LevelThe scale at which evaluation is conducted: individual, group, institution, region, or nationAggregation level critically shapes the validity of the metrics applied. Indicators appropriate at one level may be misleading at another. Moreover, peer review suitability decreases with higher aggregation.
2Unit of AssessmentThe specific entity or profile being evaluated (e.g., individual researcher, lab, department)The unit of assessment is influenced by the context and nature of the research. Disciplinary and sector-specific characteristics must be considered to ensure relevance and fairness.
3Purpose of AssessmentThe objectives of the evaluation, such as funding, improvement, promotion, or benchmarkingThe purpose drives methodology, timeline, baseline, and criteria. Different objectives required tailored evaluation strategies.
4Context of AssessmentThe broader environment and conditions - whether institutional, systemic or national - in which research takes place.Evaluation must be sensitive to systemic, geographical, or disciplinary contexts in order to avoid bias or misinterpretation.
5Elements of Research ProcessThe stages and outputs of research, including input, process, output, and impact – both academic and non-academicEvaluation should account for the full range of research stages and outputs. It is essential to consider diverse forms of impact (e.g., social, economic, cultural) beyond scholarly output.
6Stakeholder EngagementThe involvement of those affected by, or participating in, research and its evaluation, such as funders, institutions, policy makers and the wider public, etc.Stakeholder inclusion enhances the legitimacy and comprehensiveness of the assessment and allow for consideration of both intended and unintended consequences.
#DimensionDefinitionWarnings (or Pitfalls)
1Aggregation LevelThe scale at which evaluation is conducted: individual, group, institution, region, or nationAggregation level critically shapes the validity of the metrics applied. Indicators appropriate at one level may be misleading at another. Moreover, peer review suitability decreases with higher aggregation.
2Unit of AssessmentThe specific entity or profile being evaluated (e.g., individual researcher, lab, department)The unit of assessment is influenced by the context and nature of the research. Disciplinary and sector-specific characteristics must be considered to ensure relevance and fairness.
3Purpose of AssessmentThe objectives of the evaluation, such as funding, improvement, promotion, or benchmarkingThe purpose drives methodology, timeline, baseline, and criteria. Different objectives required tailored evaluation strategies.
4Context of AssessmentThe broader environment and conditions - whether institutional, systemic or national - in which research takes place.Evaluation must be sensitive to systemic, geographical, or disciplinary contexts in order to avoid bias or misinterpretation.
5Elements of Research ProcessThe stages and outputs of research, including input, process, output, and impact – both academic and non-academicEvaluation should account for the full range of research stages and outputs. It is essential to consider diverse forms of impact (e.g., social, economic, cultural) beyond scholarly output.
6Stakeholder EngagementThe involvement of those affected by, or participating in, research and its evaluation, such as funders, institutions, policy makers and the wider public, etc.Stakeholder inclusion enhances the legitimacy and comprehensiveness of the assessment and allow for consideration of both intended and unintended consequences.
#DimensionDefinitionWarnings (or Pitfalls)
1Aggregation LevelThe scale at which evaluation is conducted: individual, group, institution, region, or nationAggregation level critically shapes the validity of the metrics applied. Indicators appropriate at one level may be misleading at another. Moreover, peer review suitability decreases with higher aggregation.
2Unit of AssessmentThe specific entity or profile being evaluated (e.g., individual researcher, lab, department)The unit of assessment is influenced by the context and nature of the research. Disciplinary and sector-specific characteristics must be considered to ensure relevance and fairness.
3Purpose of AssessmentThe objectives of the evaluation, such as funding, improvement, promotion, or benchmarkingThe purpose drives methodology, timeline, baseline, and criteria. Different objectives required tailored evaluation strategies.
4Context of AssessmentThe broader environment and conditions - whether institutional, systemic or national - in which research takes place.Evaluation must be sensitive to systemic, geographical, or disciplinary contexts in order to avoid bias or misinterpretation.
5Elements of Research ProcessThe stages and outputs of research, including input, process, output, and impact – both academic and non-academicEvaluation should account for the full range of research stages and outputs. It is essential to consider diverse forms of impact (e.g., social, economic, cultural) beyond scholarly output.
6Stakeholder EngagementThe involvement of those affected by, or participating in, research and its evaluation, such as funders, institutions, policy makers and the wider public, etc.Stakeholder inclusion enhances the legitimacy and comprehensiveness of the assessment and allow for consideration of both intended and unintended consequences.