Can innovation be measured? A framework of how measurement of innovation engages attention in firms

https://doi.org/10.1016/j.jengtecman.2018.04.003Get rights and content

Abstract

Many firms manage the innovation process by using metrics. Yet, whether measurement supports or hinders innovation continues to be a topic of debate. To shed new light on this debate, this paper presents a conceptual framework of how measurement engages attention in firms. We draw on attention based theory and conceptualize innovation measurement as an attention-focusing device. We identify two ideal types of measurement practices. i) Directional Measurement: which is based on few and unidirectional metrics and encourages exploitative innovation efforts. ii) Conversational Measurement: which is based on multiple and ambiguous metrics and encourages exploration. We extend theory building in the technology and accounting literatures by theorizing the role of metrics and measurement for attention and by discussing the implications of such attentional engagement for innovation performance. In so doing, we engage closely with the managerial task of managing innovation while simplifying its conditions, thereby providing actionable advice.

Introduction

Many firms manage the innovation process by using metrics (Chan et al., 2008). This makes the measurement of innovation an important topic in the technology- and innovation management literature (Richtnér et al., 2017). The extent to which measurement is beneficial for innovation, however, continues to be a topic of debate (Criscuolo et al., 2017; Chiesa, 1999). Even after decades of research, results are mixed. One line of research suggests that measurement can be beneficial to innovation (e.g. Markham and Lee, 2013). Scholars in this stream have argued that measurement help managers to audit structural antecedents, processes and outcomes, thus ensuring that innovation is sufficiently supported and efficiently performed. Another line of research suggests that measurement discourage managers from pursuing more ground-breaking innovation (Criscuolo et al., 2017). Here, studies have shown that innovation measurement obstructs or hinders innovation since it pushes organizational members to focus their attention too narrowly (Abernethy and Brownell, 1997; Amabile et al., 1996; Tushman, 1997).

The purpose of this paper is to shed new light on this debate by proposing a framework on how innovation can be measured. Drawing on attention based theory (Ocasio, 1997, 2011) we conceptualize innovation measurement as an attention-focusing device. We identify two ideal types of measurement practices: Directional Measurement, which is based on the use of few and unidirectional metrics, and Conversational Measurement, which is based on the use of multiple and ambiguous metrics. Our framework specifies mechanisms through which directional and conversational measurement affect attention and, in extension, innovation performance. We discuss how different levels of ambiguity, meaning that there is unclarity such as that it is difficult to interpret or distinguish issues and action alternatives, requires different types of measurement practices. Our core argument is that situations of low ambiguity call for directional measurement since this allows a sustained and persistent focus of attention. Situations of higher ambiguity, on the other hand, call for conversational measurement. This is because conversational measurement engages attention in a bottom-up process, allowing organizational members to consider multiple issues and action alternatives simultaneously.

On the basis of our framework, we develop actionable advice on how innovation can be better measured to produce desirable outcomes. The paper builds upon and contributes to two streams of literature: the technology- and innovation management (e.g. Boly et al., 2014; Richtnér et al., 2017) and the managerial accounting literature (e.g. Davila et al., 2009; Bisbe and Malagueño, 2015; Carlsson-Wall and Kraus, 2015). Both literatures provide important insights into the different contingencies surrounding innovation measurement. By theorizing the role of metrics and measurement for attention, our framework extends both literatures by proposing different ways in how the use of metrics can engage attention; and by discussing the implications of such attentional engagement for innovation performance. In so doing, we engage even more closely with the managerial task of measurement and innovation while simplifying its conditions, thereby providing actionable advice to managers.

Section snippets

Measuring innovation: a literature background

Following the Oslo manual, we define innovation as the implementation of a new or significantly improved product (good or service), or process, a new marketing method, or a new organizational method in business practices, workplace organization or external relations. We adopt a broad definition of innovation performance, acknowledging that innovation can have a range of performance implications within and across firms, from effects on turnover and market share, to improved productivity or

Innovation measurement as an attention focusing device

To address the need for developing theory and engage with the managerial task and condition of innovation, we develop in the following sections a conceptual framework that delineates how measurement of innovation shapes attention. In Section 3, we introduce the attention based view and we discuss the role of attention under different levels of ambiguity. In Section 4, we present a framework of how measurement engages attention in firms and the implications of that attentional engagement for

A framework for improved measurement of innovation

After having proposed the role of attention for innovative work in the prior section, we now turn to the question of how measurement engages attention and the implications of such attentional engagement for innovation performance. According to the attention based theory, attention is situated (Ocasio, 1997, p. 188). This means that the situational context in which individuals are embedded determine which issues and answers an individual focus upon. The organizational context, its routines,

Discussion

One of the most fundamental dilemmas that firms face is finding a balance between structure and flexibility in innovation work (Brattström et al., 2012, 2015). In this paper, we address innovation measurement, a managerial practice where this dilemma is at the forefront. As our core contribution to the technology and innovation management literature, we build on current insights about the contingencies that makes different metrics optimal (e.g. Boly et al. 2014; Richtnér et al., 2017). We

Conclusion

Most firms measure innovation (Markham and Lee, 2013). Yet, several studies have suggested that measurement leads organizational members too far down the path of exploitation, while undermining creativity and the exploration of new opportunities. In this paper, we develop a theoretical framework of how different bundles (directional vs. conversational) of metrics influence attention. This allows us to propose that measurement can have both explorative and exploitative effects. Our research

References (95)

  • V. Chiesa et al.

    Development of a technical innovation audit

    J. Prod. Innov. Manage.

    (1996)
  • V. Chiesa

    Technology development control styles in multinational corporations: a case study

    J. Eng. Technol. Manage.

    (1999)
  • T. Davila

    An empirical study on the drivers of management control systems’ design in new product development

    Account. Organ. Soc.

    (2000)
  • M. Hall

    The effect of comprehensive performance measurement systems on role clarity, psychological empowerment and managerial performance

    Account. Organ. Soc.

    (2008)
  • M. Hall

    Accounting information and managerial work

    Account. Organ. Soc.

    (2010)
  • H. Hellsmark et al.

    The role of pilot and demonstration plants in technology development and innovation policy

    Res. Policy

    (2016)
  • M. Hobday

    The project-based organisation: an ideal form for managing complex products and systems?

    Res. Policy

    (2000)
  • B. Jørgensen et al.

    Accounting and strategising: a case study from new product development

    Account. Org. Soc.

    (2010)
  • C.H. Loch et al.

    Implementing a strategy‐driven performance measurement system for an applied research group

    J. Prod. Innov. Manage.

    (2002)
  • M.H. Meyer

    Locus of organizational control in the development of knowledge-based systems

    J. Eng. Technol. Manage.

    (1991)
  • J. Mouritsen et al.

    Short and long translations: management accounting calculations and innovation management

    Account. Organ. Soc.

    (2009)
  • B. Nixon

    Research and development performance measurement: a case study

    Manag. Account. Res.

    (1998)
  • K.S. Pawar et al.

    Performance measurement for product design and development in a manufacturing environment’

    Int. J. Prod. Econ.

    (1999)
  • S. Revellino et al.

    Accounting as an engine: the performativity of calculative practices and the dynamics of innovation

    Manag. Account. Res.

    (2015)
  • R. Simons

    Accounting control systems and business strategy: an empirical analysis

    Account. Organ. Soc.

    (1987)
  • S. Tessier et al.

    A conceptual development of Simons' levers of control framework

    Manag. Account. Res.

    (2012)
  • M. Wouters et al.

    Developing performance-measurement systems as enabling formalization: a longitudinal field study of a logistics department

    Account. Organ. Soc.

    (2008)
  • R. Adams et al.

    Innovation management measurement: a review

    Int. J. Manag. Rev.

    (2006)
  • P.S. Adler et al.

    Two types of bureaucracy: enabling and coercive

    Admin. Sci. Q.

    (1996)
  • T. Ahrens et al.

    Accounting for flexibility and efficiency: a field study of management control systems in a restaurant chain

    Contemp. Account. Res.

    (2004)
  • T. Amabile

    How to kill creativity

    Harv. Bus. Rev.

    (1998)
  • T.M. Amabile et al.

    Assessing the work environment for creativity

    Acad. Manag. J.

    (1996)
  • N. Anderson et al.

    Innovation and creativity in organizations a state-of-the-science review, prospective commentary, and guiding framework

    J. Manage.

    (2014)
  • S.W. Anderson et al.

    Designing quality into products: the use of accounting data in new product development

    Account. Horiz.

    (1998)
  • S.D. Anthony et al.

    Mapping Your Innovation Strategy

    (2006)
  • I. Barreto et al.

    Toward a theory of intraorganizational attention based on desirability and feasibility factors

    Strateg. Manage. J.

    (2013)
  • J. Bisbe et al.

    How control systems influence product innovation processes: examining the role of entrepreneurial orientation

    Account. Bus. Res.

    (2015)
  • M. Bourne et al.

    Designing, implementing and updating performance measurement systems

    Int. J. Operat. Prod. Manag.

    (2000)
  • A. Brattström et al.

    Similar, yet different: a comparative analysis of the role of trust in radical and incremental product innovation

    Int. J. Innov. Manag.

    (2015)
  • W.G. Bremser et al.

    Utilizing the balanced scorecard for r&d performance measurement

    R&D Manag.

    (2004)
  • M.G. Brown et al.

    Measuring r&d productivity

    Res.-Technol. Manag.

    (1998)
  • L.B. Cardinal et al.

    Balancing and rebalancing in the creation and evolution of organizational control

    Organ. Sci.

    (2004)
  • V. Chan et al.

    Assessing Innovation Metrics: Mckinsey Global Survey Results

    (2008)
  • V. Chiesa et al.

    Exploring the differences in performance measurement between research and development: evidence from a multiple case study

    R&D Manag.

    (2007)
  • V. Chiesa et al.

    Performance measurement in R&D: exploring the interplay between measurement objectives, dimensions of performance and contextual factors

    R&D Manag.

    (2009)
  • V. Chiesa et al.

    Exploring management control in radical innovation projects

    Eur. J. Innov. Manag.

    (2009)
  • R.G. Cooper

    Perspective: the innovation dilemma: how to innovate when the market is mature

    J. Prod. Innov. Manage.

    (2011)
  • Cited by (34)

    • Measuring R&D team's focus towards innovation: Scale development and empirical validation

      2023, Journal of Engineering and Technology Management - JET-M
    • Building an innovation strategy and culture on the structural characteristics of the mining industries

      2022, Resources Policy
      Citation Excerpt :

      Innovation needs metrics to measure. Various metrics are used to quantify the innovation performance of an industry or organization, such as R&D expenditures, productivity (output per person hour or day), the revenue increase obtained from innovation, the rate of revenue related to innovation to total revenue, the number of ideas, the number of patents, and the special questionnaire for company surveys (Hu et al., 2018; Brattström et al., 2018; Loh and Mortara, 2017; Ozorhon et al., 2016). Labour productivity is also frequently used as a criterion of innovativeness.

    • Communication and trust in small business innovation management

      2023, Communication, Leadership, and Trust in Organizations
    View all citing articles on Scopus
    1

    These authors gratefully acknowledge funding from VINNOVA and Hedorfs Fund. The authors also want to thank Jennie Björk, Mats Magnusson and anynomous reviewers for valuable comments.

    View full text