Next Article in Journal
Field Study on the Efficiency of a Methane Degradation Layer Composed of Fine Fraction Soil from Landfill Mining
Next Article in Special Issue
Where to Go with Corporate Sustainability? Opening Paths for Sustainable Businesses through the Collaboration between Universities, Governments, and Organizations
Previous Article in Journal
Improving Coherence in a Cross-Border Public Transport System: Lessons from the Greater Copenhagen Region
Previous Article in Special Issue
Integrating Sustainability in the Quality Assessment of EHEA Institutions: A Hybrid FDEMATEL-ANP-FIS Model
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An ANP-Balanced Scorecard Methodology to Quantify the Impact of TQM Elements on Organisational Strategic Sustainable Development: Application to an Oil Firm

by
Carla Andrade Arteaga
1,2,
Raúl Rodríguez-Rodríguez
3,*,
Juan-José Alfaro-Saiz
3 and
María-José Verdecho
3
1
Doctorate School, Universidad de Guayaquil, Guayaquil 090613, Ecuador
2
Doctorate School, Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain
3
Research Centre on Production Management and Engineering (CIGIP), Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain
*
Author to whom correspondence should be addressed.
Sustainability 2020, 12(15), 6207; https://doi.org/10.3390/su12156207
Submission received: 2 July 2020 / Revised: 22 July 2020 / Accepted: 30 July 2020 / Published: 1 August 2020
(This article belongs to the Special Issue Synergies between Quality Management and Sustainable Development)

Abstract

:
This paper presents a methodology for quantifying the impact of Total Quality Management TQM elements on organisational strategic sustainable development, integrating within it the well-known strategic management tool of Balanced Scorecard to represent the strategic part of the organisations, and the multi-criteria technique Analytic Network Process (ANP) to identify and quantify the mentioned impact. Additionally, the application of TQM generates directly some organisational improvements—or outputs—which help model a decisional ANP network constituted by all three building blocks—TQM elements, strategic objectives and outputs—and their interrelationships. The application of the methodology to an oil firm carried out by an expert group offered, from a decision-making point of view, meaningful results that were developed following three different analyses: Global analysis, which identified the global weight of each variable; Analysis of Influences, which established sound cause–effect relationships between the variables to identify the elements—TQM and outputs—that are more important to achieve the strategic objectives; and the Integrated analysis, which pointed out which TQM elements should be fostered in order to achieve the most important sustainable strategic objectives. Finally, it is suggested to apply the methodology to other types of size and sector activity organisations, as well as to use other techniques that introduce fuzzy elements.

1. Introduction

In the current environment of globalisation, technological development, processes of political and economic integration and the development of new potential participants, economies and companies are forced to be increasingly competitive in creating value for customers. In this sense, one key way to achieve competitiveness is to encourage individuals, institutions and companies, whether small or medium, to implement Total Quality Management (TQM) elements as a way to learn to identify the mechanisms that improve organisational performance and business competitiveness [1,2].
Then, even though there are few works that have reported either an inverse [3] or no relationship [4,5] between the implementation of TQM elements and business performance, it is generally assumed that such an implementation can improve organisational performance to a certain level, because it is possible to find many scientific works that support this affirmation [6,7,8,9,10,11,12,13,14,15,16,17]. However, the quantification of the impact of TQM on organisational performance and, more directly, on organisational strategic sustainable development is an exciting and up-to-date research topic, which is dealt with in this paper.
The structure of this paper is as follows: Section 2 presents a literature review, highlighting the research gap; Section 3 explains the developed methodology; Section 4 presents the main results of application it to an oil organisation; Section 5 summarises the main theoretical and practical contributions, as well as limitations and future research; finally, Section 5 highlights the main conclusions.

2. Literature Review

2.1. TQM and Performance Measurement

The works that measure the impact of TQM elements on business performance can be classified, from a performance measurement point of view, into (i) non structured works and (ii) structured works.
The non-structured works are those that cover the impact of TQM elements either on a specific part or on the whole organisation, without following any type of performance measurement framework guideline. Regarding the former, the main works focus on studying the impact of applying TQM on the performance of some key areas of the organisation, such as financial [18,19,20], customer [21,22,23] or process [24,25,26]. Additionally, there are other works that measure the impact of TQM on more than one key area at the same time; for example, [6] studied the relationships among TQM, plant performance and customer satisfaction. On the other hand, regarding the latter, there are some works that have studied the impact of TQM on the organisation as a whole from different perspectives [27,28,29,30,31]. However, none of these works followed a structured approach in terms of a performance measurement framework, which means that the results achieved are limited from a global and strategic point of view. In general, it is possible to conclude that these works are unbalanced ones, providing a partial vision of the impact of TQM on the organisation.
Regarding the structured works, these measure the impact of TQM elements on the whole business organisation. At the same time, they follow some type of performance measurement framework guideline, which enables them to have a structured approach leading to better determination of the impact of the application of TQM on the organisational performance. In this sense, references [32] pointed out the suitability of using a performance measurement system (PMS) and, more effectively, the Balanced Scorecard (BSC) [33] to capture the relationship between TQM and organisational performance, because such systems provide a balanced and global vision of business performance. The BSC is the most well-known and widely used performance measurement system, as it defines, from the point of view of organisational strategy, the main strategic objectives to be accomplished. These strategic objectives are defined via the main key areas of an organisation that they cover: Financial, Customers, Processes, and Learning and Growth. This structure allows a properly balanced approach, whereby the lower perspectives (Processes and Learning and Growth) support the upper ones (Financial and Customer).
Within this structured works section, it is worth mentioning the study in [34], wherein the impact of TQM elements on a set of balanced key performance indicators in the automotive industry is determined. Further, [35] investigated the impact of soft TQM on financial performance from a BSC non-financial perspective, affirming that there was a meaningful impact on the financial performance due to the effect of the customer perspective, which was directly impacted by the soft TQM elements.
As such, there are no studies that have fully investigated the impact of TQM elements, both soft and hard, on strategic organisational performance using a whole Balanced Scorecard approach.

2.2. Strategic Sustainable Development and Performance Measurement

Currently, more and more organisations have sustainability among their priorities, and include it within their strategic goals. They invest resources in achieving a strategic sustainable organisational development and design, and apply different performance measurement approaches so as to measure to what extent such investments are paying off. Then, focusing on the BSC, which already incorporates a perspective for defining strategic economic (financial) objectives, it is possible to find works that either have or have not incorporated perspectives in addition to the classic four in order to measure both social and environmental sustainability [36]. Then, the works that have included these two additional perspectives [37,38,39,40] affirm that the strategic objectives defined from the environmental perspective should be related to circular economy, so as to reduce waste, reuse materials, reduce energy consumption, etc. [41]. On the other hand, the strategic objectives of the social perspective should have to do with improving decent work and labor conditions, product responsibility, health and safety, equal opportunities, etc. [42]. Furthermore, when using the classic BSC to measure the strategic sustainable development of an organisation, it will be possible to classify some of the strategic objectives of the different perspectives into one of the three sustainability dimensions, making it then possible to measure not the whole sustainable development of the organisation, but the strategic sustainable development as it is reflected in the strategic objectives defined within the BSC. The main advantage of this approach is that it does not require the addition of any other perspective to the traditional ones of the BSC; additionally, if the strategic objectives are defined via a sustainability approach, the entire BSC will be perfectly integrated without much effort. Finally, from a practical point of view, management teams usually use the traditional version of the BSC.

2.3. TQM and Sustainable Development

Further, TQM’s main aim is to foster continuous improvement across an organisation, which is a long-term objective and directly impacts sustainability [43,44]. There are few works that deal with identifying how TQM affects the sustainable strategic development of an organisation [45]. In this sense, reference [46] affirms that TQM meaningfully affects corporate sustainability and develops a TQM-Knowledge management proposal, and [47] concludes that TQM has a significant and positive impact on corporate green performance. In any case, none of the previous works follow any structured performance measurement framework.

2.4. Quantitative Techniques

In order to quantify the impact of TQM elements on organisational strategic sustainable performance, it is necessary to choose an effective technique in order to be able to properly take into account all of the existing relationships. Such a technique could be a statistical one, if there were historical data available to correlate the degree of achievement of the strategic objectives derived from the Balanced Scorecard with the application of TQM elements. However, the results obtained would be difficult to interpret and trust, because data coming from the application of TQM is usually available in operative-tactical contexts, whereas the objectives of the organisation are defined in the highest strategic context. Therefore, the technique to be used should be one that is able to provide additional information to organisations in multi-criteria situations, wherein all of the variables of the network are important and it is possible to get to know each variable’s weight in the network, and to establish how some of them have an influence on achieving others [48]. Additionally, as a consequence of modelling the problem as a decisional multi-criteria network, it is possible to introduce other elements whose impact could be important from a decision-making point of view. In this research, they are called outputs that, as will be presented in the Methodology section, are improvements generated from applying the TQM elements.
Regarding the specific technique to be used, some previous studies have successfully applied the Analytic Network Process [49] to similar problems; reference [50] applied ANP to link intangible assets and organisational performance within a Balanced Scorecard context, and reference [51] used ANP to manage collaborative relationships and their impacts on strategic performance. Additionally, ANP does not follow any hierarchy like other multi-criteria methods, such as AHP [52], which allows one to carry out a sensitivity analysis in an easier way.

2.5. Research Gap

All of the above reflections contextualise the problem associated with quantifying the impact of TQM elements on strategic sustainable organisational development, and justify the development of an effective and easily replicable methodology. In this sense, this paper covers a research gap as it: (i) develops a BSC-ANP-based methodology to quantify the impact of TQM elements on strategic sustainable organisational development, which will take into account the impact of TQM elements, both soft and hard, on strategic organisational performance using a whole BSC, instead considering only the impact of soft TQM on financial performance through the non-financial BSC perspectives of [35]; (ii) uses a structured approach, the BSC, to manage the organisational strategic sustainable development, as defined within such a BSC, which is a novel approach within the TQM and sustainable development area [46,47]; (iii) it applies a multi-criteria technique, the ANP, to quantify the impact of TQM on elements of organisational strategic sustainable development.

3. The ANP-BSC Methodology

The main objective of this paper is to develop and implement a methodology to quantify the impact of applying TQM elements to business strategic performance within a sustainability context. More directly, the business strategic performance will be measured through a Balanced Scorecard, and it will contain the organisation’s strategic sustainable development. The phases of the proposed methodology are presented next.
Phase 1. Definition of the PMS and the expert group
Firstly, it is necessary to create an expert group that will go through the whole decision-making process when implementing the methodology. This expert group will ideally be multi-disciplinary and formed of workers from different levels of the organisation, especially from both the strategic level, which is driven by the top-managers and directors of the organisation, and the tactical level, made up of workers who exercise greater control in TQM practices. In addition, it is also advisable to include external consultants, when possible, to broaden the internal vision.
On the other hand, it is necessary that the organisation has already defined a PMS in order to be able to identify the strategic objectives, which will cover the entire organisation with a balanced approach. In this sense, it is possible to use different PMSs, i.e., the well-known BSC [33], the Performance Prism [53], or other PMSs specifically designed for integrating sustainability (i.e., [37,38,39,40]). Then, the expert group will identify the strategic objectives that can be classified into one of the sustainability dimensions, therefore representing the strategic sustainable development of the organisation.
Phase 2. Selection of the TQM elements and definition of the outputs
The application of TQM elements will allow both the achievement of the organisational strategic objectives and also the creation of some important OIutputs or improvements, which represent direct organisational improvements. Then, a network with relationships between its three main building blocks—PMS, TQM elements and outputs—is established, as Figure 1 illustrates.
In general, the application of TQM elements will directly influence the achievement of the strategic objectives, both sustainable and non-sustainable, defined within the PMS. Additionally, such TQM elements will create some organisational improvements called outputs, i.e., cost reduction, quality assurance or customer satisfaction. At the same time, the achievement of the strategic objectives will also influence both the TQM elements and the outputs. Finally, the outputs will also influence the achievement of the strategic objectives.
Then, from a TQM elements perspective, it can be concluded that its application influences both directly and indirectly (through the generation of outputs) the achievement of the strategic objectives of the organisation, and the generation of important organisational improvements (outputs). Regarding the former, it will be possible to specify which TQM elements and outputs meaningfully influence the achievement of the strategic objectives in general, and the sustainable development of the organisation in particular, as represented by the sustainable strategic objectives previously identified in Phase 1.
Therefore, in this step the expert group should do the following:
Select the TQM elements that will be included in the study. They should select the TQM elements that are already implemented in the organisation.
Define the outputs that they consider will be created as a consequence of applying the TQM elements previously selected.
Phase 3. Determination and quantification of meaningful influences
Once the network has been defined, regarding the three building blocks, it is time to determine and quantify the main meaningful influences within the network. To this end, the Analytic Network Process (ANP) is applied. According to [49], the ANP is a method that comprises four main steps, which are presented next.
Step 1. Determination of the components, network elements and their relationships
This is the most important step of the method [54], as decision-makers should design the network as a decision problem. In doing so, they need to (i) identify the decision criteria and alternatives (decision elements); (ii) group these elements into clusters with common features; and iii) analyse the dependences between network elements, as either inner-dependences (between elements of the same cluster) or outer-dependences (between elements of different clusters). Then, a zero–one matrix, the Dependence matrix, is developed, where a 0 means that decision-makers believe that there is no dependence of one element on other, and 1 is the opposite case. Furthermore, a 1 in a concrete row–column intersection of the Dependence matrix means that the element in the row has some meaningful influence on the element in the column.
Step 2. Determination of the inner priorities
The aim is to identify the relative weight of each element within a cluster. To do so, pairwise comparisons between these elements are carried out, obtaining the eigenvectors for each element, which results in the Unweighted supermatrix.
Step 3. Determination of the outer priorities
A procedure based on pair-wise comparisons between clusters is applied and the cluster weights are obtained which, when multiplied by the Unweighted supermatrix, outputs the Weighted supermatrix, which is a column-stochastic matrix.
Step 4. Determination of the Limit supermatrix
This is created by raising the Weighted supermatrix to successive powers until it converges, which is achieved when all the columns of the Limit supermatrix have the same values. The values of these equal columns represent the global priorities of the elements of the network.
Phase 4. Analysis of results
Once the results of the ANP have been obtained, they are analysed from different linked perspectives to develop the following three analyses: (1) Global analysis; (2) Analysis of Influences; (3) Integrated analysis.
The Global analysis is based on the global importance of each of the elements of the network as calculated in the Limit supermatrix. Then, it will be possible to establish a ranking of importance that classifies the elements following an ABC analysis. Further, structural block ABC analysis will be carried out, which will show which elements are key (class A) for each of these blocks.
On the other hand, the Analysis of Influences is based on the results of the Weighted supermatrix, which points out the cause–effect relationships between elements of the network. Then, directly from the supermatrix, a graph will be drawn which points out the most important elements from both the TQM elements and the outputs that are meaningfully influencing the strategic objectives of the organisation, paying special attention to the sustainable ones. Therefore, it will be possible to identify the elements that are more important in achieving these strategic objectives.
Finally, the Integrated analysis, which is based on the results of both the Global analysis and the Analysis of Influences, will present to decision-makers which TQM elements they should foster in order to achieve the most important sustainable strategic objectives. With the construction of a coverage table, the analysis will identify the minimum combination of TQM elements that covers and influences all of the sustainable strategic objectives.
Phase 5. Sensitivity analysis
A sensitivity analysis is carried out in this phase in order to determine the robustness of the ANP model when perturbations are introduced either on the Dependence matrix or on the Weighted matrix.
Finally, it important to point out that the free software “Superdecisions V2.10” has been used to support phases 3–5 of this research.

4. Results

The methodology was applied to an organisation of the Petroleum, Fuels and Mining sector of Ecuador. This organisation has about 125 workers and in 2019 billed about USD 16,000.000. Its main activities are to exercise industrial activity in the elaboration and distribution of oils and lubricants. The organisation is structured into different departments (mainly Finance, Operations, Sales, Purchasing, Human Resource Management and IT) and had already defined, for the year 2020, a Balanced Scorecard following the strategic line of Operational Excellence. Next, the main results of the application are presented.
Phase 1. Definition of the PMS and the expert group
An expert group was formed with six people from the organisation and one external consultant; the first author of the paper was this external consultant. Regarding the internal personnel, there were two top managers from Operations and Strategy and four middle-class technical managers from Operations. This expert group applied the methodology for 5 months, holding meetings every 2 weeks.
On the other hand, the organisation had already defined and implemented a Balanced Scorecard, whose strategic objectives are shown in Table 1. In this table, the expert group identified the strategic objectives that could be classified into one of the sustainability dimensions. Then, it can be seen from the 12 strategic objectives defined in Table 1 that 8 fit in one of the sustainability dimensions, 3 in the economic, 2 in the environmental, and 3 in the social. These strategic objectives constitute the structural block of the Balanced Scorecard, and the four perspectives are the four clusters of such a block.
Phase 2. Selection of the TQM elements and definition of the outputs
The expert group selected 20 TQM elements that the organisation was already implementing, which are shown in Table 2. Additionally, the expert group defined the next nine outputs or organisational improvements that will be generated as a result of implementing the 20 TQM elements: (1) Cost reduction, (2) Process improvement, (3) Quality assurance, (4) Efficiency in product delivery, (5) Increase in motivation and satisfaction level, (6) Increase in customer satisfaction, (7) Sales increase, (8) Productivity increase and (9) Information and knowledge. From a sustainable development viewpoint, some of these outputs are next classified into one of the sustainability dimensions:
  • Economic. Output 1 (Cost reduction), Output 7 (Sales increase), Output 9 (Information and knowledge);
  • Environmental. Output 4 (Efficiency in product delivery), Output 3 (Quality assurance), Output 2 (Process improvement);
  • Social. Output 5 (Increase in motivation and satisfaction level).
Output 6 (Increase in customer satisfaction) will help in achieving the economic objectives of the organisation, whereas Output 8 (Productivity increase) is not included within the environmental classification, since depending on how such an increment of productivity will be achieved (either following green production practices or not), Output 8 would either be aligned with the environmental dimension or not.
Phase 3. Determination and quantification of meaningful influences
In this step, the expert group applied the different steps of the ANP. Then, in step 1 they designed the ANP network as a decision problem, obtaining a total of 9 clusters divided into criterion clusters (Financial perspective, Customer perspective, Process perspective, and Learning and Growth perspective) and alternative clusters (Quality control, Quality management, Lean production, Collaborative factors and outputs). Table 2 presents the elements grouped by clusters of the structural blocks of TQM elements and outputs.
Then, the Dependence matrix was constructed (see Figure 2), from which both the inner- and the outer-dependences were established, and the ANP network was fully designed (see Figure 3).
As a result of applying Steps 2, 3 and 4, the Unweighted supermatrix, Weighted supermatrix and the Limit supermatrix were generated. Regarding the latter, this is the one that provides a global vision of the individual importance of each element of the network, and it is presented in Table 3 following a ranked ABC classification approach.
Phase 4. Analysis of results
Global Analysis
From a global point of view, the most important elements of the network are those that have achieved an A class in the Limit supermatrix. Then, from Table 3, it is possible to observe that the three most important elements are the two financial objectives (FO1 and FO2) and the process objective (PO1), which together account for about one third of the total importance. These three are strategic sustainability objectives that indicate the degree of importance that sustainability has in this network and, more broadly, in the organisation from a strategic standpoint. Further, the rest of the A class elements are six outputs, one customer perspective strategic objective and three TQM elements. Regarding the latter, the LP4 (Error-proof), the QC1 (Simulation) and the QC2 (Sampling) have come up as the most important ones globally. On the other hand, the outputs Out2 (Process improvement) and Out8 (Productivity increase) present a high percentage of global importance, 8.58% and 8.24%, respectively.
Then, the B class stands for about 16% of the global value, and it is composed of elements from the three structural blocks, mainly from the TQM: QPI2, LP2, LP3, LP2 and QPI4.
Finally, the C class accounts for about 5% of the global network importance, and it contains the rest of the TQM elements, which are mainly the ones from the Collaborative factors and Quality planning and Improvement clusters, many of the strategic objectives from the Customer perspective and Learning and Growth, and only one output.
Then, a similar ABC classification at the structural block level has been carried out, which is shown next.
In this sense, Figure 4 shows the elements of the BSC structural block ranked according to their global importance in the Limit supermatrix. Then, the A class is formed by only sustainability strategic objectives (FO1, PO1 and FO2), whereas the other sustainability strategic objectives are ranked in either the B class (OCO2, LGO2 and LGO4) or in the C class (LGO1 and LGO3).
Then, it is possible to affirm that the Economic sustainability strategic objectives are of either key importance (A class), represented by the FO1 and FO2, or relative importance (B class), represented by the LGO2. The Environmental sustainability strategic objectives are either of key importance (PO1) or relative importance (OCO2), and the Social ones are either of relative importance (LGO4) or low importance (C class), represented by the LGO1 and the LGO3.
Similarly, regarding the TQM structural block, Figure 5 shows its elements according to their global importance in the Limit supermatrix. Then, the key TQM elements, those classified into the A class, are LP4, QC1, QC2, QPI2, LP2, LP3 and LP1. This indicates that the clusters of Lean Production and Quality Control are the most important within the TQM structural blocks. On the other hand, the B elements are QPI4, QPI3, CF7 and CF2, which can be reported as being of relative importance; the rest of the TQM elements are of low importance.
Influences analysis
From the obtained Weighted supermatrix (Figure 6), it is possible to conduct a cause–effect analysis that points out the effect of both the TQM elements and the outputs on the strategic objectives of the network.
In this sense, a graph has been produced (see Figure 7) that shows such relationships found in the Weighted supermatrix, wherein a continuous line reflects direct meaningful relationships and a discontinuous line represents a medium–low impact relationship.
Focusing on the sustainability strategic objectives, highlighted in green colour, and the influences that these undergo from both TQM elements and outputs, it is possible to affirm the following:
  • Economic dimension. FO1 and FO2 receive a strong influence only from LP4 (Error-proof), whereas LGO2 is strongly influenced by both the TQM element QC1 (Simulation) and the Out7 (Sales increase). Additionally, FO1 and FO2 are influenced with a medium–low intensity by the TQM elements LP2 (Automation), LP3 (Just in time) and QPI2 (Continuous improvement), and by Out2 (Process improvement), Out3 (Quality assurance), Out4 (Efficiency in product delivery), Out7 (Productivity increase) and Out9 (Information and knowledge). On the other hand, LGO2 is relatively influenced by QC2 (Sampling), QPI2 (Continuous improvement) and Out 1 (Cost reduction), Out3 (Quality assurance) and Out 4 (Efficiency in product delivery).
  • Environmental dimension. PO1 is strongly influenced by QC1 (Simulation) and relatively influenced by all of the TQM elements, QC2 (Sampling), LP2 (Automation), LP4 (Error-proof), QPI4 (Recognition and reward systems), Out7 (Productivity increase) and Out8 (Interoperability). On the other hand, CO2 is strongly influenced by LP4 (Error-proof) and relatively influenced by al; of the TQM elements LP2 (Automation), LP1 (Redistribution of work), QPI1 (Customer needs), CF4 (Information sharing process), and Out2 (Process improvement) and Out9 (Information and knowledge).
  • Social dimension. LGO1 and LGO4 are strongly influenced by the TQM element QC2 (Sampling) and by Out5 (Increase in motivation and satisfaction level), Out7 (Sales increase), Out8 (Productivity increase) and Out9 (Information and knowledge). On the other hand, LGO1, LGO3 and LGO4 are moderately influenced by both of the TQM elements, CF4 (Information sharing process), CF5 (Joint decisions), CF6 (Process alignment), QPI3 (Training), QPI5 (Recognition and rewards systems) and Out1 (Cost reduction), Out5 (Increase in motivation and satisfaction level) and Out6 (Information and knowledge).
Then, decision-makers have additional information when interpreting these cause–effects relationships. For instance, if the main aim is to reach FO1, then they should firstly foster the following:
  • Strong direct influences, which are produced by LP4 (Error-proof);
  • Medium–low direct influences, which are produced by LP2 (Automation), LP3 (Just in time) and QPI2 (Continuous improvement).
Integrated analysis
Table 4 shows the result of integrating both the Global analysis and the Influences analysis. Then, following a coverture method, it is possible to reduce the dimensionality of the options to the minimum and offer to decision-makers the following recommendations, when they are willing to foster and achieve both the key and non-key sustainability strategic objectives:
  • The organisation should primarily focus on, foster and maintain the TQM elements of LP4 (Error-proof) and QC1 (Simulation), which will strongly influence the three key sustainable strategic objectives (FO1, PO1, and FO2);
  • Secondly, they should foster and augment the investment in LP2 (Automation) and either LP3 (Just in time) or QPI2 (Continuous improvement), which will moderately influence the achievement of the three key sustainable strategic objectives (FO1, PO1 and FO2).
  • Thirdly, they should foster QC2 (Sampling), which strongly affects both the LGO4 and LGO1.
Finally, they should foster either CF4 (Information sharing process) or QPI3 (Training), which moderately influences LGO3.
Phase 5. Sensitivity analysis
At this point, several single, double and triple perturbations of different variables of the network were carried out. As an illustration, in Figure 8 it is possible to observe the evolutions, continuous and linear, of the nine outputs when an input was perturbed. Such behaviour was achieved when perturbing all of the 20 TQM elements. This means that the ANP network was stable and consistent when the TQM elements were perturbed.

5. Discussion

This section highlights not only the theoretical and the practical contributions of this research, but also its limitations and the scope for future work.
This paper offers a theory about TQM and performance measurement, as it develops a novel methodology that quantifies, by applying the ANP, the impact of TQM elements, both soft and hard, on strategic organisational performance using a whole BSC, which complements the work of [35]. This approach provides a global vision with a global impact, instead of a partial one, as expressed by [33]. Additionally, this research also extends the works that jointly deal with both TQM and sustainable development, as these works focus on either investigating how TQM affects corporate sustainability [46] or on identifying the positive impact of TQM on corporate green performance [47], whereas the current research uses a structured approach, the BSC, to manage the organisational strategic sustainable development, as defined within such a BSC. Finally, this research also complements the current literature by: (i) defining organisational improvements or outputs, generated as a result of applying TQM elements; (ii) modelling the problem as a decisional ANP network, which can be broken down into three levels (building blocks, clusters and elements); (iii) defining three types of analysis (Global analysis, Influences analysis and Integrated analysis) of the results to show the impact of TQM on organisational sustainable development.
On the other hand, from a practical point of view, the main contributions of this research are: (i) providing decision-makers with additional information to better understand how, and to what extent, investing in TQM elements contributes to achieving strategic sustainable organisational objectives; (ii) determining which TQM elements are the most important ones when aiming to achieve the organisation’s sustainable strategic objectives; (iii) ranking the sustainable strategic objectives according to their importance for the organisation; (iv) prioritising the TQM elements in order to reach the most important sustainable strategic objectives, pointing out in which of them the organisation should invest, not only first, but also more.
The limitations of this study are related to: (i) the application of the proposed methodology to only one organisation; (ii) the influence of TQM elements on the strategic objectives and outputs that come from the evaluations of the group of experts, deriving from them the Dependence matrix and, therefore, the relationships between the variables. A group of experts made up of different members could evaluate these relationships differently, which would lead to other results; (iii) the BSC is used to classify the strategic objectives into the sustainability dimensions, and other performance measurement frameworks could be used; (iv) the use of the BSC to manage the organisational strategic sustainable development, as defined within such a BSC with the strategic objectives, which does not measure the whole organisational sustainable development.
Future research works could: (i) use other performance measurement frameworks, then making it possible to analyse and compare the results obtained with the two approaches, the BSC and the new one; (ii) apply the methodology to organisations of other sizes, either smaller or larger ones; (iii) apply the methodology to service organisations; (iv) use other multi-criteria techniques that incorporate fuzzy elements, such as Fuzzy-ANP.

6. Conclusions

The link between TQM and strategic sustainable development is a valuable and current research theme, as it would allow a better understanding of how and to what extent investing in TQM elements contributes to achieving strategic sustainable organisational objectives. In this sense, this paper aimed to contribute to this issue by developing an effective and easily replicable methodology for quantifying such an impact, and presenting the main results achieved from its application to a real world organisation. Furthermore, the methodology was based on the ANP multi-criteria technique in order to quantify not only the role of TQM elements in achieving the organisational strategic objectives defined within a Balanced Scorecard context, but also the generation of improvements or outputs as a direct result of implementing these TQM elements.
This methodology defined three structural blocks (TQM elements, strategic objectives and outputs), which contained different clusters, as well as the existing relationships of all the elements of these clusters, thus modelling a decisional ANP network. Once the corresponding matrices (Unweighted, Weighted, Limit and Cluster) were solved, three different analyses were carried out: the Global analysis, to discover the individual global importance of each variable of the network; the Analysis of Influences, to determine the cause–effect relationships between elements of the network; and the Integrated analysis, to highlight which TQM elements should be fostered in order to achieve the most important sustainable strategic objectives. Finally, a sensitivity analysis to assess the robustness of the ANP network was carried out.
In this sense, the application of the methodology to an oil organisation of about 125 workers mainly revealed the most important variables of the network, and prioritised the TQM elements in order to reach the most important sustainable strategic objectives.

Author Contributions

All authors have meaningfully collaborated in developing all the sections of the paper. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Hardie, N. The effects of quality on business performance. Qual. Manag. J. 1997, 4, 65–83. [Google Scholar] [CrossRef]
  2. Lee, C.L.; Yang, H.J. Organization structure, competition and performance measurement systems and their joint effects on performance. Manag. Account. Res. 2011, 22, 84–104. [Google Scholar] [CrossRef]
  3. Davis, T. Breakdowns in total quality management. Int. J. Manag. 1997, 14, 13–23. [Google Scholar]
  4. Powell, T.C. Total quality management as competitive advantage: A review and empirical study. Strateg. Manag. J. 1995, 16, 15–37. [Google Scholar] [CrossRef]
  5. Westphal, J.D.; Gulati, R.; Shortell, S.M. The institutionalization of total quality management: The emergence of normative TQM adoption and the consequences for organizational legitimacy and performance. Acad. Manag. Procedia 1996, 14, 249–253. [Google Scholar] [CrossRef]
  6. Choi, T.; Eboch, K. The TQM paradox: Relations among TQM practices, plant performance, and customer satisfaction. J. Oper. Manag. 1998, 17, 59–75. [Google Scholar] [CrossRef]
  7. Brah, S.A.; Tee, S.S.; Madhu Rao, B. Relationship between TQM and performance of Singapore companies. Int. J. Qual. Reliab. Manag. 2002, 19, 356–379. [Google Scholar] [CrossRef]
  8. Shenaway, E.E.; Baker, T.; Lemak, D.J. A meta-analysis of the effect of TQM on competitive advantage. Int. J. Qual. Reliab. Manag. 2007, 25, 42–61. [Google Scholar]
  9. Zakuan, N.M.; Yusof, S.M.; Laosirihongthong, T.; Shaharoun, A.M. Proposed relationship of TQM and organisational performance using structured equation modelling. Total Qual. Manag. 2010, 21, 185–203. [Google Scholar] [CrossRef]
  10. Corredor, P.; Goñi, S. TQM and performance: Is the relationship so obvious? J. Bus. Res. 2011, 64, 830–838. [Google Scholar] [CrossRef] [Green Version]
  11. Talib, F.; Rahman, Z.; Qureshi, M.N. An empirical investigation of relationship between total quality management practices and quality performance in Indian service companies. Int. J. Qual. Reliab. Manag. 2013, 30, 280–291. [Google Scholar] [CrossRef]
  12. Ahmad, M.; Zakuan, N.; Jusoh, A.; Takala, J. Relationship of tqm and business performance with mediators of spc, lean production and tpm. Procedia-Soc. Behav. Sci. 2012, 65, 186–191. [Google Scholar] [CrossRef] [Green Version]
  13. Kaur, P.; Sharma, S.K. Evaluating the relationship and influence of critical success factors of tqm on business performance: Evidence from SMEs of manufacturing sector. IUP J. Oper. Manag. 2014, 13, 17–30. [Google Scholar]
  14. García-Bernal, J.; Ramírez-Alesón, M. Why and how tqm leads to performance improvements. Qual. Manag. J. 2015, 22, 23–37. [Google Scholar] [CrossRef]
  15. Farish, K.; Anil, A.P.; Satish, K. Effect of TQM Practices on Financial Performance through Innovation Performance-In Indian Manufacturing Context. Int. Res. J. Eng. Technol. 2017, 4, 2649–2655. [Google Scholar]
  16. Qasrawi, B.T.; Almahamid, S.M.; Qasrawi, S.T. The impact of TQM practices and KM processes on organisational performance: An empirical investigation. Int. J. Qual. Reliab. Manag. 2017, 34, 1034–1055. [Google Scholar] [CrossRef]
  17. Singh, V.; Kumar, A.; Singh, T. Impact of TQM on organisational performance: The case of Indian manufacturing and service industry. Oper. Res. Perspect. 2018, 5, 199–217. [Google Scholar] [CrossRef]
  18. Hendricks, K.B.; Singhal, V.R. Firm characteristics, total quality management, and financial performance. J. Oper. Manag. 2001, 19, 269–285. [Google Scholar] [CrossRef]
  19. Hansson, J.; Eriksson, H. The impact of TQM on financial performance. Meas. Bus. Excell. 2002, 6, 44–54. [Google Scholar] [CrossRef] [Green Version]
  20. York, K.M.; Miree, C.E. Causation or covariation: An empirical re-examination of the link between TQM and financial performance. J. Oper. Manag. 2004, 22, 291–311. [Google Scholar] [CrossRef]
  21. Gündüz, G.Ş.; Gündüz, Ç. Examining the perception of quality and customer satisfaction within the framework of total quality management in textile sector. Tekst. Ve Konfeksiyon 2017, 27, 101–107. [Google Scholar]
  22. Jaiswal, K.; Garg, M. Impact of TQM on employee and customer satisfaction-A study on tricity region. 8M: J. Indian Manag. Strategy 2018, 23, 28–35. [Google Scholar] [CrossRef]
  23. Hwang, G.H. The impact of TQM practices on employee satisfaction in the ICT customer service and retail distribution organizations. J. Korean Soc. Equip. Manag. 2015, 20, 39–50. [Google Scholar]
  24. Zimon, D. The impact of TQM philosophy for the improvement of logistics processes in the supply chain. Int. J. Qual. Res. 2017, 11, 3–16. [Google Scholar]
  25. Ooi, K.B.; Arumugam, V.; Teh, P.L.; Chong, A.Y.L. TQM practices and its association with production workers. Ind. Manag. Data Syst. 2008, 108, 909–927. [Google Scholar] [CrossRef] [Green Version]
  26. Dubey, R.; Gunasekaran, A. Exploring soft TQM dimensions and their impact on firm performance: Some exploratory empirical results. Int. J. Prod. Res. 2015, 53, 371–382. [Google Scholar] [CrossRef]
  27. Basu, R.; Bhola, P. Impact of quality management practices on performance stimulating growth Empirical evidence from Indian IT enabled service SMEs. Int. J. Qual. Reliab. Manag. 2016, 33, 1179–1201. [Google Scholar] [CrossRef]
  28. Shafiq, M.; Lasrado, F.; Hafeez, K. The effect of TQM on organisational performance: Empirical evidence from the textile sector of a developing country using SEM. Total Qual. Manag. Bus. Excell. 2019, 30, 31–52. [Google Scholar] [CrossRef]
  29. Feng, J.; Prajogo, D.I.; Tan, K.C.; Sohal, A.S. The impact of TQM practices on performance: A comparative study between Australian and Singaporean organizations. Eur. J. Innov. Manag. 2006, 9, 269–278. [Google Scholar] [CrossRef]
  30. Kaynak, H. The relationship between total quality management practices and their effects on firm performance. J. Oper. Manag. 2003, 21, 405–435. [Google Scholar] [CrossRef]
  31. Wei, J.T.; Chang, Y.W.; Zhang, X.; Wu, H.H.; Tang, Y.T. Performance measurement systems, TQM and multi-level firm performance: A person–organisation fit perspective. Total Qual. Manag. Bus. Excell. 2019, 30, 1578–1595. [Google Scholar] [CrossRef] [Green Version]
  32. Hoque, Z. Total quality management and the balanced scorecard approach: A critical analysis of their potential relationships and directions for research. Crit. Perspect. Account. 2003, 14, 553–566. [Google Scholar] [CrossRef]
  33. Kaplan, R.S.; Norton, D.P. The Balanced Scorecard. Measures that Drive Performance. Available online: https://steinbeis-bi.de/images/artikel/hbr_1992.pdf (accessed on 31 July 2020).
  34. Sabbagh, O.; Ab Rahman, M.N.; Ismail, W.R.; Hussain, W.; Hirwani, W.M. The impact of TQM practices on key performance indicators: Empirical evidence from automotive dealerships. Econ. Manag. 2019, 22, 115–129. [Google Scholar] [CrossRef]
  35. Albuhisi, A.M.; Abdallah, A.B. The impact of soft TQM on financial performance: The mediating roles of non-financial balanced scorecard perspectives. Int. J. Qual. Reliab. Manag. 2018, 35, 1360–1379. [Google Scholar] [CrossRef]
  36. Figge, F.; Hahn, T.; Schaltegger, S.; Wagner, M. The sustainability balanced scorecard: Linking sustainability management to business strategy. Bus. Strategy Environ. 2002, 11, 269–284. [Google Scholar] [CrossRef]
  37. Ferreira, L.; Silva, C.; Garrido Azevedo, S. An environmental balanced scorecard for supply chain performance measurement. Benchmarking Int. J. 2016, 23, 1398–1422. [Google Scholar] [CrossRef]
  38. Motevali-Haghighi, S.; Torabi, S.A.; Ghasemi, R. An integrated approach for performance evaluation in sustainable supply chain networks. J. Clean. Prod. 2016, 137, 579–597. [Google Scholar] [CrossRef]
  39. Valenzuela, L.; Maturana, S. Designing a three-dimensional performance measurement system for the wine industry: A Chilean example. Agric. Syst. 2016, 142, 112–121. [Google Scholar] [CrossRef]
  40. Qorri, A.; Mujki, Z.; Kraslawski, A. A conceptual framework for measuring sustainability performance of supply chains. J. Clean. Prod. 2018, 189, 570–584. [Google Scholar] [CrossRef]
  41. Verdecho, M.-J.; Alarcón-Valero, F.; Pérez-Perales, D.; Alfaro-Saiz, J.-J.; Rodríguez-Rodríguez, R. A methodology to select suppliers to increase sustainability within supply chains. Cent. Eur. J. Oper. Res. 2020. [Google Scholar] [CrossRef]
  42. Popovic, T.; Kraslawski, A.; Barbosa-Póvoa, A.; Carvalho, A. Quantitative indicators for social sustainability assessment of society and product responsibility aspects in supply chains. J. Int. Stud. 2017, 10, 9–36. [Google Scholar] [CrossRef] [PubMed]
  43. Siva, V.; Gremyr, I.; Bergquist, B.; Garvare, R.; Zobel, T.; Isaksson, R. The support of Quality Management to sustainable development: A literature review. J. Clean. Prod. 2016, 138, 148–157. [Google Scholar] [CrossRef] [Green Version]
  44. Li, D.; Zhao, Y.; Zhang, L.; Chen, X.; Cao, C. Impact of quality management on green innovation. J. Clean. Prod. 2018, 170, 462–470. [Google Scholar] [CrossRef]
  45. Todorut, R. Sustainable development of organizations through total quality management. Procedia Soc. Behav. Sci. 2012, 62, 927–931. [Google Scholar] [CrossRef] [Green Version]
  46. Abbas, J. Impact of TQM on corporate sustainability through the mediating effect of knowledge management. J. Clean. Prod. 2020, 20, 118806. [Google Scholar] [CrossRef]
  47. Abbas, J. Impact of TQM on corporate green performance through the mediating role of corporate social responsibility. J. Clean. Prod. 2019, 242, 118458. [Google Scholar] [CrossRef]
  48. Perez Ruano, J.L.; Rodríguez-Rodríguez, R.; Alfaro-Saiz, J.J.; Verdecho, M.J. An ANP-Based Network to Measure the Impact of Lean Production on Organisational Performance. J. Ind. Eng. Manag. 2018, 11, 222–228. [Google Scholar] [CrossRef]
  49. Saaty, T.L. The Analytic Network Process. In Decision Making with Interdependence and Feedback, 2nd ed.; RWS Publications: Pittsburgh, PA, USA, 2001. [Google Scholar]
  50. Boj, J.J.; Rodríguez-Rodríguez, R.; Alfaro-Saiz, J.J. An ANP-multi-criteria-based methodology to link intangible assets and organizational performance in a Balanced Scorecard context. Decis. Support Syst. 2014, 68, 98–110. [Google Scholar] [CrossRef]
  51. Verdecho, M.J.; Alfaro-Saiz, J.J.; Rodríguez-Rodríguez, R.; Ortiz Bas, A. A multi-criteria approach for managing inter-enterprise collaborative relationships. Omega 2012, 40, 249–263. [Google Scholar] [CrossRef]
  52. Saaty, T.L. Fundamentals of Decision Making and Priority Theory with the Analytic Hierarchy Process, 1st ed.; RWS Publications: Pittsburgh, PA, USA, 1994. [Google Scholar]
  53. Neely, A.; Adams, C.; Crowe, P. The performance prism in practice. Meas. Bus. Excel. 2001, 5, 6–12. [Google Scholar] [CrossRef]
  54. Saaty, T.L.; Shih, H.S. Structures in decision-making: On the subjective geometry of hierarchies and networks. Eur. J. Oper. Res. 2009, 199, 867–872. [Google Scholar] [CrossRef]
Figure 1. Relationships between the building blocks.
Figure 1. Relationships between the building blocks.
Sustainability 12 06207 g001
Figure 2. Dependence matrix.
Figure 2. Dependence matrix.
Sustainability 12 06207 g002
Figure 3. ANP network (from Superdecisions TM).
Figure 3. ANP network (from Superdecisions TM).
Sustainability 12 06207 g003
Figure 4. BSC elements ordered by ABC classes.
Figure 4. BSC elements ordered by ABC classes.
Sustainability 12 06207 g004
Figure 5. TQM elements ordered by ABC classes.
Figure 5. TQM elements ordered by ABC classes.
Sustainability 12 06207 g005
Figure 6. Weighted supermatrix (partial).
Figure 6. Weighted supermatrix (partial).
Sustainability 12 06207 g006
Figure 7. Graphical representation of cause–effect relationships.
Figure 7. Graphical representation of cause–effect relationships.
Sustainability 12 06207 g007
Figure 8. Evolution of the outputs when perturbing a TQM element.
Figure 8. Evolution of the outputs when perturbing a TQM element.
Sustainability 12 06207 g008
Table 1. Strategic objectives.
Table 1. Strategic objectives.
PerspectiveObjectiveDenominationGoalSustainability Dimension
Financial perspectiveCost reductionFO17%Economic
Increase net benefitFO22%Economic
Customer perspectiveIncrease product availabilityCO14%
Increase quality level by maintaining or reducing environmental impactOCO21%Environmental
Increase customer loyaltyCO310%
Increase new customersCO410%
Improve customer satisfactionCO53%
Process perspectiveImprove circular economy practices efficiency: Recycle, reuse, reduce and recover (4Rs)PO17% (aggregated value)Environmental
Learning and GrowthIncrease staff competenciesLGO12%Social
Increase and maximize the technological infrastructureLGO220%Economic
Increase staff satisfaction LGO310%Social
Increase development and learning programsLGO42%Social
Table 2. Elements and cluster of TQM and outputs.
Table 2. Elements and cluster of TQM and outputs.
ClusterTQM Elements
Quality control (QC)Simulation (QC1), Sampling (QC2)
Quality planning and improvement (QPI)Customer needs (QPI1), Continuous improvement (QPI2), Training (QPI3), Quality standards (QPI4),
Recognition and rewards systems (QPI5), Staff turnover (QPI6)
Lean Production (LP)Redistribution of work (LP1), Automation with human touch (LP2), Just in time (LP3), Error-proof (LP4)
Collaboration factors (CF)Trust (CF1), Supply chain design (CF2), Cooperation (CF3), Information sharing processes (CF4), Joint decisions (CF5), Multi-disciplinary teams (CF6), Process alignment (CF7), Interoperability (CF8)
Outputs (Out)Cost reduction (Out1), Process improvement (Out2), Quality assurance (Out3), Efficiency in product delivery (Out4), Increase in motivation and satisfaction level (Out 5), Increase in customer satisfaction (Out6), Sales increase (Out7), Productivity increase (Out8), Information and knowledge (Out9)
Table 3. Limit supermatrix values ordered following an ABC classification.
Table 3. Limit supermatrix values ordered following an ABC classification.
Structural BlockClusterElementGlobal PriorityGlobal Normalised PriorityAccumulatedPareto Classes
BscFinancial PerspectiveFO10.118711.87%11.87%A
BscProcesses PerspectivePO10.109710.97%22.84%A
BscFinancial PerspectiveFO20.1050810.51%33.35%A
OutputsOutputsOut20.085758.58%41.93%A
OutputsOutputsOut80.082438.24%50.17%A
OutputsOutputsOut30.049764.98%55.15%A
TQMLean ProductionLP40.041214.12%59.27%A
BSCCustomer PerspectiveCO20.038753.88%63.14%A
OutputsOutputsOut70.037243.72%66.87%A
TQMQuality ControlQC10.035693.57%70.44%A
OutputsOutputsOut40.030863.09%73.52%A
TQMQuality ControlQC20.027162.72%76.24%A
OutputsOutputsOut10.025222.52%78.76%A
OutputsOutputsOut90.025012.50%81.26%B
TQMQuality PlanningQPI20.023522.35%83.61%B
BscCustomer PerspectiveCO10.019851.99%85.60%B
TQMLean ProductionLP20.018311.83%87.43%B
BscL&G PerspectiveLGO20.018191.82%89.25%B
OutputsOutputsOut60.016351.64%90.89%B
TQMLean ProductionLP30.012471.25%92.13%B
TQMLean ProductionLP10.011711.17%93.30%B
TQMQuality PlanningQPI40.009380.94%94.24%B
BscL&G PerspectiveLGO40.008350.84%95.08%C
OutputsOutputsOut50.006490.65%95.73%C
BscCustomer PerspectiveCO50.006020.60%96.33%C
TQMQuality PlanningQPI30.004410.44%96.77%C
TQMCollaboration FactorsCF70.004380.44%97.21%C
BscCustomer PerspectiveCO30.004270.43%97.63%C
TQMCollaboration FactorsCF20.003950.40%98.03%C
BscL&G PerspectiveLGO10.003920.39%98.42%C
BscCustomer PerspectiveCO40.003610.36%98.78%C
TQMQuality PlanningQPI10.002210.22%99.00%C
TQMQuality PlanningQPI60.001920.19%99.19%C
TQMCollaboration FactorsCF80.001890.19%99.38%C
BscL&G PerspectiveLGO30.001730.17%99.56%C
TQMQuality PlanningQPI50.001320.13%99.69%C
TQMCollaboration FactorsCF10.001160.12%99.80%C
TQMCollaboration FactorsCF40.001090.11%99.91%C
TQMCollaboration FactorsCF60.00060.06%99.97%C
TQMCollaboration FactorsCF30.000260.03%100.00%C
TQMCollaboration FactorsCF50.000130.01%100.00%C
Table 4. Integration of the Global and the Influences analysis.
Table 4. Integration of the Global and the Influences analysis.
Sustainability
Dimension
ObjectiveStrongly
Influenced by
Moderately
Influenced by
1 (Key)EconomicFO1. Cost reductionLP4LP2, LP3, QPI2
2 (Key)EnvironmentalPO1. 4Rs (Recycle, reuse, reduce and recover)QC1LP2, LP4, QPI4
3 (Key)EconomicFO2. Increase net benefitLP4LP3, QPI2
4EnvironmentalCO2. Increase quality level by maintaining or reducing environmental impactLP4LP1, LP2, QPI1,
QC1, CF4
5EconomicLGO2. Increase and maximise the technological infrastructureQC1QC2, QPI2
6SocialLGO4. Increase development and learning programsQC2-
7SocialLGO1. Increase staff competenciesQC2QPI5, QC1,
CF5, CF6
8SocialLGO3. Increase staff satisfaction-QPI3, CF4

Share and Cite

MDPI and ACS Style

Andrade Arteaga, C.; Rodríguez-Rodríguez, R.; Alfaro-Saiz, J.-J.; Verdecho, M.-J. An ANP-Balanced Scorecard Methodology to Quantify the Impact of TQM Elements on Organisational Strategic Sustainable Development: Application to an Oil Firm. Sustainability 2020, 12, 6207. https://doi.org/10.3390/su12156207

AMA Style

Andrade Arteaga C, Rodríguez-Rodríguez R, Alfaro-Saiz J-J, Verdecho M-J. An ANP-Balanced Scorecard Methodology to Quantify the Impact of TQM Elements on Organisational Strategic Sustainable Development: Application to an Oil Firm. Sustainability. 2020; 12(15):6207. https://doi.org/10.3390/su12156207

Chicago/Turabian Style

Andrade Arteaga, Carla, Raúl Rodríguez-Rodríguez, Juan-José Alfaro-Saiz, and María-José Verdecho. 2020. "An ANP-Balanced Scorecard Methodology to Quantify the Impact of TQM Elements on Organisational Strategic Sustainable Development: Application to an Oil Firm" Sustainability 12, no. 15: 6207. https://doi.org/10.3390/su12156207

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop