Skip to main content
SpringerLink
Log in

Generalizing DTW to the multi-dimensional case requires an adaptive approach

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

In recent years Dynamic Time Warping (DTW) has emerged as the distance measure of choice for virtually all time series data mining applications. For example, virtually all applications that process data from wearable devices use DTW as a core sub-routine. This is the result of significant progress in improving DTW’s efficiency, together with multiple empirical studies showing that DTW-based classifiers at least equal (and generally surpass) the accuracy of all their rivals across dozens of datasets. Thus far, most of the research has considered only the one-dimensional case, with practitioners generalizing to the multi-dimensional case in one of two ways, dependent or independent warping. In general, it appears the community believes either that the two ways are equivalent, or that the choice is irrelevant. In this work, we show that this is not the case. The two most commonly used multi-dimensional DTW methods can produce different classifications, and neither one dominates over the other. This seems to suggest that one should learn the best method for a particular application. However, we will show that this is not necessary; a simple, principled rule can be used on a case-by-case basis to predict which of the two methods we should trust at the time of classification. Our method allows us to ensure that classification results are at least as accurate as the better of the two rival methods, and, in many cases, our method is significantly more accurate. We demonstrate our ideas with the most extensive set of multi-dimensional time series classification experiments ever attempted.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Notes

  1. Project webpage: https://sites.google.com/site/dtwAdaptive.

References

  • Aach J, Church GM (2001) Aligning gene expression time series with time warping algorithms. Bioinformatics 17(6):495–508

    Article  Google Scholar 

  • Akl A, Valaee S (2010) Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, & compressive sensing. In: Proceedings of the IEEE international conference on acoustics speech and signal processing (ICASSP), pp 2270–2273

  • Albertus S (1734) Locupletissimi rerum naturalium thesauri accurata descriptio, et iconibus artificiosissimis expressio, per universam physices historiam: Opus, cui, in hoc rerum genere, nullum par exstitit

  • Al-Jawad A, Adame MR, Romanovas M, Hobert M, Maetzler W, Traechtler M, Moeller K, Manoli Y (2012) Using multi-dimensional dynamic time warping for tug test instrumentation with inertial sensors. In: Proceedings of the IEEE conference on multisensor fusion and integration for intelligent systems (MFI), pp 212–218

  • Assent I, Wichterich M, Krieger R, Kremer H, Seidl T (2009) Anticipatory DTW for efficient similarity search in time series databases. Proc VLDB Endow 2(1):826–837

    Article  Google Scholar 

  • Bashir M, Kempf J (2008) Reduced dynamic time warping for handwriting recognition based on multidimensional time series of a novel pen device. Int J Intell Syst Technol WASET 3(4):194

    Google Scholar 

  • Das Ehrenbuch der Fugger (The secret book of honour of the Fugger) -BSB Cgm 9460, Augsburg, ca. 1545–1548 mit Nachträgen aus späterer Zeit

  • de Mello RF, Gondra I (2008) Multi-dimensional dynamic time warping for image texture similarity., Advances in Artificial Intelligence-SBIASpringer, Berlin, pp 23–32

    Google Scholar 

  • Ding H, Trajcevski G, Scheuermann P, Wang X, Keogh E (2008) Querying and mining of time series data: experimental comparison of representations and distance measures. Proc VLDB Endow 1(2):1542–1552

    Article  Google Scholar 

  • Ermes M, Parkka J, Mantyjarvi J, Korhonen I (2008) Detection of daily activities and sports with wearable sensors in controlled and uncontrolled conditions. IEEE Trans Inf Technol Biomed 12(1):20–26

    Article  Google Scholar 

  • Gillian N, Knapp RB, O’Modhrain S (2011) Recognition of multivariate temporal musical gestures using n-dimensional dynamic time warping. In: Proceedings of the 11th international conference on new interfaces for musical expression (NIME), pp 337–342

  • Hao Y, Shokoohi-Yekta M, Papageorgiou G, Keogh E (2013) Parameter-free audio motif discovery in large data archives. In: Proceedings of the 13th IEEE international conference on data mining (ICDM), pp 261–270

  • Hu B, Chen Y, Keogh EJ (2013) Time series classification under more realistic assumptions. In: Proceedings of the SIAM international conference on data mining (SDM), pp 578–586

  • Hu B, Chen Y, Zakaria J, Ulanova L, Keogh E (2013) Classification of multi-dimensional streaming time series by weighting each classifier’s track record. In: Proceedings of the 13th IEEE international conference on data mining (ICDM), pp 281–290

  • Kale N, Lee J, Lotfian R, Jafari R (2012) Impact of sensor misplacement on dynamic time warping based human activity recognition using wearable computers. In: Proceedings of the ACM conference on wireless health, p 7

  • Kela J, Korpipää P, Mäntyjärvi J, Kallio S, Savino G, Jozzo L, Marca D (2006) Accelerometer-based gesture control for a design environment. Pers Ubiquitous Comput 10(5):285–299

    Article  Google Scholar 

  • Keogh E, Wei L, Xi X, Lee SH, Vlachos M (2006) LB\(\_\)Keogh supports exact indexing of shapes under rotation invariance with arbitrary representations and distance measures. In: Proceedings of the 32nd international conference on very large data bases (VLDB), pp 882–893

  • Keogh E, Kasetty S (2003) On the need for time series data mining benchmarks: a survey and empirical demonstration. Data Min Knowl Discov 7(4):349–371

    Article  MathSciNet  Google Scholar 

  • Ko MH, West G, Venkatesh S, Kumar M (2005) Online context recognition in multisensor systems using dynamic time warping. In: Proceedings of the IEEE international conference on intelligent sensors, sensor networks and information processing (ISSNIP), pp 283–288

  • Kruskal JB, Liberman M (1983) The symmetric time-warping problem: from continuous to discrete. Time warps, string edits and macromolecules: the theory and practice of sequence comparison. Addison-Wesley Publication, Reading, pp 125–161

    Google Scholar 

  • Lara OD, Labrador MA (2013) A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tutor 15(3):1192–1209

    Article  Google Scholar 

  • Liu J, Zhong L, Wickramasuriya J, Vasudevan V (2009) uWave: accelerometer-based personalized gesture recognition and its applications. Pervasive Mob Comput 5(6):657–675

    Article  Google Scholar 

  • McGlynn D, Madden MG (2011) An ensemble dynamic time warping classifier with application to activity recognition., Research and Development in Intelligent Systems XXVIISpringer, London, pp 339–352

    Google Scholar 

  • Papapetrou P, Athitsos V, Potamias M, Kollios G, Gunopulos D (2011) Embedding-based subsequence matching in time-series databases. ACM Trans Database Syst 36(3):17

    Article  Google Scholar 

  • Petitjean F, Inglada J, Gançarski P (2012) Satellite image time series analysis under time warping. IEEE Trans Geosci Remote Sens 50(8):3081–3095

    Article  Google Scholar 

  • Quinlan JR (1986) Induction of decision trees. Machine Learning 1(1):81–106

    Google Scholar 

  • Rabiner L, Juang BH (1993) Fundamentals of speech recognition. Prentice Hall, Upper Saddle River

    MATH  Google Scholar 

  • Rakthanmanon T, Campana B, Mueen A, Batista G, Westover B, Zhu Q, Zakaria J, Keogh E (2013) Addressing big data time series: mining trillions of time series subsequences under dynamic time warping. ACM Trans Knowl Discov Data 7(3):10

    Article  Google Scholar 

  • Ratanamahatana CA, Keogh E (2004) Everything you know about dynamic time warping is wrong. In: Third workshop on mining temporal and sequential data, pp 1–11

  • Rawassizadeh R, Price BA, Petre M (2014) Wearables: has the age of smartwatches finally arrived? Commun ACM 58(1):45–47

    Article  Google Scholar 

  • Ridgely Robert S, Tudor G Field (2009) Guide to the songbirds of South America the Passerines., Mildred Wyatt-Wold Series in OrnithologyUniversity of Texas Press, Austin

    Google Scholar 

  • Sakurai Y, Faloutsos C, Yamamuro M (2007) Stream monitoring under the time warping distance. In: Proceedings of the 23rd IEEE international conference on data engineering (ICDE), pp 1046–1055

  • Shieh J, Keogh E (2009) iSAX: disk-aware mining and indexing of massive time series datasets. Data Min Knowl Discov 19(1):24–57

    Article  MathSciNet  Google Scholar 

  • Shokoohi-Yekta M (2015) Meaningful rule discovery and adaptive classification of multi-dimensional time series data. Dissertation, University of California

  • Shokoohi-Yekta M, Chen Y, Campana B, Hu B, Zakaria J, Keogh E (2015) Discovery of meaningful rules in time series. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, Sydney, pp 1085–1094

  • Shokoohi-Yekta M, Wang J, Keogh E (2015) On the non-trivial generalization of dynamic time warping to the multi-dimensional case. IN: Proceedings of the SIAM international conference on data mining (SDM), Vancouver, pp 39–48

  • Tang J, Dannenberg RB (2014) Extracting commands from gestures: gesture spotting and recognition for real-time music performance. Sound, music, and motion. Springer, Berlin, pp 72–85

    Google Scholar 

  • Ten Holt GA, Reinders MJT, Hendriks EA (2007) Multi-dimensional dynamic time warping for gesture recognition. In: Thirteenth annual conference of the Advanced School for Computing and Imaging

  • Wang J, Balasubramanian A, Mojica de la Vega L, Green JR, Samal A, Prabhakaran B (2013) Word recognition from continuous articulatory movement time-series data using symbolic representations. In: ACL/ISCA interspeech workshop on speech and language processing for assistive technologies, Grenoble, pp 119–127

  • Wang J, Samal A, Green JR (2014) Preliminary test of a real-time, interactive silent speech interface based on electromagnetic articulograph. In: ACL/ISCA workshop on speech and language processing for assistive technologies, Baltimore, pp 38–45

  • Wang J, Samal A, Green JR, Rudzicz F (2012) Whole-word recognition from articulatory movements for silent speech interfaces. In: Proceedings of the interspeech, Portland, pp 1327–1330

  • Ye L, Keogh E (2009) Time series shapelets: a new primitive for data mining. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining, pp 947–956

  • Yuksel SE, Wilson JN, Gader PD (2012) Twenty years of mixture of experts. IEEE Trans Neural Netw Learn Syst 23(8):1177–1193

    Article  Google Scholar 

  • Yunusova Y, Green JR, Mefferd A (2009) Accuracy assessment for AG500, electromagnetic articulograph. J Speech Lang Hear Res 52(2):547–555

    Article  Google Scholar 

  • Zhu Q, Keogh E (2010) Mother fugger: mining historical manuscripts with local color patches. In: Proceedings of the 10th IEEE international conference on data mining (ICDM), pp 699–708

Download references

Acknowledgments

This research was supported by NSF IIS-1161997, NIH R01 DC013547, the Bill and Melinda Gates Foundation, the Vodafone’s Wireless Innovation Project and a gift from Samsung.

Author information

Authors and Affiliations

  1. Apple Inc., Cupertino, USA

    Mohammad Shokoohi-Yekta

  2. Facebook, Menlo Park, USA

    Bing Hu

  3. Samsung Research America, Mountain View, USA

    Hongxia Jin

  4. University of Texas at Dallas, Richardson, USA

    Jun Wang

  5. University of California, Riverside, Riverside, USA

    Eamonn Keogh

Authors
  1. Mohammad Shokoohi-Yekta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongxia Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eamonn Keogh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Shokoohi-Yekta.

Additional information

Responsible editor: Johannes Fuernkranz.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shokoohi-Yekta, M., Hu, B., Jin, H. et al. Generalizing DTW to the multi-dimensional case requires an adaptive approach. Data Min Knowl Disc 31, 1–31 (2017). https://doi.org/10.1007/s10618-016-0455-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-016-0455-0

Keywords

Search

Navigation