Weiqi Li
Skip Abstract Section
Abstract
This tutorial presents a novel search system—the Attractor-Based Search System (ABSS)—that can solve the Traveling Salesman Problem very efficiently with optimality guarantee. From the perspective of dynamical systems, a heuristic local search algorithm for an NP-complete combinatorial problem is a discrete dynamical system. In a local search system, an attractor drives the search trajectories into the vicinity of a globally optimal point in the solution space, and the convergence of local search trajectories makes the search system become a global and deterministic system. The attractor contains a small set of the most promising solutions to the problem. The attractor can reduce the problem size exponentially, and thus make the exhaustive search feasible. Therefore, this new search paradigm is called optimizing with attractor. The ABSS consists of two search phases: local search phase and exhaustive search phase. The local search process is used to quickly construct the attractor in the solution space, and the exhaustive search process is used to completely search the attractor to identify the optimal solution. Therefore, the exact optimal solution can be found quickly by combining local search and exhaustive search. This tutorial introduces the concept of an attractor in a local search system, and describes the process of optimizing with the attractor, using the Traveling Salesman Problem as the study platform.
- [1] . 2003. Local Search in Combinatorial Optimization. Princeton University Press, Princeton.Google Scholar
Cross Ref
- [2] . 1997. Chaos: Introduction to Dynamical Systems. Springer, New York.Google ScholarCross Ref
- [3] . 2006. The Traveling Salesman Problem: A Computational Study. Princeton University Press, Princeton, NJ, USA.Google ScholarDigital Library
- [4] . Concorde website. (March 2015). Retrieved November 15, 2023 from https://www.math.uwaterloo.ca/tsp/concorde/index.htmlGoogle Scholar
- [5] . 2009. Computational Complexity: A Modern Approach. Cambridge University Press, Cambridge.Google ScholarDigital Library
- [6] . 1964. Attractor in Dynamical Systems. NASA Technical Report NASA-CR-59858.Google Scholar
- [7] . 1978. On the branching factor of the alpha-beta pruning algorithm. Artificial Intelligence 10, 2 (1978), 173–199. Google ScholarDigital Library
- [8] . 2008. Multi-start and path relinking methods to deal with multi-objective knapsack problem. Annals of Operations Research 157, 1 (2008), 105–133. .Google ScholarCross Ref
- [9] . 1998. Mathematics for Dynamic Modeling. Academic Press, San Diego.Google Scholar
- [10] . 2007. On the invariance of ant colony optimization. IEEE Transactions on Evolutionary Computation 11, 6 (2007), 732–742. Google ScholarDigital Library
- [11] . 2003. Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM Computing Surveys 35, 3 (2003), 268–308. Google ScholarDigital Library
- [12] . 1994. New adaptive multistart techniques for combinatorial global optimizations. Operations Research Letter 16, 2 (1994), 101–113. Google ScholarDigital Library
- [13] . 2016. Introduction to Dynamical Systems. Cambridge University Press, Cambridge.Google Scholar
- [14] . 2011. Approximation-guided evolutionary multi-objective optimization. In Proceedings of the 22nd International Conference on Artificial Intelligence, Vol. 2, 1198–1203. Google ScholarCross Ref
- [15] . 2018. A Modern Introduction to Dynamical Systems. Oxford University Press, Oxford.Google Scholar
- [16] . 1999. New results on the old k-opt algorithm for the traveling salesman problem. SIAM Journal on Computing 28, 6 (1999), 1998–2029. Google ScholarDigital Library
- [17] . 2007. Evolutionary Algorithms for Solving Multi-Objective Problems. Springer, Berlin.Google ScholarDigital Library
- [18] . 2004. Multiobjective Programming and Planning. Courier Dover Publications, Mineola.Google Scholar
- [19] Stephen Cook. 2022. The P versus NP Problem. Clay Mathematics Institute. http://www.claymath.org/sites/default/files/pvsnp.pdfGoogle Scholar
- [20] . 2012. In Pursuit of the Traveling Salesman: Mathematics at the Limits of Computation. Princeton University Press, Princeton.Google Scholar
- [21] . 1998. Combinatorial Optimization. John Wiley & Sons, New York.Google ScholarDigital Library
- [22] . 2001. Introduction to Algorithms. MIT Press, Cambridge.Google ScholarDigital Library
- [23] . 2009. Introduction to Algorithms. MIT Press, Cambridge.Google ScholarDigital Library
- [24] . 1999. New Ideas in Optimization. McGraw-Hill, London.Google ScholarDigital Library
- [25] . 2006. Elements of Information Theory. Wiley, New York.Google ScholarDigital Library
- [26] . 1958. A method for solving traveling-salesman problem. Operations Research 6, 6 (1958), 791–812. Google ScholarDigital Library
- [27] . 2011. Attractors and basins of dynamical systems. Electronic Journal of Qualitative Theory of Differential Equations 20, 20 (2011), 1–11. Google ScholarCross Ref
- [28] . 1992. Optimization, Learning and Natural Algorithms. Ph.D. Thesis, Dip Electronica e Informazione, Politecnico Di Milano, Italy.Google Scholar
- [29] . 1997. Ant colony system: A cooperative learning approach to the traveling salesman problem. IEEE Transactions on Evolutionary Computation 1, 1 (1997), 53–66. Google ScholarDigital Library
- [30] . 2004. Ant Colony Optimization. The MIT Press, Cambridge.Google ScholarCross Ref
- [31] . 2000. Theory of Computational Complexity. John Wiley & Sons, New York.Google ScholarCross Ref
- [32] . 2008. History of optimization. In Encyclopedia of Optimization. (Eds.). Springer, Boston. Google ScholarCross Ref
- [33] . 1998. The branching factor of regular search space. In Proceedings of the 15thNational Conference on Artificial Intelligence. 292–304.Google Scholar
- [34] . 2005. Multicriteria Optimization (2nd ed.). Springer, New York.Google ScholarDigital Library
- [35] . 1989. A probabilistic heuristic for a computationally difficult set covering problem. Operations Research Letters, 8, 2 (1989), 67–71. Google ScholarDigital Library
- [36] . 1995. Greedy randomized adaptive search procedure. Journal of Global Optimization 2 (1995), 1–27. Google ScholarCross Ref
- [37] . 1995. A note on the complexity of local search problems. Information Processing Letters 53, 2 (1995), 69–75. Google ScholarDigital Library
- [38] . 1988. The design, analysis, and implementation of heuristics. Management Science 34, 3 (1988), 263–265. Google ScholarDigital Library
- [39] . 2009. The status of the P versus NP problem. Communications of the ACM 52, 9 (2009), 78–86. Google ScholarDigital Library
- [40] . 2013. The Golden Ticket—P, NP, and the Search for the Impossible. Princeton University Press. Princeton.Google Scholar
- [41] . 2017. Markov Chains: From Theory to Implementation and eExperimentation. John Wiley & Sons, New York.Google ScholarCross Ref
- [42] . 1991. Information Theory and Reliable Communication. Wiley, New York.Google Scholar
- [43] . 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freemen and Company, New York.Google ScholarDigital Library
- [44] . 2019. Handbook of Metaheuristics,Vol. 272. International Series in Operations Research & Management Science. Springer, New York. Google ScholarCross Ref
- [45] . 1997. The traveling salesman problem: New solvable cases and linkages with the development of approximation algorithms. Journal of the Operational Research Society 48, 5 (1997), 502–510. Google ScholarCross Ref
- [46] . 2008. Computational Complexity: A Conceptual Perspective. Cambridge University Press, Cambridge.Google ScholarCross Ref
- [47] . 2010. P, NP, and NP-Completeness. Cambridge University Press, Cambridge.Google ScholarDigital Library
- [48] . 2019. Stochastic global optimization algorithms: A systematic formal approach. Information Science 472 (2019), 53–76. Google ScholarCross Ref
- [49] . 1983. Characterization of strange attractor. Physical Review Letters 50, 5 (1983), 346–349. Google ScholarCross Ref
- [50] . 2007. The Traveling Salesman Problem and Its Variations. Springer, New York, NY.Google ScholarCross Ref
- [51] . 2011. Efficient multi-start strategies for local search algorithms. Journal of Artificial Intelligence Research 41, 2 (2011), 407–444. Google ScholarCross Ref
- [52] . 2001. Revisiting the big valley search space structure in the TSP. Journal of Operational Research Society 62, 2 (2011), 305–312. Google ScholarCross Ref
- [53] . 1971. An overview of the theory of computational complexity. Journal of the ACM 18, 3 (1971), 444–475. Google ScholarDigital Library
- [54] . 1970. Monte Carlo sampling methods using Markov chains and their applications. Biametnika 57, 1 (1970), 97–109. Google ScholarCross Ref
- [55] . 2021. Attractors in dynamical systems. In Ergodic Dynamics. Graduate Texts in Mathematics, Vol. 289. Springer, New York. Google ScholarCross Ref
- [56] . 2000. An effective implementation of the Lin–Kernighan traveling salesman heuristic. European Journal of Operational Research 126, 1 (2000), 106–130. Google ScholarCross Ref
- [57] . 2009. General k-opt submoves for the Lin–Kernighan TSP heuristic. Mathematical Programming Computation 1 (2009), 119–163. Google ScholarCross Ref
- [58] . 1964. Topics in Algebra. Blaisdell Publishing Company, Waltham, MA.Google Scholar
- [59] . 2004. Stochastic Local Search: Foundations and Applications. Morgan Kaufmann, New York.Google ScholarDigital Library
- [60] . 1995. Handbook of Global Optimization: Nonconvex Optimization and Its Application. Springer, New York.Google ScholarCross Ref
- [61] . 2000. Introduction to Global Optimization: Nonconvex Optimization and Its Applications. Springer, New York.Google ScholarCross Ref
- [62] . 1990. Local optimization and the Traveling Salesman Problem. In Automata, Languages and Programming (ICALP1990). Lecture Notes in Computer Science, Vol. 443. Springer, Berlin, 446–461. Google ScholarCross Ref
- [63] . 2007. Experimental analysis of heuristics for the STSP. In The Traveling Salesman Problem and Its Variations. Combinatorial Optimization. (Eds.). Springer, Boston, 369–443. Google ScholarCross Ref
- [64] . 1985. Configuration space analysis of traveling salesman problems. Journal de Physique 46, 8 (1985), 1277–1292. Google ScholarCross Ref
- [65] . 2010. Nonlinear Markov Process and Kinetic Equations. Cambridge University Press, Cambridge.Google ScholarCross Ref
- [66] . 1985. Depth-first iterative deepening: An optimal admissible tree search. Artificial Intelligence 27, 1 (1985), 97–109. Google ScholarDigital Library
- [67] . 2018. Combinatorial Optimization: Theory and Algorithms. Springer, New York.Google ScholarCross Ref
- [68] . 2011. Handbook of Monte Carlo Methods. John Wiley & Sons, New York.Google ScholarCross Ref
- [69] . 1999. GRASP and path relinking for a 2-player straight line crossing minimization. INFORMS Journal on Computing 11, 1 (1999), 44–52. Google ScholarCross Ref
- [70] . 2010. Local and global convergence. Numerical Analysis for Statisticians (Statistics and Computing). Springer, New York, 277–296. Google ScholarCross Ref
- [71] . 1985. The Traveling Salesman Problems: A Guided Tour of Combinatorial Optimization. John Wiley & Sons, New York.Google Scholar
- [72] . 1965. Computer solution of the traveling salesman problem. The Bell System Technical Journal 44, 10 (1965), 2245–2268. Google ScholarCross Ref
- [73] . 1973. An effective heuristic algorithm for the traveling-salesman problem. Operations Research 21, 2 (1973), 498–516. Google ScholarDigital Library
- [74] . 1996. Niching Methods for Genetic Algorithms. University of Illinois Press, Champaign.Google ScholarDigital Library
- [75] . 2015. Multi-start methods. In Handbook of Heuristics. Springer, Boston, MA, USA, 1–21. Google ScholarCross Ref
- [76] . 2010. Advanced multi-start methods. In Handbook of Metaheuristics. Springer, Boston, 265–281. Google ScholarCross Ref
- [77] . 2013. Multi-start methods for combinatorial optimization. European Journal of Operational Research 226, 1 (2013), 1–8. Google ScholarCross Ref
- [78] . 1992. Large step Markov chains for the TSP incorporating local search heuristics. Operations Research Letters 11, 4 (1992), 219–224. Google ScholarDigital Library
- [79] . 2016. Implementation challenge for TSP heuristics. In Encyclopedia of Algorithms. Springer, New York, 952–954. Google ScholarCross Ref
- [80] . 2009. Markov Chains and Stochastic Stability. Cambridge University Press, Cambridge.Google ScholarCross Ref
- [81] . 1986. A replica analysis of the traveling salesman problem. Journal de Physique 47, 8 (1986), 1285–1296. Google ScholarCross Ref
- [82] . 2004. How to Solve It: Modern Heuristics. Springer, Berlin.Google ScholarCross Ref
- [83] . 2001. Qualitative Theory of Dynamical Systems. Marcel Dekker, New York.Google ScholarCross Ref
- [84] . 1999. Nonlinear Multiobjective Optimization. Kluwer Academic Publishers, Bordrecht.Google Scholar
- [85] . 1985. On the concept of attractor. Communications in Mathematical Physics 99, 2 (1985), 177–196. Google ScholarCross Ref
- [86] . 2010. Collected Papers of John Milnor VI: Dynamical Systems (1953–2000). American Mathematical Society, Providence.Google Scholar
- [87] . 2003. Million city traveling salesman problem solution by divide and conquer clustering with adaptive resonance neural networks. Neural Networks 16, 5-6 (2003), 827–832. Google ScholarDigital Library
- [88] . 2013. A powerful genetic algorithm using edge assembly crossover for the traveling salesman problem. INFORMS Journal on Computing 25, 2 (2013), 346–363. Google ScholarDigital Library
- [89] . 2010. Artificial Ants. Wiley, New York.Google Scholar
- [90] . 2016. Deconstructing the big valley search space hypothesis. In Proceedings of Evolutionary Computation in Combinatorial Optimization (EvoCOP ’16). Lecture Notes in Computer Science, Vol. 9595. Springer. Google ScholarCross Ref
- [91] . 1993. Computational Complexity. Pearson, New York.Google Scholar
- [92] . 1977. On the complexity of local search for the traveling salesman problem. SIAM Journal on Computing 6, 1 (1977), 76–83. Google ScholarDigital Library
- [93] . 1999. Combinatorial Optimization: Algorithms and Complexity. Dover Publications, New York.Google Scholar
- [94] . 2002. Handbook of Global Optimization Volume 2: Nonconvex Optimization and Its Application. Springer, New York.Google ScholarCross Ref
- [95] . 1982. The solution for the branching factor of the alpha-beta pruning algorithm and its optimality. Communications of the ACM 25, 8 (1982), 559–564. .Google ScholarDigital Library
- [96] . 1984. Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addison-Wesley, New York.Google ScholarDigital Library
- [97] . 2004. An Introduction to Binary Search Trees and Balanced Trees. Free Software Foundation, Inc., Boston.Google Scholar
- [98] . 1993. Modern Heuristic Techniques for Combinatorial Problems. John Wiley & Sons, New York.Google ScholarDigital Library
- [99] . 1999. Landscapes, operators and heuristic search. Annals of Operations Research 86 (1999), 473–490. Google ScholarCross Ref
- [100] . 2011. Traveling salesman problem heuristics: Leading methods, implementations and latest advances. European Journal of Operational Research 211, 3 (2011), 427–441. .Google ScholarCross Ref
- [101] . 2010. GRASP and path relinking for the max-min diversity problem. Computer & Operations Research 37, 3 (2010), 498–508. Google ScholarDigital Library
- [102] . 2010. Greedy randomized adaptive search procedures: Advances, hybridizations, and applications. In Handbook of Metaheuristics. Springer, New York, 283–319. Google ScholarCross Ref
- [103] . 2016. Optimization by GRASP. Springer, New York.Google ScholarCross Ref
- [104] . 2016. Path-relinking. In Optimization by GRASP. Springer, New York, 167–188. Google ScholarCross Ref
- [105] . 1976. P-complete approximation problem. Journal of the ACM 23, 3 (1976), 555–565. Google ScholarDigital Library
- [106] . 2017. Improving an exact solver for the traveling salesman problem using partition crossover. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’17). 337–344. Google ScholarDigital Library
- [107] . 1976. Some theoretical implications of local optimization. Mathematical Programming 10, 1 (1976) 354–366. Google ScholarDigital Library
- [108] . 2006. Stochastic Optimization. Springer, New York.Google Scholar
- [109] . 1948. A mathematical theory of communication. Bell System Technical Journal 27 (1948), 379–423 and 623–656. Google ScholarCross Ref
- [110] . 2012. Niching in evolutionary algorithms. In Handbook of Natural Computing. Springer, Berlin. Google ScholarCross Ref
- [111] . 2011. Improving convergence of evolutionary multi-objective optimization with local search: A concurrent-hybrid algorithm. Natural Computing 10 (2011), 1407–1430. Google ScholarCross Ref
- [112] . 1992. The history and status of the P verses NP question. In Proceedings of the 24th Annual ACM Symposium on Theory of Computing. 603–618. Google ScholarDigital Library
- [113] . 2006. Introduction to the Theory of Computation. Thomson Course Technology, Boston.Google ScholarDigital Library
- [114] . 1986. Statistical mechanics and the traveling salesman problem. Europhysics Letter 2, 12 (1986), 919–923. Google ScholarCross Ref
- [115] . 1994. A bibliography of heuristic search research through 1992. IEEE Transactions on Systems, Man, and Cybernetic 24, 2 (1994), 268–293. Google ScholarCross Ref
- [116] . 2009. Metaheuristics: From Design to Implementation. Wiley, New York.Google ScholarCross Ref
- [117] . 1989. Heuristic methods and applications: A categorized survey. European Journal of Operations Research 43, 1 (1989), 88–110. Google ScholarCross Ref
- [118] . 2008. Stochastic Global Optimization. Springer, New York.Google Scholar
- [119] . 2010. Handbook of Multicriteria Analysis (Applied Optimization, 103). Springer, New York.Google ScholarCross Ref
- [120] . 1995. An effective tour construction and improvement procedure for the traveling salesman problem. Operations Research 43, 6 (1995), 1049–1057. Google ScholarDigital Library
Index Terms
- Optimizing with Attractor: A Tutorial
Recommendations
A new approach to the traveling salesman problem
ACM SE '22: Proceedings of the 2022 ACM Southeast ConferenceThe intrinsic difficulty of the Traveling Salesman Problem (TSP) is associated with the combinatorial explosion of potential solutions in the solution space. The significant difficulty with search algorithms for the TSP is how to quickly find the ...
Solving the maximum clique problem by k-opt local search
SAC '04: Proceedings of the 2004 ACM symposium on Applied computingThis paper presents a local search algorithm based on variable depth search, called the k-opt local search, for the maximum clique problem. The k-opt local search performs add and drop moves, each of which can be interpreted as 1-opt move, to search a k-...
Approximate Local Search in Combinatorial Optimization
Local search algorithms for combinatorial optimization problems are generally of pseudopolynomial running time, and polynomial-time algorithms are not often known for finding locally optimal solutions for NP-hard optimization problems. We introduce the ...
Comments