Skip to main content
SpringerLink
Log in

Efficient Hop-constrained s-t Simple Path Enumeration

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

Graph is a ubiquitous structure representing entities and their relationships applied in many areas such as social networks, web graphs, and biological networks. One of the fundamental tasks in graph analytics is to investigate the relations between two vertices (e.g., users, items, and entities) such as how a vertex A influences another vertex B, or to what extent A and B are similar to each other, based on the graph topology structure. For this purpose, we study the problem of hop-constrained s-t simple path enumeration in this paper, which aims to list all simple paths from a source vertex s to a target vertex t with hop-constraint k. We first propose a polynomial delay algorithm, namely BC-DFS, based on a barrier-based pruning technique. Then, a join-oriented algorithm, namely JOIN, is designed to further enhance the query response time. On the theoretical side, BC-DFS is a polynomial delay algorithm with O(km) time per output where m is the number of edges in the graph. This time complexity is similar to the best known theoretical result for the polynomial delay algorithms of this problem. On the practical side, our comprehensive experiments on 15 real-life networks demonstrate the superior performance of the BC-DFS algorithm compared to the state-of-the-art techniques. It is also reported that the JOIN algorithm can further significantly enhance the query response time. In this paper, we also study the hop-constrained path enumeration problem with diversity constraint and propose a block-oriented algorithm, namely SCB. To further speed up the computation, hybrid lower bounds based on reverse shortest-path tree are also developed, namely SCB+. The experiments show our proposed methods significantly improve the query time and scalability comparing with baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Notes

  1. Note that we do not need to maintain u.bar after u is pushed to the stack \({\mathcal {S}}\), and u.bar is correctly calculated when u is unstacked.

  2. It can find a new valid result by using a BFS on the induced graph, where the blocked vertices B are removed.

  3. If \(d=2\), it is the case for block-DFS with \(O(m*\delta )\).

  4. http://konect.uni-koblenz.de/networks/

  5. http://networkrepository.com/networks.php

  6. http://snap.stanford.edu/data/

References

  1. Abraham, I., Delling, D., Goldberg, A.V., Werneck, R.F.: Alternative routes in road networks. Journal of Experimental Algorithmics (JEA) 18, 1–1 (2013)

    MathSciNet  MATH  Google Scholar 

  2. Akgün, V., Erkut, E., Batta, R.: On finding dissimilar paths. European Journal of Operational Research 121(2), 232–246 (2000)

    Article  Google Scholar 

  3. Birmelé, E., Ferreira, R.A., Grossi, R., Marino, A., Pisanti, N., Rizzi, R., Sacomoto, G.: Optimal listing of cycles and st-paths in undirected graphs. In SODA, pages 1884–1896, (2013)

  4. Borodin, A., Lee, H.C., Ye, H.C.: Max-sum diversification, monotone submodular functions and dynamic updates. In Proceedings of the 31st ACM SIGMOD-SIGACT-SIGAI symposium on Principles of Database Systems, pages 155–166, (2012)

  5. Chang, L., Lin, X., Qin, L., Yu, J.X., Pei, J.: Efficiently computing top-k shortest path join. In EDBT 2015-18th International Conference on Extending Database Technology, Proceedings, (2015)

  6. Chen, X., Lai, L., Qin, L., Lin, X., Liu, B.: A framework to quantify approximate simulation on graph data. arXiv preprint arXiv:2010.08938, (2020)

  7. Cheng, Y., Li, J., Sterbenz, J.P.: Path geo-diversification: Design and analysis. In 2013 5th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), pages 46–53. IEEE, (2013)

  8. Chondrogiannis, T., Bouros, P., Gamper, J., Leser, U., Blumenthal, D.B.: Finding k-dissimilar paths with minimum collective length. In Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pages 404–407, (2018)

  9. Chondrogiannis, T., Bouros, P., Gamper, J., Leser, U., Blumenthal, D.B.: Finding k-shortest paths with limited overlap. The VLDB Journal, pages 1–25, (2020)

  10. Clarke, C.L., Kolla, M., Cormack, G.V., Vechtomova, O., Ashkan, A., Büttcher, S., MacKinnon, I.: Novelty and diversity in information retrieval evaluation. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pages 659–666, (2008)

  11. de Queirós Vieira Martins, E., Pascoal, M.M.B.: A new implementation of yen’s ranking loopless paths algorithm. 4OR, 1(2):121–133, (2003)

  12. Deng, D., Tao, Y., Li, G.: Overlap set similarity joins with theoretical guarantees. In Proceedings of the 2018 International Conference on Management of Data, pages 905–920, (2018)

  13. Drosou, M., Pitoura, E.: Search result diversification. ACM. SIGMOD Record 39(1), 41–47 (2010)

    Article  Google Scholar 

  14. Drosou, M., Pitoura, E.: Disc diversity: result diversification based on dissimilarity and coverage. arXiv preprint arXiv:1208.3533, (2012)

  15. Fang, Y., Cheng, R., Li, X., Luo, S., Hu, J.: Effective community search over large spatial graphs. Proceedings of the VLDB Endowment 10(6), 709–720 (2017)

    Article  Google Scholar 

  16. Fang, Y., Cheng, R., Luo, S., Hu, J.: Effective community search for large attributed graphs. Proceedings of the VLDB Endowment 9(12), 1233–1244 (2016)

    Article  Google Scholar 

  17. Fang, Y., Cheng, R., Luo, S., Hu, J., Huang, K.: C-explorer: browsing communities in large graphs. Proceedings of the VLDB Endowment 10(12), 1885–1888 (2017)

    Article  Google Scholar 

  18. Fang, Y., Huang, X., Qin, L., Zhang, Y., Zhang, W., Cheng, R., Lin, X.: A survey of community search over big graphs. The VLDB Journal, (2019)

  19. Fang, Y., Wang, Z., Cheng, R., Wang, H., Hu, J.: Effective and efficient community search over large directed graphs. IEEE Transactions on Knowledge and Data Engineering (2018)

  20. Freitas, A., da Silva, J.C.P., Curry, E., Buitelaar, P.: A distributional semantics approach for selective reasoning on commonsense graph knowledge bases. In International Conference on Applications of Natural Language to Data Bases/Information Systems, pages 21–32. Springer, (2014)

  21. Gao, J., Jin, R., Zhou, J., Yu, J.X., Jiang, X., Wang, T.: Relational approach for shortest path discovery over large graphs. Proceedings of the VLDB Endowment 5(4), 358–369 (2011)

    Article  Google Scholar 

  22. Gao, J., Qiu, H., Jiang, X., Wang, T., Yang, D.: Fast top-k simple shortest paths discovery in graphs. In Proceedings of the 19th ACM international conference on Information and knowledge management, pages 509–518. ACM, (2010)

  23. Grossi, R., Marino, A., Versari, L.: Efficient algorithms for listing k disjoint st-paths in graphs. In Latin American Symposium on Theoretical Informatics, pages 544–557. Springer, (2018)

  24. Henao-Mazo, W., Bravo-Santos, A.: Finding diverse shortest paths for the routing task in wireless sensor networks. In Proc. 7th Int. Conf. Syst. Netw. Commun., pages 53–58, (2012)

  25. Hochbaum, D.S.: The pseudoflow algorithm: A new algorithm for the maximum-flow problem. Operations Research 56(4), 992–1009 (2008)

    Article  MathSciNet  Google Scholar 

  26. Hu, P., Lau, W.C.: A survey and taxonomy of graph sampling. arXiv preprint arXiv:1308.5865, (2013)

  27. Jeong, Y.-J., Kim, T.J., Park, C.-H., Kim, D.-K.: A dissimilar alternative paths-search algorithm for navigation services: A heuristic approach. KSCE Journal of Civil Engineering 14(1), 41–49 (2010)

    Article  Google Scholar 

  28. Jin, R., Ruan, N.: Shortest path computation in large networks, Dec. 19 (2013). US Patent App. 13/899,124

  29. Jin, X., Yang, Z., Lin, X., Yang, S., Qin, L., Peng, Y.: Fast: Fpga-based subgraph matching on massive graphs. arXiv preprint arXiv:2102.10768, (2021)

  30. Johnson, D.B.: Finding all the elementary circuits of a directed graph. SIAM J. Comput. 4(1), 77–84 (1975)

    Article  MathSciNet  Google Scholar 

  31. Johnson, D.S., Papadimitriou, C.H., Yannakakis, M.: On generating all maximal independent sets. Inf. Process. Lett. 27(3), 119–123 (1988)

    Article  MathSciNet  Google Scholar 

  32. Khan, H.A., Drosou, M., Sharaf, M.A.: Dos: an efficient scheme for the diversification of multiple search results. In Proceedings of the 25th International Conference on Scientific and Statistical Database Management, pages 1–4, (2013)

  33. Koh, J.-L., Lin, C.-Y., Chen, A.L.: Finding k most favorite products based on reverse top-t queries. The VLDB journal 23(4), 541–564 (2014)

    Article  Google Scholar 

  34. Kunegis, J.: Konect: the koblenz network collection. In Proceedings of the 22nd International Conference on World Wide Web, pages 1343–1350. ACM, (2013)

  35. Lai, Z., Peng, Y., Yang, S., Lin, X., Zhang, W.: Pefp: Efficient k-hop constrained st simple path enumeration on fpga. arXiv preprint arXiv:2012.11128, (2020)

  36. Lao, N., Cohen, W.W.: Relational retrieval using a combination of path-constrained random walks. Machine Learning 81(1), 53–67 (2010)

    Article  MathSciNet  Google Scholar 

  37. Leser, U.: A query language for biological networks. Bioinformatics, 21:ii33–9, 10 (2005)

  38. Leskovec, J., Krevl, A.: SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data, (2014)

  39. Lim, Y., Kim, H.: A shortest path algorithm for real road network based on path overlap. Journal of the Eastern Asia Society for Transportation Studies 6, 1426–1438 (2005)

    Google Scholar 

  40. Liu, B., Yuan, L., Lin, X., Qin, L., Zhang, W., Zhou, J.: Efficient (a, \(\beta \))-core computation: an index-based approach. In The World Wide Web Conference, pages 1130–1141. ACM, (2019)

  41. Liu, H., Jin, C., Yang, B., Zhou, A.: Finding top-k shortest paths with diversity. IEEE Transactions on Knowledge and Data Engineering 30(3), 488–502 (2017)

    Article  Google Scholar 

  42. Mazumder, S., Liu, B.: Context-aware path ranking for knowledge base completion. In IJCAI, pages 1195–1201, (2017)

  43. Peng, Y., Zhang, Y., Lin, X., Qin, L., Zhang, W.: Answering billion-scale label-constrained reachability queries within microsecond. Proceedings of the VLDB Endowment 13(6), 812–825 (2020)

    Article  Google Scholar 

  44. Peng, Y., Zhang, Y., Lin, X., Zhang, W., Qin, L., Zhou, J.: Towards bridging theory and practice: hop-constrained st simple path enumeration. Proceedings of the VLDB Endowment 13(4), 463–476 (2019)

    Article  Google Scholar 

  45. Peng, Y., Zhang, Y., Zhang, W., Lin, X., Qin, L.: Efficient probabilistic k-core computation on uncertain graphs. In 2018 IEEE 34th International Conference on Data Engineering (ICDE), pages 1192–1203. IEEE, (2018)

  46. Qiu, X., Cen, W., Qian, Z., Peng, Y., Zhang, Y., Lin, X., Zhou, J.: Real-time constrained cycle detection in large dynamic graphs. PVLDB 11(12), 1876–1888 (2018)

    Google Scholar 

  47. Rizzi, R., Sacomoto, G., Sagot, M.: Efficiently listing bounded length st-paths. In IWOCA, pages 318–329, (2014)

  48. Roditty, L., Zwick, U.: Replacement paths and k simple shortest paths in unweighted directed graphs. In International Colloquium on Automata, Languages, and Programming, pages 249–260. Springer, (2005)

  49. Rossi, R.A., Ahmed, N.K.: The network data repository with interactive graph analytics and visualization. In AAAI, (2015)

  50. Sommer, C.: Shortest-path queries in static networks. ACM Comput. Surv., 46(4):45:1–45:31, (2014)

  51. Talarico, L., Sörensen, K., Springael, J.: The k-dissimilar vehicle routing problem. European Journal of Operational Research 244(1), 129–140 (2015)

    Article  MathSciNet  Google Scholar 

  52. Valiant, L.G.: The complexity of enumeration and reliability problems. SIAM Journal on Computing 8(3), 410–421 (1979)

    Article  MathSciNet  Google Scholar 

  53. Voss, C., Moll, M., Kavraki, L.E.: A heuristic approach to finding diverse short paths. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 4173–4179. IEEE, (2015)

  54. Wang, K., Cao, X., Lin, X., Zhang, W., Qin, L.: Efficient computing of radius-bounded k-cores. In 2018 IEEE 34th International Conference on Data Engineering (ICDE), pages 233–244. IEEE, (2018)

  55. Wang, S., Cheema, M.A., Zhang, Y., Lin, X.: Selecting representative objects considering coverage and diversity. In Second International ACM Workshop on Managing and Mining Enriched Geo-Spatial Data, pages 31–38, (2015)

  56. Yang, Z., Lai, L., Lin, X., Hao, K., Zhang, W.: Huge: An efficient and scalable subgraph enumeration system. arXiv preprint arXiv:2103.14294, (2021)

  57. Yen, J.Y.: Finding the k shortest loopless paths in a network. Management Science 17(11), 712–716 (1971)

    Article  MathSciNet  Google Scholar 

  58. Yu, C., Lakshmanan, L., Amer-Yahia, S.: It takes variety to make a world: diversification in recommender systems. In Proceedings of the 12th international conference on extending database technology: Advances in database technology, pages 368–378, (2009)

  59. Yu, J.X., Cheng, J.: Graph reachability queries: A survey. In Managing and Mining Graph Data, pages 181–215. (2010)

  60. Yuan, L., Qin, L., Lin, X., Chang, L., Zhang, W.: Diversified top-k clique search. The VLDB Journal 25(2), 171–196 (2016)

    Article  Google Scholar 

  61. Yue, D., Wu, X., Wang, Y., Li, Y., Chu, C.: A review of data mining-based financial fraud detection research. In International Conference on Wireless Communications, Networking and Mobile Computing, pages 5519 – 5522, 10 (2007)

  62. Zhang, Y., Callan, J., Minka, T.: Novelty and redundancy detection in adaptive filtering. In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, pages 81–88, (2002)

  63. Zheng, K., Wang, H., Qi, Z., Li, J., Gao, H.: A survey of query result diversification. Knowledge and Information Systems 51(1), 1–36 (2017)

    Article  Google Scholar 

  64. Ziegler, C.-N., McNee, S.M., Konstan, J.A., Lausen, G.: Improving recommendation lists through topic diversification. In Proceedings of the 14th international conference on World Wide Web, pages 22–32, (2005)

Download references

Acknowledgements

Xuemin Lin is supported by 2018YFB1003504, NSFC6123 2006, ARC DP180103096 and DP170101628. Ying Zhang is supported by ARC DP180103096 and FT170100128. Wenjie Zhang is supported by ARC DP180103096. Lu Qin is supported by ARC DP160101513.

Author information

Authors and Affiliations

  1. University of New South Wales, Sydney, Australia

    You Peng, Xuemin Lin & Wenjie Zhang

  2. Centre for AI, University of Technology Sydney, Sydney, Australia

    Ying Zhang & Lu Qin

  3. Alibaba Group, Hangzhou, China

    Jingren Zhou

Authors
  1. You Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuemin Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lu Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jingren Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to You Peng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, Y., Lin, X., Zhang, Y. et al. Efficient Hop-constrained s-t Simple Path Enumeration. The VLDB Journal 30, 799–823 (2021). https://doi.org/10.1007/s00778-021-00674-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-021-00674-5

Keywords

Search

Navigation