Skip to main content
SpringerLink
Log in

Utility distribution matters: enabling fast belief propagation for multi-agent optimization with dense local utility function

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Belief propagation algorithms including Max-sum and its variants are important methods for multi-agent optimization. However, they face a significant scalability challenge as the computational overhead grows exponentially with respect to the arity of each utility function. To date, a number of acceleration algorithms for belief propagation algorithms were proposed. These algorithms maintain a lower bound on total utility and employ either a domain pruning technique or branch and bound to reduce the search space. However, these algorithms still suffer from low-quality bounds and the inability of filtering out suboptimal tied entries. In this paper, we first show that these issues are exacerbated and can considerably degenerate the performance of the state-of-the-art methods when dealing with the problems with dense utility functions, which widely exist in many real-world domains. Built on this observation, we then develop several novel acceleration algorithms that alleviate the effect of densely distributed local utility values from the perspectives of both bound quality and search space organization. Specifically, we build a search tree for each distinct local utility value to enable efficient branch and bound on tied entries and tighten a running lower bound to perform dynamic domain pruning. That is, we integrate both search and pruning to iteratively reduce the search space. Besides, we propose a discretization mechanism to offer a tradeoff between the reconstruction overhead and the pruning efficiency. Finally, a K-depth partial tree-sorting scheme with different sorting criteria is proposed to reduce the memory consumption. We demonstrate the superiorities of our algorithms over the state-of-the-art acceleration algorithms from both theoretical and experimental perspectives.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. We hereafter drop the notion of m for sake of simplicity.

  2. We have omitted \(x_1=R\) from the search tree.

  3. Recall that a path from the root to an internal node in a search tree specifies a partial assignment. We therefore slightly abuse \(est_{s}(PA)\) to denote the estimation of PA in \(tree(v_i,s)\) which is stored in the node that corresponds to PA.

  4. Note that the target variable is omitted from the search tree.

  5. The implementation of compared baselines and our proposed algorithms can be found in https://github.com/dyc941126/ARTGD2P.

References

  1. Aji, S. M., & McEliece, R. J. (2000). The generalized distributive law. IEEE Transactions on Information Theory, 46(2), 325–343.

    Article  MathSciNet  Google Scholar 

  2. Atlas, J., Warner, M., & Decker, K. (2008). A memory bounded hybrid approach to distributed constraint optimization. In Proceedings of 10th international workshop on distributed constraint reasoning (pp. 37–51).

  3. Chen, D., Deng, Y., Chen, Z., Zhang, W., & He, Z. (2020). HS-CAI: A hybrid DCOP algorithm via combining search with context-based inference. In AAAI (pp. 7087–7094).

  4. Chen, Z., Deng, Y., Wu, T., & He, Z. (2018). A class of iterative refined max-sum algorithms via non-consecutive value propagation strategies. Autonomous Agents and Multi-Agent Systems, 32(6), 822–860.

    Article  Google Scholar 

  5. Chen, Z., Jiang, X., Deng, Y., Chen, D., & He, Z. (2019). A generic approach to accelerating belief propagation based incomplete algorithms for DCOPs via a branch-and-bound technique. In AAAI (pp. 6038–6045).

  6. Chen, Z., Zhang, W., Deng, Y., Chen, D., & Li, Q. (2020). RMB-DPOP: Refining MB-DPOP by reducing redundant inference. In AAMAS (pp. 249–257).

  7. Cohen, L., Galiki, R., & Zivan, R. (2020). Governing convergence of max-sum on DCOPs through damping and splitting. Artificial Intelligence, 279, 103212.

    Article  MathSciNet  Google Scholar 

  8. Dechter, R., & Mateescu, R. (2007). AND/OR search spaces for graphical models. Artificial Intelligence, 171(2–3), 73–106.

    Article  MathSciNet  Google Scholar 

  9. Deng, Y., & An, B. (2020). Speeding up incomplete GDL-based algorithms for multi-agent optimization with dense local utilities. In IJCAI (pp. 31–38).

  10. Farinelli, A., Rogers, A., Petcu, A., & Jennings, N.R. (2008). Decentralised coordination of low-power embedded devices using the max-sum algorithm. In AAMAS (pp. 639–646).

  11. Fioretto, F., Yeoh, W., Pontelli, E., Ma, Y., & Ranade, S. J. (2017). A distributed constraint optimization (DCOP) approach to the economic dispatch with demand response. In AAMAS (pp. 999–1007).

  12. Freuder, E. C., & Quinn, M. J. (1985). Taking advantage of stable sets of variables in constraint satisfaction problems. IJCAI, 85, 1076–1078.

    Google Scholar 

  13. Gershman, A., Meisels, A., & Zivan, R. (2009). Asynchronous forward bounding for distributed COPs. Journal of Artificial Intelligence Research, 34, 61–88.

    Article  MathSciNet  Google Scholar 

  14. Givoni, I. E., & Frey, B. J. (2009). A binary variable model for affinity propagation. Neural Computation, 21(6), 1589–1600.

    Article  MathSciNet  Google Scholar 

  15. Hirayama, K., Miyake, K., Shiotani, T., & Okimoto, T. (2019). DSSA+: Distributed collision avoidance algorithm in an environment where both course and speed changes are allowed. International Journal on Marine Navigation and Safety of Sea Transportation, 13(1), 117–123.

    Article  Google Scholar 

  16. Hirayama, K., & Yokoo, M. (1997). Distributed partial constraint satisfaction problem. In CP (pp. 222–236).

  17. Hirayama, K., & Yokoo, M. (2005). The distributed breakout algorithms. Artificial Intelligence, 161(1–2), 89–115.

    Article  MathSciNet  Google Scholar 

  18. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National academy of Sciences, 102(46), 16569–16572.

    Article  Google Scholar 

  19. Hoang, K.D., Fioretto, F., Yeoh, W., Pontelli, E., & Zivan, R. (2018). A large neighboring search schema for multi-agent optimization. In CP (pp. 688–706).

  20. Katagishi, H., & Pearce, J.P. (2007). KOPT: Distributed DCOP algorithm for arbitrary k-optima with monotonically increasing utility. In DCR workshop.

  21. Khan, M.M., Tran-Thanh, L., & Jennings, N.R. (2018). A generic domain pruning technique for GDL-based DCOP algorithms in cooperative multi-agent systems. In AAMAS (pp. 1595–1603).

  22. Kim, Y., Krainin, M., & Lesser, V. (2011). Effective variants of the max-sum algorithm for radar coordination and scheduling. In WI/IAT (pp. 357–364).

  23. Kim, Y., & Lesser, V. (2013). Improved max-sum algorithm for DCOP with n-ary constraints. In AAMAS (pp. 191–198).

  24. Kim, Y., & Lesser, V. (2014). DJAO: A communication-constrained DCOP algorithm that combines features of ADOPT and Action-GDL. In AAAI (pp. 2680–2687).

  25. Komodakis, N., & Paragios, N. (2009). Beyond pairwise energies: Efficient optimization for higher-order mrfs. In CVPR (pp. 2985–2992). IEEE.

  26. Kschischang, F. R., Frey, B. J., Loeliger, H. A., et al. (2001). Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 47(2), 498–519.

    Article  MathSciNet  Google Scholar 

  27. Li, S., Negenborn, R. R., & Lodewijks, G. (2016). Distributed constraint optimization for addressing vessel rotation planning problems. Engineering Applications of Artificial Intelligence, 48, 159–172.

    Article  Google Scholar 

  28. Litov, O., & Meisels, A. (2017). Forward bounding on pseudo-trees for DCOPs and ADCOPs. Artificial Intelligence, 252, 83–99.

    Article  MathSciNet  Google Scholar 

  29. Macarthur, K.S., Stranders, R., Ramchurn, S.D., & Jennings, N.R. (2011). A distributed anytime algorithm for dynamic task allocation in multi-agent systems. In AAAI (pp. 701–706).

  30. Maheswaran, R. T., Pearce, J. P., & Tambe, M. (2004). Distributed algorithms for DCOP: A graphical-game-based approach. In ISCA PDCS (pp. 432–439).

  31. Marinescu, R., & Dechter, R. (2009). AND/OR branch-and-bound search for combinatorial optimization in graphical models. Artificial Intelligence, 173(16–17), 1457–1491.

    Article  MathSciNet  Google Scholar 

  32. Modi, P. J., Shen, W. M., Tambe, M., & Yokoo, M. (2005). ADOPT: Asynchronous distributed constraint optimization with quality guarantees. Artificial Intelligence, 161(1–2), 149–180.

    Article  MathSciNet  Google Scholar 

  33. Monteiro, T. L., Pujolle, G., Pellenz, M. E., Penna, M. C., & Souza, R. D. (2012). A multi-agent approach to optimal channel assignment in WLANs. In WCNC (pp. 2637–2642).

  34. Netzer, A., Grubshtein, A., & Meisels, A. (2012). Concurrent forward bounding for distributed constraint optimization problems. Artificial Intelligence, 193, 186–216.

    Article  MathSciNet  Google Scholar 

  35. Nguyen, D. T., Yeoh, W., Lau, H. C., & Zivan, R. (2019). Distributed Gibbs: A linear-space sampling-based DCOP algorithm. Journal of Artificial Intelligence Research, 64, 705–748.

    Article  MathSciNet  Google Scholar 

  36. Okamoto, S., Zivan, R., & Nahon, A. (2016). Distributed breakout: Beyond satisfaction. In IJCAI (pp. 447–453).

  37. Ottens, B., Dimitrakakis, C., & Faltings, B. (2017). DUCT: An upper confidence bound approach to distributed constraint optimization problems. ACM Transactions on Intelligent Systems and Technology, 8(5), 69:1-69:27.

    Article  Google Scholar 

  38. Pearce, J.P., & Tambe, M. (2007). Quality guarantees on k-optimal solutions for distributed constraint optimization problems. In IJCAI (pp. 1446–1451).

  39. Pepyne, D., Westbrook, D., Philips, B., Lyons, E., Zink, M., & Kurose, J. (2007). A design for distributed collaborative adaptive sensing of the atmosphere. Tech. rep. University of Massachusetts.

  40. Petcu, A., & Faltings, B. (2005). A scalable method for multiagent constraint optimization. In IJCAI (pp. 266–271).

  41. Petcu, A., & Faltings, B. (2007). MB-DPOP: A new memory-bounded algorithm for distributed optimization. In IJCAI (pp. 1452–1457).

  42. Pujol-Gonzalez, M., Cerquides, J., Meseguer, P., Rodríguez-Aguilar, J. A., & Tambe, M. (2013). Engineering the decentralized coordination of uavs with limited communication range. In CAEPIA (pp. 199–208).

  43. Rogers, A., Farinelli, A., Stranders, R., & Jennings, N. R. (2011). Bounded approximate decentralised coordination via the max-sum algorithm. Artificial Intelligence, 175(2), 730–759.

    Article  MathSciNet  Google Scholar 

  44. Rollon, E., & Larrosa, J. (2012). Improved bounded max-sum for distributed constraint optimization. In CP (pp. 624–632). Springer.

  45. Rollon, E., & Larrosa, J. (2014). Decomposing utility functions in bounded max-sum for distributed constraint optimization. In CP (pp. 646–654). Springer.

  46. Stranders, R., Farinelli, A., Rogers, A., & Jennings, N. R. (2009). Decentralised coordination of mobile sensors using the max-sum algorithm. In IJCAI (pp. 299–304).

  47. Sultanik, E. A., Lass, R. N., & Regli, W. C. (2008). DCOPolis: A framework for simulating and deploying distributed constraint reasoning algorithms. In AAMAS (pp. 1667–1668).

  48. Tarlow, D., Givoni, I., & Zemel, R. (2010). HOP-MAP: Efficient message passing with high order potentials. In AISTAT (pp. 812–819).

  49. Vinyals, M., Rodriguez-Aguilar, J.A., & Cerquides, J. (2009). Generalizing DPOP: Action-GDL, a new complete algorithm for DCOPs. In AAMAS (pp. 1239–1240).

  50. Weiss, Y., & Freeman, W. T. (2001). On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs. IEEE Transactions on Information Theory, 47(2), 736–744.

    Article  MathSciNet  Google Scholar 

  51. Yeoh, W., Felner, A., & Koenig, S. (2010). BnB-ADOPT: An asynchronous branch-and-bound DCOP algorithm. Journal of Artificial Intelligence Research, 38, 85–133.

    Article  Google Scholar 

  52. Yokoo, M., Durfee, E. H., Ishida, T., & Kuwabara, K. (1998). The distributed constraint satisfaction problem: Formalization and algorithms. IEEE Transactions on knowledge and data engineering, 10(5), 673–685.

    Article  Google Scholar 

  53. Zhang, W., Wang, G., Xing, Z., & Wittenburg, L. (2005). Distributed stochastic search and distributed breakout: Properties, comparison and applications to constraint optimization problems in sensor networks. Artificial Intelligence, 161(1–2), 55–87.

    Article  MathSciNet  Google Scholar 

  54. Zivan, R., Lev, O., & Galiki, R. (2020). Beyond trees: Analysis and convergence of belief propagation in graphs with multiple cycles. In AAAI (pp. 7333–7340).

  55. Zivan, R., Okamoto, S., & Peled, H. (2014). Explorative anytime local search for distributed constraint optimization. Artificial Intelligence, 212, 1–26.

    Article  MathSciNet  Google Scholar 

  56. Zivan, R., Parash, T., Cohen, L., Peled, H., & Okamoto, S. (2017). Balancing exploration and exploitation in incomplete min/max-sum inference for distributed constraint optimization. Autonomous Agents and Multi-Agent Systems, 31(5), 1165–1207.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISG-RP-2019-0013), National Satellite of Excellence in Trustworthy Software Systems (Award No: NSOE-TSS2019-01), and NTU.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore

    Yanchen Deng & Bo An

Authors
  1. Yanchen Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo An
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanchen Deng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This paper is an extension to our IJCAI paper [9]. Beside additional examples, experiments and proofs, we also present a K-depth partial tree-sorting scheme reducing the memory consumption by limiting the depth of search trees, which is not included in the IJCAI paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, Y., An, B. Utility distribution matters: enabling fast belief propagation for multi-agent optimization with dense local utility function. Auton Agent Multi-Agent Syst 35, 24 (2021). https://doi.org/10.1007/s10458-021-09511-z

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10458-021-09511-z

Keywords

Search

Navigation