Abstract
We consider the problem of learning Relational Logistic Regression (RLR). Unlike standard logistic regression, the features of RLR are first-order formulae with associated weight vectors instead of scalar weights. We turn the problem of learning RLR to learning these vector-weighted formulae and develop a learning algorithm based on the recently successful functional-gradient boosting methods for probabilistic logic models. We derive the functional gradients and show how weights can be learned simultaneously in an efficient manner. Our empirical evaluation on standard data sets demonstrates the superiority of our approach over other methods for learning RLR.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We use the term example to mean the grounded target literal. Hence \(y_i = 1\) denotes that the grounding \(\mathsf {Q}(\mathbf {X})=1\) i.e., the grounded target predicate is true. Following standard Bayesian networks terminology, we denote the parents \(\mathcal {A}(\mathsf {Q})\) to include the set of formulae \(\psi \) that influence the current predicate \(\mathsf {Q}\).
We use formulae and clauses interchangeably from hereon.
References
Blockeel H (1999) Top–down induction of first order logical decision trees. AI Commun 12(1–2):119–120
Craven M, McCallum A, PiPasquo D, Mitchell T, Freitag D (1998) Learning to extract symbolic knowledge from the world wide web. Proceedings of the 15th AAAI conference on artificial intelligence, pp 509–516
Dietterich TG, Ashenfelter A, Bulatov Y (2004) Training conditional random fields via gradient tree boosting. In: Proceedings of the twenty–first international conference on machine learning, p 28
Domingos P, Lowd D (2009) Markov logic: an interface layer for artificial intelligence, vol 3. Morgan & Claypool Publishers
Fatemi B, Kazemi SM, Poole D (2016) A learning algorithm for relational logistic regression: preliminary results. arXiv preprint arXiv:1606.08531
Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat, pp 1189–1232
Gutmann B, Kersting K (2006) Tildecrf: conditional random fields for logical sequences. In: European conference on machine learning, Springer, Berlin, pp 174–185
Getoor L, Taskar B, eds. (2007) Introduction to statistical relational learning. MIT Press, Cambridge
Harper FM, Konstan JA (2015) The movielens datasets: history and context. ACM Trans Interact Intell Systems 5(4):1–19
Heckerman D, Meek C, Koller D (2007) Probabilistic entity–relationship models, PRMs, and plate models. Introduction to statistical relational learning pp 201–238
Huynh TN, Mooney RJ (2008) Discriminative structure and parameter learning for markov logic networks. In: Proceedings of the 25th international conference on machine learning, pp 416–423
Kazemi SM, Poole D (2018) Relnn: a deep neural model for relational learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 32
Kazemi SM, Buchman D, Kersting K, Natarajan S, Poole D (2014a) Relational logistic regression: the directed analog of markov logic networks. In: AAAI workshop: statistical relational artificial intelligence, Citeseer
Kersting K, De Raedt L (2007) Bayesian logic programming: theory and tool. In: Taskar B, Getoor L (ed). An introduction to statistical relational learning, MIT Press
Kersting K, Driessens K (2008) Non–parametric policy gradients: a unified treatment of propositional and relational domains. In: Proceedings of the 25th international conference on machine learning, pp 456–463
Khot T, Natarajan S, Kersting K, Shavlik J (2011) Learning markov logic networks via functional gradient boosting. In: 2011 IEEE 11th international conference on data mining, IEEE, pp 320–329
Kimmig A, Bach S, Broecheler M, Huang B, Getoor L (2012) A short introduction to probabilistic soft logic. In: Proceedings of the NIPS workshop on probabilistic programming: foundations and applications, pp 1–4
Kok S, Domingos P (2005) Learning the structure of markov logic networks. In: Proceedings of the 22nd international conference on machine learning, pp 441–448
Kok S, Domingos P (2009) Learning markov logic network structure via hypergraph lifting. In: Proceedings of the 26th annual international conference on machine learning, pp 505–512
Koller D (1999) Probabilistic relational models. In: International conference on inductive logic programming, Springer, pp 3–13
Lowd D, Domingos P (2007) Efficient weight learning for markov logic networks. In: European conference on principles of data mining and knowledge discovery, Springer, pp 200–211
Malec M, Khot T, Nagy J, Blasch E, Natarajan S (2016) Inductive logic programming meets relational databases: an application to statistical relational learning
McCullagh P (1984) Generalized linear models. Eur J Oper Res 16(3):285–292
Mihalkova L, Mooney RJ (2007) Bottom–up learning of markov logic network structure. In: Proceedings of the 24th international conference on machine learning, pp 625–632
Muggleton S (1995) Inverse entailment and Progol. New Gener Comput 13(3–4):245–286
Muggleton S (1996) Learning from positive data. In: International conference on inductive logic programming, Springer, pp 358–376
Muggleton S, De Raedt L (1994) Inductive logic programming: theory and methods. J Log Program 19:629–679
Natarajan S, Tadepalli P, Dietterich TG, Fern A (2008) Learning first-order probabilistic models with combining rules. Ann Math Artif Intell 54(1):223–256
Natarajan S, Joshi S, Tadepalli P, Kersting K, Shavlik J (2011) Imitation learning in relational domains: a functional–gradient boosting approach. In: IJCAI proceedings–international joint conference on artificial intelligence, Citeseer, vol 22, p 1414
Natarajan S, Khot T, Kersting K, Gutmann B, Shavlik J (2012) Gradient-based boosting for statistical relational learning: the relational dependency network case. Mach Learn 86(1):25–56
Natarajan S, Prabhakar A, Ramanan N, Bagilone A, Siek K, Connelly K (2017) Boosting for postpartum depression prediction. In: 2017 IEEE/ACM international conference on connected health: applications, systems and engineering technologies (CHASE), IEEE, pp 232–24
Poole D, Buchman D, Kazemi SM, Kersting K, Natarajan S (2014) Population size extrapolation in relational probabilistic modelling. In: International conference on scalable uncertainty management, Springer, pp 292–305
Popescul A, Ungar LH (2003) Structural logistic regression for link analysis. Departmental Papers (CIS) p 133
Raedt LD, Kersting K, Natarajan S, Poole D (2016) Statistical relational artificial intelligence: logic, probability, and computation. Synth Lect Artif Intell Mach Learn 10(2):1–189
Richardson M, Domingos P (2006) Markov logic networks. Mach Learn 62(1–2):107–136
Srinivasan A (2001) The aleph manual. http://web.comlab.ox.ac.uk/oucl/research/areas/machlearn/Aleph/
Taskar B, Abbeel P, Wong MF, Koller D (2007) Relational markov networks. Introduction to statistical relational learning, pp 175–200
Yang S, Khot T, Kersting K, Kunapuli G, Hauser K, Natarajan S (2014) Learning from imbalanced data in relational domains: a soft margin approach. In: 2014 IEEE international conference on data mining, IEEE, pp 1085–1090
Yang S, Korayem M, AlJadda K, Grainger T, Natarajan S (2017) Combining content-based and collaborative filtering for job recommendation system: a cost-sensitive statistical relational learning approach. Knowl-Based Syst 136:37–45
Acknowledgements
SN & NR gratefully acknowledge AFOSR award FA9550-18-1-0462 and the support of relationalAI. We thank the anonymous reviewers for their insightful comments and in significantly improving the paper. Any opinions, findings, and conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the view of the AFOSR, or the US government.
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible editor: Hendrik Blockeel.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ramanan, N., Kunapuli, G., Khot, T. et al. Structure learning for relational logistic regression: an ensemble approach. Data Min Knowl Disc 35, 2089–2111 (2021). https://doi.org/10.1007/s10618-021-00770-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10618-021-00770-8