Skip to main content
Log in

Predicting Visual Political Bias Using Webly Supervised Data and an Auxiliary Task

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The news media shape public opinion, and often, the visual bias they contain is evident for careful human observers. This bias can be inferred from how different media sources portray different subjects or topics. In this paper, we model visual political bias in contemporary media sources at scale, using webly supervised data. We collect a dataset of over one million unique images and associated news articles from left- and right-leaning news sources, and develop a method to predict the image’s political leaning. This problem is particularly challenging because of the enormous intra-class visual and semantic diversity of our data. We propose two stages of training to tackle this problem. In the first stage, the model is forced to learn relevant visual concepts that, when joined with document embeddings computed from articles paired with the images, enable the model to predict bias. In the second stage, we remove the requirement of the text domain and train a visual classifier from the features of the former model. We show this two-stage approach that relies on an auxiliary task leveraging text, facilitates learning and outperforms several strong baselines. We present extensive quantitative and qualitative results analyzing our dataset. Our results reveal disparities in how different sides of the political spectrum portray individuals, groups, and topics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Our dataset, code, and additional materials are available here: http://www.cs.pitt.edu/~chris/politics.

  2. https://github.com/dragnet-org/dragnet.

  3. http://dlib.net/dnn_mmod_face_detection_ex.cpp.html.

References

  • Akyürek, A. F., Guo, L., Elanwar, R., Ishwar, P., Betke, M., & Wijaya, D. T. (2020). Multi-label and multilingual news framing analysis. In Proceedings of the 58th annual meeting of the association for computational linguistics (pp. 8614–8624).

  • Alayrac, J. B., Recasens, A., Schneider, R., Arandjelović, R., Ramapuram, J., De Fauw, J., Smaira, L., Dieleman, S., & Zisserman, A. (2020). Self-supervised multimodal versatile networks. In Neural Information Processing Systems (NeurIPS).

  • Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S., & Zhang, L. (2018) Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Angermeyer, M. C., & Schulze, B. (2001). Reinforcing stereotypes: How the focus on forensic cases in news reporting may influence public attitudes towards the mentally ill. International Journal of Law and Psychiatry.

  • Baly, R., Karadzhov, G., Alexandrov, D., Glass, J., & Nakov, P. (2018). Predicting factuality of reporting and bias of news media sources. In Proceedings of the conference on empirical methods in natural language processing (pp. 3528–3539).

  • Baumer, E., Elovic, E., Qin, Y., Polletta, F., & Gay, G. (2015) Testing and comparing computational approaches for identifying the language of framing in political news. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1472–1482).

  • Bechavod, Y., & Ligett, K. (2017). Penalizing unfairness in binary classification. arXiv preprint arXiv:1707.00044.

  • Bolukbasi, T., Chang, K. W., Zou, J. Y., Saligrama, V., & Kalai, A. T. (2016). Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems (NIPS)

  • Borghi, G., Pini, S., Grazioli, F., Vezzani, R., & Cucchiara, R. (2018) Face verification from depth using privileged information. In British Machine Vision Conference (BMVC). Springer.

  • Burns, K., Hendricks, L.A., Darrell, T., & Rohrbach, A. (2018) Women also snowboard: Overcoming bias in captioning models. In European Conference on Computer Vision (ECCV).

  • Card, D., Boydstun, A., Gross, J. H., Resnik, P., & Smith, N. A. (2015). The media frames corpus: Annotations of frames across issues. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (Volume 2: Short Papers) (pp. 438–444).

  • Chattopadhay, A., Sarkar, A., Howlader, P., Balasubramanian, V. N. (2018) Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In Winter conference on applications of computer vision (WACV) (pp. 839–847). IEEE.

  • Chen, T. H., Liao, Y. H., Chuang, C. Y., Hsu, W. T., Fu, J., & Sun, M. (2017). Show, adapt and tell: Adversarial training of cross-domain image captioner. In Proceedings of the IEEE international conference on computer vision (ICCV).

  • Chen, X., & Gupta, A. (2015). Webly supervised learning of convolutional networks. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 1431–1439).

  • Chen, Y. C., Li, L., Yu, L., Kholy, A. E., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020). Uniter: Learning universal image-text representations. In European Conference on Computer Vision (ECCV).

  • Cinbis, R. G., Verbeek, J., & Schmid, C. (2016). Weakly supervised object localization with multi-fold multiple instance learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 39(1), 189–203.

    Article  Google Scholar 

  • Cohen, R., & Ruths, D. (2013). Classifying political orientation on twitter: It’s not easy! In Seventh International Association for the Advancement of Artificial Intelligence (AAAI) Conference on Weblogs and Social Media.

  • Colleoni, E., Rozza, A., & Arvidsson, A. (2014). Echo chamber or public sphere? predicting political orientation and measuring political homophily in twitter using big data. Journal of Communication, 64(2), 317–332.

    Article  Google Scholar 

  • Conover, M. D., Gonçalves, B., Ratkiewicz, J., Flammini, A., & Menczer, F. (2011). Predicting the political alignment of twitter users. In IEEE Third International Conference on Privacy, Security, Risk and Trust (PASSAT) and IEEE Third International Conference on Social Computing (SocialCom) (pp. 192–199). IEEE.

  • Dai, B., Fidler, S., Urtasun, R., & Lin, D. (2017) Towards diverse and natural image descriptions via a conditional gan. In Proceedings of the IEEE international conference on computer vision (ICCV).

  • Doersch, C., Singh, S., Gupta, A., Sivic, J., & Efros, A. (2012). What makes Paris look like Paris? ACM Transactions on Graphics, 31(4), 10.

    Article  Google Scholar 

  • Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., & Darrell, T. (2015). Long-term recurrent convolutional networks for visual recognition and description. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Dong, Z., Shi, C., Sen, S., Terveen, L., & Riedl, J. (2012). War versus inspirational in forrest gump: Cultural effects in tagging communities. In Sixth international AAAI conference on weblogs and social media.

  • Edsall, T. B. (2012). Studies: Conservatives are from mars, liberals are from venus. https://www.theatlantic.com/politics/archive/2012/02/studies-conservatives-are-from-mars-liberals-are-from-venus/252416/.

  • Eickhoff, C. (2018). Cognitive biases in crowdsourcing. In Proceedings of the Eleventh ACM international conference on web search and data mining (pp. 162–170). ACM.

  • Eisenschtat, A., & Wolf, L. (2017). Linking image and text with 2-way nets. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Elliott, D., & Kádár, Á. (2017). Imagination improves multimodal translation. In Proceedings of the eighth international joint conference on natural language processing (Volume 1: Long Papers) (pp. 130–141).

  • Faghri, F., Fleet, D. J., Kiros, J. R., & Fidler, S. (2018) Vse++: Improved visual-semantic embeddings. In British Machine Vision Conference (BMVC).

  • Garbe, W. (2019). Symspell. https://github.com/wolfgarbe/SymSpell.

  • Gilens, M. (1996). Race and poverty in Americapublic misperceptions and the American news media. Public Opinion Quarterly, 60(4), 515–541.

    Article  Google Scholar 

  • Glorot, X., & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics (AISTATS) (pp. 249–256).

  • Gomez, L., Patel, Y., Rusinol, M., Karatzas, D., Jawahar, C. V. (2017). Self-supervised learning of visual features through embedding images into text topic spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Happer, C., & Philo, G. (2013). The role of the media in the construction of public belief and social change. Journal of Social and Political Psychology, 1(1), 321–336.

    Article  Google Scholar 

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 770–778).

  • He, T., Zhang, Z., Zhang, H., Zhang, Z., Xie, J., & Li, M. (2019). Bag of tricks for image classification with convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp 558–567).

  • Hessel, J., Lee, L., & Mimno, D. (2018). Quantifying the visual concreteness of words and topics in multimodal datasets. In North American Association for Computational Linguistics.

  • Hoffman, J., Gupta, S., & Darrell, T. (2016). Learning with side information through modality hallucination. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 826–834). IEEE.

  • Honnibal, M., & Montani, I. (2017). spacy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing (to appear).

  • Hussain, Z., Zhang, M., Zhang, X., Ye, K., Thomas, C., Agha, Z., Ong, N., & Kovashka, A. (2017). Automatic understanding of image and video advertisements. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Jae Lee, Y., Efros, A. A., & Hebert, M. (2013). Style-aware mid-level representation for discovering visual connections in space and time. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 1857–1864).

  • Jiang, L., Meng, D., Zhao, Q., Shan, S., & Hauptmann, A. G. (2015). Self-paced curriculum learning. In Twenty-ninth association for the advancement of artificial intelligence (AAAI) conference on artificial intelligence (Vol. 2, p. 6).

  • Jiang, L., Zhou, Z., Leung, T., Li, L. J., & Fei-Fei, L. (2018). Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In Proceedings of the international conference on machine learning (ICML) (pp. 2309–2318).

  • Jin, Z., Cao, J., Zhang, Y., Zhou, J., & Tian, Q. (2016). Novel visual and statistical image features for microblogs news verification. IEEE Transactions on Multimedia, 19(3), 598–608.

  • Johnson, J., Karpathy, A., Fei-Fei, L. (2016). Densecap: Fully convolutional localization networks for dense captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  • Joo, J., Li, W., Steen, F. F., & Zhu, S. C. (2014) Visual persuasion: Inferring communicative intents of images. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Joo, J., Steen, F. F., & Zhu, S. C. (2015). Automated facial trait judgment and election outcome prediction: Social dimensions of face. In Proceedings of the IEEE international conference on computer vision (ICCV).

  • Karimi, H., & Tang, J. (2019). Learning hierarchical discourse-level structure for fake news detection. In Proceedings of the 2019 conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (pp. 3432–3442).

  • Khattar, D., Goud, J. S., Gupta, M., & Varma, V. (2019) Mvae: Multimodal variational autoencoder for fake news detection. In The World Wide Web Conference (pp. 2915–2921).

  • King, D. E. (2009). Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research, 10, 1755–1758.

    Google Scholar 

  • Kingma, D. P., & Ba, J. (2015) Adam: A method for stochastic optimization. In Proceedings of the international conference on learning representations (ICLR).

  • Kiros, R., Salakhutdinov, R., & Zemel, R. S. (2015). Unifying visual-semantic embeddings with multimodal neural language models. In: TACL.

  • Kosinski, M. (2021). Facial recognition technology can expose political orientation from naturalistic facial images. Scientific Reports, 11(1), 1–7.

    Article  Google Scholar 

  • Lambert, J., Sener, O., & Savarese, S. (2018). Deep learning under privileged information using heteroscedastic dropout. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and documents. In Proceedings of the international conference on machine learning (ICML) (pp. 1188–1196).

  • Li, H., Ellis, J. G., Zhang, L., & Chang, S. F. (2018). Patternnet: Visual pattern mining with deep neural network. In Proceedings of the 2018 ACM on international conference on multimedia retrieval (pp. 291–299). ACM.

  • Li, Y., Liu, L., Shen, C., & Van Den Hengel, A. (2017). Mining mid-level visual patterns with deep cnn activations. International Journal of Computer Vision (IJCV), 121(3), 344–364.

    Article  MathSciNet  Google Scholar 

  • Liu, S., Guo, L., Mays, K., Betke, M., & Wijaya, D. T. (2019). Detecting frames in news headlines and its application to analyzing news framing trends surrounding us gun violence. In Proceedings of the 23rd conference on computational natural language learning (CoNLL) (pp. 504–514).

  • Liu, Z., Luo, P., Wang, X., & Tang, X. (2015). Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision (ICCV).

  • Lu, J., Batra, D., Parikh, D., & Lee, S. (2019). Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Advances in neural information processing systems (NeurIPS) (pp. 13–23).

  • Lu, J., Yang, J., Batra, D., & Parikh, D. (2016). Hierarchical question-image co-attention for visual question answering. In Advances in Neural information processing systems (NIPS) (pp. 289–297).

  • Malkov, Y. A., & Yashunin, D. A. (2016) Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).

  • Merrill, J. B. (2016). Liberal, moderate or conservative? see how facebook labels you. The New York Times https://www.nytimes.com/2016/08/24/us/politics/facebook-ads-politics.html.

  • Messing, S., Jabon, M., & Plaut, E. (2016). Bias in the flesh: Skin complexion and stereotype consistency in political campaigns. Public Opinion Quarterly, 80(1), 44–65.

    Article  Google Scholar 

  • Minghui Liao, B. S., & Bai, X. (2018). TextBoxes++: A single-shot oriented scene text detector. IEEE Transactions on Image Processing, 27(8), 3676–3690. https://doi.org/10.1109/TIP.2018.2825107.

    Article  MathSciNet  MATH  Google Scholar 

  • Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2017). Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing.

  • Morin, F., & Bengio, Y. (2005). Hierarchical probabilistic neural network language model. In Tenth international workshop on artificial intelligence and statistics (AISTATS) (Vol. 5, pp. 246–252). Citeseer.

  • Motiian, S., Piccirilli, M., Adjeroh, D. A., & Doretto, G. (2016). Information bottleneck learning using privileged information for visual recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1496–1505). IEEE.

  • Muñoz, C. L., & Towner, T. L. (2017). The image is the message: Instagram marketing and the 2016 presidential primary season. Journal of Political Marketing, 16(3–4), 290–318.

    Article  Google Scholar 

  • Nguyen D, Trieschnigg D, Doğruöz AS, Gravel R, Theune M, Meder T, De Jong F (2014) Why gender and age prediction from tweets is hard: Lessons from a crowdsourcing experiment. In: Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers (COLING), pp 1950–1961

  • Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. NYU Press.

  • Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2019). Social data: Biases, methodological pitfalls, and ethical boundaries. Frontiers in Big Data, 2, 13.

    Article  Google Scholar 

  • Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2015). Is object localization for free?-weakly-supervised learning with convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 685–694).

  • Orr, G. B. (1997). Removing noise in on-line search using adaptive batch sizes. In Advances in neural information processing systems (pp. 232–238).

  • Otterbacher, J., Checco, A., Demartini, G., & Clough, P. (2018). Investigating user perception of gender bias in image search: The role of sexism. In The 41st international ACM SIGIR conference on research & development in information retrieval (pp. 933–936). ACM.

  • Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., & Lerer, A. (2017). Automatic differentiation in pytorch. In Advances in neural information processing systems workshops (NIPS-W).

  • Peck, T., & Boutelier, N. (2018). Big political data. https://www.isidewith.com/polls.

  • Pedersoli, M., Lucas, T., Schmid, C., & Verbeek, J. (2017). Areas of attention for image captioning. In Proceedings of the IEEE international conference on computer vision (ICCV).

  • Peng, Y. (2018). Same candidates, different faces: Uncovering media bias in visual portrayals of presidential candidates with computer vision. Journal of Communication, 68(5), 920–941.

    Article  Google Scholar 

  • Pennacchiotti, M., & Popescu, A. M. (2011). A machine learning approach to twitter user classification. In Fifth international association for the advancement of artificial intelligence (AAAI) conference on weblogs and social media.

  • Pentina, A., Sharmanska, V., & Lampert, C. H. (2015). Curriculum learning of multiple tasks. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 5492–5500).

  • Peters, M. E., & Lecocq, D. (2013). Content extraction using diverse feature sets. In Proceedings of the 22nd international conference on world wide web (WWW) (pp. 89–90). ACM.

  • Philo, G. (2008). Active audiences and the construction of public knowledge. Journalism Studies, 9(4), 535–544.

    Article  Google Scholar 

  • Potthast, M., Kiesel, J., Reinartz, K., Bevendorff, J., & Stein, B. (2018). A stylometric inquiry into hyperpartisan and fake news. In Proceedings of the 56th annual meeting of the association for computational linguistics (Volume 1: Long Papers) (pp. 231–240).

  • Recasens, M., Danescu-Niculescu-Mizil, C., Jurafsky, D. (2013). Linguistic models for analyzing and detecting biased language. In Proceedings of the 51st annual meeting of the association for computational linguistics (Volume 1: Long Papers) (Vol. 1, pp. 1650–1659).

  • Řehůřek, R., Sojka, P. (2010). Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 workshop on new challenges for NLP frameworks, ELRA, Valletta, Malta (pp. 45–50). http://is.muni.cz/publication/884893/en.

  • Richard, A., Kuehne, H., & Gall, J. (2017). Weakly supervised action learning with RNN based fine-to-coarse modeling. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 754–763).

  • Ryu, H. J., Mitchell, M., & Adam, H. (2017). Improving smiling detection with race and gender diversity. arXiv preprint arXiv:1712.00193.

  • Schill, D. (2012). The visual image and the political image: A review of visual communication research in the field of political communication. Review of Communication, 12(2), 118–142.

    Article  Google Scholar 

  • Schreiber, D., Fonzo, G., Simmons, A. N., Dawes, C. T., Flagan, T., Fowler, J. H., & Paulus, M. P. (2013). Red brain, blue brain: Evaluative processes differ in democrats and republicans. PLOS ONE, 8(2), 1–6. https://doi.org/10.1371/journal.pone.0052970.

    Article  Google Scholar 

  • Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 815–823).

  • Sen, S., Giesel, M. E., Gold, R., Hillmann, B., Lesicko, M., Naden, S., Russell, J., Wang, Z. K., & Hecht, B. (2015). Turkers, scholars, arafat and peace: Cultural communities and algorithmic gold standards. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing (pp. 826–838). ACM.

  • Sharmanska, V., Quadrianto, N., & Lampert, C. H. (2013). Learning to rank using privileged information. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 825–832). IEEE.

  • Sicre, R., Avrithis, Y. S., Kijak, E., & Jurie, F. (2017). Unsupervised part learning for visual recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 3116–3124).

  • Singh, S., Gupta, A., & Efros, A. A. (2012). Unsupervised discovery of mid-level discriminative patches. In Proceedings of the European conference on computer vision (ECCV) (pp. 73–86). Springer.

  • Tan, H., & Bansal, M. (2019). Lxmert: Learning cross-modality encoder representations from transformers. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 5103–5114).

  • Thomas, C., & Kovashka, A. (2018). Persuasive faces: Generating faces in advertisements. In Proceedings of the British machine vision conference (BMVC).

  • Thomas, C., & Kovashka, A. (2019). Predicting the politics of an image using webly supervised data. In Advances in neural information processing systems (NeurIPS) (pp. 3625–3637).

  • Thomas, C., & Kovashka, A. (2020). Preserving semantic neighborhoods for robust cross-modal retrieval. In European Conference on Computer Vision (ECCV) (pp. 317–335). Springer.

  • Vapnik, V., & Izmailov, R. (2015). Learning using privileged information: Similarity control and knowledge transfer. Journal of Machine Learning Research (JMLR), 16(2023–2049), 2.

    MathSciNet  MATH  Google Scholar 

  • Venugopalan, S., Anne Hendricks, L., Rohrbach, M., Mooney, R., Darrell, T., & Saenko, K. (2017). Captioning images with diverse objects. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015). Show and tell: A neural image caption generator. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 3156–3164).

  • Vogel, K., Shane, S., & Kingsley, P. (2018). How vilification of george soros moved from the fringes to the mainstream. https://www.nytimes.com/2018/10/31/us/politics/george-soros-bombs-trump.html. Accessed January 15, 2020.

  • Volkova, S., Coppersmith, G., & Van Durme, B. (2014). Inferring user political preferences from streaming communications. In Proceedings of the 52nd annual meeting of the association for computational linguistics (Volume 1: Long Papers) (Vol. 1, pp. 186–196).

  • Wang, L., Xiong, Y., Lin, D., & Van Gool, L. (2017a). Untrimmednets for weakly supervised action recognition and detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 4325–4334).

  • Wang, Y., Feng, Y., Hong, Z., Berger, R., & Luo, J. (2017b). How polarized have we become? a multimodal classification of trump followers and clinton followers. In International conference on social informatics.

  • Wang, Y., Li, Y., & Luo, J. (2016). Deciphering the 2016 us presidential campaign in the twitter sphere: A comparison of the trumpists and clintonists. In Tenth international association for the advancement of artificial intelligence (AAAI) conference on web and social media (pp. 723–726).

  • Wei, Y., Shen, Z., Cheng, B., Shi, H., Xiong, J., Feng, J., & Huang, T. (2018). Ts2c: Tight box mining with surrounding segmentation context for weakly supervised object detection. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 434–450).

  • Wong, F. M. F., Tan, C. W., Sen, S., & Chiang, M. (2016). Quantifying political leaning from tweets, retweets, and retweeters. IEEE Transactions on Knowledge and Data Engineering, 28(8), 2158–2172.

    Article  Google Scholar 

  • Xi, N., Ma, D., Liou, M., Steinert-Threlkeld, Z. C., Anastasopoulos, J., & Joo, J. (2020). Understanding the political ideology of legislators from social media images. Proceedings of the International AAAI Conference on Web and Social Media, 14, 726–737.

    Google Scholar 

  • Xiong, C., Zhong, V., & Socher, R. (2017). Dynamic coattention networks for question answering. In Proceedings of the international conference on learning representations (ICLR).

  • Ye, K., Honarvar Nazari, N., Hahn, J., Hussain, Z., Zhang, M., & Kovashka, A. (2019). Interpreting the rhetoric of visual advertisements. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI). https://doi.org/10.1109/TPAMI.2019.2947440.

  • Ye, K., & Kovashka, A. (2018). Advise: Symbolism and external knowledge for decoding advertisements. In Proceedings of the European Conference on Computer Vision (ECCV).

  • Ye, K., Zhang, M., Kovashka, A., Li, W., Qin, D., Berent, J. (2019). Cap2det: Learning to amplify weak caption supervision for object detection. In Proceedings of the IEEE international conference on computer vision (ICCV).

  • Yoon, J., Joo, J., Park, E., & Han, J. (2020). Cross-domain classification of facial appearance of leaders. In International conference on social informatics (pp. 440–446). Springer.

  • Zamir, A. R., Wu, T. L., Sun, L., Shen, W. B., Shi, B. E., Malik, J., & Savarese, S. (2017). Feedback networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1808–1817). IEEE.

  • Zhang, J., Wu, Q., Zhang, J., Shen, C., & Lu, J. (2019) Mind your neighbours: Image annotation with metadata neighbourhood graph co-attention networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2956–2964).

  • Zhang, Y., David, P., & Gong, B. (2017). Curriculum domain adaptation for semantic segmentation of urban scenes. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) (pp. 2020–2030).

  • Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K. W. (2017). Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the conference on empirical methods in natural language processing (EMNLP).

  • Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2921–2929).

  • Zhou, F., De la Torre, F., & Cohn, J. F. (2010). Unsupervised discovery of facial events. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2574–2581). IEEE.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Christopher Thomas.

Additional information

Communicated by Judy Hoffman.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This material is based upon work supported by the National Science Foundation under Grant Numbers 1566270 and 1718262. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. Funding was also provided by a Nvidia hardware grant.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thomas, C., Kovashka, A. Predicting Visual Political Bias Using Webly Supervised Data and an Auxiliary Task. Int J Comput Vis 129, 2978–3003 (2021). https://doi.org/10.1007/s11263-021-01506-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-021-01506-3

Keywords

Navigation