Определение демографических атрибутов пользователей микроблогов

При заполнении полей профиля в различных интернет-сервисах пользователи зачастую по ошибке или преднамеренно не указывают значения некоторых демографических атрибутов, таких как пол, возраст, семейное положение, уровень образования, религиозные и политические взгляды. Вместе с тем, информация об атрибутах пользователей позволяет существенно повысить эффективность систем рекомендации, интернет-маркетинга и других приложений, предполагающих персонализацию результатов. В статье предлагается метод автоматического определения демографических атрибутов пользователей социального сервиса микроблогов Twitter по текстам их сообщений и другой доступной информации из профилей. Метод основан на алгоритме машинного обучения, его отличительными особенностями являются полностью автоматическое построение исходного набора данных для обучения и тестирования, а также поддержка широкого набора языков и демографических атрибутов. Экспериментальные исследования показали высокое качество результатов определения пола, возраста и семейного положения пользователя для наиболее популярных языков: английского, русского, немецкого, французского, итальянского и испанского. Кроме того, для английского языка поддерживается также определение уровня образования, а также религиозных и политических взглядов пользователя.

демографические характеристики, демографические атрибуты, социальные сети, микроблоги, обработка текстов на естественном языке, анализ содержимого, компьютерная лингвистика, машинное обучение

1. Sloan L. Knowing the Tweeters: Deriving Sociologically Relevant Demographics from Twitter. [Текст] / L. Sloan [et al.] – Sociological Research Online. – 2013. – Т. 18. – №. 3. – p. 7.

2. Tang C. What’s in a name: A study of names, gender inference, and gender behavior in facebook. [Текст] / C. Tang [et al.] – Database Systems for Adanced Applications. – Springer Berlin Heidelberg, 2011. – pp. 344–356.

3. Miller Z. Gender Prediction on Twitter Using Stream Algorithms with N-Gram Character Features. [Текст] / Z. Miller, B. Dickinson, W. Hu – International Journal. – 2012. – Т. 2.

4. Deitrick W. Gender identification on twitter using the modified balanced winnow. [Текст] / W. Deitrick [et al.] – Communications and Network. – 2012. – Т. 4. – №. 3. – pp. 189–195.

5. Burger J. D. Discriminating gender on Twitter. [Текст] / J. D. Burger [et al.] – Proceedings of the Conference on Empirical Methods in Natural Language Processing. – Association for Computational Linguistics, 2011. – pp. 1301–1309.

6. Schwartz H. A. Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach. [Текст] / H. A. Schwartz [et al.] – PloS one. – 2013. – Т. 8. – №. 9. – p. 73791.

7. Filippova K. User demographics and language in an implicit social network. [Текст] – Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. – Association for Computational Linguistics, 2012. – pp. 1478–1488.

8. Cheng N. Author gender identification from text. [Текст] / N. Cheng, R. Chandramouli, K. P. Subbalakshmi – Digital Investigation. – 2011. – Т. 8. – №. 1. – pp. 78–88.

9. Rao D. Classifying latent user attributes in twitter. [Текст] / D. Rao [et al.] – Proceedings of the 2nd international workshop on Search and mining user-generated contents. – ACM, 2010. – pp. 37–44.

10. Rao D. Hierarchical Bayesian Models for Latent Attribute Detection in Social Media. [Текст] / D. Rao [et al.] – ICWSM. – 2011.

11. Mukherjee A. Improving gender classification of blog authors. [Текст] / A. Mukherjee, B. Liu – Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. – Association for Computational Linguistics, 2010. – pp. 207–217.

12. Liu W. What’s in a Name? Using First Names as Features for Gender Inference in Twitter. [Текст] / W. Liu, D. Ruths – 2013 AAAI Spring Symposium Series. – 2013.

13. Al Zamal F. Homophily and Latent Attribute Inference: Inferring Latent Attributes of Twitter Users from Neighbors. [Текст] / F. Al Zamal, W. Liu, D. Ruths – ICWSM. – 2012.

14. Garera N. Modeling latent biographic attributes in conversational genres. [Текст] / N. Garera, D. Yarowsky – Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. – Association for Computational Linguistics, 2009. – Vol. 2, pp. 710–718.

15. Schler J. Effects of Age and Gender on Blogging. [Текст] / J. Schler [et al.] – AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs. – 2006. – pp. 199–205.

16. Goswami S. Stylometric analysis of bloggers’ age and gender. [Текст] / S. Goswami, S. Sarkar, M. Rustagi – Third International AAAI Conference on Weblogs and Social Media. – 2009.

17. Nguyen D. Author age prediction from text using linear regression. [Текст] / D. Nguyen, N. A. Smith, C. P. Rosé – Proceedings of the 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities. – Association for Computational Linguistics, 2011. – pp. 115–123.

18. van Heerden C. Combining regression and classification methods for improving automatic speaker age recognition. [Текст] / C. van Heerden [et al.] – Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on. – IEEE, 2010. – pp. 5174–5177.

19. Nguyen D. “How Old Do You Think I Am?”: A Study of Language and Age in Twitter. [Текст] / D. Nguyen [et al.] – Seventh International AAAI Conference on Weblogs and Social Media. – 2013.

20. Peersman C. Predicting age and gender in online social networks. [Текст] / C. Peersman, W. Daelemans, L. Van Vaerenbergh – Proceedings of the 3rd international workshop on Search and mining user-generated contents. – ACM, 2011. – pp. 37–44.

21. Rosenthal S. Age prediction in blogs: A study of style, content, and online behavior in pre-and post-social media generations. [Текст] / S. Rosenthal, K. McKeown. – Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. – Association for Computational Linguistics, 2011. – Vol. 1, pp. 763–772.

22. Pennacchiotti M. A Machine Learning Approach to Twitter User Classification. [Текст] / M. Pennacchiotti, A. M. Popescu – ICWSM. – 2011.

23. Conover M. D. Predicting the political alignment of twitter users. [Текст] / M. D. Conover [et al.] – Privacy, security, risk and trust (passat), 2011 ieee third international conference on and 2011 ieee third international conference on social computing (socialcom). – IEEE, 2011. – pp. 192–199.

24. Eisenstein J. A latent variable model for geographic lexical variation [Текст] / J. Eisenstein [et al.] – Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. – Association for Computational Linguistics, 2010. – pp. 1277–1287.

25. Cheng Z. You are where you tweet: a content-based approach to geo-locating twitter users. [Текст] / Z. Cheng, J. Caverlee, K. Lee – Proceedings of the 19th ACM international conference on Information and knowledge management. – ACM, 2010. – pp. 759–768.

26. Al Zamal F., Liu W., Ruths D. Homophily and Latent Attribute Inference: Inferring Latent Attributes of Twitter Users from Neighbors //ICWSM. – 2012.

27. Rao D. et al. Classifying latent user attributes in twitter //Proceedings of the 2nd international workshop on Search and mining user-generated contents. – ACM, 2010. – С. 37-44.

28. Burger J. D. et al. Discriminating gender on Twitter //Proceedings of the Conference on Empirical Methods in Natural Language Processing. – Association for Computational Linguistics, 2011. – С. 1301-1309.

29. Kótyuk G., Buttyán L. A machine learning based approach for predicting undisclosed attributes in social networks //Pervasive Computing and Communications Workshops (PERCOM Workshops), 2012 IEEE International Conference on. – IEEE, 2012. – С. 361-366.

30. Caruana, G. A survey of emerging approaches to spam filtering [Текст] / G. Caruana, M. Li // ACM Computing Surveys (CSUR), Vol. 44, No.2, February 2012. pp. 1-27.

31. Stafford, G. An Evaluation of the Effect of Spam on Twitter Trending Topics [Электронный ресурс] — Электрон. дан. - США, [2013] — Режим доступа: http://homepages.gac.edu/~lyu/Grant_paper.pdf, свободный. — Англ.

32. Martinez-Romo, J. Detecting malicious tweets in trending topics using a statistical analysis of language [Текст] / J. Martinez-Romo, L. Araujo // Expert Systems with Applications, Vol. 40, No.8, June 2013. pp. 2992-3000.

33. Almeida, T. A. Advances in spam filtering techniques. In Computational Intelligence for Privacy and Security [Текст] / T. A. Almeida, A.Yamakami // Computational Intelligence for Privacy and Security, Vol. 394, 2012. pp. 199-214.

34. Wang, A. H. Machine Learning for the Detection of Spam in Twitter Networks [Текст] / A. H. Wang // e-Business and Telecommunications, Vol. 222, 2012. pp. 319-333.

35. Ahmed, F. Generic Statistical Approach for Spam Detection [Текст] / F.Ahmed, M. A. Abulaish // Computer Communications, Vol. 36, June 2013. pp. 1120-1129.

36. Thomas, K. Suspended Accounts in Retrospect: An Analysis of Twitter Spam [Текст] / K. Thomas, C. Grier, V. Paxson, D. Song // Proceedings of the Internet Measurement Conference 2011 (IMC 2011) , Berlin, Germany, November 2-4. 2011. pp. 243-258.

37. Sridharan, V. Twitter games: how successful spammers pick targets [Текст] / V. Sridharan, V. Shankar, M. Gupta // Proceedings of the 28th Annual Computer Security Applications Conference, Orlando, Florida, USA, December 3-7. 2012. pp. 389-398.

38. Левенштейн, В.И. Двоичные коды с исправлением выпадений, вставок и замещений символов [Текст] / В.И. Левенштейн // Доклады Академий Наук СССР, 1965, Т. 163, №4. C. 845-848.

39. Lin P.C. A study of effective features for detecting long-surviving Twitter spam accounts [Текст] / P.C. Lin, P.M. Huang // The 15th International Conference on Advanced Communications Technology, Phoenix Park, PyeongChang, South Korea, January 27-30. 2013. pp. 841 - 846

40. Романов А.С., Мещеряков Р.В. Определение пола автора короткого электронного сообщения // Компьютерная лингвистика и интеллектуальные технологии: По материалам ежегодной Междунар. конф. «Диалог» (Бекасово, 25–29 мая 2011 г.). М. : Изд-во РГГУ, 2011. Вып. 10 (17). С. 620–626