Saif M. Mohammad (he/him)

Senior Research Scientist,
National Research Council Canada

Email: saif.mohammad@nrc-cnrc.gc.ca


Picture: Palau De La Música Catalana (Palace of Catalan Music). Architect: Antoni Gaudi.
A postcard from my first academic conference, ACL-2004.

Research Focus: Emotions and Language, Lexical Semantics, Computational Creativity, Natural Language Processing.

BIO

Dr. Saif M. Mohammad is a Senior Research Scientist at the National Research Council Canada (NRC). He received his Ph.D. in Computer Science from the University of Toronto. Before joining NRC, Saif was a Research Associate at the Institute of Advanced Computer Studies at the University of Maryland, College Park. His research interests are in Computational Linguistics and Natural Language Processing (NLP), especially Lexical Semantics, Emotions in Language, Sentiment Analysis, Computational Creativity, Fairness in NLP, Psycholinguistics, and Information Visualization. He has published ~100 scientific articles (journal articles, book chapters, and conference papers). He has served in various capacities at prominent journals and conferences, including: action editor for Computational Linguistics, chair of the Canada--UK symposium on Ethics in AI, co-chair of SemEval 2017-19 (the largest platform for semantic evaluations), workshops co-chair for ACL 2020, co-organizer of WASSA 2017 and 2018 (a sentiment analysis workshop), and area chair for ACL, NAACL, and EMNLP (in the areas of sentiment analysis, lexical semantics, and fairness in NLP). His team developed a sentiment analysis system which ranked first in shared task competitions. His word--emotion resources, such as the NRC Emotion Lexicon, are used for analyzing affect in text. His work has garnered media attention, including articles in Time, SlashDot, LiveScience, io9, The Physics arXiv Blog, PC World, and Popular Science.

You can watch a video of the talk at the Alan Turing Institute (London, March 2019) summarizing recent work.

Detailed CV         How my name is pronounced:        Photo


RECENT NEWS AND PUBLICATIONS

Blog Posts

Ethics Sheets for AI Tasks. July 5, 2021.    
Video

Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysis. July 5, 2021.

Paper

Ethics Sheets for AI Tasks. Saif M. Mohammad. arXiv preprint arXiv:2107.01183. July 2021.

Invited Talks

Ethics Sheets for AI Tasks and a Case Study for Automatic Emotion Recognition. The University of British Columbia Language Sciences Talks, Vancouver, Canada. July 15, 2021.
Slides      Video

Emotion Dynamics of Fictional Characters using Large Emotion Lexicons. The 17th IGEL Conference. June 23, 2021.
Slides     

Ethics Sheets for AI Tasks and a Case Study for Automatic Emotion Recognition. The Alan Turing Institute, London, UK. May, 2021.

Paper

Ruddit: Norms of Offensiveness for English Reddit Comments. Rishav Hada, Sohi Sudhir, Pushkar Mishra, Helen Yannakoudakis, Saif M. Mohammad, and Ekaterina Shutova. In Proceedings of the 59th Annual Meeting of the Association of Computational Linguistics (ACL-2021), August 2021. 
Paper (pdf)    BibTeX     Code and Data

Journal Papers

Emotion Dynamics in Movie Dialogues. Will E. Hipson and Saif M. Mohammad. arXiv preprint arXiv:2103.01345. March 2021. (To appear in PLOS One, 2021)
Paper (pdf)    BibTeX     Code

Examining the Language of Solitude vs. Loneliness in Tweets.  Will E. Hipson, Svetlana Kiritchenko, Robert J. Coplan, Saif M. Mohammad. Journal of Social and Personal Relationships. March 2021. 
Paper (pdf)    BibTeX

Book Chapter

Sentiment Analysis: Automatically Detecting Valence, Emotions, and Other Affectual States from Text. Saif M. Mohammad, arXiv:2005.11882, Jan 2021.
To Appear as a Book chapter in The 2nd Edition of Emotion Measurement, Elsevier, 2021.
PDF     BibTeX

Talk

What is a Research Ethics Statement and Why does it Matter? Saif M. Mohammad.
Slides

I presented some of these ideas at the  EMNLP 2020 Plenary Panel on Publishing in the Era of Responsible AI: How Can we be Proactive? Considerations and Implications. Panelists: Emily M. Bender, Rosie Campbell, Allan Dafoe, Pascale Fung, Meg Mitchell, Saif M. Mohammad.

Paper

Practical and Ethical Considerations in the Effective use of Emotion and Sentiment Lexicons.  Saif M. Mohammad. arXiv preprint arXiv:2011.03492. December 2020. 
Paper (pdf)    BibTeX

Recognition

Nominated for the outstanding acheivement in research excellence award by National Research Council Canada. November 2020. (1 of 6 nominees)

Invited Talk

Gender Gap in Natural Language Processing Research: Disparities in Authorship and Citations. Women+@DCS Seminar Series, University of Sheffield, October 28 2020, Sheffield, UK.
Slides      Video

Papers

Lexichrome: Text Construction and Lexical Discovery with Word-Color Associations Using Interactive Visualization. Chris K. Kim, Uta Hinrichs, and Saif M. Mohammad, and Chris Collins. In Proceedings of DIS 2020: Conference on Designing Interactive Systems, July 2020, Eindhoven, Netherlands.
Paper (pdf)    BibTeX    Video    Interactive Visualization

Examining Citations of Natural Language Processing Literature. Saif M. Mohammad. In Proceedings of the 58th Annual Meeting of the Association of Computational Linguistics (ACL-2020), July 2020, Seattle, USA.

  • Examines nine questions pertaining to broad trends in citations of NLP papers (across time, across venue types, across paper types, across areas, etc.).

Paper (pdf)    BibTeX       Presentation       Project Home Page    Interactive Visualizations      Medium Blog Posts

Gender Gap in Natural Language Processing Research: Disparities in Authorship and Citations. Saif M. Mohammad. In Proceedings of the 58th Annual Meeting of the Association of Computational Linguistics (ACL-2020). July 2020. Seattle, USA. 

  • Summary: Examines eight questions pertaining to disparities across male and female NLP researchers (in authorship and citations).

Paper (pdf)    BibTeX       Video       Presentation       Project Home Page     Medium Blog Posts

PoKi: A Large Dataset of Poems by Children. Will E. Hipson, and Saif M. Mohammad. In Proceedings of the 12th Language Resources and Evaluation Conference (LREC-2020), May 2020, Marseille, France.
Paper (pdf)    BibTeX     Project Home Page and Data

NLP Scholar: An Interactive Visual Explorer for Natural Language Processing Literature. Saif M. Mohammad. In Proceedings of the 58th Annual Meeting of the Association of Computational Linguistics (ACL-2020), July 2020, Seattle, USA.

  • Presents an interactive visualization tool to help users find (related) work published in the ACL Anthology.

Paper (pdf)    BibTeX       Presentation      Project Home Page    Interactive Visualizations      Medium Blog Posts

SOLO: A Corpus of Tweets for Examining the State of Being Alone. Svetlana Kiritchenko, Will Hipson, Robert Coplan, and Saif M. Mohammad. In Proceedings of the 12th Language Resources and Evaluation Conference (LREC-2020), May 2020, Marseille, France.
Paper (pdf)    BibTeX       Data

WordWars: A Dataset to Examine the Natural Selection of Words. Saif M. Mohammad. In Proceedings of the 12th Language Resources and Evaluation Conference (LREC-2020), May 2020, Marseille, France.
Paper (pdf)    BibTeX       Data      Project Home Page and Visualizations

NLP Scholar: A Dataset for Examining the State of NLP Research. Saif M. Mohammad. In Proceedings of the 12th Language Resources and Evaluation Conference (LREC-2020), May 2020, Marseille, France.

  • Presents the NLP Scholar Dataset -- a single unified source of information from both the ACL Anthology (AA) and Google Scholar for tens of thousands of NLP papers.Presents initial work on analyzing the volume of  research in NLP over the years, identifies some the most cited papers in AA, as well as outlines a list of potential applications of the dataset.

Paper (pdf)    BibTeX      Project Home Page and Data    Interactive Visualizations      Medium Blog Posts

The State of NLP Literature: A Diachronic Analysis of the ACL Anthology.  Saif M. Mohammad. arXiv preprint arXiv:1911.03562. November 2019. 

  • A manuscript that brings together the analyses of NLP papers first presented in the four State of NLP blog posts.

Paper (pdf)    BibTeX      Project Home Page and Data     Interactive Visualizations      Medium Blog Posts

Applied AI Ethics. Report on Canada-United Kingdom Symposia on Ethics in AI in Ottawa, Canada and London, UK. de Bruijn, B., Désillets, A., Fraser, K., Kiritchenko, S., Mohammad, S., Vinson, N., Bloomfield, P., Brace, H., Brzoska, K., Elhalal, A., Ho, K., Kinsey, L., McWhirter, R., Nazare, M., and Ofuri-Kuragu, E. Digital Catapult, London, UK / NRC, Ottawa, Canada, 2019.
Paper (pdf)     Symposium Website

Recognition

Awarded a "Top Writer for Science on Medium" status. November 2019. (One of 50 writers who had this tag at the time. There were about 63K stories in the Science category on Medium at the time.)

Invited Talks

The Search for Emotions, Creativity, and Fairness in Language. Data Systems Seminar Series, University of Waterloo, October 28 2019, Waterloo, Canada.
Slides    Video

Fairness and Emotions in Language. The Globe and Mail. October 29, 2019, Toronto, Canada.

Hosting the Responsible AI Summit, October 24 2019, Montreal, Canada.

Creativity and Emotions in Language. Invited talk and panel on AI and Creativity in Government at the Creative Marketplace Lab on Data, Skills and Technology, Department of Canadian Heritage, September 30 2019, Gatineau, Canada.

The Search for Emotions, Creativity, and Fairness in Language. Mila - Quebec AI Institute, August 2019, Montreal, Canada.
Slides 

The Search for Emotions, Creativity, and Fairness in Language. Keynote speech at the 8th KDD Workshop on Issues of Sentiment Discovery and Opinion Mining (WISDOM-2019), August 2019, Anchorage, Alaska.
Slides  

Web/Press Mentions

Gender and Racial Bias in Cloud NLP Sentiment APIs, Aug 21, 2019. Article looking into race and gender biases in the Google and AWS cloud sentiment analysis APIs using the Equity Evaluation Corpus and the techniques we published in 2018.

Journal Paper

AffectiveTweets: a Weka Package for Analyzing Affect in Tweets. Felipe Bravo-Marquez, Eibe Frank, Bernhard Pfahringer, Saif M. Mohammad. Journal of Machine Learning Research, 20(92):1−6, 2019.
Paper (pdf)    BibTeX      Code

Papers

Big Bird: A Large, Fine-Grained, Bigram Relatedness Dataset for Examining Semantic Composition. Shima Asaadi, Saif M. Mohammad, and Svetlana Kiritchenko. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL-2019), June 2019, Minnesota, USA.
Paper (pdf)    BibTeX       Poster       Data       Project Home Page and Visualizations       Code

How do we feel when a robot dies? Emotions expressed on Twitter before and after hitchBOT’s destruction. Kathleen C. Fraser, Frauke Zeller, David Harris Smith, Saif M. Mohammad, and Frank Rudicz. In Proceedings of the NAACL workshop on computational approaches to subjectivity, sentiment, and social media analysis (WASSA-19), June 2019, Minneapolis, USA.
Paper (pdf)    BibTeX       Slides       Visualizations

Invited Talks

Examining Fairness in Language Through Emotions. Keynote speech at the 2019 Canada--UK Workshop on Practical AI Ethics, March 12, 2019, London, UK.
Slides

The Search for Emotions, Creativity, and Fairness in Language. The Alan Turing Institute. March 11, 2019. London, UK.
Slides    Video

Examining Fairness in Language Through Emotions. The 2019 Canada--UK Symposium on Ethics in AI, Feb 21, 2019, Ottawa, Canada.
Slides    Video

Journal Paper

The Natural Selection of Words: Finding the Features of Fitness. Peter D. Turney and Saif M. Mohammad. PLoS One, 14 (1):e0211512. January 2019.
Paper (pdf)    BibTeX     Code

Professional Community Involvement

Chair of the 2019 Canada--UK Symposium on Ethics in AI, Feb 21--22, Ottawa, Canada.

Area chair for Sentiment Analysis and Argumentation Mining, EMNLP-IJCNLP-2019, Hong Kong.

Area Chair for ACL 2019 in the area of Sentiment Analysis and Argument Mining.

Area chair for Semantics in NLP Applications, *Sem-2019, Minneapolis, Minnesota, USA.

Area Chair for Ethics, Bias and Fairness at NAACL-2019, Minneapolis, USA.

NAACL-2019 has a special theme encouraging papers on ethics, bias, and airness.

Co-chair of SemEval 2019, Minneapolis, Minnesota, USA.

Mentor, Student Research Workshop, ACL-2019, Florence, Italy

Mentor, Student Research Workshop, NAACL-2019, Minneapolis, Minnesota, USA.

Invited Talks

The Search for Emotions, Creativity, and Fairness in Language. Venue: Keynote speech at the 6th evaluation campaign EVALITA 2018. December 12, 2018. Turin, Italy.

The Search for Emotions, Creativity, and Fairness in Language. Venue: Department of Computational Linguistics, Heidelberg University. December 13, 2018. Heidelberg, Germany.

The Search for Emotions, Creativity, and Fairness in Language. Venue: Invited Speaker Series. Pickering Centre for Human Development, Carleton University. November 16, 2018. Ottawa, Canada.

The Search for Emotions, Creativity, and Fairness in Language. Short talk to 9th graders on the occasion of Bring-Your-Ninth-Grader-to-Work Day at the National Research Council Canada. November 14, 2018. Ottawa, Canada

The Search for Emotions, Creativity, and Fairness in Language. At the Natural Language Processing Seminar and AI Seminar, University of Michigan, Ann Arbor. October 2, 2018. Ann Arbor, Michigan.

Journal Paper

Data and systems for medication-related text classification and concept normalization from Twitter: Insights from the Social Media Mining for Health (SMM4H)-2017 shared task. Abeed Sarker, Maksim Belusov, Jasper Friedrichs, Kai Hakala, Sifei Han, Svetlana Kiritchenko, Farrokh Mehryary, Anthony Rios, Tung Tran, Berry de Bruijn, Filip Ginter, Ramakanth Kavuluru, Debanjan Mahata, Saif M. Mohammad, Goran Nenadic, Graciela Gonzalez-Hernandez. Journal of the American Medical Informatics Association (JAMIA), 25(10):1274--1283, October 2018.

Official Rankings: Our team (NRC-Canada) ranked first in the AMIA Shared Task on detecting adverse drug reactions in tweets.

Invited Talks

The Search for Emotions, Creativity, and Fairness in Language. Department of Computer Science, University of Utah. August 6, 2018. Salt Lake City, Utah.

The Search for Emotions, Creativity, and Fairness in Language. Keynote speech at the ACL Workshop on Natural Language Processing for Social Media. July 20, 2018. Melbourne, Australia.

The Search for Emotions, Creativity, and Fairness in Language. Department of Computer Science, University of Melbourne, July 20, 2018. Melbourne, Australia.

The Search for Emotions, Creativity, and Fairness in Language. The University of Waikato, Computing & Mathematical Sciences. July 10, 2018. Hamilton, New Zealand.

Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. Invited talk at the Second ACL Workshop on Ethics in Natural Language Processing, New Orleans, LA, USA, June 2018.

The Search for Emotions in Language. Google Research. May 25, 2018. Montreal, Canada.

Professional Community Involvement

Co-organizer of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, to be held in conjunction with EMNLP-2018, Oct 31–Nov 1, Brussels, Belgium.

Papers

Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words. Saif M. Mohammad. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, July 2018.
Paper (pdf)    BibTeX    Project Page and Data     Presentation    Video    Poster

Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. Svetlana Kiritchenko and Saif M. Mohammad. In Proceedings of *Sem, New Orleans, LA, USA, June 2018.
Paper (pdf)    BibTeX    Project Page and Data     Presentation

Agree or Disagree: Predicting Judgments on Nuanced Assertions. Michael Wojatzki, Torsten Zesch, Saif M. Mohammad, and  Svetlana Kiritchenko. In Proceedings of *Sem, New Orleans, LA, USA, June 2018.
Paper (pdf)    BibTeX        Project Page and Data        Presentation

Semeval-2018 Task 1: Affect in tweets. Saif M. Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2018), New Orleans, LA, USA, June 2018.
Paper (pdf)    BibTeX       Data and Visualization     Presentation

SemEval-2018 Task 1: Affect in Tweets Webpage
75 teams and about 200 participants.

DeepMiner at SemEval-2018 Task 1: Emotion Intensity Recognition Using Deep Representation Learning. Habibeh Naderi, Svetlana Kiritchenko, Saif M. Mohammad, and Stan Matwin. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2018), New Orleans, LA, USA, June 2018.
Paper (pdf)    BibTeX  

WikiArt Emotions: An Annotated Dataset of Emotions Evoked by Art. Saif M. Mohammad and Svetlana Kiritchenko. In Proceedings of the 11th Edition of the Language Resources and Evaluation Conference (LREC-2018), May 2018, Miyazaki, Japan.
Paper (pdf)    BibTeX       Poster       Project Page and Data

Word Affect Intensities. Saif M. Mohammad. In Proceedings of the 11th Edition of the Language Resources and Evaluation Conference (LREC-2018), May 2018, Miyazaki, Japan.
Paper (pdf)    BibTeX       Presentation       Project Page and Data

Understanding Emotions: A Dataset of Tweets to Study Interactions between Affect Categories. Saif M. Mohammad and Svetlana Kiritchenko. In Proceedings of the 11th Edition of the Language Resources and Evaluation Conference (LREC-2018), May 2018, Miyazaki, Japan.
Paper (pdf)    BibTeX       Presentation       Shared Task Page and Data

Quantifying Qualitative Data for Understanding Controversial Issues. Michael Wojatzki, Saif M. Mohammad, Torsten Zesch, and Svetlana Kiritchenko. In Proceedings of the 11th Edition of the Language Resources and Evaluation Conference (LREC-2018), May 2018, Miyazaki, Japan.
Paper (pdf)    BibTeX        Presentation      Project Page and Data

Invited Talks

The Search for Emotions in Language. Featured speaker at the Distinguished Computational Linguistics Lecture at Rochester Institute of Technology. March 1, 2018. Rochester, NY, USA.

The Search for Emotions in Language. University of Toronto. Feb 27, 2018. Toronto, Canada.

Finding Emotions in Language. Information Systems Sprott School of Business, Carleton University. November 13, 2017. Ottawa, Canada.

Emotion-Aware Machines. Short talk to 9th graders on the occasion of Bring-Your-Ninth-Grader-to-Work Day at the National Research Council Canada. November 1, 2017. Ottawa, Canada.

Professional Community Involvement

Co-chair for SemEval-2018.

Organizer of SemEval-2018 Task 1: Affect in Tweets.
75 teams and about 200 participants.

Papers

NRC-Canada at SMM4H Shared Task: Classifying Tweets Mentioning Adverse Drug Reactions and Medication Intake. Svetlana Kiritchenko, Saif M. Mohammad, Jason Morin, and Berry de Bruijn (2017). In Proceedings of the Social Media Mining for Health Applications Workshop at AMIA-2017, Washington, DC, USA, 2017.
Paper (pdf)   BibTeX    Our System Homepage

Official Rankings: Our team (NRC-Canada) ranked first in the AMIA Shared Task on detecting adverse drug reactions in tweets.  

WASSA-2017 Shared Task on Emotion Intensity. Saif M. Mohammad and Felipe Bravo-Marquez. In Proceedings of the EMNLP 2017 Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media (WASSA), September 2017, Copenhagen, Denmark.
Paper (pdf)    BibTex     Data and Shared Task    Presentation

Emotion Intensities in Tweets. Saif M. Mohammad and Felipe Bravo-Marquez. In Proceedings of the Sixth Joint Conference on Lexical and Computational Semantics (*Sem), August 2017, Vancouver, Canada.
Paper (pdf)    BibTex     Data and Shared Task    AffcetiveTweets package    Presentation

Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation. Svetlana Kiritchenko and Saif M. Mohammad. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL-2017), Vancouver, Canada, 2017.
Paper (pdf)    BibTeX       Data      Poster      

Word Affect Intensities. Saif M. Mohammad. arXiv preprint arXiv:1704.08798, April 2017.
Paper (pdf)   

Invited Talks

Affect Associations in Creative Language. Seminar Series for the Doctorate in Cognitive Informatics at Université du Québec à Montréal (UQAM), April 13, 2017, Montreal, Canada.
Presentation

Affect Associations in Creative Language. Keynote speech at the COLING 2016 workshop on Computational Modeling of People's Opinions, Personality, and Emotions in Social Media (PEOPLES 2016), December 12, 2016, Osaka, Japan.
Presentation

Affect Associations: The Building Blocks of Sentiment Analysis. Keynote speech at the 20th International Conference on Asian Language Processing (IALP 2016), November 21-23, 2016, National Cheng Kung University, Tainan, Taiwan.
Presentation

Affect Associations in Creative Language. Keynote speech at the 2016 IR Workshop at the Academia Sinica, November 24, 2016, Taipei, Taiwan.
Presentation

Emotion-Aware Machines. Short talk to 9th graders on the occasion of Bring-Your-Ninth-Grader-to-Work Day at the National Research Council Canada. November 1, 2016. Ottawa, Canada.

Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best–Worst Scaling. Invited talk at the TAMALE seminar series, September 15, 2016, University of Ottawa. (Joint work with, and talk given by, Svetlana Kiritchenko.)
Presentation

Metaphor as a Medium for Emotion: An Empirical Study. Invited talk at the TAMALE seminar series, September 15, 2016, University of Ottawa. (Joint work with Ekaterina Shutova and Peter Turney.)
Presentation

Professional Community Involvement

Co-chair for SemEval-2017.

Co-organizer of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA-2017), to be held in conjunction with EMNLP-2017, Copenhagen, Denmark.

Organizer of shared task on detecting emotion intensity at WASSA-2017.

Area chair for Sentiment Analysis and Opinion Mining at ACL-2017, Vancouver, Canada.

Area chair for Sentiment Analysis and Opinion Mining at EMNLP-2017, Copenhagen, Denmark.

Event

A symphony orchestra performed music composed using the NRC Emotion Lexicon under the glass of the Louvre museum in Paris on Sept. 20, 2016. Click here for a video of the performance.
Articles published in the Washington Post, CBS News, Columbia Tribune, and others.

Journal Paper

Stance and Sentiment in Tweets. Saif M. Mohammad, Parinaz Sobhani, and Svetlana Kiritchenko. Special Section of the ACM Transactions on Internet Technology on Argumentation in Social Media, 2017, 17(3).
Paper (pdf)    BibTeX       Data and Visualization

Papers

Metaphor as a Medium for Emotion: An Empirical Study. Saif M. Mohammad, Ekaterina Shutova, and Peter Turney. In Proceedings of the Joint Conference on Lexical and Computational Semantics (*Sem), August 2016, Berlin, Germany.
Paper (pdf)    BibTeX     Presentation       Data and Interactive Visualization

Detecting Stance in Tweets And Analyzing its Interaction with Sentiment. Parinaz Sobhani, Saif M. Mohammad, and Svetlana Kiritchenko. In Proceedings of the Joint Conference on Lexical and Computational Semantics (*Sem), August 2016, Berlin, Germany.
Paper (pdf)    BibTeX    Presentation       Data and Visualization

Determining Word-Emotion Associations from Tweets by Multi-Label Classification. Felipe Bravo-Marquez, Eibe Frank, Saif Mohammad, and Bernhard Pfahringer. In Proceedings of the 2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI'16), October 2016, Omaha, Nebraska, USA.
Paper (pdf)    BibTeX    Data (scroll to section on this paper)

Invited Talk

Sentiment and Emotions in Tweets. Department of Geography & Geoinformation Science at George Mason University. May 29, 2016, FairFax, Virginia.
Presentation

Professional Community Involvement

Publicity Chair for the 2016 meeting of the 2016 Conference on Empirical Methods on Natural Language Processing. (EMNLP-2016).

Area Chair for Sentiment and Opinion Mining at EMNLP 2016.

Book Chapter

Challenges in Sentiment Analysis. Saif M. Mohammad, A Practical Guide to Sentiment Analysis, Springer, 2016.
Pre-print version (pdf)    BibTeX

Papers

Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best-Worst Scaling. Svetlana Kiritchenko and Saif M. Mohammad. In Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2016, San Diego, California.
Paper (pdf)    BibTeX    Presentation     Data   

Sentiment Composition of Words with Opposing Polarities. Svetlana Kiritchenko and Saif M. Mohammad. In Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 2016, San Diego, California.
Paper (pdf)    BibTeX    Poster     Data: Opposing Polarity Sentiment Lexicon    Interactive Visualization

The Effect of Negators, Modals, and Degree Adverbs on Sentiment Composition. Svetlana Kiritchenko and Saif M. Mohammad, In Proceedings of the NAACL 2016 Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media (WASSA), June 2016, San Diego, California.
Paper (pdf)    BibTeX    Presentation     Data (Sentiment Composition Lexicon of Negators, Modals, and Adverbs (SCL-NMA)) and Visualization

Semeval-2016 Task 6: Detecting Stance in Tweets. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. In Proceedings of the International Workshop on Semantic Evaluation (SemEval ’16). June 2016. San Diego, California.
Paper (pdf)    BibTeX    Presentation    Task Website

Semeval-2016 Task 7: Determining Sentiment Intensity of English and Arabic Phrases. Svetlana Kiritchenko, Saif M. Mohammad, and Mohammad Salameh. In Proceedings of the International Workshop on Semantic Evaluation (SemEval ’16). June 2016. San Diego, California.
Paper (pdf)    BibTeX    Presentation    Task Website

A Practical Guide to Sentiment Annotation: Challenges and Solutions. Saif M. Mohammad, In Proceedings of the NAACL 2016 Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media (WASSA), June 2016, San Diego, California.
Paper (pdf)    BibTeX    Presentation   

Sentiment Lexicons for Arabic Social Media. Saif M. Mohammad, Mohammad Salameh, and Svetlana Kiritchenko. In Proceedings of the 10th edition of the Language Resources and Evaluation Conference, May 2016, Portorož (Slovenia).
Paper (pdf)    BibTeX    Presentation    Video        Data: Arabic Sentiment Lexicons

A Dataset for Detecting Stance in Tweets. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. In Proceedings of the 10th edition of the Language Resources and Evaluation Conference, May 2016, Portorož (Slovenia).
Paper (pdf)    BibTeX    Presentation     Data: Stance Dataset    Interactive Visualization

Happy Accident: A Sentiment Composition Lexicon for Opposing Polarities Phrases. Svetlana Kiritchenko and Saif M. Mohammad. In Proceedings of the 10th edition of the Language Resources and Evaluation Conference, May 2016, Portorož (Slovenia).
Paper (pdf)    BibTeX    Poster    Data: Opposing Polarity Sentiment Lexicon    Interactive Visualization

Book Chapter

Sentiment Analysis: Detecting Valence, Emotions, and Other Affectual States from Text. Saif M. Mohammad, Emotion Measurement, 2016.
Pre-print version (pdf)    BibTeX
This is a survey on automatic methods for affect analysis.

Interactive Visualization and Paper

Imagisaurus: An Interactive Visualizer of Valence and Emotion in the Roget’s Thesaurus. Saif M. Mohammad. In Proceedings of the EMNLP 2015 Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media (WASSA), September 2015, Lisbon, Portugal.
Paper (pdf)    BibTeX    Interactive Visualization

Data

The NRC Emotion Lexicon is now available in over 20 languages.

Journal Paper

How Translation Alters Sentiment. Saif M. Mohammad, Mohammad Salameh, and Svetlana Kiritchenko, Journal of Artificial Intelligence Research, January 2016, 55:95-130.
Paper (pdf)    BibTeX    Data: Arabic Sentiment Lexicons

Professional Community Involvement

Organizing these shared task competitions under the aegis of SemEval-2016 (see webpage for schedule):

Journal Paper

Developing a Successful SemEval Task in Sentiment Analysis of Twitter and Other Social Media Texts.Preslav Nakov, Sara Rosenthal, Svetlana Kiritchenko, Saif M. Mohammad, Zornitsa Kozareva, Alan Ritter, Veselin Stoyanov, and Xiaodan Zhu. Language Resources and Evaluation. January 2016, 50(1):35-65.
Paper (pdf)    Preprint Version (pdf)    BibTeX

Tutorial

Computational Analysis of Affect and Emotion in Language. Saif M. Mohammad and Cecilia Ovesdotter Alm. Tutorial at the 2015 Conference on Empirical Methods on Natural Language Processing, September 2015, Lisboa, Portugal.
Presentation       Annotated Bibliography       Extended Bibliography      Proposal

Papers

SemEval-2015 Task 10: Sentiment Analysis in Twitter. Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif M Mohammad, Alan Ritter, and Veselin Stoyanov. In Proceedings of the ninth international workshop on Semantic Evaluation Exercises (SemEval-2015), June 2015, Denver, Colorado.
Paper (pdf)    BibTeX

Sentiment After Translation: A Case-Study on Arabic Social Media Posts. Mohammad Salameh, Saif M. Mohammad, and Svetlana Kiritchenko, In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL-2015), June 2015, Denver, Colorado.
Paper (pdf)    BibTeX   Data: rabic Sentiment Lexicons

Visualization

Explore the interactive visualization for the NRC Word-Emotion Association Lexicon.

Symposium

My N is Ten Million: Using Social Media to Track Emotion, Mental Health, and Measure Personality Across Entire Populations. Gregory J Park, Saif M Mohammad, and Johannes C Eichstaedt. A symposium at the International Convention of Psychological Science (ICPS), March 2015, Amsterdam, The Netherlands.

Tutorial

Sentiment Analysis of Social Media Texts. Saif M. Mohammad and Xiaodan Zhu. Tutorial at the 2014 Conference on Empirical Methods on Natural Language Processing, October 2014, Doha, Qatar.
Presentation   Video    Proposal

Professional Community Involvement

I am serving as Publicity Chair for the 2015 meeting of the North American Association for Computational Linguistics (NAACL-2015). I am also the Area Chair for Sentiment and Opinion Mining.

Invited Talk

The Words are Alive: Associations with Sentiment, Emotions, Colours, and Music. Invited talk at Language Technology Institute, Carnegie Mellon University, September 2014, Pittsburgh, PA.
Presentation    Video

Journal Paper

Sentiment, Emotion, Purpose, and Style in Electoral Tweets. Saif M. Mohammad, Svetlana Kiritchenko, Xiaodan Zhu, and Joel Martin. Information Processing and Management, Volume 51, Issue 4, July 2015, Pages 480–499.
Paper (pdf)    BibTeX     AnnotatedData

Invited Talk

Words: Evaluative, Emotional, Colourful, Musical! Keynote speech at the ACL 2014 Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media (WASSA), June 27, 2014, Baltimore, MD.
Presentation

Journal Papers

Experiments with Three Approaches to Recognizing Lexical Entailment. Peter D. Turney, Saif M. Mohammad, Natural Language Engineering, 21(3):437-476, May 2015.
Paper (pdf)    BibTeX

Using Hashtags to Capture Fine Emotion Categories from Tweets. Saif M. Mohammad, Svetlana Kiritchenko, Computational Intelligence, 31( 2):301-326, May 2015.
Paper (pdf)    BibTeX

Professional Community Involvement

One of the organizers of SemEval-2015 Task 10 - Sentiment Analysis in Twitter. Of special interest to me is subtask E - Determining strength of association of Twitter terms with positive sentiment (or, degree of prior polarity). Task description, trial data, test data, and other details available here.

Journal Paper

Sentiment Analysis of Short Informal Texts. Svetlana Kiritchenko, Xiaodan Zhu and Saif Mohammad. Journal of Artificial Intelligence Research, volume 50, pages 723-762, August 2014.
Paper (pdf)    BibTeX

Papers

NRC-Canada-2014: Detecting Aspects and Sentiment in Customer Reviews, Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif M. Mohammad. In Proceedings of the eighth international workshop on Semantic Evaluation Exercises (SemEval-2014), August 2014, Dublin, Ireland.
Paper (pdf)    BibTeX     Poster

Official Rankings: Our team (NRC-Canada) ranked first in three of the six subtasks. About 30 teams participated.

NRC-Canada-2014: Recent Improvements in the Sentiment Analysis of Tweets, Xiaodan Zhu, Svetlana Kiritchenko, and Saif M. Mohammad. In Proceedings of the eighth international workshop on Semantic Evaluation Exercises (SemEval-2014), August 2014, Dublin, Ireland.
Paper (pdf)    BibTeX     Poster

Official Rankings: Our team (NRC-Canada) ranked first in five of the ten subtask-domain combinations. About 40 teams participated.

An Empirical Study on the Effect of Negation Words on Sentiment. Xiaodan Zhu, Hongyu Guo, Saif Mohammad and Svetlana Kiritchenko. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, June 2014, Baltimore, MD.
Paper (pdf)    BibTeX

Semantic Role Labeling of Emotions in Tweets. Saif M. Mohammad, Xiaodan Zhu, and Joel Martin, In Proceedings of the ACL 2014 Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media (WASSA), June 2014, Baltimore, MD.
Paper (pdf)    BibTeX     AnnotatedData

Generating Music from Literature. Hannah Davis and Saif M. Mohammad, In Proceedings of the EACL Workshop on Computational Linguistics for Literature, April 2014, Gothenburg, Sweden.
Paper (pdf)    BibTeX    TransProse Website

Notable Press Mentions: The Physics arXiv Blog, March 20, 2014, TIME, May 7, 2014, PC World, May 15, 2014, Popular Science, May 14, 2014, io9, May 12, 2014, LiveScience, May 11, 2014.

 

 

 

 


QUICK LINKS

Presence in Research Networks:

Lexicons:

Other:

Pinned Blog Posts

  • March 2020, Ten Years of the NRC Word-Emotion Association Lexicon
    • Ten years back, with some anticipation and lots of excitement, Peter Turney and I introduced the NRC Word-Emotion Association Lexicon. So grateful that, over the years, so many people have put their hopes and trust in it. Such joy to see them boldly shine a light on the human condition — the positives and the negatives; not shying away even from sadness, fear, and anger. This blog bost puts a spotlight on ten favorites (includes fun video and audio clips as well as links to papers and popular press articles):

  • October 2019, The State of NLP Literature
    • This series of posts presents a diachronic analysis of the ACL Anthology — Or, as I like to think of it, making sense of NLP Literature through pictures.
 

 

PROFESSIONAL ACTIVITIES

2012--ongoing

  • Standing reviewer for Transactions of ACL (TACL) journal

2022

  • Senior Area Chair for Ethics and NLP, ACL-2022.

2021

2020

  • Action Editor, Computational Linguistics

  • Senior Area Chair for Sentiment Analysis, Stylistic Analysis, and Argument Mining, EMNLP-2020, Punta Cana, Dominican Republic

  • Area Chair for Sentiment Analysis, Stylistic Analysis, and Argument Mining, StarSem-2020, Barcelona, Spain

  • Reviewer for Nature

  • Workshops Co-chair ACL-2020, Seattle, USA

  • Area chair for Lexical Semantics, ACL-2020, Seattle, USA

  • Member of the COLING 2020 Ethics Advisory Group

2019

  • Area chair for Sentiment Analysis and Argumentation Mining, EMNLP-IJCNLP-2019, Hong Kong, China

  • Area chair for Sentiment Analysis and Argumentation Mining, ACL-2019, Florence, Italy

  • Area chair for Ethics, Bias and Fairness, NAACL-2019, Minneapolis, Minnesota, USA

  • Area chair for Semantics in NLP Applications, *Sem-2019, Minneapolis, Minnesota, USA
  • General Chair of the 2019 Canada–UK Symposium on Ethics in AI, Ottawa, Canada

  • Co-chair of SemEval 2019, Minneapolis, Minnesota, USA

  • Mentor, Student Research Workshop, ACL-2019, Florence, Italy

  • Mentor, Student Research Workshop, NAACL-2019, Minneapolis, Minnesota, USA

2018

  • Co-chair of SemEval-2018, New Orleans, Louisiana, USA

  • Co-organizer of WASSA-2018, Brussels, Belgium

  • Mentor, Student Research Workshop, NAACL-2018, New Orleans, Louisiana, USA

  • Mentor, Student Research Workshop, ACL-2018, Melbourne, Australia.

2017

  • Co-chair of SemEval-2017, Vancouver, Canada

  • Co-organizer of WASSA-2017, Copenhagen, Denmark

  • Area chair for Sentiment and Opinion Mining, ACL-2017, Vancouver, Canada

  • Area chair for Sentiment and Opinion Mining, EMNLP-2017, Copenhagen, Denmark

  • Mentor, Student Research Workshop, ACL-2017, Vancouver, Canada.

2016

  • Area chair for Sentiment and Opinion Mining, EMNLP-2016, Austin, Texas, USA

  • Publicity chair, EMNLP-2016, Austin, Texas, USA

2015

  • Area chair for Sentiment and Opinion Mining, NAACL-2015, Denver, Colorado

  • Publicity chair, NAACL-2015, Denver, Colorado

  • Mentor, Student Research Workshop, NAACL-2015, Denver, Colorado

2014, 2012, 2011

  • Mentor, Student Research Workshops, ACL-2014, NAACL-2012, ACL-2011

Most years 2008–present

  • Program committee member for top NLP journals and conferences, including TACL, ACL, NAACL, EMNLP, CL, CI, NLE, JAIR, TSLP, IJCNLP, ICWSM, and IJCAI.

 

 

NOTABLE WEB/PRESS MENTIONS

  • Universo Online, Dec 5, 2019. Gender bias in sentiment analysis systems and gender disparities in language.
  • Popular Science, May 14, 2014: Robot Reads Novels, Writes Songs about Them.

  • io9, May 12, 2014: Researchers Train Computers to Manipulate Human Emotions with Art.

  • LiveScience, May 11, 2014: 'TransProse' Software Creates Musical Soundtracks from Books.

  • TIME, May 7, 2014: This Is What Classic Novels Sound Like When a Computer Turns Them Into Piano Music.

  • SlashDot, March23, 2014: Algorithm Composes Music By Text Analyzing the World's Best Novels.

  • The Physics arXiv Blog, March 20, 2014: The Music Composed By An Algorithm Analysing The World’s Best Novels.

  • Glass Hammer, December 3, 2013: Are Your Emails Communicating a Lack of Confidence?

  • Singularity Hub, November 10, 2013: Algorithm Tracks Literary Emotion in Shakespeare, the Brothers Grimm.

  • The Physics ArXiv, October 4, 2013: Data Mining Reveals the Emotional Differences in Emails Written by Men and Women.

  • SlashDot, October 4, 2013: Data Mining Reveals the Emotional Differences In Emails From Men and Women.

  • The Physics ArXiv, October 1, 2013: Text Analyser Reveals Emotional Temperature of Novels and Fairy Tales.
    • Also in EduBits

  • SlashDot, October 1, 2013: Text Analyzer Reveals Emotional 'Temperature' of Novels and Fairy Tales

  • The New Scientist, September 27: What your email style says about your personality
    • Also in Times of India, MSN, Pharmacon, Galileo, Amic, and others

  • Article in MIT Technology Review, September 5, 2013: How Mechanical Turkers Crowdsourced a Huge Lexicon of Links Between Words and Emotion.
  • TIME, August 14, 2013: Main Tweet: Researchers Dig Into The Intersection of Politics and Twitter.

 
 


Organization of International Shared Task Competitions

  • Co-chair of SemEval 2019, Minneapolis, Minnesota, USA

  • Co-chair of SemEval-2018, New Orleans, Louisiana, USA
    (12 tasks, 196 papers, ~1000 participants)

  • Organizer of SemEval-2018 Task 1: Affect in Tweets (75 teams and about 200 participants)

  • Co-chair of SemEval-2017, Vancouver, Canada
    (12 tasks, 181 papers, ~900 participants)

  • Organizer of shared task on detecting emotion intensity at WASSA-2017

  • Co-organizer of Semeval-2016 Task 6: Detecting Stance in Tweets

  • Co-organizer of Semeval-2016 Task 7: Determining Sentiment Intensity of English and Arabic Phrases

  • Co-organizer of SemEval-2015 Task 10: Sentiment Analysis in Twitter

  • Co-organizer of SemEval-2012 Task 2: Measuring Degrees of Relational Similarity
 
..