“One-Size-Fits-All”? Examining Expectations around What Constitute “Fair” or “Good” NLG System Behaviors
Li Lucy, Su Lin Blodgett, Milad Shokouhi, Hanna Wallach, Alexandra Olteanu
NAACL 2024 | June 2024
Alexandra Olteanu
Principal Researcher
Publications
Publications by Year
-
-
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices
Yu Lu Liu, Meng Cao, Su Lin Blodgett, Jackie Chi Kit Cheung, Alexandra Olteanu, Adam Trischler
Findings of EMNLP 2023 | December 2023
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann (HE/HIM), Hanna Wallach
ACL 2023 | July 2023
AHA! Facilitating AI Impact Assessment by Generating Examples of Harms
Zana Buçinca, Chau Minh Pham, Maurice Jakesch, Marco Tulio Ribeiro, Alexandra Olteanu, Saleema Amershi
June 2023
Can Workers Meaningfully Consent To Workplace Wellbeing Technologies?
Shreya Chowdhary, Anna Kawakami, Mary L. Gray, Jina Suh, Alexandra Olteanu, Koustuv Saha
2023 ACM Conference on Fairness, Accountability, and Transparency | June 2023
The Kitmus Test: Evaluating Knowledge Integration From Multiple Sources In Natural Language Understanding Systems
Akshatha Arodi, Martin Pömsl, Kaheer Suleman, Adam Trischler, Alexandra Olteanu, Jackie Chi Kit Cheung
ACL 2023 | May 2023
Sensing Wellbeing in the Workplace, Why and For Whom? Envisioning Impacts with Organizational Stakeholders
Anna Kawakami, Shreya Chowdhary, Shamsi T. Iqbal, Q. Liao, Alexandra Olteanu, J. Suh, Koustuv Saha, Alexandra Olteanu, Jina Suh
Proceedings of the ACM on Human-Computer Interaction | March 2023, Vol 7: pp. 1-33
-
How Different Groups Prioritize Ethical Values for Responsible AI
Maurice Jakesch, Zana Buçinca, Saleema Amershi, Alexandra Olteanu
FAccT 2022 | May 2022
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications
Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, Alexandra Olteanu
NAACL 2022 | May 2022
Responsible Language Technologies: Foreseeing and Mitigating Harms
Su Lin Blodgett, Q. Vera Liao, Alexandra Olteanu, Rada Mihalcea, Michael Muller, Morgan Klaus Scheuerman, Chenhao Tan, Qian Yang
CHI 2022 | April 2022
-
ADEPT: An Adjective-Dependent Plausibility Task
Ali Emami, Ian Porada, Alexandra Olteanu, Kaheer Suleman, Adam Trischler, Jackie Chi Kit Cheung
ACL-IJCNLP 2021 | August 2021
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach
ACL-IJCNLP 2021 | August 2021
Responsible computing during COVID-19 and beyond
Solon Barocas, Asia J. Biega, Margarita Boyarskaya, Kate Crawford, Hal Daumé III, Miro Dudík, Benjamin Fish, Mary L. Gray, Brent Hecht, Alexandra Olteanu, Forough Poursabzi-Sangdeh, Luke Stark, Jennifer Wortman Vaughan, Hanna Wallach, Marion Zepf
Communications of the ACM | July 2021, Vol 67(7): pp. 30-32
https://doi.org/10.1145/3466612
Measuring algorithmically infused societies
Claudia Wagner, Markus Strohmaier, Alexandra Olteanu, Emre Kiciman, Noshir Contractor, Tina Eliassi-Rad
Nature | June 2021, Vol 595: pp. 197-204
“I Can’t Reply with That”: Characterizing Problematic Email Reply Suggestions
Ronald Robertson, Alexandra Olteanu, Fernando Diaz, Milad Shokouhi, Peter Bailey
CHI Conference on Human Factors in Computing Systems (CHI ’21) | May 2021
-
Overcoming Failures of Imagination in AI Infused System Development and Deployment
Margarita Boyarskaya, Alexandra Olteanu, Kate Crawford
In the Navigating the Broader Impacts of AI Research Workshop at NeurIPS 2020 | December 2020
When Are Search Completion Suggestions Problematic?
Alexandra Olteanu, Fernando Diaz, Gabriella Kazai
Computer Supported Collaborative Work and Social Computing (CSCW) | August 2020
CSCW 2020 Honorable Mention
CSCW 2020 Honorable Mention
On the Social and Technical Challenges of Web Search Autosuggestion Moderation
T. J. Hazen, Alexandra Olteanu, Gabriella Kazai, Fernando Diaz, Michael Golebiewski
July 2020
External Information Sharing on Health Forums: An Exploration
Dana Nguyen, Alexandra Olteanu, Emre Kiciman
Intl. Conf. on Web and Social Media (ICWSM) | June 2020
-
FACTS-IR: Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval
Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, Michael D. Ekstrand
ACM SIGIR Forum | December 2019, Vol 53(2): pp. 20
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Alexandra Olteanu, Carlos Castillo, Fernando Diaz, Emre Kiciman
Frontiers in Big Data | July 2019, Vol 2
Publications by Research Area
-
“One-Size-Fits-All”? Examining Expectations around What Constitute “Fair” or “Good” NLG System Behaviors
Li Lucy, Su Lin Blodgett, Milad Shokouhi, Hanna Wallach, Alexandra Olteanu
NAACL 2024 | June 2024
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices
Yu Lu Liu, Meng Cao, Su Lin Blodgett, Jackie Chi Kit Cheung, Alexandra Olteanu, Adam Trischler
Findings of EMNLP 2023 | December 2023
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications
Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, Alexandra Olteanu
NAACL 2022 | May 2022
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach
ACL-IJCNLP 2021 | August 2021
“I Can’t Reply with That”: Characterizing Problematic Email Reply Suggestions
Ronald Robertson, Alexandra Olteanu, Fernando Diaz, Milad Shokouhi, Peter Bailey
CHI Conference on Human Factors in Computing Systems (CHI ’21) | May 2021
When Are Search Completion Suggestions Problematic?
Alexandra Olteanu, Fernando Diaz, Gabriella Kazai
Computer Supported Collaborative Work and Social Computing (CSCW) | August 2020
CSCW 2020 Honorable Mention
CSCW 2020 Honorable Mention
On the Social and Technical Challenges of Web Search Autosuggestion Moderation
T. J. Hazen, Alexandra Olteanu, Gabriella Kazai, Fernando Diaz, Michael Golebiewski
July 2020
-
AHA! Facilitating AI Impact Assessment by Generating Examples of Harms
Zana Buçinca, Chau Minh Pham, Maurice Jakesch, Marco Tulio Ribeiro, Alexandra Olteanu, Saleema Amershi
June 2023
Can Workers Meaningfully Consent To Workplace Wellbeing Technologies?
Shreya Chowdhary, Anna Kawakami, Mary L. Gray, Jina Suh, Alexandra Olteanu, Koustuv Saha
2023 ACM Conference on Fairness, Accountability, and Transparency | June 2023
The Kitmus Test: Evaluating Knowledge Integration From Multiple Sources In Natural Language Understanding Systems
Akshatha Arodi, Martin Pömsl, Kaheer Suleman, Adam Trischler, Alexandra Olteanu, Jackie Chi Kit Cheung
ACL 2023 | May 2023
Sensing Wellbeing in the Workplace, Why and For Whom? Envisioning Impacts with Organizational Stakeholders
Anna Kawakami, Shreya Chowdhary, Shamsi T. Iqbal, Q. Liao, Alexandra Olteanu, J. Suh, Koustuv Saha, Alexandra Olteanu, Jina Suh
Proceedings of the ACM on Human-Computer Interaction | March 2023, Vol 7: pp. 1-33
How Different Groups Prioritize Ethical Values for Responsible AI
Maurice Jakesch, Zana Buçinca, Saleema Amershi, Alexandra Olteanu
FAccT 2022 | May 2022
Responsible Language Technologies: Foreseeing and Mitigating Harms
Su Lin Blodgett, Q. Vera Liao, Alexandra Olteanu, Rada Mihalcea, Michael Muller, Morgan Klaus Scheuerman, Chenhao Tan, Qian Yang
CHI 2022 | April 2022
Responsible computing during COVID-19 and beyond
Solon Barocas, Asia J. Biega, Margarita Boyarskaya, Kate Crawford, Hal Daumé III, Miro Dudík, Benjamin Fish, Mary L. Gray, Brent Hecht, Alexandra Olteanu, Forough Poursabzi-Sangdeh, Luke Stark, Jennifer Wortman Vaughan, Hanna Wallach, Marion Zepf
Communications of the ACM | July 2021, Vol 67(7): pp. 30-32
https://doi.org/10.1145/3466612
Measuring algorithmically infused societies
Claudia Wagner, Markus Strohmaier, Alexandra Olteanu, Emre Kiciman, Noshir Contractor, Tina Eliassi-Rad
Nature | June 2021, Vol 595: pp. 197-204
“I Can’t Reply with That”: Characterizing Problematic Email Reply Suggestions
Ronald Robertson, Alexandra Olteanu, Fernando Diaz, Milad Shokouhi, Peter Bailey
CHI Conference on Human Factors in Computing Systems (CHI ’21) | May 2021
When Are Search Completion Suggestions Problematic?
Alexandra Olteanu, Fernando Diaz, Gabriella Kazai
Computer Supported Collaborative Work and Social Computing (CSCW) | August 2020
CSCW 2020 Honorable Mention
CSCW 2020 Honorable Mention
-
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann (HE/HIM), Hanna Wallach
ACL 2023 | July 2023
AHA! Facilitating AI Impact Assessment by Generating Examples of Harms
Zana Buçinca, Chau Minh Pham, Maurice Jakesch, Marco Tulio Ribeiro, Alexandra Olteanu, Saleema Amershi
June 2023
Can Workers Meaningfully Consent To Workplace Wellbeing Technologies?
Shreya Chowdhary, Anna Kawakami, Mary L. Gray, Jina Suh, Alexandra Olteanu, Koustuv Saha
2023 ACM Conference on Fairness, Accountability, and Transparency | June 2023
The Kitmus Test: Evaluating Knowledge Integration From Multiple Sources In Natural Language Understanding Systems
Akshatha Arodi, Martin Pömsl, Kaheer Suleman, Adam Trischler, Alexandra Olteanu, Jackie Chi Kit Cheung
ACL 2023 | May 2023
How Different Groups Prioritize Ethical Values for Responsible AI
Maurice Jakesch, Zana Buçinca, Saleema Amershi, Alexandra Olteanu
FAccT 2022 | May 2022
ADEPT: An Adjective-Dependent Plausibility Task
Ali Emami, Ian Porada, Alexandra Olteanu, Kaheer Suleman, Adam Trischler, Jackie Chi Kit Cheung
ACL-IJCNLP 2021 | August 2021
Responsible computing during COVID-19 and beyond
Solon Barocas, Asia J. Biega, Margarita Boyarskaya, Kate Crawford, Hal Daumé III, Miro Dudík, Benjamin Fish, Mary L. Gray, Brent Hecht, Alexandra Olteanu, Forough Poursabzi-Sangdeh, Luke Stark, Jennifer Wortman Vaughan, Hanna Wallach, Marion Zepf
Communications of the ACM | July 2021, Vol 67(7): pp. 30-32
https://doi.org/10.1145/3466612
Overcoming Failures of Imagination in AI Infused System Development and Deployment
Margarita Boyarskaya, Alexandra Olteanu, Kate Crawford
In the Navigating the Broader Impacts of AI Research Workshop at NeurIPS 2020 | December 2020
External Information Sharing on Health Forums: An Exploration
Dana Nguyen, Alexandra Olteanu, Emre Kiciman
Intl. Conf. on Web and Social Media (ICWSM) | June 2020
FACTS-IR: Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval
Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, Michael D. Ekstrand
ACM SIGIR Forum | December 2019, Vol 53(2): pp. 20
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Alexandra Olteanu, Carlos Castillo, Fernando Diaz, Emre Kiciman
Frontiers in Big Data | July 2019, Vol 2
-
Can Workers Meaningfully Consent To Workplace Wellbeing Technologies?
Shreya Chowdhary, Anna Kawakami, Mary L. Gray, Jina Suh, Alexandra Olteanu, Koustuv Saha
2023 ACM Conference on Fairness, Accountability, and Transparency | June 2023
Responsible computing during COVID-19 and beyond
Solon Barocas, Asia J. Biega, Margarita Boyarskaya, Kate Crawford, Hal Daumé III, Miro Dudík, Benjamin Fish, Mary L. Gray, Brent Hecht, Alexandra Olteanu, Forough Poursabzi-Sangdeh, Luke Stark, Jennifer Wortman Vaughan, Hanna Wallach, Marion Zepf
Communications of the ACM | July 2021, Vol 67(7): pp. 30-32
https://doi.org/10.1145/3466612
Measuring algorithmically infused societies
Claudia Wagner, Markus Strohmaier, Alexandra Olteanu, Emre Kiciman, Noshir Contractor, Tina Eliassi-Rad
Nature | June 2021, Vol 595: pp. 197-204
Overcoming Failures of Imagination in AI Infused System Development and Deployment
Margarita Boyarskaya, Alexandra Olteanu, Kate Crawford
In the Navigating the Broader Impacts of AI Research Workshop at NeurIPS 2020 | December 2020
-
Overcoming Failures of Imagination in AI Infused System Development and Deployment
Margarita Boyarskaya, Alexandra Olteanu, Kate Crawford
In the Navigating the Broader Impacts of AI Research Workshop at NeurIPS 2020 | December 2020
-
When Are Search Completion Suggestions Problematic?
Alexandra Olteanu, Fernando Diaz, Gabriella Kazai
Computer Supported Collaborative Work and Social Computing (CSCW) | August 2020
CSCW 2020 Honorable Mention
CSCW 2020 Honorable Mention
On the Social and Technical Challenges of Web Search Autosuggestion Moderation
T. J. Hazen, Alexandra Olteanu, Gabriella Kazai, Fernando Diaz, Michael Golebiewski
July 2020
FACTS-IR: Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval
Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, Michael D. Ekstrand
ACM SIGIR Forum | December 2019, Vol 53(2): pp. 20
Publications by Type
-
“One-Size-Fits-All”? Examining Expectations around What Constitute “Fair” or “Good” NLG System Behaviors
Li Lucy, Su Lin Blodgett, Milad Shokouhi, Hanna Wallach, Alexandra Olteanu
NAACL 2024 | June 2024
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices
Yu Lu Liu, Meng Cao, Su Lin Blodgett, Jackie Chi Kit Cheung, Alexandra Olteanu, Adam Trischler
Findings of EMNLP 2023 | December 2023
FairPrism: Evaluating Fairness-Related Harms in Text Generation
Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann (HE/HIM), Hanna Wallach
ACL 2023 | July 2023
Can Workers Meaningfully Consent To Workplace Wellbeing Technologies?
Shreya Chowdhary, Anna Kawakami, Mary L. Gray, Jina Suh, Alexandra Olteanu, Koustuv Saha
2023 ACM Conference on Fairness, Accountability, and Transparency | June 2023
The Kitmus Test: Evaluating Knowledge Integration From Multiple Sources In Natural Language Understanding Systems
Akshatha Arodi, Martin Pömsl, Kaheer Suleman, Adam Trischler, Alexandra Olteanu, Jackie Chi Kit Cheung
ACL 2023 | May 2023
How Different Groups Prioritize Ethical Values for Responsible AI
Maurice Jakesch, Zana Buçinca, Saleema Amershi, Alexandra Olteanu
FAccT 2022 | May 2022
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications
Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, Alexandra Olteanu
NAACL 2022 | May 2022
Responsible Language Technologies: Foreseeing and Mitigating Harms
Su Lin Blodgett, Q. Vera Liao, Alexandra Olteanu, Rada Mihalcea, Michael Muller, Morgan Klaus Scheuerman, Chenhao Tan, Qian Yang
CHI 2022 | April 2022
ADEPT: An Adjective-Dependent Plausibility Task
Ali Emami, Ian Porada, Alexandra Olteanu, Kaheer Suleman, Adam Trischler, Jackie Chi Kit Cheung
ACL-IJCNLP 2021 | August 2021
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach
ACL-IJCNLP 2021 | August 2021
“I Can’t Reply with That”: Characterizing Problematic Email Reply Suggestions
Ronald Robertson, Alexandra Olteanu, Fernando Diaz, Milad Shokouhi, Peter Bailey
CHI Conference on Human Factors in Computing Systems (CHI ’21) | May 2021
Overcoming Failures of Imagination in AI Infused System Development and Deployment
Margarita Boyarskaya, Alexandra Olteanu, Kate Crawford
In the Navigating the Broader Impacts of AI Research Workshop at NeurIPS 2020 | December 2020
When Are Search Completion Suggestions Problematic?
Alexandra Olteanu, Fernando Diaz, Gabriella Kazai
Computer Supported Collaborative Work and Social Computing (CSCW) | August 2020
CSCW 2020 Honorable Mention
CSCW 2020 Honorable Mention
External Information Sharing on Health Forums: An Exploration
Dana Nguyen, Alexandra Olteanu, Emre Kiciman
Intl. Conf. on Web and Social Media (ICWSM) | June 2020
-
Sensing Wellbeing in the Workplace, Why and For Whom? Envisioning Impacts with Organizational Stakeholders
Anna Kawakami, Shreya Chowdhary, Shamsi T. Iqbal, Q. Liao, Alexandra Olteanu, J. Suh, Koustuv Saha, Alexandra Olteanu, Jina Suh
Proceedings of the ACM on Human-Computer Interaction | March 2023, Vol 7: pp. 1-33
Responsible computing during COVID-19 and beyond
Solon Barocas, Asia J. Biega, Margarita Boyarskaya, Kate Crawford, Hal Daumé III, Miro Dudík, Benjamin Fish, Mary L. Gray, Brent Hecht, Alexandra Olteanu, Forough Poursabzi-Sangdeh, Luke Stark, Jennifer Wortman Vaughan, Hanna Wallach, Marion Zepf
Communications of the ACM | July 2021, Vol 67(7): pp. 30-32
https://doi.org/10.1145/3466612
Measuring algorithmically infused societies
Claudia Wagner, Markus Strohmaier, Alexandra Olteanu, Emre Kiciman, Noshir Contractor, Tina Eliassi-Rad
Nature | June 2021, Vol 595: pp. 197-204
FACTS-IR: Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval
Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, Michael D. Ekstrand
ACM SIGIR Forum | December 2019, Vol 53(2): pp. 20
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Alexandra Olteanu, Carlos Castillo, Fernando Diaz, Emre Kiciman
Frontiers in Big Data | July 2019, Vol 2
-
AHA! Facilitating AI Impact Assessment by Generating Examples of Harms
Zana Buçinca, Chau Minh Pham, Maurice Jakesch, Marco Tulio Ribeiro, Alexandra Olteanu, Saleema Amershi
June 2023
-
On the Social and Technical Challenges of Web Search Autosuggestion Moderation
T. J. Hazen, Alexandra Olteanu, Gabriella Kazai, Fernando Diaz, Michael Golebiewski
July 2020