Crisis Data: An AI Ethics Case Study

Author: Irina Raicu

Publisher: Markkula Center for Applied Ethics

Publication Year: 2022

Summary: The following case study describes how there is an ethical concern with keeping data linked to mental health and using this data to receive monetary gain. The mental health hotline took the text messages from people who had reached out to them and made them anonymous. From there, they partnered with another company to use those messages to have a more comprehensive guide on how to provide care for those suffering from similar illnesses. This became a hot topic because there was pushback that the messages could be linked back to the original senders and therefore could impact their lives. It was also discussed how the organization was legally allowed to do this was through making users agree to their Terms, which linked to a 50-page document. This was considered to be somewhat ethically misleading due to the nature of the hotline, and critics questioned whether users following through in these instances are fully understood consent.