Bradley Merrill Thompson, Member of the Firm in the Health Care & Life Sciences practice, in the firm’s Washington, DC, office, authored an article in The Journal of Robotics, Artificial Intelligence & Law, titled “Unpacking Averages: Searching for Bias in Word Embeddings Trained on Food and Drug Administration Regulatory Documents.”

Following is an excerpt (see below to download the full version in PDF format):

Often when we talk about bias in word embeddings, we are talking about such things as bias against race or sex. This article talks about bias a little bit more generally to explore attitudes we have that are manifest in the words we use about any number of topics.

Bias Evaluation Using Sentiment Analysis

There are many different ways to evaluate potential bias in word embeddings, but I did not want to do a survey article where I talked briefly about all of them. Instead, I thought I would pick just one approach for illustration. The one I picked is perhaps the simplest, which is an evaluation of the word embeddings using a model for positive versus negative sentiment. In other words, I am looking to see whether particular word embeddings have a largely positive or negative connotation.

If words that should be regarded similarly have significantly different sentiments or connotations, that would be evidence of bias. In other words, if the word “Black” as an adjective for people has a largely negative connotation while the word “white” as an adjective for people has a largely positive connotation, that would be some evidence that the embeddings, trained on what people have written, have absorbed from that training data a bias against Black people.

However, I am not going to use race as my example in the analysis below. For one thing, race is rarely discussed in the documents that I am going to examine—Food and Drug Administration (FDA) documents—apart from a handful of documents specifically on race. I will leave you to draw your own conclusions from that. Instead, I am going to look for bias in other topics.

Related Materials

Jump to Page

Privacy Preference Center

When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.

Strictly Necessary Cookies

These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work. These cookies do not store any personally identifiable information.

Performance Cookies

These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.