By Richard LeComte

LEXINGTON, Ky. – Sometimes it’s a question of context – or, in this case, the context of questions.

Jennifer Cramer, associate professor of linguistics at the University of Kentucky’s College of Arts & Sciences, specializes in sociolinguistics – the study of how social factors, including race, gender and class, affect language. She’s studied differences in language in Southern regions of the United States in particular.

Over the past five years, she’s found a practical outlet for her research: evaluating questions on an exam given to certify family physicians. She uses her skills to determine if a particular question might be confusing to test-takers because of cultural contexts.

“When we look at an assessment tool, we have to know what we're assessing,” Cramer said. “Linguists work with educational groups all the time when they’re creating tests that have language-related questions. Let’s say you were taking a test and you were supposed to pick, ‘May I go to the bathroom?’ as opposed to ‘Can I go to the bathroom?’ They're not actually testing you on your knowledge of grammatical English. They’re testing you on what’s proper in your cultural context.”

Cramer is working with The American Board of Family Medicine, which has its headquarters in Lexington. She’s part of a panel of experts who examine the tests the board gives annually to doctors who desire certification. These tests evaluate whether doctors have absorbed the latest information in their field and have retained key elements of their medical training.

“We invited Jennifer to participate in our Differential Item Functioning Panel largely as a matter of quality,” said Thomas R. O’Neill, vice president of psychometric services at the board. “We think that the questions on our exam should be tied to family medicine knowledge and clinical decision making.  It shouldn't be tied to aspects of culture that are not part of family medicine in an important way.”

The panel examines how doctors answer certain questions on the certification exam.

“We start off by conducting something called differential item functioning analysis,” he said. “Essentially it finds out that, after you have adjusted for the ability of the people taking the exam, do people from different race, ethnicity or gender groups have the same probability of answering that question correctly as others? If it is by a magnitude that is notable and unlikely to happen by chance, we flag those questions.”

That’s where Cramer comes in.Along with content experts who are also members of the possibly affected minority, she uses her expertise in linguistics to evaluate a flagged question for cultural tripwires. One example she gives is using the word “spelunking” in a question about treating someone injured in a cave accident. “Spelunking,” which means cave exploration, may be a word easily recognized by people of one culture but not another.

"This was brought up because essentially it requires you to not only know the medicine, but also to know what spelunking is,” she said. “You could simply replace the word, so you don't need the jargony word. In that case, you can say cave exploring or a cave explorer. You don't need to say spelunking.”

The panel then assesses whether the language of the question may be biased toward a different ethnic group or culture. They also assess whether the wording is pointing at a key aspect of medical practice.

“If the panel can't identify a source of bias, then we move on and that question is fine,” O’Neill said. “If the panel can identify a possible source of bias, then they determine if that source is an important aspect of family medicine. If it is, then the question is retained. If it is not, then the question is recommended for being rewritten or deleted from the question bank.”

Cramer’s efforts on the panel help ensure that the questions on the exam are asking about clinical content in a way that accounts for differences in cultural and language usage.   It’s a great example of linguistics in action – something that Cramer stresses to her students looking for real-world uses for their skills.

“I'm always curious about how the institutions that exist in our society potentially get misunderstood because of the cultural differences in language,” she said. “Those are places where linguistics has an application.”