Linguistic Biases in Letters of Recommendation for Radiation Oncology Residency Applicants from 2015 to 2019.

We aimed to investigate whether implicit linguistic biases exist in letters of recommendation (LORs) for applicants to radiation oncology (RO) residency. LORs (n = 487) written for applicants (n = 125) invited to interview at a single RO residency program from the 2015 to 2019 application cycles were included for analysis. Linguistic Inquiry and Word Count (LIWC) software was used to evaluate LORs for length and a dictionary of predetermined themes. Language was evaluated for gender bias using a publicly available gender bias calculator. Non-parametric tests were used to compare linguistic domain scores. The median number of the LORs per applicant was 4 (range 3-5). No significant differences by applicant gender were detected in LIWC score domains or gender bias calculator (P > 0.05). However, LORs for applicants from racial/ethnic backgrounds underrepresented in medicine were less likely to include standout descriptors (P = 0.008). Male writers were less likely to describe applicant characteristics related to patient care (P < 0.0001) and agentic personality (P = 0.006). LORs written by RO were shorter (P < 0.0001) and included fewer standout descriptors (P = 0.014) but were also more likely to include statements regarding applicant desirability (P = 0.045) and research (P = 0.008). While language was globally male-biased, assistant professors were less likely than associate professors (P = 0.0064) and full professors (P = 0.023) to use male-biased language. Significant linguistic differences were observed in RO residency LORs, suggesting that implicit biases related to both applicants and letter writers may exist. Recognition, and ideally eradication, of such biases are crucial for fair and equitable evaluation of a diverse applicant pool of RO residency candidates.

Chapman BV ，Rooney MK ，Ludmir EB ，De La Cruz D ，Salcedo A ，Pinnix CC ，Das P ，Jagsi R ，Thomas CR Jr ，Holliday EB ... -  《-》

Don't Judge a Letter by its Title: Linguistic Analysis of Letters of Recommendation by Author's Academic Rank.

This study analyzed the linguistic differences in letters of recommendation (LORs) for general surgery residency applicants written by authors of various academic ranks. Given that many general surgery residency programs require a LOR from the Chair of surgery, this study also examined whether LORs written by the Chair demonstrate linguistic differences to support this practice. A single institution, retrospective review analyzed LORs from two application cycles of general surgery residency applicants who were selected for interview at a large academic institution. Word count (WC) and linguistic characteristics of LORs were analyzed with a previously developed institution-specific dictionary using the Linguistic Inquiry and Word Count software (LIWC2015; Pennebaker Conglomerates, Inc., Austin, Texas). WC and linguistic characteristics of LORs reported as frequencies of terms within twenty-four categories were examined based on the letter authors' academic rank. Further examination compared LORs written by a Chair of surgery with those written by non-Chairs. A single large, Midwestern academic general surgery residency program PARTICIPANTS: Four hundred and sixty-five letters of recommendation received during two interview cycles were included for analysis. A total of 465 LORs written by assistant (n = 82), associate (n = 94), and full professors (n = 289) were included in the study. No statistically significant difference was noted in the WC of LORs based on the letter writers' academic ranks (p = 0.95). Assistant professors utilized grindstone, communal, and technical skill terms with higher frequencies compared to associate professors and full professors. LORs written by assistant professors demonstrated the highest authentic variable score followed by associate professors then full professors (4.94, 3.92, 3.28, p < 0.01). LORs written by Chairs (n = 128) had lower authentic variable scores compared to LORs written by non-Chairs (n = 337; 2.71 vs. 3.91, p = 0.001). Only 50 (39%) LORs written by Chairs indicated working directly with the applicant, and sub-group analysis demonstrated a higher authentic variable score in this group compared with LORs written by Chairs who did not indicate having worked directly with the applicant (3.51 vs. 2.5, p = 0.01). Linguistic analysis of LORs for general surgery residency applicants demonstrated minor yet statistically significant differences based on the author's academic rank. If applicants can obtain linguistically similar LORs from surgeons of any academic rank, but less authentic LORs from writers with higher academic ranks, these LORs may be less valuable for the residency programs when evaluating applicants. Based on the subgroup analysis, less than 40% of Chair LORs indicated that the Chair worked directly with the applicant, calling into question the utility of the Chair LORs as meaningful evaluation of applicants. Further study to compare LORs of applicants selected and not selected for interview may add additional insight into linguistic differences in LORs written by authors of different academic ranks.

Han AY ，French JC ，Tu C ，Obiri-Yeboah D ，Lipman JM ，Prabhu AS ... -  《-》

Linguistic Differences by Gender in Letters of Recommendation for Maternal-Fetal Medicine Fellowship Applicants.

As in many realms of academia and medicine, in obstetrics and gynecology, women experience gender bias in residency evaluations and academic promotions. More specifically, women in Maternal-Fetal Medicine (MFM) are underrepresented within departmental leadership positions. As a means of identifying spaces where bias may exist, multiple investigators have previously reported on gender bias in letters of recommendation (LORs) for residency and subspecialty training programs. We aimed to determine if linguistic differences exist in LORs for self-identified male and female applicants to MFM fellowship at an academic institution. This was a retrospective single-site cohort study from 2019 to 2021. Data collected included applicant's age, self-reported race/ethnicity and gender, geographic region of residency, step 1 and 2 scores, scholarly and volunteer activities, and number of LORs. The Linguistic Inquiry and Word Count (LIWC) software, a validated text analysis program, was used to characterize LOR linguistic content. Multivariable analysis was used to compare letter characteristics to applicant demographics. A total of 212 applications were reviewed, including 808 LORs. Women comprised 76.9% of applicants, and men 23.1%. Most applicants identified as non-Hispanic White (52.8%). Men were more likely to be international medical graduates (20 vs. 6%, p ≤ 0.01), and women reported more volunteer activities (7.1 ± 5.1 vs. 5.5 ± 4.3, p = 0.04). There were no differences in step scores, number of research projects, or number of LORs. Multivariable analysis controlling for applicant race, step 1 score, and gender of letter writer revealed that letters written for males contained significantly more references to the word category cognitive processes (7.4 ± 0.2 vs. 7.1 ± 0.1, p = 0.046), specifically in reference to the subcategories of certainty and differentiation. We identified linguistic differences in LORs written for MFM applicants, suggesting potential bias in the style of writing for male and female physicians applying to this field. · Gender bias exists in the evaluation and promotion of women in medicine.. · We sought to determine whether it also exists in letters of recommendation for MFM fellowship.. · Previous studies have examined gender bias in letters of recommendation for other specialties.. · Linguistic bias was detected in letters of recommendation for MFM fellowship from 2019 through 2021.. · We found evidence of linguistic differences based on gender of applicant and letter writers..

Rosenthal E ，Tappy E ，Pan E ，Verma D ，Wang A ，Brown LS ，Santiago-Muñoz P ，Florian-Rodriguez M ... -  《-》

Do gender and racial differences exist in letters of recommendation for obstetrics and gynecology residency applicants?

Studies in several fields demonstrate gender and racial differences in descriptions of applicants in letters of recommendation. However, gender and racial biases in obstetrics and gynecology residency letters of recommendation are understudied. This study aimed to determine whether there are differences in the letters of recommendation written for medical students applying for obstetrics and gynecology residency based on applicant gender and underrepresented in medicine status. We analyzed all letters of recommendation submitted to a single obstetrics and gynecology residency program during the 2019-2020 application cycle using a computerized text analysis software to evaluate the presence of 25 word categories. Multivariable regression was used to compare differences in the frequency of word categories by gender and underrepresented in medicine status. Concurrently, we performed a qualitative content analysis of letters from a random sample of applicants balanced for gender and underrepresented in medicine status and explored emerging themes until thematic saturation was reached. We analyzed 3060 letters written for 834 applicants, 721 (87%) of whom were women and 198 (24%) of whom were underrepresented in medicine. Men authored 1605 letters (53%). There was no difference in word categories used in letters written for men and women applicants on multivariable analysis. Similarly, there was no word category difference between letters written for applicants who were underrepresented in medicine compared with White and Asian applicants. However, women letter writers used more communal (relationship-oriented) words compared with men letter writers (P<.001). Moreover, on the qualitative analysis of letters from a random sample of 30 applicants, comments about personality traits were frequent in all letters. Comments on surgical skills, work ethic, and leadership were found more in letters for White and Asian applicants, and phrases that were doubt raisers were found more in letters written for underrepresented medicine applicants. On linguistic analysis, letters of recommendation written for obstetrics and gynecology residency applicants were similar overall compared with applicant race and gender.

Brown O ，Mou T ，Lim SI ，Jones S ，Sade S ，Kwasny MJ ，Mueller MG ，Kenton K ... -  《-》

Assessment of gender differences in letters of recommendation for pharmacy residency applicants.

Letters of recommendation (LORs) are highly regarded components of pharmacy residency applications, as they provide insight into an applicant's character and capabilities. In other medical fields, differences in language have been reported for letters written for female and male applicants; however, data on gender differences in LORs for pharmacy residency applications are currently lacking. LORs for applicants to our institution's postgraduate year 1 pharmacy residency program for the 2019-2020 academic year were extracted and processed by a natural language processing service. Words within 18 categories were identified and counted for each LOR. Total word count was also compared. Of the 473 LORs included for analysis, 320 (67.7%) were written for female applicants and 153 (32.3%) were written for male applicants. Approximately two-thirds of all writers were women for both female and male applicants. In comparing letters for women and men, there was a statistically significant difference in the percentage of LORs that contained terms in categories described as gendered, solitary/reserved, and desire. There was no statistically significant difference in total word count or in the presence of words in other categories such as grindstone, standout, agentic, or communal. When controlling for grade point average, writer gender, duration that the writer knew the applicant, and the writer's professional position, there were no changes to the statistical findings. Letters written for female and male applicants were largely similar with regard to length and word categories utilized. While no clear gender bias was found when evaluating pharmacy residency LORs, writers must continue to assess their implicit biases and how those biases might affect a candidate's application.

Rice ML ，Leung JG ，Mara KC ，Leung SB ... -  《-》