We are recruiting PhD students at @Cornell ! Our research covers a diverse range of topics, including perception and internal representations, cultural evolution, large-scale online experiments, cross-cultural studies, and computational music cognition. Please spread the word!

Finally, we demonstrate how our data can be used to benchmark methods for semantic alignment. We found that Bilingual Lexical Induction surpasses other (geometric) methods, suggesting that it would be appropriate for applications such as machine translation. 12/

We also visualize the map of correspondence between human and GPT descriptors of conversation tone in kNN structures, finding an asymmetric mapping as shown below. High robustness (dark color) indicates strong correspondence. 11/

Using an MDS of human-GPT correlations, we project the conversation tones into a shared embedding space. We found divergences in fundamental dimensions (eg., valence and arousal) and close alignment in other directions. 10/

We also found that there are difference between conversational tones that are highly aligned such as joyful, pleased, and happy, whereas tones like reflective, apologetic and proud are much less aligned. 9/

Using QoF, we calculated the correlation between ratings of sentences for each pair of conversational tones, creating a similarity matrix. We found a close agreement between these results and a separated experiment collecting humans’ similarity judgments. 8/

The second tool is Quality-of-Fit rating (QoF), to rate each sentence on the strength of a selected list of top 40 conversation tones elicited from both humans and LLMs via Sampling with People. 7/

To explore the sentences we sampled using SP, we also use UMAP to compute sentence embeddings. We find that regions of this sentence embedding scatter plot correspond to specific conversational topics and tones, such as the correspondence of (vi) to a more relational tone. 6/

We found that humans and GPT are different in their use of conversational tones. We observed a large difference in the frequency of usage of tones such as excited, grateful, and concerned. 5/

The first tool is Sampling with People (SP). It recruits humans to alternate between annotating tones for sentences and creating sentences for given tones, forming an unbiased sample of both, as SP implements a Gibbs’ Sampler with people. We repeat the process for LLMs. 4/

By combining advancements in cognitive science and machine learning, we propose a series of tools that address human-AI alignment in conversational tones. 3/