In successful communication, speakers tailor their language to the context and listeners make inferences about the speaker’s knowledge. Several current accounts propose that both speakers and listeners accomplish this by rational analysis of the statistics in the environment, including their partner. Here we examine perspective-taking behaviour in a dyadic conversation task, where the same individuals act in the role of both speaker and listener. We model perspective-taking in both production and comprehension, taking into account the dyadic situation. Our findings suggest that conversational partners weight their own perspective more than the partner’s when speaking, and the partner’s perspective more than their own when listening. We also find that in both production and comprehension, conversational partners change the weighting of perspectives over time, moving towards relying more on the partner’s perspective at the expense of their own perspective. Surprisingly, we find little evidence that listeners or speakers adapt to the idiosyncratic statistics of their partner’s linguistic behaviour.