One particular topic in the literature on Frege’s conception of sense relates to two apparently contradictory theses held by Frege: the isomorphism of thought and language on one hand and the expressibility of a thought by different sentences on the other. I will divide the paper into five sections. In (1) I introduce the problem of the tension in Frege’s thought. In (2) I discuss the main attempts to resolve the conflict between Frege’s two contradictory claims, showing what is wrong with some of them. In (3), I analyze where, in Frege’s writings and discussions on sense identity, one can find grounds for two different conceptions of sense. In (4) I show how the two contradictory theses held by Frege are connected with different concerns, compelling Frege to a constant oscillation in terminology. In (5) I summarize two further reasons that prevented Frege from making the distinction between two conceptions of sense clear: (i) the antipsychologism problem and (ii) the overlap of traditions in German literature contemporary to Frege about the concept of value. I conclude with a hint for a reconstruction of the Fregean notion of ‘thought’ which resolves the contradiction between his two theses.