Philosophers of mathematics agree that the only interpretation of arithmetic that takes that discourse at 'face value' is one on which the expressions 'N', '0', '1', '+', and 'x' are treated as proper names. I argue that the interpretation on which these expressions are treated as akin to free variables has an equal claim to be the default interpretation of arithmetic. I show that no purely syntactic test can distinguish proper names from free variables, and I observe that any semantic (...) test that can must beg the question. I draw the same conclusion concerning areas of mathematics beyond arithmetic. This paper is a greatly extended version of my response to Stewart Shapiro's paper in the conference 'Structuralism in physics and mathematics' held in Bristol on 2–3 December, 2006. (shrink)