There are two separate things:
1) How we think about mathematical objects; and
2) How we formalize mathematical objects within set theory.
People such as Archimedes, Eudoxus, Newton, Euler, and Gauss did brilliant math long before the development of set theory. Clearly set-theoretic formalization is not a prerequisite of doing math.
The formalization of mathematical objects in the context of set theory is a relatively recent historical development, having taken place between say the late nineteenth century and much of the twentieth. During that period of time mathematicians found a need for more rigorous foundations, and set theory turned out to be highly useful for this purpose. It's not necessarily the last word on foundations; nor do we spend much time caring about the formalization most of the time.
For example if we are in calculus class and we encounter the function defined by $f(x) = x^2$, we typically think of it as a machine or black box that inputs a real number and outputs the square of the input. Nobody thinks of it as a set of ordered pairs.
On the other hand if we're learning analytic geometry for the first time, given $f(x) = x^2$ we make a table containing a few sample pairs $(x, x^2)$, then we plot those points in the $x$-$y$ plane, and we see that the dots seem to form a parabola. When we graph a function we are implicitly using the idea of a function as a set of ordered pairs!
The best way to think about this is that we have two points of view, the intuition and the formalism. We go back and forth between them as needed. Sometimes a function is a machine or a mapping or a correspondence, and sometimes it's a set of ordered pairs. Whichever point of view helps us at any given moment.
One can ask about the nature of the relationship between the intuitive idea of a function (or any other mathematical object) and the set-theoretic symbology that represents or models that object. This is a question of philosophy.
Perhaps functions and numbers and sets exist in some abstract Platonic realm, and our symbols are accurate representations of them. Or perhaps our symbols are helpful but inaccurate representations. Or perhaps there are no functions or numbers or sets at all, just strings of symbols manipulated in accordance with formal rules. In that latter point of view, math is just a game like chess. Nobody thinks the laws of the universe are written in chess, but many people think the laws of the universe are written in math. Why are the rules of math so different than the rules of chess?
Many smart people have thought deeply about these issues. But when they think about these things they are doing philosophy, not math.
The takeaway is that when we do math, we use both intuition and formalism, whichever is most helpful at any given moment. And we don't think about the philosophy, unless we are doing philosophy.
