Since logic and set theory themselves  at least as I imbibed them at the bosom of
Quine  are
all about ontology, this may seem a queer thing to single out. One reason to do so is that the question of the ontological status of various abstract entities is, in mathematics, seen through a glass darkly (typically going undiscussed unless challenged), whereas here  face to face.
This will, like its sister essay
The Ontology of Physics, be simply a corkboard for posting stray thoughts and choice quotes. For a more general take on all this, click
here. As for the ontology of mathematics proper, we need no special post, since that is an ongoing theme in the "Theologia Mathematica" series. For that, click
here.
Here we go.
Frege took all classes as rockbottom objects on a par with individuals.
 W.V.O. Quine, “On Frege’s Way Out” (1954)
This is the bold, the manly path, of Cantorian Realism. We do not merely accept sets (classes, collections) as, well, okay, “real” in some pale, some
Meinongian sense, though in no wise privileged to belly up to the bar as equals with such unquestionably real individuals as Piglet: No, they are
rockbottom objects, pardner, fit to drink with any man.
*
Commercial Break
A private
detective confronts the uncanny;
an ecclesiastical
mystery:
Murphy Calls In a
Specialist
*
~
Substitutional quantification makes good sense … no matter what substitution class we take  even that whose sole member is the lefthand parenthesis. To conclude that entities are being assumed that trivially, and that far out, is simply to drop ontological questions.
 W.V.O. Quine, “Existence and Quantification”
Quine footnotes this: “Lesniewski’s example, from a conversation of 1933 in Warsaw.”
The heart stops cold. 1933 in in central Europe  not a good place and time. Yet to have been Quine, even Quine, young and thrusting, in Warsaw with Lesniewski  very heaven!
Quine goes on:
Lesniewski did not himself relate his kind of quantification [i.e., substitutional as opposed to objectual] to ontological commitments.
And indeed was right to do so, since
Where substitutional quantification serves, ontology lacks point.
~
Andrew Gleason, Fundamentals of Abstract Analysis (1966), p. 1, offers a notably practical definition of set from the standpoint of a working mathematician:
A set is any collection of mathematical objects which is sufficiently well defined to be the subject of logical analysis.
Since the word collection itself is often used by mathematicians as a synonym of set, you might deem this definition circular. But here collection is being used entirely informally, referring to our pretheoretic intuitions; he could just as well said bunch or (even better) passel.
Indeed, from this perspective, Gleason is offering what amounts to an
ostensive definition. It is comparable to such a classic ostensive scenario as this: “See that creature over there? Assuming that it is not a mirage or an animatronic gimmick, that is what we refer to as a
capybara.”
~
In Set Theory, you start with any object, which can be anything. Indeed, even one will do: it can be zero, x, or even Piglet, for soon you contrast thesetcontainingPiglet, and off you go. Indeed, in the most abstract settheory, you don’t even need objects, just start from the empty set. Even so, the enterprise smacks of ontology, since you have … let’s call them ‘thingies’, so as not to be dragged into any philosophical presuppositions about objects (thingies being more like the CheshireCatsmile memories of objects), but even so, we are forming sets which consist of such thingies, and ask which contain which, and which are equal to which, and how many of them are there  tangible things like that.
In Category Theory, you put childish things aside, and say Goodby to All That. As one practitioner puts it:
Since a category consists of arrows, our subject could also be described as learning how to live without elements, using arrows instead.
 Saunders MacLane, Categories for the Working Mathematician (1971; 2^{nd} ed. 1998), p. vii
[Update  Lent 2013  Let us try, for a time, to live without elements...]
~
Of all the sciences  nay, of all human cognitive activities  mathematics is ontologically the most venturesome. For among its key tools is the ancient method of
reductio ad absurdum or
modus tollens. Here we work  calmly and logically  with Impossible Posits; and when the smoke has cleared, all is once again as it should be, and we know something new.
Thus, take the question of how many prime numbers exist. Lots, no doubt; but it is not initially obvious that they go on forever. The bigger a number gets, the harder it is for it to pass through the inflexible Sieve of Eratosthenes: the mesh gets finer and finer as the number of primes  of your possible submultiples  grows and grows.
A direct way of proving the infinitude of the primes would be to come up with a formula that could serve as a primal generator: plug in a number, out pops a prime. But no such formula is known, and probably none exists.
So the standard move, known already to the ancient Greeks (men like gods, like very gods) is to turn on your heel and spin about and say: Fine! Be like that! Let’s assume that there are not infinitely many primes. (Your finitist opponent  a fat and greasy Nominalist  emits an oily smile; but he soon shall taste the wrath of Modus Tollens.) So there are only finitely many (smile, nod); so there’s a biggest one (nod, but a fading smile, as our finitist realizes that something is about to go very, very wrong); let us call this largest one M. (Suspecting a trap, the finitist objects; you don’t make an issue of it; let’s call it N instead.)
Our mathematician now has what proves to be a powerful weapon: N, the Biggest Prime in the Universe. That no such number exists (as we soon shall find out) does not lessen its devastating effect, while we hold it in mind and operate with it  working in an antiuniverse, as it were, on the far side of Alice’s lookingglass, and yet where otherwise all the usual laws continue to hold.
So we form the factorial of
N and add a unit:
N! + 1. It is larger than
N; and yet it must itself be prime, since, by elementary arithmetic, no number (let alone a prime number) no larger than
N, can divide it. Our quixotic posit of a largest prime has managed to unhorse itself.

Modus Tollens, skewering an Impossible Posit 
Note that we really have been operating according to God’s own laws of logic, yet operating  temporarily, like Jack Bauer saving the day by working in a radioactive chamber  in a universe not made by His hand: namely, one in which a largest prime exists. Like the Devil’s inventions, this universe selfdestructs  but in a fashion that is constructive for ourselves.
"Your bait of falsehood takes this carp of truth."
 Hamlet
This process of assuming a falsity to come up with a positive result is really quite extraordinary. It is generally stated that, if a logical system allows you to derive a certain statement as well as its negation, then the whole system falls apart, since then you can derive anything at all, (p & ~p) => q being analytic. But obviously the structure of mathematics is more resilient than that.
In vain might one seek such fruitful use of counterfactuals outside of math. It is as though, in order to solve some particularly perplexing crime, one were to falsely accuse someone and put him on trial; then, in the course of the proceedings, the forensics and crossexaminations, the real truth would out. (Come to think of it, that is exactly the plot of many a courtroom drama.)
[Footnote] G.
H. Hardy, in A Mathematician’s Apology, gave the classic expression of
the audacity of this move:
The proof is by reductio ad absurdum, and reductio
ad absurdum, which Euclid loved so much, is one of a mathematician’s finest
weapons. It is a far finer
gambit than any chess gambit: a chess player may offer the sacrifice
of a pawn or even a piece, but a
mathematician offers the game.
A serioushumorous example of positing an impossible object  yet here, not with modus tollens in mind, but as a kind of calculational convenience:
Assume that the bubbles in a foam are reglar polyhedrons, whose faces are regular polygons with n sides, and that the angles between these sides are all 109° 28’. Since no such object exists, let us call it the ‘follyhedron’, and pretend that it does anyway.
 Ian Stewart, How to Cut a Cake (2006), p. 126
[Metafootnote]
Logic … operate[s] in terms of quite unidentified but none the less definite ‘objects’,
which then do or do not fall into
classes … I do not know whether it can deal with jellylike or cloudlike
entities, with mushy viscous messes that do not break up into manifest units. I suspect that nothing is beyond the
technical ingenuity of men …
 Ernest Gellner, Language and
Solitude (posthum. 1998), p. 53
An expert review of Dennis Hesseling, Gnomes in the
Fog: The Reception of Brower’s
Intuitionism in the 1920s (2003), by Sam Adams on Amazon, summarizes:
Brouwer's position arises from the
assertion that the ontology of mathematics is determined solely by
epistemology, and thus a statement which has not been determined to be true or
to be false cannot be determined to refer to a state of affairs … Since
mathematical meaning derives from emergent, epistemologically generated
ontology, it follows that logic itself is not a formative and central structure
of cognition, but rather is itself an emergent, structural overlay.