When they're not being cast, spells occupy a different dimension than us. That dimension overlaps with our own, because all dimensions overlap. It's a plane composed not of atoms or molecules, but consciousness. Wizards call it the ethereal plane. It "looks" empty to our eyes because we cannot see the stable, self-sufficient elements of sentience that compose it.
Another word for a stable, self-sufficient element of sentience is "spell". (Although, the things we think of as spells are just the ones that we have found practical uses for, such as invisibility. Spells that do not seem to have a practical use are discarded, but in the ethereal plane, you will find many such spells that "don't do anything". The difficulty of inventing a spell is not getting a spell to work, it's finding a spell that will do something useful in a reliable, predictable way. This is why wizards who research spells are always blowing themselves up.)
The power of a spell relies on the tension between the ethereal plane and the material. A spell's energy, to generate heat or light or power, doesn't seem to come from any material source, because it doesn't. The ethereal plane powers spellcasting. Sentience itself powers magic (and sentience is about as difficult to define as magic is).
Now, a human is mostly meat. But there is some sentience in there too (almost always located in the brain). And there are some ethereal creatures that are inverse: mostly sentience but with a little bit of meat as well. A human who has learned to use his sentience as a tool--who has weaponized his sentience--in order to affect magic is called a wizard.
(The ethereal plane has counterparts to materials and energy, too. Just as the fundamentals of matter can be said to be atoms and quarks, the fundamentals of consciousness can be said to be emotions and qualia. A prick of coldness on non-existent skin. A spurt of jealously, isolated an undefined. We don't think of these things as sentient or aware--they aren't--but they are the building blocks.)
(And there are analogues for bigger things, too. The material plane has mountains of granite and oceans of water. The ethereal plane has vast plains of anger, crystallized and insensate. There are also self propagating fractals of mathematics, logical cysts of multiplication that branch off and up until they crumble under their own fragility. What numbers are these trees of mathematics multiplying? Why, they aren't multiplying anything at all, merely the naked concept.")
Humans have a hard time thinking about these things as separate, although they are. For a long time people thought of the human meat body as a whole. They thought of it as something complete, independent, and not too complex. Only after the microscope was invented did we start to see ourselves as a summary of billions of cooperating cells, each one virtually an animal in it's own right.
And populations of humans cells can continue to thrive and live on, even long after the human has died. (See: Henrietta Lacks, who is both immortal and dead). Spells can do this, too. In the flash of their casting, they can destabilize, mutate, and grow like a cancer. These spells swell with the rich effluvium of the wizard's cerebrospinal fluid, and grow thick from the ejaculate of his mind. THE SPELL GROWS MEAT. These degenerate, autonomous spells are what we call demons.
But I digress.
Memorizing a spell is not like memorizing a series of noises and hand motions. It's like inviting a spell into your brain by creating a suitable environment for it to reside. It's like building a trap for an external fragment of sentience. It's like creating an impression in our dimensional fabric so that water on the other side can pool there. It's like a gravity well.
That's why casting a spell "erases it from your memory"--because it's not erasing anything. It's merely the dissipation of a certain pattern of consciousness.
To put it another way, it's like weaving a netted bag (out of your neurons) to catch (invite) a fish (spell). You can't handle this fish with your hands, so your only way to affect that fish is through handing the bag. And then when you want to hit something with the fish, you throw it, releasing both the bag and the fish. (Why do all of my metaphors get stupid at the end?)
This is why why wizard neurons literally tie themselves in knots. This is why wizard brains twist themselves into maddening shapes, and carve the inside of the skull. This is why wizard heads are valuable even if they are severed. This is why wizards invariably go mad. (The only wizards who don't go mad are the ones who've managed to cast the fewest spells.)
And this is why wizards are some of the most ignorant people on the planet:
Because spellcasting requires a very specific microenvironment in a very small part of the wizard's brain, the act of "memorizing" a spell requires cultivation of certain mental traits. Not only must wizards learn otherworldly esoterica, but they must also believe some of it as well.
People often contrast arcane spell with divine miracles by claiming that the former is powered by knowledge and belief and the latter is powered by emotions and belief. As far as reductionistic, crude simplifications go, this one is pretty accurate.
And so wizards believe such strange things because they must. If they stopped believing in these things, they would cease being wizards. No spell would ever come to roost in a brain that hasn't contorted itself into a suitable nest! The regular sulci and gyri of sailors and scholars are but vulgar and transient to spells.
And so wizards believe that there are barnacles that turn into geese. That black cats herald doom. That certain circular patterns can trap demons inside when made from silver. That crows can carry away souls. That the planet is hollow and is lit by a miniature star. That the speak with dead spell actually allows communication with the spirit of one who has passed.
They guard their thoughts by following strange traditions. They filter their perception of reality by isolating themselves in tower and in monasteries. In their books they have built a false history of the world with false maps and false assumptions. And yet the same wizards who can level a city block with a few words are also the ones who have no idea how boats float or babies are concieved.