It may be a rhetorical question, but it's one I've wrestled with before so I'll answer it. I think it's because, in an attempt to think of a name for the motherboard, all its attachments, and the case housing it, "CPU" was what a group of people started to use in the absence of a better option, and now a lot more people use the term because it's understood to refer to this even if it's not technically accurate. Alternate terms have other problems:
"Box" -- too generic -- could be referring to the cardboard box in which the computer arrived
"PC" -- 1) Refers to the whole computer; 2) in another twist of misused terms, this tends to refer to the alternative to a Mac
"Case" -- Usually refers to the unit without its contents; also too generic
"Unit" -- too generic again
"CPU housing" -- no clear meaning -- does this refer to the case alone, and technically the case doesn't house just the CPU, so it's a confusing term
"CPU Box" -- close... I tend to use this, but people tend to prefer one word terms, and it's still a little awkward technically speaking.
"Computer" -- Not specific enough when one wants to refer to just the CPU Box and not include the monitor and all the attached components.
So what would you call it?
I used to be pedantic about using accurate terminology, but finally I accepted the fact that words are used for more than just their discrete denotations. They slowly assume/acquire the meanings associated with their connotations, determined by the contexts in which they are used. A person knows the meaning of a word primarily by the context in which it is used, and often times the definition of a word doesn't quite cover that, and we geeks complain when that happens

. The word gets used in other contexts where a better word wasn't available or was too hard to think of, and words kind of change meaning.