May not break, but it can hurt like hell.
I "took" a few hooks to the head by ducking my head during a drunk boxing match with a buddy. Didn't hurt me too much, but after the flurry, I saw him wince and clench his fists as if he'd hurt them. He didn't break them though.
On another note though...body shots work SO much better bareknuckled.