When temperature (T) is lower than the T_c, the energy of spin parallel orientation is higher than spin antiparallel orientation. From the well-known expressions of entropy of mixing, the entropy for spin-parallel orientation is lower than that of spin antiparallel orientation. One expects a static structure of spin antiparallel orientation when T becomes lower than T_c. However, the observed ferromagnetism is just the opposite phenomena. Clearly there is a big flaw in statistical physics, which is the Gibbs paradox statement (Entropy of mixing to form solid assemblages, liquid and gas mixtures or any other analogous assemblages such as mixed quantum states, decreases with the increase in the property similarity of the subsystems). A new entropy theory says that the entropy of mixing increases continuously with the increase in similarity [1,2]. Indistinguishability and maximum symmetry corresponds to the maximal entropy of mixing. The logarithmic relations of entropy and symmetry were established [1]. Indistinguishability and symmetry is an ubiquitous driving force for all spontaneous physical and chemical processes. This new theory conforms very well with and explains perfectly all the symmetry breaking phenomena. [1] S. -K. Lin, J. Chem. Inf. Comp. Sci. 36 (1996) 367-376. [2] S. -K. Lin, J. Theor. Chem., 1 (1996) 135-150; S. -K. Lin, Molecules, 1 (1996) 57-67. (Downloadable from www.mdpi.org/lin.htm).