Abstract—The aim of this study was to evaluate and compare the reliability of a fully automatic cephalometric analysis software with manual cephalometric tracing. The lateral cephalograms of 108 orthodontic patients were selected. Eight linear (Pg-NB, Co-A, Co-Gn, U1-NA, L1-NB, Lower lip to E-plane, S-Go, and N-Me) and 9 angular (NS-Ba, SNA, SNB, NS-MP, FH-FO, U1-NA, L1-NB, L1-MP, and U1-L1) measurements were used in this study. The cephalometric analyses were performed by both manual method and automatic software. The differences between two methods were compared by paired t-test with p<0.05. Analysis of interexaminer calibration of the measurement revealed a high reliability. The result showed that there were statistically significant differences (p<0.05) in 13/17 parameters between the two methods, which consisted of 6 linear parameters (Pg-NB, Co-A, Co-Gn, U1-NA, L1-NB, and Lower lip to E-plane) and 7 angular parameters (SNA, NS-MP, FH-FO, U1-NA, L1-NB, L1-MP, and U1-L1). Only 4 parameters (NS-Ba, SNB, S-Go, and N-Me) did not show any significant differences. It is summarized that 76.47% (13/17 parameters) of cephalometric measurements performed automatically by the dental imaging software showed statistically significant differences when compared with the manual method. The automatic software could not reliably locate all cephalometric landmarks. Hence, the clinicians should not rely on the fully automatic analysis mode since the algorithm of the software still needs improvement for the higher accuracy in locating the cephalometric landmarks. Thus, to obtain accurate results, manual adjustments to the automatically located cephalometric landmarks are recommended.