Detection of the facial skin condition enables people to better understand skin problems and helps them select appropriate treatment methods and maintenance products. We collected facial images of different skin problems by using an image optical sensor. To overcome the problem of insufficient training data, the collected dataset was expanded through data augmentation. In the field of computer vision, deep learning is often used for solving image recognition problems with high accuracy. Therefore, we used a convolutional neural network (CNN) to detect facial images of different skin problems. To overcome the difficulty of parameter selection and increase the recognition rate and robustness of the CNN, the parameters of the CNN were optimized using the Taguchi method. Eight control factors in the convolutional layer and the L36 orthogonal array (OA) were used in experiments. Analysis of variance was used for statistical analysis in the design of the experiments to obtain the optimal parameter combination for the developed CNNs. The experimental results indicate that the CNN optimized using the Taguchi method had an accuracy of 86.95%. The accuracy of the optimized CNN was 7.24% higher than that of the original CNN. The experimental results prove that the proposed hyperparameter optimization method can effectively improve the accuracy of network detection.