Pedestrian detection based on vision sensors is a hot and difficult issue in the field of autonomous driving. The large amount of data processing leads to high requirements for the robustness and real-time performance of the employed algorithm. The aggregate channel feature (ACF) algorithm is one of the widely recognized fast pedestrian detection algorithms, but there are many missed detections when the target is occluded or small. In response to this problem, we propose a pedestrian detection algorithm based on a combination of a five-layer convolutional neural network structure and an AdaBoost classifier (CNN–AdaBoost). The model was trained using Caltech and INRIA datasets, and detection experiments were performed using collected videos. The results show that the error detection rate of the proposed algorithm is greatly reduced compared with that of the ACF algorithm, but the detection speed is basically unchanged. Compared with the locally decorrelated channel features (LDCF) algorithm, the proposed algorithm achieves similar detection accuracy but the detection efficiency is greatly improved.