A pulse array image sensor (PAIS) is a bionic image sensor that converts light intensity into a pulse sequence to reduce the data volume. The traditional tracking method is not suitable for the sparse data form because of the absence of grayscale information. A trajectory tracking method based on a Bayesian classifier is proposed to maximize the property of the pulse data. First, candidate points with the smallest interval distances are selected in the area of interest. Then the total pulse numbers in a specified period at different positions are gathered to compose the positive and negative feature vectors, which are used to train a naive Bayesian classifier. The classifier can exactly obtain positions from the candidate points, and features in the new tracking positions train and update the classifier. The two-step filtering target point tracking algorithm using the interval distance and Bayesian classifier requires only the raw pulse data without processing, which can maximize the advantages of the special data format and improve computing efficiency. In this way, the position information can be directly obtained from pulse data, and the problem of wasting computing resources on reconstructing grayscale images and treating them in the traditional way is avoided. Experiments were performed on both real filmed data and the public event camera datasets. Our method can obtain trajectories with a high accuracy and long tracking time. The tracking errors are in single digits. In the comparison experiments, although our method has a smaller data volume than other models that obtain both frame and event data, the results still show that it has a comparable performance to the state-of-the-art methods.