To fully use the contextual information of hyperspectral images (HSIs), we propose a U-shaped network model combined with attention mechanism to achieve image-level HSI classification. First, the entire HSI is input into the network for end-to-end training, and the classification results of the entire scene are directly output. Then, the context information is used to improve the classification accuracy, while reducing many redundant calculations. Second, to improve the classification accuracy, considering two dimensions (i.e., space and channel), a hybrid attention module, mixing spatial and channel, is designed. Third, three datasets of the University of Pavia, Indian Pines, and Salinas are selected for the classification experiments. The experimental results show that, compared with other methods, the proposed method can obtain higher classification accuracy, and its training and testing efficiency is higher.