Abstract: With the rapid development of remote sensing engineering technology in our country, the acquisition of remote sensing image data has increased sharply, and the complexity of the image background has also increased. However, the traditional remote sensing image target detection algorithm has low accuracy and weak generalization ability. The accuracy of CNN is gradually difficult to meet the requirements. Aiming at this problem, a processing architecture based on aerospace-grade FPGA for forward inference acceleration of convolutional neural networks is proposed, and the YOLOv5s network model is selected as the benchmark algorithm for the design of the processing architecture. Since the main body of YOLOv5s is composed of a large number of convolutional layers, the center of gravity of the accelerator architecture design is located in the convolutional layer. In the design of the architecture, the parallel expansion of input channels and output channels and the optimization strategy of data pipeline control are used to effectively improve the Real-time processing performance in the inference phase. The experimental results show that when using this processing architecture to accelerate the inference stage of YOLOv5s, the operating frequency of the convolution module can reach 200MHz, its computing performance is as high as 394.4GOP/s, the power consumption of the FPGA is 14.662W, and the average computing efficiency of the DSP calculation matrix It is as high as 96.29%, indicating that using FPGA for hardware acceleration of convolutional neural networks in on-board platforms with limited resources and power consumption has significant advantages.