In order to improve the quality and efficiency of the bright temperature reconstruction image of the synthetic aperture microwave radiometer, this paper takes the gray value of remote sensing image with channel amplitude and phase error and random error as the original bright temperature image, uses convolutional neural network (CNN) to supervise and learn the mapping relationship between bright temperature image and visibility function, and reconstructs bright temperature image according to the learned mapping relationship. It is compared with the traditional hexagon Fourier transform(HFFT). In terms of visual effect, the CNN network inversion method has clearer boundary and better effect. In terms of evaluation indexes, RMSE values of HFFT inversion method and CNN network inversion method are 15.80K and 10.93K, and PSNR values are 23.88dB and 27.09dB, respectively. Therefore, compared with HFFT inversion method, CNN network inversion method has less error in reconstruction results, effectively reduces Gibbs effect, and can better restore the original bright temperature image.