Aiming at the low efficiency and accuracy of traditional methods in mask optimization of integrated circuits, a mask optimization algorithm based on deep learning is proposed. Firstly, an improved generation countermeasure network (GAN) model is designed. By introducing multi-scale feature fusion module and attention mechanism, the detail generation ability of mask pattern in lithography proximity effect correction is improved. Secondly, a dynamic weight adjustment strategy is proposed to adaptively balance the optimal weights in different regions during the training process. In the aspect of FPGA acceleration, a customized hardware architecture is designed for the computing bottleneck of the optimization algorithm. The experimental results show that, compared with the traditional OPC algorithm, this method significantly shortens the optimization time, improves the speedup ratio on the FPGA platform, and maintains a low edge placement error (EPE).