Wearable sensors show great potential for exercise energy management. This paper addresses the issues of insufficient sample data and low estimation accuracy in existing studies. First, an exercise training monitoring system with real-time capabilities is implemented utilizing multi-modal wearable sensors, and the monitoring data are preprocessed to remove redundant features by the regression coefficient method. The preprocessed data are augmented in light of an enhanced generative adversarial network to expand the sample data. Then an improved residual neural network model is proposed to reduce the computational complexity by decomposition convolution and group regularization, and finally the motion energy estimation results are obtained by softmax. Simulation results indicate that the measurement error of the proposed method is within 8% for each physiological index, and the accuracy of energy consumption estimation is 92.71%, which significantly improves the management effect of exercise energy consumption.