Uncertainty in the demand and supply of social public resources leads to increased competition, which increases the complexity and collaboration cost of resource allocation. Therefore, this paper introduces a multi-agent reinforcement learning (MARL) algorithm to optimize the dynamic allocation mechanism of social public resources. Firstly, considering the problem of partial loss of information due to environmental unobservability in a multi-intelligent body environment, a MARL algorithm based on prophet-guided (MADRLPG) is proposed. On this basis, a flexible overlapping organization framework under spatio-temporal constraints is constructed. In addition, a dynamic resource allocation algorithm under spatiotemporal constraints is proposed to solve the unbalanced use of resources; meanwhile, a collaborative strategy generation algorithm is proposed to reduce resource competition. Finally, the simulation results show that the resource utilization rate of the proposed method is 97%, which can effectively alleviate the resource constraint problem.