With the increasing penetration of renewable energy, power grid operators are observing both fast and large fluctuations in power and voltage profiles on a daily basis. Fast and accurate control actions derived in real time are vital to ensure system security and economics. To this end, solving alternating current (AC) optimal power flow (OPF) with operational constraints remains an important yet challenging optimization problem for secure and economic operation of the power grid. This paper adopts a novel method to derive fast OPF solutions using state-of-the-art deep reinforcement learning (DRL) algorithm, which can greatly assist power grid operators in making rapid and effective decisions. The presented method adopts imitation learning to generate initial weights for the neural network (NN), and a proximal policy optimization algorithm to train and test stable and robust artificial intelligence (AI) agents. Training and testing procedures are conducted on the IEEE 14-bus and the Illinois 200-bus systems. The results show the effectiveness of the method with significant potential for assisting power grid operators in real-time operations.