Abstract: To enable the autonomous underwater vehicles (AUV) to show high accuracy and stability in tracking complex trajectory, an AUV optimal trajectory tracking method is proposed by using deep reinforcement learning. Firstly, the control model is built based on the Actor deep neural network and the Critic deep neural network. The Actor network is trained to adapt action and the Critic network is trained to evaluate the training outcome of the Actor network. Secondly, proper reward function is constructed to make the deep reinforcement learning algorithm feasibly in underwater vehicles dynamics model. Lastly, the judgment of successful networks training is a set based on the standard deviation of reward functions to ensure the stability of AUV within certain accuracy. Simulations are carried out and we prove that this algorithm performance better than PID control in trajectory tracking in a complex trajectory.