Abstract: To study the machine learning-based portfolio optimization problem with environment, social and gover-nance factors under realistic constraints, firstly, Support Vector Regression (SVR), LASSO Regression and Elastic Net, are used to predict the closing price of the stock, which is used to calculate the portfolio's mean and variance, the weighted sum of stock ESG scores is used to construct portfolio ESG objective; then four realistic constraints are considered: transaction costs, threshold constraints, borrowing and lending constraints and cardinality constraints. Based on these considerations, a Mean-Variance-ESG (M-V-ESG) portfolio selection model is proposed. Next, the optimization problem corresponding to this model is solved by the genetic algorithm. Finally, the performance of the model is analyzed in an empirical study using the 30 constituents of the China Securities 300 Index. The investment performance of the M-V-ESG model, the modified Mean-Variance model (MM-V) that retains only four realistic constraints (transaction costs, threshold constraints, borrowing and lending constraints, and cardinality constraints) after removing the ESG constraint from the M-V-ESG model, and the naive portfolio model is compared through out-of-sample testing. The empirical results show that the M-V-ESG model outperforms the equally weighted portfolio in both Sharpe ratio and ESG score. While the M-V-ESG model achieves a higher ESG score than the MM-V model, its Sharpe ratio is lower than that of the MM-V model. The results indicate that incorporating machine learning models and ESG factor into portfolio selection can improve portfolio performance and its ESG score.