Determining the accurate values of the return flow fraction from different uses is essential for optimal allocation, management and operation of water resources. The goal of this research was estimation of return flow coefficients from domestic, industrial and agricultural demands and also contribution of these return flows in surface flow. For this purpose, calibration technique and Genetic Algorithm were used. MODSIM is a river basin management decision support model that was used as a simulation model. In this research by Custom Run capability of MODSIM, i.e. coding in VB programming language into the model, the calibration of considered coefficients was provided. The calibration was done for 3 different objective function. Results showed that the most appropriate and logical coefficient was obtained in the 3rd calibration state, i.e. calibration with an objective function that included RMSE of surface flow discharge and RMSE of groundwater level. The return flow fractions of domestic, industrial and agricultural demands for Shian sub-basin in the western part of Iran are obtained 87, 76 and 18 percent, respectively.