This paper presents the deep learning assisted distorted Born iterative method (DBIM) for permittivity reconstruction of dielectric objects. The inefficiency of DBIM to reconstruct strong scatterers can be overcome if it is supported with Convolutional Neural Network (CNN). A novel approach, cascaded CNN is used to obtain a fine resolution estimate of the permittivity distribution. The CNN is trained using images consisting of MNIST digits, letters, and circular objects. The proposed model is tested on synthetic data with a different signal-to-noise ratio (SNR) and various contrast profiles. Thereafter, it is verified by means of experimental data provided by the Institute of Fresnel, France. Reconstruction results show that the proposed inversion method outperforms the conventional DBIM method in terms of accuracy as well as convergence rate.