It is well known that urban impervious surface (IS) has a warming effect on urban land surface temperature (LST). However, the influence of an IS’s structure, components, and spatial distribution on LST has rarely been quantitatively studied within strictly urban areas. Using ETM+ remote sensing images from the downtown area of Shanghai, China in 2010, this study characterized and quantified the influence of the IS spatial pattern on LST by selecting the percent cover of each IS cover feature and ten configuration metrics. The IS fraction was estimated by linear spectral mixture analysis (LSMA), and LST was retrieved using a mono-window algorithm. The results indicate that high fraction IS cover features account for the majority of the study area. The high fraction IS cover features are widely distributed and concentrated in groups, which is similar with that of high temperature zones. Both the percent composition and the configuration of IS cover features greatly affect the magnitude of LST, but the percent composition is a more important factor in determining LST than the configuration of those features. The significances and effects of the given configuration variables on LST vary greatly among IS cover features.