Many researchers found high potential of adopting building photovoltaic (PV) systems in urban areas, especially on building rooftop, to improve the sustainability of urban environment. However, the optimal energy output performance and economic benefit of the PV system are affected by the usable roof area, PV array layout, and shading effect considering high city density. This study aims to understand the effects of these design parameters in the urban environment of rooftop PV’s economic performance. This study carries out a case study in the urban area of Melbourne with 90 PV designs under three shading conditions to generate 270 scenarios. Through a lifecycle cost-benefit analysis, including net present value (NPV), NPV per kW, internal return rate (IRR), and payback year, the results can help in developing a comprehensive understanding of the economic performance of rooftop PV designs that cover most of the urban areas of Melbourne. The optimal PV design scenarios for the urban environment are identified, thereby providing investors and industry professionals with useful information on value-for-money PV design. Meanwhile, the maximum shading loss that makes the PV systems financially unfeasible is investigated, and design scenarios with greatest ability to sustain the shading effect are identified. This research can also support the policy makers’ decision on the development and deployment of the roof PV systems in urban planning.