With the development of farmland rental market in China, the role of farmland rental need to be further explored. This study focuses on the effects and mechanisms of farmland rental on the long-term investment (land-related and non-land-related investment) of rural households. We conducted an empirical test using the two-stage instrumental variable method (2SLS-IV) based on survey data of 1718 rural households in China. The results indicate that the effects of farmland rental vary with the type of long-term investment. Specifically, it positively affects non-land-related long-term investment but has no significant effect on land-related investment. Moreover, farmland rental affects non-land-related investments through three mechanisms: expanding the scale of operations, improving farmland fragmentation, and increasing the proportion of agricultural income. In addition, the effects of farmland rental on non-land-related investment are heterogeneous. For land tenants who are grain planters with high-quality and titled farmland in developed villages, the effects are significant. These findings deepen our understanding of the relationship between farmland rental and agricultural investment in rural households.