The increasing pervasiveness of social media has been matched by growing concerns regarding their potential impact on democracy and public debate. While some theorists have claimed that ICTs and social media would bring about a new independent public sphere and increase exposure to political divergence, others have warned that they would lead to polarization, through the formation of echo chambers. The issue of social media echo chambers is both crucial and widely debated. This article attempts to provide a comprehensive account of the scientific literature on this issue, highlighting the different approaches, their similarities, differences, benefits and drawbacks, and offering a consolidated and critical perspective that can hopefully support future research in this area. Concretely, it presents the results of a systematic review of 55 studies investigating the existence of echo chambers on social media, identifying patterns across their foci, methods and findings, and shedding light on the contradictory nature of the literature. We found that the results of research on this issue seem largely influenced by methodological and data collection choices. Indeed, articles that found clear evidence of echo chambers on social media were all based on digital trace data, while those that found no evidence were all based on self-reported data. Future studies should take into account the potential biases of the different approaches and the significant potential of combining self-reported data with digital trace data.