We Compared Real Podcast Downloads to Rephonic’s Estimates – Here’s What We Found

Podcasting is infamously opaque when it comes to understanding listener numbers.

And this is because, not only is podcasting super fragmented compared to other media, but there’s also a real lack of standardization when it comes to what actually counts as a download.

For some platforms, a ‘listen’ is when someone actively downloads an episode, or streams it for more than 30 seconds; for others, hitting that ‘subscribe’ button results in an auto-download of every new episode regardless of whether you’ve listened to it yet or not.

But despite the undeniable grey areas involved, podcasters, brands, and agencies still need to rely on third-party listener data to help decide which shows are potentially suitable collaborators.

So it’s good to know that there are people out there developing tools to try and make the industry a bit more transparent. These tools can help us better understand the podcasting ecosystem and what listener behaviour looks like across it. And the leading tool in this area right now is undoubtedly Rephonic.

But we wanted to know how close public estimate tools like Rephonic are to the real thing.

And so, with the help of a few podcasting pals, we ran a test to find out!

What is Rephonic?

Rephonic is a podcast analytics platform and research tool that helps podcasters, brands, advertisers, and networks analyse and better understand the podcast ecosystem.

The tool provides insights into podcasts and their audiences, like listener numbers and locations, so users can decide which shows to collaborate with.

Rephonic reports that it logs data for more than 3 million podcasts, and it’s probably the best tool on the market for finding cross-promotion opportunities and keeping track of what your audience also listens to. The 3D audience graph tool is easily the platform’s neatest feature, which uses data pulled from ‘listeners also subscribed to’ to create a visual map of connected shows that you can click to dive into the detail. Here’s what it looks like:

Podcasters might use Rephonic when looking to…

• Choose new podcasts to do guest swaps and cross-promotion. You can use the search function to find shows within your podcast niche, and it will provide listener data and contact details for you to easily reach out to. 
• Understand your own podcast’s position within the ecosystem. Rephonic allows you to view your performance in comparison to other shows in your niche and also see which similar shows your audience are also listening to

We were curious to know just how accurate the listener data Rephonic logs really is, so we ran a test. Here’s what we did, and what we found out.

The Podcast Download Test: What We Did

To run the test, we took a sample of 16 podcasts. The shows varied widely in size and were taken from various podcast categories and genres. We based our Size Tier on Buzzsprout’s platform stats, which are, in turn, based on the number of downloads new episodes get in their first week of release.

In order to include a podcast in the test, we needed to contact the host directly so that we could access their actual download number data and compare it with Rephonic’s.

Of the podcasters we contacted, 16 shows agreed to participate. We excluded two outliers that returned wildly inaccurate results (one had an inaccuracy rate of -1395.3%!), so the sample size became 14 shows in total. 

Sharing real download numbers is still uncomfortable for many creators, and so we agreed that this data would keep them anonymous.

The Results: What We Found

Here’s a quick overview of the results we gathered from our test in comparing Rephonic’s download numbers with actual numbers gathered directly from podcasters:

• Average accuracy: 51.6%
• Median accuracy: 50.85%
• Highest accuracy: 98.9%
• Lowest (non-outlier): 5.7%
• Over 70% accuracy: 5 of 14 podcasts (~36%)

So, in a nutshell, we found out that Rephonic’s download estimates provide a very rough approximation.

• We found that the results were directionally useful, in that they give you a rough idea of how many downloads a show gets, especially for podcasts in the 500–2000 range, but you’d still want to ask the hosts directly to confirm their numbers. 
• However, we’d say the results we gathered were not accurate enough for ad pricing, pitch decks, or precise planning. 4 of the 14 podcasts we tested had an accuracy of 70% or more.
• We also found that the results were unreliable for podcasts with very low or very high downloads, where estimates were wildly off. This includes the two outlier shows that we decided to exclude from the final results, as they were too inaccurate. 

Why These Results Make Sense

A ‘very rough approximation’ might sound like Rephonic’s download estimates aren’t so useful, but this is far from the case. And, given the opaque nature of podcast downloads, 50-ish % accuracy isn’t so surprising.

As mentioned earlier in this article, there’s a real lack of public visibility when it comes to download numbers, and also a lack of standardisation across different platforms in terms of what qualifies as a listen. For example, Apple Podcasts only recently updated its approach to automatic downloads—previously, episodes were counted as downloads simply because a user was subscribed, regardless of whether they listened. With recent changes, auto-downloads are now paused for inactive listeners, leading to more accurate download metrics

Rephonic also states that it uses OP3 (the Open Podcast Prefix Project to verify the number of listeners per episode. While OP3 is an excellent open-source data platform, hosts need to proactively choose to participate, so it’s likely only a small fraction of Rephonic’s 3 million shows actually do this. 

I contacted the founder of Rephonic for comment, and what he said reflects much of what we saw in the test:

“Rephonic’s download estimates are primarily designed to help PR professionals, brands, and marketers quickly gauge the approximate size of a podcast’s audience. The main value is giving users a reliable way to prioritise which podcasts they approach for guest appearances, sponsorships, or general research – without needing exact listener figures upfront.

We always encourage users of the platform to reach out to the host directly to get the real numbers.

While we believe the estimates are useful and the best in the industry (and it’s a really hard problem to estimate these figures because podcasting is so fragmented!) they do tend to be worse for (a) non-English language shows and (b) shows where the audience is quite small, say 1000 listeners per episode or less, because there is usually less data that we can collect for those.

We also provide lots of other real metrics (not estimates) like Spotify followers, ratings, reviews and chart rankings. Those help you get a better overall understanding of the size of the podcast’s audience.”

— James Potter, Founder of Rephonic

Rephonic: A Place for Estimates, A Need for Real Data

Despite our test results suggesting that Rephonic’s download estimates are only 52% accurate overall, the tool’s usefulness, particularly at scale, can not be disputed.

Our recommendation would be to treat Rephonic estimates as a starting point but not a final figure, and validate numbers with actual data whenever possible.

Nothing beats real numbers when making key decisions like which shows to collaborate with, but Rephonic provides an excellent kick-off point to help you narrow your searches and focus on the shows most suited to your goals.