If you’ve ever compared VPN speed claims across review sites, you’ve probably noticed the numbers rarely agree. One site says a provider averages 480 Mbps; another says the same provider barely breaks 200 Mbps on the same server. The gap usually isn’t dishonesty — it’s methodology. Here’s exactly how we test speed at Goliry, and why we built our process the way we did.
Why a Single Speed Test Isn’t Good Enough
A single speed test, run once, tells you almost nothing reliable. Network conditions fluctuate constantly — your ISP’s local congestion, the VPN server’s current load, even the specific speed-test server you happen to connect to, all shift from minute to minute. A provider could look excellent or terrible purely by bad luck on test timing. That’s simply not a fair or useful basis for a recommendation people are going to act on.
Our Core Testing Principles
1. Multiple test locations, not one
We test from several different physical locations with different baseline ISP connections — not just one office with one fiber line. A VPN that performs beautifully from a single gigabit connection in one city might behave very differently for someone on a 100 Mbps cable connection in a completely different region. Testing from a variety of starting conditions gives a much more representative picture of what real users will actually experience.
2. Repeated testing across time of day
Because server load fluctuates dramatically between peak and off-peak hours, we run tests at multiple points throughout the day and across multiple days — including evenings and weekends, when server congestion tends to be highest. A provider’s average score reflects a realistic mix of “good” and “bad” moments, not a cherry-picked best-case scenario.
3. Baseline measurement immediately before and after
Every VPN test is bracketed by a non-VPN baseline speed test on the exact same device and connection, run immediately before and immediately after the VPN test. This lets us calculate a percentage of retained speed rather than a raw number, which is far more meaningful — a raw “150 Mbps” score is meaningless without knowing whether the baseline was 160 Mbps or 900 Mbps.
4. Multiple protocols, tested separately
We never average across protocols and call it “the VPN’s speed.” Every provider is tested individually on each protocol it offers — WireGuard, OpenVPN (UDP and TCP separately), and IKEv2 where available — because, as we’ve documented extensively, protocol choice alone can swing results by 30% or more.
The Metrics We Actually Measure
| Metric | Why It Matters |
|---|---|
| Download speed | Determines streaming quality, download times, general browsing feel |
| Upload speed | Critical for video calls, cloud backups, content creators |
| Ping / latency | Determines responsiveness for gaming and real-time applications |
| Jitter | Variability in latency — high jitter causes choppy calls and lag spikes even when average ping looks fine |
| Packet loss | Dropped data that has to be resent, silently killing throughput |
| Connection stability | Whether the tunnel holds steady over a sustained 30+ minute session |
Nearby vs Long-Distance Server Testing
We deliberately test each provider on both a nearby server (same country or region) and a long-distance server (a different continent) because these represent two very different real-world use cases: someone using a VPN simply for privacy and security while browsing normally, versus someone specifically trying to appear as though they’re located somewhere far away. A provider can excel at one and struggle at the other, and lumping both scenarios into a single score would hide that distinction from readers who need it.
Sustained Load Testing, Not Just Snapshots
A ten-second speed test captures a brief snapshot, but real usage — a two-hour movie, a long file transfer, a multi-hour gaming session — puts sustained pressure on a connection over time. We run extended sessions specifically to catch problems that only appear after several minutes: gradual throttling, connection drops, or servers that perform well initially but degrade under continued load.
Controlling for What We Can’t Eliminate
No testing methodology can fully eliminate every variable — the broader internet is a shared, constantly shifting system. What we can do is control everything within our reach and be transparent about what remains outside it:
- Same testing hardware across all providers in a given comparison round
- Same underlying ISP connection for any single comparison batch
- Testing windows spread across enough time to smooth out one-off anomalies
- Re-testing any surprising outlier result before publishing it, to rule out a fluke
Why This Level of Detail Matters to You
A recommendation built on a single test run on a good day, from one location, on one protocol, isn’t a recommendation — it’s a coin flip dressed up as data. When we tell you a provider “retains roughly 90% of baseline speed on WireGuard from a nearby server, dropping to around 65% on a long-distance connection,” that number reflects dozens of individual test runs across multiple conditions, not a lucky snapshot. It’s the difference between a number that sounds impressive in a headline and a number you can actually plan your streaming setup, work-from-home connection, or gaming rig around.
What We’re Still Improving
Testing methodology is never finished. We continue to expand the number of physical testing locations we use, add new protocols as they gain adoption, and refine how we weight sustained-load results against quick-snapshot results in our final scoring. If you ever see a Goliry speed result that doesn’t match your own experience, we genuinely want to hear about it — real-world reports from readers with different ISPs, regions, and hardware are one of the most valuable feedback loops we have for catching blind spots in our own lab conditions.
How We Handle Outliers and Anomalies
Every large testing dataset contains outliers — a single test run that comes back dramatically better or worse than everything around it. Rather than including or excluding these automatically, every outlier gets manually reviewed before it factors into a published score. Sometimes an outlier reveals something genuinely useful, like a provider’s server briefly going down for maintenance or an unusual local network issue on our end. Other times, it turns out to be a real and repeatable result — in which case we re-test multiple additional times specifically to confirm whether it’s a consistent pattern worth reporting or a genuine one-off fluke. Numbers that can’t be reproduced on a second and third attempt don’t make it into our published averages.
Cross-Referencing Against Independent Tools
We don’t rely on a single speed-testing tool for our measurements. Different speed test services occasionally show slightly different results due to differences in their own server infrastructure and testing methodology, so we cross-reference results across multiple independent testing tools during each round, rather than trusting any single service’s numbers in isolation. When results from different tools diverge significantly, that discrepancy itself becomes useful information, sometimes pointing to a testing-tool-specific quirk rather than a genuine VPN performance issue.
How Scores Translate Into Our Written Reviews
The raw numbers from our testing process feed into the specific, concrete language you see in our written reviews — we deliberately avoid vague descriptors like “fast” or “great speeds” without backing them up with an actual percentage of retained baseline speed and the specific conditions that number reflects. When you read a Goliry review stating a specific speed retention percentage, that figure is traceable directly back to this testing process, not a marketing summary pulled from a provider’s own claims.
Transparency About Our Limitations
We test from a defined, though expanding, set of physical locations and standard consumer-grade hardware configurations. We don’t currently test from every possible country, every possible ISP, or every possible device configuration that exists — that would be an effectively infinite testing matrix. What we aim for instead is a representative, honest sample large enough that our published results generalize reasonably well to typical users, while being upfront that individual results can and will vary based on your specific location, ISP, and hardware.
Frequently Asked Questions
How often do you re-test VPN providers?
Provider infrastructure changes over time — new servers, protocol updates, network upgrades — so we periodically re-run our full testing process rather than treating any single round of results as permanent.
Do VPN providers know when you’re testing them?
No. We test as ordinary paying or trial customers, under the same conditions any regular user would experience, specifically to avoid any possibility of preferential treatment during testing.
Why do your numbers sometimes differ from other review sites?
Differences in testing locations, dates, protocols tested, and methodology all contribute to variation across review sites. We publish our methodology openly so you can weigh our numbers with full context rather than blind trust.
Do you accept provider input on testing results before publishing?
Providers are never given the opportunity to review or influence a speed result before it’s published. We may reach out to a provider afterward if a result seems anomalous, purely to rule out a technical issue on our end, but the published figures reflect our own independent measurements throughout.
The Bottom Line
Good VPN speed testing isn’t about running one test and publishing a big number. It’s about layering enough tests, across enough conditions, closely enough to how people actually use these tools day to day, that the final result holds up under real-world scrutiny. That’s the standard we hold every review to — and it’s why our numbers sometimes look a little less flattering than a provider’s own marketing claims, but a lot more trustworthy.
Leave a Reply