Jun 5, 2016

I recently moved to a new place (because the rental market is cray cray here). Despite how annoying it is to move around a benefit of the new place is that it has fiber so I wasted no time and got a connection from Telia.

I plugged in the ISP shipped router and while browsing through the admin interface I noticed an IPv6 address showed up. All excited I checked my devices but no one was getting v6 addresses assigned. The tools in the admin interface where also completely unable to understand IPv6 addresses and when running the diagnostics their router reported it failed to contact the IPv6 gateway.

Enabling IPv6 through 6rd

Slightly disappointed I unplugged hte ISPs router and plugged in my own. I use an Asus RT-AC66U with the Asuswrt-Merlin firmware. I enabled 6rd but nothing much happened. I figured they weren’t announcing the 6rd configuration optinos over DHCP and left it at that, with the intent to come back later armed with Google search results to get this to work.

I got to tweaking some other settings in the mean time to fix my throughput and accidentally clicked on the v6 tab. Much to my surprise all of a sudden it had configured IPv6 and I was getting a /64 available to use. Looks like all I had to do was wait a bit longer.

I told the router to start doing router advertisements and within seconds all my capable devices on my network had received an IPv6 address and were using it to report in. Great success!

Ping all the things

Once you get v6, what you gonna do? Well, run pings! So I pinged Google over IPv4 and IPv6 and to my surprise over IPv4 it took 2ms, over IPv6 32ms. Even for 6rd that seemed like a bit too much.

The traceroutes however revealed why I’m seeing this difference. Over IPv4 this is what an mtr looks like:

$ sudo mtr -4 google.com

                                                Packets
 Host                                         Loss%   Snt   Avg
 1. router.asus.com                            0.0%    10   1.1
 2. gw7-no10.tbcn.telia.com                    0.0%    10   2.4
 3. hy-peer4-link.se.telia.net                 0.0%    10   1.9
 4. s-b6-link.telia.net                        0.0%     9   2.4
 5. google-ic-301506-s-b2.c.telia.net          0.0%     9   3.5
 6. 213-155-151-186.customer.teliacarrier.com  0.0%     9   2.4

That’s rather curious. Even though we’re tracing to Google we seem to never leave Telia’s network. Over IPv6 we get a different story:

$ sudo mtr -6 google.com

 Host                                   Loss%   Snt   Avg
 1. 2001:2002:d542:df70::1               0.0%     4   1.4
 2. 2001:2000:4020:14::2                66.7%     4   2.1
 3. 2001:2000:4020:15::1                 0.0%     4   2.9
 4. s-b5.se.telia.net                    0.0%     4   3.1
 5. google-ic-314684-s-b5.c.telia.net    0.0%     3   3.0
 6. 2001:4860::1:0:26eb                  0.0%     3  32.9
 7. 2001:4860:0:1::3e5                   0.0%     3  32.3
 8. arn06s07-in-x0e.1e100.net            0.0%     3  31.0

It seems like at the edge Google and Telia are doing something more over IPv4. This is likely to be what is known as the Google Global Cache:

Google Global Cache (GGC) enables your company to optimize network infrastructure costs associated with delivering Google and YouTube content to your users by serving this content from inside your network.

Over IPv6 however it seems like we’re entirely bypassing GGC and going out and fetching the content ourselves from Google.

In browser experience it makes a difference though it’s barely noticeable. If I curl google.com over IPv4 it always takes about 10ms whereas the first request over IPv6 tends to be around 60ms, any subsequent request is just as speedy.

Once you’ve pinged Google, time to try someone else! Facebook was about 10ms faster over IPv6 than IPv4 and Netflix curiously dropped ping over v4 but was happy to reply over IPv6.

Akamai and Telia seem to do something like GGC too both over IPv4 and IPv6, it stays steady at 2ms. They seem to be doing this for many other CDNs too, like Fastly. This actually results in a notable speedup whenever I’m fetching things from PyPi and a bunch of other package repositories.

Lessons learned

This little round of ping and mtr proved to be far more interesting than I originally expected.

  • Though ping will give you an indication of latency there’s no guarantee that your connection makes it all the way out when things like content caching networks are involved. So take that latency ping gives you with a pinch of salt and check what you’re actually measuring
  • Content caches are actually pretty awesome for the end user/customer

On paper my old 500/50 connection with Com Hem should be much faster than the 100/100 I get through Telia and using speed checks indeed say so. However, Telia’s use of content caches and probably some additional benefits of being a sister of Telia Carrier, the Tier 1, results in a significantly faster internet experience.

network  ipv6  6rd