Telekom and Cloudflare

03. Dec. 2024

My servers are connected to domains via a Cloudflare tunnel and thus accessible from the Internet. I noticed that access to my websites has become extremely slow. However, the problem only occurred from time to time. I could not determine exactly who was responsible for the slowdown. However, I found that the problem only occurs when accessing from an Internet connection provided by Deutsche Telekom.¹

After some research, it seems that there are problems between Cloudflare and Deutsche Telekom, and for this reason, Internet traffic is routed in such a way that it does not take the shortest route to the servers, but is first routed across the Atlantic and then back again.

I was able to easily verify this fact with traceroute. In order to reach a website via a connected domain, the (DNS) requests have to be routed accordingly, and how they are routed becomes visible with traceroute:

traceroute zangs.com      
traceroute: Warning: zangs.com has multiple addresses; using 104.21.64.1
traceroute to zangs.com (104.21.64.1), 64 hops max, 40 byte packets
 1  fritz.box (192.168.1.1)  5.130 ms  4.936 ms  6.522 ms
 2  p3e9bf29f.dip0.t-ipconnect.de (62.155.242.159)  15.971 ms  10.540 ms  13.028 ms
 3  nyc-sb6-i.nyc.us.net.dtag.de (62.154.5.202)  105.114 ms  102.245 ms  171.182 ms
 4  nyc-sb6-i.nyc.us.net.dtag.de (62.154.5.202)  104.956 ms  102.469 ms  328.916 ms
 5  80.156.160.213 (80.156.160.213)  112.465 ms  106.346 ms  108.509 ms
 6  if-ae-0-2.tcore3.njy-newark.as6453.net (216.6.90.14)  166.028 ms  280.382 ms  110.772 ms
 7  66.198.70.2 (66.198.70.2)  122.046 ms  237.385 ms  104.912 ms
 8  162.158.61.109 (162.158.61.109)  105.668 ms
    162.158.61.113 (162.158.61.113)  116.404 ms  133.956 ms
 9  104.21.64.1 (104.21.64.1)  244.904 ms  107.535 ms  107.440 ms

Here you can clearly see that the Internet gateway forwards to t-ipconnect.de which routes to nyc.us.net.dtag.de, which according to the URL is in New York City. This results in a significant delay of more than 100 ms, and makes accessing my website zangs.com correspondingly slower.

What makes things even stranger is that my website gerfficient.com is set up with Cloudflare Pro and is redirected from the gateway to m.de.net.dtag.de, which, according to the URL, is in Munich²:

traceroute gerfficient.com
traceroute: Warning: gerfficient.com has multiple addresses; using 104.26.15.172
traceroute to gerfficient.com (104.26.15.172), 64 hops max, 40 byte packets
 1  fritz.box (192.168.2.1)  5.557 ms  6.467 ms  5.761 ms
 2  p3e9bf29f.dip0.t-ipconnect.de (62.155.242.159)  14.157 ms  14.897 ms  357.228 ms
 3  m-ef2-i.m.de.net.dtag.de (62.153.181.22)  11.980 ms  11.637 ms  11.254 ms
 4  80.150.168.185 (80.150.168.185)  16.521 ms  17.300 ms  695.404 ms
 5  cloudflare-gw.cr0-muc1.ip4.gtt.net (141.136.100.98)  12.903 ms  15.582 ms  29.041 ms
 6  104.26.15.172 (104.26.15.172)  15.122 ms  12.466 ms  16.446 ms

And a check of the above, but from a different location that is also connected to the Internet via Deutsche Telekom, leads to the same result:

mtr -r gerfficient.com
Start: 2024-12-03T10:36:38+0100
HOST: 215f4efa5a56                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 172.19.0.1                 0.0%    10    0.1   0.1   0.1   0.1   0.0
  2.|-- 192.168.1.1                0.0%    10    1.0   1.1   1.0   1.5   0.2
  3.|-- p3e9bf208.dip0.t-ipconnec  0.0%    10    5.7   5.8   5.4   6.5   0.3
  4.|-- nyc-sb6-i.NYC.US.NET.DTAG  0.0%    10   96.4 111.9  96.4 151.9  20.4
  5.|-- 80.156.160.213             0.0%    10  144.3 109.4  94.9 144.3  14.7
  6.|-- if-ae-0-2.tcore3.njy-newa  0.0%    10  100.3 100.1  99.6 100.8   0.4
  7.|-- 66.198.70.2                0.0%    10  109.8 106.0 101.9 113.4   4.5
  8.|-- 162.158.61.113             0.0%    10  124.7 116.2 102.3 147.6  16.4
  9.|-- 188.114.96.3               0.0%    10  116.4 113.2 102.1 154.6  17.1
mtr -r zangs.com      
Start: 2024-12-03T10:38:04+0100
HOST: 215f4efa5a56                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 172.19.0.1                 0.0%    10    0.1   0.1   0.1   0.1   0.0
  2.|-- 192.168.1.1                0.0%    10    1.1   5.6   0.9  28.2   9.7
  3.|-- p3e9bf208.dip0.t-ipconnec  0.0%    10   37.2  13.9   5.0  48.9  15.7
  4.|-- m-ef2-i.M.DE.NET.DTAG.DE   0.0%    10    8.1   8.1   7.7   9.2   0.4
  5.|-- 80.150.168.185             0.0%    10   13.5  14.5  12.4  23.9   3.4
  6.|-- cloudflare-gw.cr0-muc1.ip 70.0%    10    7.8   8.4   7.3  10.1   1.5
  7.|-- 104.26.14.172              0.0%    10    7.0   7.5   7.0   8.5   0.4

This slowdown has been occurring since January 2024, according to other Cloudflare users:

ICMP history Quelle: Cloudflare Community (falke1337)³

And now that I've switched to a different Internet provider, the connection is much more stable and faster:

traceroute zangs.com
traceroute: Warning: zangs.com has multiple addresses; using 104.21.48.1
traceroute to zangs.com (104.21.48.1), 64 hops max, 40 byte packets
 1  192.168.1.1 (192.168.1.1)  2.136 ms  1.962 ms  1.437 ms
 2  10.19.6.67 (10.19.6.67)  13.850 ms
    10.19.6.66 (10.19.6.66)  9.935 ms
    10.19.6.67 (10.19.6.67)  8.631 ms
 3  xe-0-1-13.core-mbb.komro.net (37.156.85.228)  8.196 ms
    37.156.85.230 (37.156.85.230)  10.971 ms
    xe-0-1-13.core-mbb.komro.net (37.156.85.228)  9.619 ms
 4  ae1.core-ow.komro.net (37.156.85.180)  13.086 ms
    et-0-0-1.pe-muc-equinix01.komro.net (37.156.85.189)  15.052 ms
    ae1.core-ow.komro.net (37.156.85.180)  10.618 ms
 5  de-cix-munich.as13335.net (185.1.208.9)  13.629 ms
    et-0-0-1.pe-muc-equinix01.komro.net (37.156.85.189)  10.753 ms
    de-cix-munich.as13335.net (185.1.208.9)  12.972 ms
 6  de-cix-munich.as13335.net (185.1.208.9)  11.868 ms
    104.21.48.1 (104.21.48.1)  17.115 ms
    de-cix-munich.as13335.net (185.1.208.9)  13.027 ms

Gerfficient