The Men & Mice Blog

The RIPE-javik logs: Day 3

Posted by Carsten Strotmann on 5/23/19 7:11 AM

ripe day 3carsten@menandmice:~$ cat ~/ripe/ripejavik-day3.txt | blog-publish

Wednesday was a hands-on kind of day at RIPE 78. Attending the OpenSource Working Group yielded lots of interesting information, and we’ve interviewed some RIPE 78 participants for our upcoming podcasts. (Watch this space!)

Open Source Working Group

The Working Group started with two different solutions for a similar task, both very interesting.

The first presentation was about building Network Labs using OpenSource tools. Wolfgang Tremmel from German Internet Exchange DE-CIX reported his experiences with using Docker Linux containers to build a training lab for BGP training. He used a Docker container with FRRouting (an open source routing software rooted on Quagga) and exposed the terminal command line of each container via ttyd to the net.

In this configuration, the training participants only need a web browser to access the lab machines. The lab can either run local in the training room or on some cloud service. Getting IPv6 to work with Docker can be challenging, and Wolfgang ran into problems there. I personally would recommend podman or systemd-nspawn as an IPv6 friendly alternative to Docker.

In the same presentation slot, Sander Steffann talked about his experiences with his router labs. While the focus in Wolfgang’s training is the routing protocol itself (and less the routing software used), Sander has a lab that allows the students to try out real commercial router software such as Cisco, Juniper, or Microtik.

Sander is using the GNS3 project that is able to emulate or virtualize commercial router hardware to run the router firmware unmodified. While GNS3 itself is open source, the router firmware needed is not. Emulation is costly, especially for more modern router machines, so his lab needed very powerful machines. Sander combined GNS3 with a nice, web-based management system that would display instructions and information about the routing labs.


The second presentation was from Max Rottenkolber, who was talking about his open source project, a high-performance VPN solution for x86_64 machines. This Site-to-Site VPN software is called Vita and is built upon Snabb, a high-performance network stack running in userspace.

While it is running on top of Linux, it does not use the Linux network stack, instead accessing the network cards hardware from userspace directly. While doing this, Snabb can be used to create applications that are very optimized for network throughput. Vita (and Snabb) are mainly built with the Lua programming language, and the code is compiled to optimized x84_64 machine code using a Just-in-Time (JIT) compiler. Because Vita is bypassing the kernel, it can fully control the hardware and squeeze maximum performance out of the system.

The project is still in development, and the medium-term goal is to be able to encrypt 100 Gbps line-rate traffic (with 60byte packets). Because VPN gateways running Vita are dedicated servers, and because all networking is done in userspace, almost no kernel syscalls are used and the system's performance is not affected by the mitigations for the Intel CPU problems such as Spectre, Meltdown, and others.

Lightning talks

In the lightning talks session, Sander Steffann was asking the RIPE community for help with the NAT64check website he operates. The service allows users to enter the URL of a particular website, and run tests over IPv4, IPv6, and NAT64 in order to check:

  • whether the website is actually reachable in each case,
  • whether identical web pages are returned,
  • and whether all the resources such as images, stylesheets, and scripts load correctly.

Sander is looking for people who are interested in joining the team that keeps this service running.


Next, Maria Jan Matejka from CZ.NIC presented an update on new developments around the BIRDv2 open source routing daemon. BIRD is a dynamic routing daemon running on Linux, BSD and other systems and implements many routing protocols like BGP, OSPF, Babel and more.

The new version has custom route attributes, a filter benchmark tool and will become faster filter in the future. There was also a "dirty hack" presented on how to auto-reload a route as an RPKI change.


The working-group closed with a discussion on industry hackathons, with presentations on both experiences from the IETF hackathons and the RIPE hackathons.

More coverage (And a podcast!)

RIPE 78 is now in full swing, with conference events and lots of off-site discussions, sight-seeing, and social happenings. We’ll continue our daily briefings throughout the week, but we’re also working on a more in-depth project: a podcast digging deeper into all things DNS, DHCP, and IPAM.

Make sure you follow Men & Mice’s social media channels and blog for the announcement!

Topics: Open Source, RIPE 78, VPN, workshop, routing

The RIPE-javik logs: Day 2

Posted by Carsten Strotmann on 5/22/19 8:31 AM

ripe day 2

carsten@menandmice:~$ cat ~/ripe/ripejavik-day2.txt | blog-publish

The second day of RIPE 78 started with the plenary, and three presentations on the topic of Distributed Denial of Service (DDoS).

DDoS

DDoS attacks are an increasing risk on the Internet. Mattijs Jonker from the University of Twente explained how DDoS attacks work. His research has revealed that many businesses have all their Internet services (website, mailserver, etc.) in a single network. In case of a DDoS attack, all services are impacted. He counted 31 thousand websites, 3.5 thousand mailservers, and 323 DNS servers that are on a single network and would suffer in case of an attack. An alternative IP address from a different network (autonomous system/AS) would make the services more resilient.

Matthias Wichtlhuber from the German Internet Exchange DE-CIX found that DDoS attackers only use certain protocols for their amplification attacks:

  • unspecified (Port 0)
  • NTP (Port 123)
  • LDAP (Port 389)
  • DNS (Port 53)
  • Chargen (Port 19)
  • Memcache (Port 11211)

Filtering these ports (in transport networks) will stop most DDoS attacks. The problem is that most ISPs cannot do fine-grained filtering. Most can only filter on networks or IP addresses, which blocks all traffic from or to a certain machine. DE-CIX has developed a new fine-grained black-holing system for DDoS attacks that is currently in beta testing.

Koen van Hove, also from the University of Twente, presented the DDOS clearinghouse: a project to collect data of DDoS attacks in a central place. The aim is to be able to research DDoS attacks and develop fast responses to them. The DDoS clearinghouse collects network measurements, identifies DDoS attacks across networks with unique fingerprints, and stores this data in a database (DDoSDB). From the database, attack information and metadata can be retrieved to help users feed fingerprint signatures into their network systems to stop DDoS attacks.

DNS

After the morning break, the main topic was DNS. David Huberman from ICANN discussed the root server system. After talking about the history of the DNS root server system, he explained that there has been no process so far for selecting new root server operators.

With over 1.120 root server instances in the world, 340 of which are in the RIPE region, the root server system is stable and there is currently no need to add additional root server operators beyond the 12 that run the 13 logical root-server addresses. ICANN is not working on a defined governance model for the root server system.

OpenINTEL

Next on the stage was Roland van Rijswijk (NLnet Labs) presenting the OpenINTEL project he has contributed to. OpenINTEL is a massive active measurement system that sends 218 million DNS queries per day from several vantage points on the Internet, resolving a defined set of DNS names. The results will be collected in a big database (Big Data helps to get research funds these days), which so far contains 3.1 trillion results since the start of the project in 2015.

The OpenINTEL system allows researchers to search for various kinds of interesting data: parent-child TTL mismatches, distribution of authoritative DNS-Servers in different AS networks, or even silly things stored in DNS TXT records (like funny IPv6 addresses or private cryptographic keys). The project can be found at https://openintel.nl

KSK roll

Like always, Geoff Huston (APNIC) delivered a highly entertaining talk, this time about the KSK roll in October 2018.

Officially, there was no impact seen for the DNSSEC validating resolvers. But some operators, like EIR in Ireland, have missed all notices about the roll in the two years leading up to it and failed to change the trust anchor of their DNS resolvers which lead to a full-day outage of their DNS resolver services. Other smaller operators were affected as well, some of which fixed the issue by disabling DNSSEC. All except two have re-enabled DNSSEC after fixing their DNS resolver configurations. Geoff also noted that the DNSSEC trust state signaling (RFC 6975 and RFC 8145) does not work reliably to detect broken KSK rolls in the root zone.

Migration from IPv4 to IPv6

In "Get Ready for Mixed World: Economic Factors Affecting IPv6 Deployment", Brenden Kürbis and Milton Mueller from the Georgia Institute of Technology talked about the economics behind network migration from IPv4 to IPv6.

The problem of IPv6 is that it is not possible to switch off IPv4 right away. Instead, IPv4 must be kept enabled for some amount of time (dual stack deployment). The cost generated due to IPv4 depletion will stay, the cost of introducing IPv6 will come on top. Only after some years will the cost benefits be visible. Depending on the growth pattern of the company and the networks, the first cost savings can appear as early as 4 to 10 years. Larger companies will have more benefit from IPv6, while smaller companies will not see economic benefits. In the following Q&A session, people from the audience challenged some of the assumptions in the research that generated this report.

DNS flag day

In the last DNS talk of the day, Petr Špaček from CZ.NIC and Ondřej Surý from ISC gave some insight into the DNS flag day in February 2019.

DNS vendors (Bind 9, Knot, PowerDNS, Unbound, and others) and large DNS resolver operators (Google, Cloudflare, Quad9, etc.) disabled workarounds for broken EDNS implementations. The workarounds were developed to help with DNS servers on the internet that had faulty implementations of the DNS protocol. However, because the workarounds existed, the operators of these faulty servers had no motivation to fix their systems. The cost of developing and maintaining the workarounds fell to the vendors of the DNS products.

For the February 2019 flag day, there was an estimated breakage of 5.68% of all DNS servers. Two large DNS operators were responsible for 66% of this breakage. The flag day was considered a success, as the pressure generated compelled the operators to fix their systems, and no other significant breakage was reported on that day.

Motivated by the success of this first flag day, the DNS server vendors plan another in 2020. No exact date has been set at the moment. On the next flag day, new DNS software releases will change the default settings for EDNS buffer size from today's 4096 bytes to a value around 1220 bytes. The goal is to prevent fragmentation of IP packets, which is known to be broken in some networks and can be a security risk. For this change, authoritative servers and DNS resolvers must be able to operate over TCP in addition to UDP. The main problem is misconfigured firewalls that block DNS over port 53/TCP.

The flag day website will be updated with detailed information about the date and will include online tests so that DNS administrators can test their systems.

More tomorrow!

RIPE 78 is a busy event, with much more going on than we were able to report here. Do visit the session archives to check the other presentations - there are plenty more good talks to dig into. We’ll be back with more RIPE coverage tomorrow!

Topics: IPv6, IPv4, DNS, DDoS, RIPE 78, OpenINTEL, KSK roll

The RIPE-javik logs: Day 1

Posted by Carsten Strotmann on 5/21/19 6:35 AM

ripe day 1

carsten@menandmice:~$ cat ~/ripe/ripejavik-day1.txt | blog-publish

The first day of RIPE 78 started with the welcome talk by RIPE chair Hans Petter Holen and the hosts of the meeting: Icelandic sea-cable provider Farice and RHnet, the university network provider of Iceland.

It was impressive to hear that Iceland has already achieved 80% Fiber-to-the-Home installation and will have 100% by 2025. In terms of Internet speed, Iceland is only second after Norway (for mobile Internet) and Singapore (for fixed line Internet).

After short talks by Andrew Sullivan, the president and CEO of ISOC, the Internet Society (the organisation that facilitates Internet Standards Processes such as those developed by the IETF, amongst other things) and Benno Overeinder for the RIPE Program Committee, probably the best known Icelander in the Internet community took to the stage: Ólafur Guðmundsson, inventor of DNSSEC and currently CTO of Internet accelerator Cloudflare.

IPv4 volatility

Ólafur’s topic was the volatility of IP addresses. While at the beginning of the Internet IP addresses were stable over a long time and could be used to identify a machine, this is not the case today. Mobile devices switch networks constantly, always getting new addresses. A smartphone can have more than 10 different IP addresses over the course of a single day, roaming across different mobile providers and wireless networks.

As Ólafur described, IP addresses cannot reliably be used to identify machines anymore. Still, many service providers, companies, and government agencies do that all the time, for

  • blacklisting,
  • geo-location,
  • to calculate online advertising prices by placing a value to the user using the IP address,
  • or to find the nearest content server.

IPv4 address brokerage makes the situation worse. Because there are no free IPv4 addresses left and many companies have not yet switched over to IPv6, IPv4 addresses are valuable (>20 US$ per address) and are for sale. When sold, these addresses change location, but providers of location databases cannot keep up with the changes and the databases become outdated and full of wrong data.

Roaming and routing

In the next talk, Alo Safari Khatouni spoke about the implication of mobile phone roaming in Europe. In his research, he has specifically looked into how IP data is being routed in roaming situations, and when the difference in latency and bandwidth impacts a roaming user’s experience.

He found no content discrimination (i.e. that certain data is being throttled during a roaming situation), but latency was certainly higher. Mobile network operators route the roaming traffic back to the home network, where it is then routed to the Internet. This means that for a customer of an US-based mobile network operator (MNO) who is in Iceland and trying to access a website in Iceland (to look up the weather conditions - vital information in Iceland!), the network data will be routed through the US MNO’s network. It’s no surprise that this is slower than staying in Iceland and accessing the data directly.

In the Q/A session following this talk, IPv6 evangelist Jan Zorz mentioned that he also experiences IPv6 Path-MTU-Discovery issues while being inside one of the MNO networks in Iceland. It may be that possibly someone is blocking ICMPv6 on the network.

ATLAS

In the first lightning talk, Christopher Amin from the RIPE NCC explained some of the security safety belts RIPE has built into the RIPE ATLAS system.

RIPE ATLAS is a network measuring networks, where ATLAS probes are distributed all around the world. These probes can be remotely controlled by researchers to make traffic measurements on the Internet from different points of the worldwide network. However, some probes are operated by private persons in their home networks and might be located in countries where access to certain Internet content is prohibited by law. Law enforcement might not be able to tell apart access from a real Internet device from that of an ATLAS probe.

To resolve this, RIPE has built in a host of security and safety measures to limit or block the access to sensitive Internet content, but also wants to add support for DNS-over-HTTPS (DoH) measurements to the ATLAS system. The problem here is that DNS-over-HTTPS looks, by design, like HTTPS traffic generated by a web browser. From the outside, one cannot see if the content requested is a website or DNS data. Enabling DoH measurements without restrictions can introduce risks for RIPE ATLAS probe operators. Christopher asked the RIPE community about their comments and how this challenge can be solved.

The second issue Christopher brought up was the use of EDNS (Extended DNS) options in ATLAS experiments. Researchers would like to test new or unspecified option values against DNS servers on the Internet, but this can lead to unexpected behaviour, even crashing DNS servers (if the DNS server software is not of high quality, which sometimes happens if network equipment vendors write their own implementations of DNS). There’s a risk in probing these EDNS options, but Christopher is not sure exactly how big the risk is.

IPv6-only

In the last lightning talk of Day One of RIPE 78, security expert Enno Rey presented his insights from an IPv6-only WLAN study that his company ERNW has conducted for a client. They found that mobile apps, especially on Apple’s iOS "just work.” (Which is no big surprise, as each app is tested by Apple to make sure it works as expected in an IPv6-only environment.)

ERNW found some applications that did not work out of the box and needed manual fixes, like the popular game "Fortnite" and its associated Epic Game Launcher. An XMPP (Jabber) component in the game only asked for IPv4 addresses (and the domain name has no IPv6 AAAA addresses), so this was naturally failing in a network without IPv4. Some other applications like Discord worked, but had some loss of functionality.

More tomorrow

This concludes our first report from RIPE 78. Check out our guide to both the event and the city, and stay tuned for more tomorrow.

Topics: IPv6, DNS, RIPE 78, ATLAS

The Men & Mice Guide to RIPE-javik

Posted by Men & Mice on 5/15/19 7:40 AM

RIPE 78 is barely a week away! We feel it's our duty, both as locals to the city and as sponsors to the event, to compile a guide to help you make the most  of your stay.

ripe

What to attend at RIPE

You're coming to attend sessions and talks at RIPE, so let us start there. There'll be an excellent lineup of speakers, making it hard to choose. May we suggest starting with Carsten Strotmann?

Carsten has been supporting customers with Unix and PC/Windows networks in Germany and abroad for more than 27 years. His specialties are Unix systems, DNS, DNSSEC and IPv6 security. He's a trainer in the field of DNS/DHCP/IPv6/Linux/Unix security for Internet Systems Consortium (ISC), Linuxhotel and Men & Mice. He also is the author of various articles on IT security topics in specialist magazines.

Carsten will give two talks at RIPE:

  1. Unwind, a Validating DNS Recursive Stub-Resolver: a short introduction on what unwind(8) is, and how this always-running, validating DNS recursive nameserver on OpenBSD can help to secure DNS name resolution for mobile devices and laptops in hostile public networks.
  2. Overview of the DNS Privacy Software landscape: new DNS privacy protocols have sparked a number of new open source software tools that make use of DNS-over-TLS and DNS-over-HTTPS - however, functionalities and software quality differ greatly. This talk will give an overview of available tools, the functions they provide and their availability on popular operating systems and also a brief look on missing pieces in the DNS privacy software landscape.

Apart from the Plenary and BoF (Bird of a Feather) Sessions and Tutorials, RIPE78 features no less than 10 Working Group sessions, on DNS, IPv6, IoT, Open Source, Anti-Abuse and more.

Talks and sessions not to be missed include:

  •        Tutorial by Enno Rey on IPv6 Security for Enterprise Organisations (Monday, 20 May)
  •        The plenary session dedicated to current DDoS threads and how to mitigate them (Tuesday, 21 May),
  •        High Performance Traffic Encryption on x86_64 (Max Rottenkolber), part of the Open Source Working Group Agenda (Wednesday, 22 May)
  •        IPv6 reliability measurements (Geoff Huston) and Large-scale Deployment of IPv6-enabled Wi-Fi Hotspots (Enno Rey) – both on Thursday, 23 May
  •        Revisiting the Root (David Hubermann, ICANN), Long-Term Active Measurements for DNS Research, and That KSK Roll (Geoff Huston) – all on Tuesday, 21 May

Diversity engagement: Women in Tech Panel at RIPE78

Sponsored by Netflix with additional support from Men & Mice and WomenTech Iceland, RIPE78 will also host a Women in Tech diversity panel discussion on the 21st of May, on which our own Paula Gould will join panelists from GRID, WuXi NextCode and Lady Brewery.

Paula is the Head of Brand & Communications for Men & Mice, and has worked with IT companies for over 15 years on go-to-market, growth, and brand strategies. She founded WomenTechIceland, and has been deeply involved in notable international women-in-business and women-in-tech initiatives for two decades.

women in tech ripe

After hours: making the most of RIPE-javik

Good times don't stop at the end of the official schedule. We’ve compiled seven useful tips to make your stay during RIPE78 as pleasant as possible. (And also financially sensible.)

  1. Leave your umbrella: OK, maybe not if it’s Cisco, but if it’s one of those hand-held thingies, you may want to just let it go. There can be unexpected gusts of winds, and unless you want to re-enact Mary Poppins, there are other better (and warmer) ways to get around.
  2. Entertainment: If you want to have a good laugh at the end of a long day of DNS and IP addresses, the Secret Cellar does comedy in English every evening in a cellar on Lækjargata, smack in the middle of downtown Reykjavík.
  3. Non-alcoholic beverages: An indisputable must in almost every Icelander’s daily life is coffee: the stronger, the blacker, the better. And if you’re feeling out of sorts on your visit, why not coffee and a cat? The Cat Café, home of outstanding coffee and four-legged creatures, offers you just that.
  4. Alcoholic beverages: If coffee is not your thing, beer easily rates as the other staple Icelandic beverage. Icelanders have caught up quickly since the lift of the beer ban in 1989: these days, everyone and their brother is making their own. Micro-breweries are literally on every other corner and these bars offer an excellent selection.
  5. Hands-on fun: Beyond food and drinks, there's much else to be enjoyed in Reykjavik. You definitely can’t go wrong with karaoke Wednesdays at Sæta Svínið (The Sweet Pig) Gastropub, or Monday’s Ping Pong Tournament in "Miami". (The one on Hverfisgata, not Florida. But complete with tropical décor and cocktails to match.)
  6. Volcanic gifts: Iceland's so rife with geothermal water that we use it to heat not only our homes, but also our pavements and driveways. We love our water. And you haven’t really been to Iceland until you’ve shot the breeze with the locals in a ‘hot pot’ at one of the many public pools scattered around the country. (Note: to enter the pool area, you are required to shower naked and wash all the right spots thoroughly and with soap.)
  7. More about water: You're quite safe to drink the water from the tap in your hotel and don't hesitate to ask for tap water in restaurants. It’s pure, tastes fantastic and doesn’t cost you a krona. Bring a refillable water bottle for refreshing hydration no matter where you go.

See you at RIPE78 (come say hello to us in person or on social media) and have a great stay in Iceland!

Topics: DNS privacy, RIPE 78, Women in Tech

Men & Mice welcomes RIPE 78 to Reykjavik

Posted by Men & Mice on 5/9/19 10:42 AM

We are developers who build software for network infrastructure people. And not just any network infrastructure, but the most fundamental parts: DNS, DHCP, and IP address management.

For that reason, and for many more, we’re more than excited to welcome RIPE 78 to our home in beautiful Reykjavik, Iceland in May this year. (Forecast is balmy and warm expected to showcase all seasons every 15 minutes or so. 🙃)

What is RIPE?

RIPE NCC is one of the five Regional Internet Registries (RIRs) dealing with the network of networks: the internet. An independent, not-for-profit membership organisation, RIPE NCC serves Europe, Central Asia, Russia and West Asia and provides internet resource allocations, registration services and coordination activities that support the operation of the Internet globally.

Formed in 1992, RIPE NCC now supports more than 12,000 members in 76 countries in its service region.

ripe map

RIPE NCC holds two General Meetings a year, where members convene to discuss a wide range of subjects related to keeping the internet up and running.  

Men & Mice @ RIPE 78

This year, for the first time ever, RIPE NCC is coming to Iceland. It’s a match made in heaven: RIPE members’ knowledge and insight meet Iceland’s’ long-running expertise in all things computing, including networks, cloud technology, and software development.

Of course, this being not only our home ground geographically, but also professionally, Men & Mice is a proud sponsor of RIPE 78 and will be participating on a number of levels.

Long-time readers of our blog will recognize the name of DNS expert Carsten Strotmann, who has previously published RIPE reports, and who has worked with Men & Mice for many years on a number of initiatives (and from time to time hosts webinars, blog posts and training sessions with us).

This time around, Carsten will not only give two talks on behalf of Men & Mice at RIPE78, but also provide you with updates on what happens at RIPE on a daily basis.

Here’s a small taste of what hot topics are waiting to be discussed at RIPE 78:

  • current DDoS threads and how to mitigate them
  • review of the 2018 DNSSEC KSK Roll in the Root Zone and the February 2019 EDNS "Flag Day"
  • IPv6 reliability
  • large-scale deployment of IPv6-enabled Wi-Fi hotspots
  • high-performance traffic encryption
  • roundtable discussion on the role of open-source in industry hackathons
  • tutorial on IPv6 security for enterprise organizations

Diversity engagement: Women in Tech Panel at RIPE78

Sponsored by Netflix with additional support from Men & Mice and WomenTechIceland, RIPE78 will also host a Women in Tech diversity panel discussion on the 21st of May, on which Paula Gould, our Head of Brand & Communications will join panelists from GRID, WuXi NextCode and Lady Brewery.  Learn more here: https://ripe78.ripe.net/diversity/women-in-tech-session/

women in tech ripe

Topics: RIPE 78, Women in Tech

Men & Mice @ Cisco Live 2019: Hybrid and Multicloud Transition

Posted by Men & Mice on 5/3/19 12:01 PM

 

Men & Mice @ Cisco Live 2019

menmice_clus2019_2234Like with previous years, Men & Mice will return to Cisco Live in San Diego, June 10-13, 2019.

Cisco solutions are a staple of the networking world. Their hardware and software services are present in almost every enterprise, creating security and efficiencies for almost every aspect of network management. They’re also a treasure trove of research and know-how, which is why so many of today's large-scale organizations continue to rely on Cisco (and Men & Mice). 

Cisco Live events are a must-attend for anyone making decisions about network innovations and transformations. 

Network growth, fueled in part by the Internet of Things and Edge computing that permeates nearly everything in our world (and soon, out of this world), is dependent on innovations to both on-premise and cloud solutions to increase resilience and uptime.  Cloud's accelerated adoption rates can likely be attributed, in part, to its maturity, meeting the strict regulations of enterprise-grade businesses, enabling implementation of nimble hybrid and multicloud infrastructure strategies to be fast-tracked.

Adding a highly compatible network overlay software, opens significant opportunities for network managers and architects to use the best-in-class services that are right for them, while leveraging the native features in both their on-premise and cloud solutions, gaining more out of their investments and positioning their network for future transition and innovation. 

Join us for our Best Practices Think Tank Session

Monday,  June 10 2019 at 3:30pm PDT

This year at Cisco Live, Paul Terrill, Men & Mice’s Director of Sales Operations, North America, will also take to the Think Tank stage to discuss New Best Practices for Future-Ready Hybrid and Multicloud Network Strategies. This session will explore new best practices and the advantages to adapting hybrid network strategies to take advantage of service-native features in all IP infrastructure solutions, whether on-premise, cloud or multicloud.

Beyond our talk, Cisco Live is also a great opportunity for us to connect with customers, old and new, and catch up on discussions with the most prominent minds of our industry.

Click here to find out more, or book a time at Cisco Live that suits you for meeting up with sales engineers from Men & Mice. Alternatively, just stop by booth #2234, any day from June 10th to June 13th, and stock up on great networking. (And nice goodies!)

 

Topics: Cisco Live

The ABC's of DNS: a select glossary from the Men & Mice training archives - Part 1

Posted by Men & Mice on 4/26/19 9:43 AM

As you’ve probably discovered by now, we have an honest passion for teaching and training. For the past 20 years, Men & Mice has been offering DNS and BIND courses across the globe. Always updated and always practical, from the start we've constructed classes to address real world challenges and solve problems that our students actually face.


Beyond this series, you can also catch us in person (outside of the training courses): we’re really proud to be sponsoring RIPE78 in Reykjavik next month!

In addition to the diversity programming, we’ll also be giving two talks, presented by Carsten Strotmann, about DNS privacy and Unwind.


And the onslaught of new challenges never stops. Public and private networks. Cloud and on-prem resources. Hybrid and multiclouds. Privacy, security, efficiency.

Being on top of our game means constantly learning.

In this new series, we'd like to give you a small taste of the Men & Mice training courses. Organized alphabetically, we'll cover a glossary of select tips, tricks, and trivia that will deepen your understanding of DNS and BIND.

Without further ado, let's get started - we have a whole alphabet to cover.

A is for "anonymizing IP addresses in logfiles"

Anonymizing IP addresses is a handy trick to know, with (DNS) privacy features often requested and businesses becoming increasingly liable for traffic to and from their servers.

ipv6loganon is a Linux command line tool for anonymizing IP addresses in HTTP server logfiles. By default your webserver (be it Apache, nginx, or something else) logs every connection.This is useful for diagnosing connection issues or find malicious actors - but during normal operations it's also a liability from a privacy standpoint.

You can type man ipv6loganon in your server terminal to see all the options. Run it as a cron job or automate some other way.

B is for "BIND features roundup"

BIND is a fantastic suite of software. Whether you consciously use it or not, it's one of the most fundamental pieces in almost any network puzzle (that's why our most popular training course is titled "DNS and BIND").

Lot of people are surprised just how many tools BIND offers. For example:

  • dig is the Swiss Army Knife of network tools. So much so, that we'll be giving it its own entry at the letter 'D' in the next post. In the meantime, read man dig in your terminal, and learn to love it.
  • delv can be used to verify DNSSEC trust. It's as easy as typing delv +v www.domain.com.
  • named-checkconf -z can be used to test manual changes to DNS zonefiles.
  • dnstap is a faster alternative to query logging. (During the training courses we go deep into how to use it.)

BIND also comes with a host of security features like DNS cookies, Response Policy Zones, Response Rate Limiting, and more. The DNSB-W and DNSB-A courses cover these in detail.

C is for "catalog zones"

C is not just for cookies, but also: catalog zones. Catalog zones are special DNS zones, used to quickly propagate DNS zones from master to slave servers. Slave servers use catalog zones to recreate member zones, and if any changes occur "upstream", they're also synced across slaves using the catalog zones.

Use catalog zones for redundancy, so if your slave servers go out of commission for any reason, you can resume normal operations by quickly spinning up backups.

Want to learn more?

In this DNS glossary series, we focus on just a handful of concepts in each post. Bite-sized, they're but the tip of the iceberg. Our training program is where all of these concepts come to exist in the right context - and you get to try your hand at putting newly learnt skills in action.

  • If you’re new to DNS, we offer the DNS & BIND Fundamentals (DNSB-F) course. It’s part of the DNS & BIND Week (DNSB-W) and serves as a shorter introduction to the world of DNS and BIND.
  • If you’re already familiar with the basics, the full five-day DNS & BIND Week (DNSB-W) course takes you deeper into DNS, including a heavy emphasis on security, stopping just short of DNSSEC (for which we offer a separate course).
  • And if you're looking for even more, we offer the DNS & BIND Advanced (DNSB-A) program, getting into the deep end of things.

Check out our training calendar for 2019, and reach out to us with any questions. 

Topics: IT best practices, DNS training, RIPE 78

How to explain Network Management to relatives and friends over the holiday (GIFs)

Posted by Greg Fazekas on 4/18/19 8:15 AM

 

Life isn’t always easy for network managers and architects. The C-suite is constantly demanding more efficiency and smoother operations, at low cost. Your colleagues are asking for more user-friendly policies and services. And you have to keep up with an ever-changing landscape of technology (infrastructure sprawl) and its ripples into your domain. (Pun absolutely intended.) Uptime and security are everything. Then, you constantly have to explain to people what you actually do for a living.

Over the holiday weekend, there’s a good chance, in addition to being asked to fix someone’s computer, phone or tablet, you’ll be asked “what is it you do again?”

How do you illustrate what you do? Maybe it’d be a lot easier to explain being a fireman, astronaut, or brain surgeon? We've pulled together some helpful GIFs to make this conversation more efficient. 

 

 

via GIPHY

Enter Ralph Breaks the Internet. (holiday movie idea!)

If ever there was an indicator that networking has permeated our everyday lives it’s an animated family movie centered around it. Some concepts are so fundamental to modern life that we aren’t even consciously thinking about them anymore.

ICYMI: Released in the fall of 2018, Ralph Breaks the Internet provided the subtext and pop culture references we all needed, while depicting basically your everyday.

 

 

via GIPHY

From the moment Ralph and Vanellope slide down the wire, to the hilarious popup advertisers and the wonderfully subtle depiction of DNS —  most every aspect of your job comes to life in a tangible, easy-to-explain-to-relatives way, every aspect of the complexities of networking in a network-driven world.

 

via GIPHY

DNS isn’t specifically named in the movie, but there are plenty of references. Knowsmore, although depicted as a search engine, certainly has his business rooted (see what we did there?) in being a DNS server of sorts. For instance, when Vanellope and Ralph decide to go to Ebay, they were automatically routed to their destination.

Ralph Also Teaches us DDoS

But if you had to showcase just one thing about your work, it could be how you have to prevent DDoS attacks against your company’s network — essentially how you have to be the hero against a million or billion Ralphs.

Explaining DNS to anyone, particularly to people not in networking (and let’s face it, even some people IN networking don’t really get DNS), is easier when you can point to the colorful transport GIFs from an animated movie. Grasping the concept of a botnet or a crippling DDoS attack is more memorable when it’s an ever-replicating bunch of clones of a funny character like Ralph. And you do get malware by clicking unscrupulous links.

tumblr_pgyjbw4Wy01s40634o6_540

via 'Disney' on Blogberth

DDoS is essentially the towering Ralphzilla of mindless objects with a single goal. Exploiting vulnerabilities in web servers, they overwhelm the system with a repeated, single query. Not only is this meant to disrupt user experience, more sinister objectives may be in play, such as bringing down firewalls.

We’ve talked a lot on this blog about DNS education. Education for both professionals — training, if you will — and for everyone, in order to understand new technologies and challenges affecting our businesses. Knowing why and how insecure networks are a liability and how important it is to defend against malicious attacks that can wreck the internet is useful for everyone.

The movie exaggerates concepts to either serve the plot or get a laugh. But the foundation for showcasing how networks and the internet work (or occasionally don’t work) is solid.

tumblr_p9ta2xlRol1tfb0neo2_540

via 'Disney' on Blogberth

Come this holiday (provided you don’t have to work because of some real-world Ralph threatening your company’s network) sit down at the family dinner, armed with GIFs and your favorite streaming service, to explain what you do and why.

And since it is a holiday weekend, here's a blog about all of the Ralph Breaks the Internet Easter Eggs. 

Image credits:Not a Real Company Productions and Disney via Giphy and Blogberth

Topics: DDoS, Disney

DNS Privacy: DNS-over-HTTPS

Posted by Men & Mice on 4/16/19 5:24 AM

DNS-over-HTTPS (DoH for short) is a standard developed by the IETF (under the RFC 8484 designation) to solve privacy concerns in DNS communication.

DNS Privacy: a primer

As we’ve talked about before, using DNS has been done in cleartext: the queries and responses between both the end user (applications, or stub resolvers, such as a browser) and the (first-hop) DNS resolver and the resolver and the DNS nameserver(s) are unencrypted by default.

While DNSSEC extensions were developed early on, they only added response integrity and not privacy. As IETF states, “either privacy was not considered a requirement for DNS traffic or it was assumed that network traffic was sufficiently private.

In recent years, however, that changed. Privacy has become a central concern and addressing it has spawned numerous solutions, such as DNS-over-HTTPS (DoH).

DNS-over-HTTPS

DoH conducts DNS operations using secure http urls and mapping DNS queries and responses into http exchanges, using default media formatting types.doh

While DoH uses existing protocols for communication, the IETF emphasizes that “[The] described approach is more than a tunnel over HTTP.” Aligned with existing http features, DNS servers and clients supporting DoH are called ‘DoH server' and ‘DoH client’ respectively, as they can be used for more than only DNS.

While DNS-over-HTTPS and DNS-over-TLS are colloquially used as different protocols, because DoH uses https it also includes TLS security. The key difference between DoH and DoT is the manner in which DNS operations are conducted.

DoH in action

DoH clients are configured with a URI template containing the url structure for DNS resolution. The client then uses a GET or POST method to send an encoded DNS query.

DNS-over-HTTPS uses standard https traffic (via port 443) to communicate. Because DNS communication is done through standard https methods and resides within https traffic, overhead for DoH is low.

DoH challenges

DNS-over-HTTPS is a newer protocol than DNS-over-TLS. DoH has had less testing and research than DoT, but because it aligns with https as an underlying transport protocol, it is less susceptible to issues other than those associated with https itself.

DoH, though still young, has successfully leveraged the existing ecosystem of native web applications and APIs. It can create more efficient (and private!) communications with DNS.

Arguments against DNS-over-HTTPS (in its current form) stem more from operational considerations. Whereas DoT can be controlled because of its use of a single and unique port, DoH is almost impossible to control or filter.

DNS privacy (DoH, as well as DoT and other solutions) is a good representation of the operational shifts network managers and architects face. DNS-over-HTTPS in particular is a solution born from a public networking mindset. Traditional corporate network operations that are increasingly dependent on cloud services and experience an influx of connected devices through IoT and BYOD, have to re-adjust.

But DNS privacy also means that the opportunity for corporate network managers and architects is more pertinent than ever before. DoH, DoT, and other solutions are young and still forming. Whereas the question before centered around ‘adoption’ of protocols, these new technologies offer a chance to ‘influence’. Ongoing participation in the conversation and debate over the merits and shortcomings of each is necessary.

Additionally, pilot programs, particularly those run in regulated, corporate environments, are invaluable to both the developers of DNS privacy solutions as well as the network managers and architects who will be charged with implementing it.

Topics: DNS privacy, DNS-over-HTTPS

DNS privacy: DNS-over-TLS

Posted by Men & Mice on 4/10/19 11:05 AM

DNS-over-TLS (DoT for short) is a standard developed by the IETF (under the RFC 7858 designation) to solve privacy concerns in DNS communication.

DNS Privacy: a primer

As we’ve talked about before, until recently,  DNS has been done in cleartext: the queries and responses between both the end user (applications, or stub resolvers, such as a browser) and the (first-hop) DNS resolver and the resolver and the DNS nameserver(s) are unencrypted by default.

While DNSSEC extensions were developed early on, they only added response integrity and not privacy. As the IETF states, “either privacy was not considered a requirement for DNS traffic or it was assumed that network traffic was sufficiently private.

In recent years, however, that changed. Privacy has become a central concern and addressing it has spawned numerous solutions, such as DNS-over-TLS.dot

DNS-over-TLS

DoT approaches privacy by encrypting DNS queries and responses between entities (predominantly between the stub resolver and the first hop resolver) using TLS (Transport Layer Security).

DoT uses a standard port (853) to initiate and accept DNS queries. It is possible to use a mutually agreed different port, but it is not the default. Once the connection is made, a TLS handshake is attempted, and after authentication the encrypted DNS communication can commence.

DNS servers supporting DoT are not accepting unencrypted data on the designated port, neither during session initiation, nor after a failed TLS authentication.

DoT overhead

Computers are powerful and efficient, but not without limits. DNS-over-TLS adds latency to DNS operations that needs to be accounted for and minimized.

DNS clients are required to adhere to a certain field length (two octets) and it is recommended to keep established, but idle, connections alive to the server. Another way to minimize latency is to pipeline multiple queries over the same TLS session. In this case, it’s the DNS client’s responsibility to match responses to queries, as they may arrive and be answered out of order.

Keeping established connections alive helps distribute the connection setup costs. Misconfigured handling of idle connections can lead to denial of service issues.

Flavors of DoT

DNS-over-TLS can be used in various ways. The IETF standard identifies opportunistic and Out-of-Band Key-Pinned privacy profiles.

Opportunistic privacy profile means the client recognizes a TLS-enabled DNS resolver and attempts to use it. If it successfully validates it, DNS-over-TLS may be used, but isn’t mandatory and the client can fall back to non-encrypted DNS.

Out-of-Band Key-Pinned privacy profile is usable where the trust between stub and recursive resolvers is already established. Enterprise DNS is one good example. With this profile, DNS clients authenticate servers by a set of (previously distributed) SPKI Fingerprints.

DoT pros and cons

DNS-over-TLS addresses privacy, but not the security of DNS operations. It is important to note that DNSSEC and DoT are not mutually exclusive, but rather compatible protocols that complement each other.

DoT is a straightforward protocol, and fairly easy to implement. TLS authentication is a mature, trusted, and well-maintained technology for encryption. But DNS-over-TLS also presents a number of challenges and concerns.

Attacks against TLS itself, such as protocol downgrade, affect DNS-over-TLS. DNS resolvers offering DoT have to be aware and be patched against TLS vulnerabilities. DNS clients can, in order to defend against person-in-the-middle attacks, discard cached data from a server stored in cleartext.

DoT isn’t fully protected against traffic analysis and SNI leaks. (Although it is in constant development to patch these vulnerabilities.) Split horizon DNS, where the DNS response may be different based on the source of the query, is also known to experience issues when used with DoT.

Network managers for both private networks and public services need to learn more about DNS privacy, DoT (and DoH and other implementations), and the solutions, and challenges, they present for their work. Education about these protocols is also important for end users — both for owning their privacy and to avoid issues resulting from unintentionally harmful configurations brought to a network.

DoT, DoH, and other protocols are in constant development, offering ways to influence their evolution. All network managers and architects, whether they’re running public or private infrastructures, should participate in pilot programs to discover and best voice and address their challenges and requirements.

Topics: DNS-over-TLS, DNS privacy

Why follow Men & Mice?

The Men & Mice blog publishes educational, informational, as well as product-related material for everyone and anyone interested in IP Address Management, DNS, DHCP, IPv6, DNSSEC and more.

Subscribe to Email Updates

Recent Posts

Posts by Topic

see all