IPv6 routing issues from VM to Host and Internet using ULAs
by peterdim from LinuxQuestions.org on (#4R57Z)
Hello, I have a host with a VM, connected to one IPv6 forwarding PC in the lab and a Firewall to the Internet.
I'm using a macvlan bridge. Here is what the setup script looks like:
Code:set -x
HWLINK=eno1
MACVLN=macvlan0
TESTHOST=www.google.com
# get network config
IP=$(ip address show dev $HWLINK | grep "inet " | awk '{print $2}')
IP6=$(ip address show dev $HWLINK | grep "inet6 " | head -n1 | awk '{print $2}')
NETWORK=$(ip -o route | grep $HWLINK | grep -v default | awk '{print $1}')
GATEWAY=$(ip -o route | grep default | awk '{print $3}')
NETWORK6=$(ip -6 -o route | grep $HWLINK | grep -v default | awk '{print $1}' | head -n1)
GATEWAY6=$(ip -6 -o route | grep default | awk '{print $3}')
# setting up $MACVLN interface
ip link add link $HWLINK $MACVLN type macvlan mode bridge
ip address add $IP dev $MACVLN
ip address add $IP6 dev $MACVLN
ip link set dev $MACVLN up
# routing table
# empty routes
ip route flush dev $HWLINK
ip route flush dev $MACVLN
ip -6 route flush dev $HWLINK
ip -6 route flush dev $MACVLN
# add routes
ip route add $NETWORK dev $MACVLN metric 0
ip -6 route add $NETWORK6 dev $MACVLN metric 0
# add the default gateway
ip route add default via $GATEWAY
ip -6 route add default via $GATEWAY6 dev $MACVLN metric 90These are the working (ping) and not working (no ping) routes using Unique Local Addresses:
Route.......IPv4....IPv6
VM-Lab.......yes....yes (disappears after a while, VM reboot fixes it somehow)
VM-Host......yes....no
VM-WWW.......yes....no
Host-Lab.....yes....yes
Host-WWW.....yes....yes
However there is VM-Lab and Host-VM connectivity with Local-Link Addresses (fe80::/64)
Here are the Host Routing Tables:
Code:$route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.3.127.1 0.0.0.0 UG 0 0 0 macvlan0
0.0.0.0 10.3.127.1 0.0.0.0 UG 100 0 0 eno1
10.3.0.0 0.0.0.0 255.255.0.0 U 0 0 0 macvlan0
10.3.0.0 0.0.0.0 255.255.0.0 U 100 0 0 eno1
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
$ route -n -6
Kernel IPv6 routing table
Destination Next Hop Flag Met Ref Use If
::1/128 :: U 256 1 0 lo
fdc8:c2cb:4586:cb11::f88a:1e90/128 :: U 100 1 0 eno1
fdc8:c2cb:4586:cb11::f88a:1e90/128 :: U 1024 1 0 macvlan0
fdc8:c2cb:4586:cb11::/64 :: U 100 17 0 eno1
fdc8:c2cb:4586:cb11::/64 :: UAe 256 2 0 macvlan0
fe80::/64 :: U 100 1 0 eno1
fe80::/64 :: U 256 1 0 macvtap0
::/0 fe80::6efd:b9ff:fe02:8223 UG 90 17 0 macvlan0
::/0 fe80::6efd:b9ff:fe02:8223 UG 100 1 0 eno1
::/0 fe80::6efd:b9ff:fe02:8223 UGDAe 1024 1 0 macvlan0
::1/128 :: UAn 0 11 0 lo
fdc8:c2cb:4586:cb11::f88a:1e90/128 :: UAn 0 18 0 eno1
fdc8:c2cb:4586:cb11::f88a:1e90/128 :: UAn 0 18 0 macvlan0
fe80::2453:9ff:fe3e:3f7a/128 :: UAn 0 11 0 macvlan0
fe80::5054:ff:fe65:2ae4/128 :: UAn 0 2 0 macvtap0
fe80::9670:2373:b9c8:c83/128 :: UAn 0 5 0 eno1
ff00::/8 :: U 256 18 0 eno1
ff00::/8 :: U 256 17 0 macvlan0
ff00::/8 :: U 256 12 0 macvtap0
::/0 :: !n -1 1 0 loVM routing tables:
Code:Kernel IPv6 routing table
Destination Next Hop Flag Met Ref Use If
::/0 :: !n -1 1 0 lo
::1/128 :: U 256 1 0 lo
fdc8:c2cb:4586:cb11::aa62:2eaa/128 :: U 256 1 0 br-data0
fdc8:c2cb:4586:cb11::/64 :: UAe 256 2 0 br-data0
fe80::/64 :: U 256 2 0 mgmt0
fe80::/64 :: U 256 1 0 br-data0
fe80::/64 :: U 256 1 0 data0
::/0 fe80::6efd:b9ff:fe02:8223 UGDAe 1024 2 0 br-data0
::1/128 :: Un 0 3 0 lo
fdc8:c2cb:4586:cb11::aa62:2eaa/128 :: Un 0 3 0 br-data0
fe80::/128 :: Un 0 3 0 mgmt0
fe80::/128 :: Un 0 3 0 br-data0
fe80::/128 :: Un 0 3 0 data0
fe80::5054:ff:fe65:2ae4/128 :: Un 0 3 0 br-data0
fe80::5054:ff:fe65:2ae4/128 :: Un 0 2 0 data0
fe80::6ca8:d2ff:fea6:d240/128 :: Un 0 4 0 mgmt0
ff00::/8 :: U 256 3 0 mgmt0
ff00::/8 :: U 256 2 0 br-data0
ff00::/8 :: U 256 1 0 data0
::/0 :: !n -1 1 0 lo
...
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.3.127.1 0.0.0.0 UG 100 0 0 br-data0
10.3.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-data0
Both VM and Host get a DHCPv6 address from fdc8:c2cb:4586:cb11::1, which is also used for IPv6 routing. Its routing table looks like this:
Code:Kernel IPv6 routing table
Destination Next Hop Flag Met Ref Use If
::1/128 :: U 256 1 0 lo
fdc8:c2cb:4586:cb11::/64 :: U 100 3 33 enp4s0
fdc8:c2cb:4586:cb11::/64 :: U 256 2 1 enp4s0
fdc8:c2cb:4586:cb12::/64 :: U 256 1 0 enp4s1
fdc8:c2cb:4586:cb12::/64 :: U 1024 1 0 enp4s1
fe80::/64 :: U 256 5 13876 enp4s1
fe80::/64 :: U 256 5 16661 enp4s0
::/0 fdc8:c2cb:4586:cb11::127:1 UG 1024 5176310 enp4s0
::1/128 :: Un 0 5 6 lo
fdc8:c2cb:4586:cb11::/128 :: Un 0 2 0 enp4s0
fdc8:c2cb:4586:cb11::1/128 :: Un 0 3 810 enp4s0
fdc8:c2cb:4586:cb12::/128 :: Un 0 2 0 enp4s1
fdc8:c2cb:4586:cb12::1/128 :: Un 0 3 22 enp4s1
fe80::/128 :: Un 0 2 0 enp4s0
fe80::/128 :: Un 0 2 0 enp4s1
fe80::6efd:b9ff:fe02:738f/128 :: Un 0 5 26829 enp4s1
fe80::6efd:b9ff:fe02:8223/128 :: Un 0 6174994 enp4s0
ff00::/8 :: U 256 5 31608 enp4s1
ff00::/8 :: U 256 5182280 enp4s0
::/0 :: !n -1 1 1 lofdc8:c2cb:4586:cb11::127:1 is the main router and firewall between our lab and the Internet. It's the same as 10.3.127.1.
I'm using a macvlan bridge. Here is what the setup script looks like:
Code:set -x
HWLINK=eno1
MACVLN=macvlan0
TESTHOST=www.google.com
# get network config
IP=$(ip address show dev $HWLINK | grep "inet " | awk '{print $2}')
IP6=$(ip address show dev $HWLINK | grep "inet6 " | head -n1 | awk '{print $2}')
NETWORK=$(ip -o route | grep $HWLINK | grep -v default | awk '{print $1}')
GATEWAY=$(ip -o route | grep default | awk '{print $3}')
NETWORK6=$(ip -6 -o route | grep $HWLINK | grep -v default | awk '{print $1}' | head -n1)
GATEWAY6=$(ip -6 -o route | grep default | awk '{print $3}')
# setting up $MACVLN interface
ip link add link $HWLINK $MACVLN type macvlan mode bridge
ip address add $IP dev $MACVLN
ip address add $IP6 dev $MACVLN
ip link set dev $MACVLN up
# routing table
# empty routes
ip route flush dev $HWLINK
ip route flush dev $MACVLN
ip -6 route flush dev $HWLINK
ip -6 route flush dev $MACVLN
# add routes
ip route add $NETWORK dev $MACVLN metric 0
ip -6 route add $NETWORK6 dev $MACVLN metric 0
# add the default gateway
ip route add default via $GATEWAY
ip -6 route add default via $GATEWAY6 dev $MACVLN metric 90These are the working (ping) and not working (no ping) routes using Unique Local Addresses:
Route.......IPv4....IPv6
VM-Lab.......yes....yes (disappears after a while, VM reboot fixes it somehow)
VM-Host......yes....no
VM-WWW.......yes....no
Host-Lab.....yes....yes
Host-WWW.....yes....yes
However there is VM-Lab and Host-VM connectivity with Local-Link Addresses (fe80::/64)
Here are the Host Routing Tables:
Code:$route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.3.127.1 0.0.0.0 UG 0 0 0 macvlan0
0.0.0.0 10.3.127.1 0.0.0.0 UG 100 0 0 eno1
10.3.0.0 0.0.0.0 255.255.0.0 U 0 0 0 macvlan0
10.3.0.0 0.0.0.0 255.255.0.0 U 100 0 0 eno1
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
$ route -n -6
Kernel IPv6 routing table
Destination Next Hop Flag Met Ref Use If
::1/128 :: U 256 1 0 lo
fdc8:c2cb:4586:cb11::f88a:1e90/128 :: U 100 1 0 eno1
fdc8:c2cb:4586:cb11::f88a:1e90/128 :: U 1024 1 0 macvlan0
fdc8:c2cb:4586:cb11::/64 :: U 100 17 0 eno1
fdc8:c2cb:4586:cb11::/64 :: UAe 256 2 0 macvlan0
fe80::/64 :: U 100 1 0 eno1
fe80::/64 :: U 256 1 0 macvtap0
::/0 fe80::6efd:b9ff:fe02:8223 UG 90 17 0 macvlan0
::/0 fe80::6efd:b9ff:fe02:8223 UG 100 1 0 eno1
::/0 fe80::6efd:b9ff:fe02:8223 UGDAe 1024 1 0 macvlan0
::1/128 :: UAn 0 11 0 lo
fdc8:c2cb:4586:cb11::f88a:1e90/128 :: UAn 0 18 0 eno1
fdc8:c2cb:4586:cb11::f88a:1e90/128 :: UAn 0 18 0 macvlan0
fe80::2453:9ff:fe3e:3f7a/128 :: UAn 0 11 0 macvlan0
fe80::5054:ff:fe65:2ae4/128 :: UAn 0 2 0 macvtap0
fe80::9670:2373:b9c8:c83/128 :: UAn 0 5 0 eno1
ff00::/8 :: U 256 18 0 eno1
ff00::/8 :: U 256 17 0 macvlan0
ff00::/8 :: U 256 12 0 macvtap0
::/0 :: !n -1 1 0 loVM routing tables:
Code:Kernel IPv6 routing table
Destination Next Hop Flag Met Ref Use If
::/0 :: !n -1 1 0 lo
::1/128 :: U 256 1 0 lo
fdc8:c2cb:4586:cb11::aa62:2eaa/128 :: U 256 1 0 br-data0
fdc8:c2cb:4586:cb11::/64 :: UAe 256 2 0 br-data0
fe80::/64 :: U 256 2 0 mgmt0
fe80::/64 :: U 256 1 0 br-data0
fe80::/64 :: U 256 1 0 data0
::/0 fe80::6efd:b9ff:fe02:8223 UGDAe 1024 2 0 br-data0
::1/128 :: Un 0 3 0 lo
fdc8:c2cb:4586:cb11::aa62:2eaa/128 :: Un 0 3 0 br-data0
fe80::/128 :: Un 0 3 0 mgmt0
fe80::/128 :: Un 0 3 0 br-data0
fe80::/128 :: Un 0 3 0 data0
fe80::5054:ff:fe65:2ae4/128 :: Un 0 3 0 br-data0
fe80::5054:ff:fe65:2ae4/128 :: Un 0 2 0 data0
fe80::6ca8:d2ff:fea6:d240/128 :: Un 0 4 0 mgmt0
ff00::/8 :: U 256 3 0 mgmt0
ff00::/8 :: U 256 2 0 br-data0
ff00::/8 :: U 256 1 0 data0
::/0 :: !n -1 1 0 lo
...
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.3.127.1 0.0.0.0 UG 100 0 0 br-data0
10.3.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-data0
Both VM and Host get a DHCPv6 address from fdc8:c2cb:4586:cb11::1, which is also used for IPv6 routing. Its routing table looks like this:
Code:Kernel IPv6 routing table
Destination Next Hop Flag Met Ref Use If
::1/128 :: U 256 1 0 lo
fdc8:c2cb:4586:cb11::/64 :: U 100 3 33 enp4s0
fdc8:c2cb:4586:cb11::/64 :: U 256 2 1 enp4s0
fdc8:c2cb:4586:cb12::/64 :: U 256 1 0 enp4s1
fdc8:c2cb:4586:cb12::/64 :: U 1024 1 0 enp4s1
fe80::/64 :: U 256 5 13876 enp4s1
fe80::/64 :: U 256 5 16661 enp4s0
::/0 fdc8:c2cb:4586:cb11::127:1 UG 1024 5176310 enp4s0
::1/128 :: Un 0 5 6 lo
fdc8:c2cb:4586:cb11::/128 :: Un 0 2 0 enp4s0
fdc8:c2cb:4586:cb11::1/128 :: Un 0 3 810 enp4s0
fdc8:c2cb:4586:cb12::/128 :: Un 0 2 0 enp4s1
fdc8:c2cb:4586:cb12::1/128 :: Un 0 3 22 enp4s1
fe80::/128 :: Un 0 2 0 enp4s0
fe80::/128 :: Un 0 2 0 enp4s1
fe80::6efd:b9ff:fe02:738f/128 :: Un 0 5 26829 enp4s1
fe80::6efd:b9ff:fe02:8223/128 :: Un 0 6174994 enp4s0
ff00::/8 :: U 256 5 31608 enp4s1
ff00::/8 :: U 256 5182280 enp4s0
::/0 :: !n -1 1 1 lofdc8:c2cb:4586:cb11::127:1 is the main router and firewall between our lab and the Internet. It's the same as 10.3.127.1.