ip rule fwmark 0xfe, what is this used for?




cumulus@leaf$ ip rule show | grep mark
100: from all fwmark 0xfe lookup main


it is installed as part of cl-mgmtvrf do_start()

sudo ip rule del fwmark 254 lookup main prio 100 2>/dev/null 1>/dev/null


am i suppose to see a entry in mangle table mark some traffic to 0xfe?

I checked mangle table, it is empty, what is this entry used for?

4 replies

Hi Eric,

You are correct that rule is installed as part of cl-mgmtvrf package in the /etc/init.d/cl-mgmtvrf script. It is an experimental feature that allows the separation of the main routing table and the routing table for the management interface eth0. Similar to VRF but I wouldn't call it such from a traditional networking VRF definition.

Once the feature is enabled it creates another Linux routing table number 252 named mgmt. (To see the full list check: /etc/iproute2/rt_tables) The main table is number 254 hence why you see the fwmark to (0xfe) 254 for default lookups to use the main routing table.

The rule after the one you asked about (101: from all iif eth0 lookup mgmt) sets traffic lookups for eth0 to be in the mgmt table. So if you were to issue a ping by default it will use the main routing table if you specify the interface such as ping -I eth0 it will use the mgmt routing table.

To answer your second question you have to dig into the feature a little more in the execution script /usr/sbin/cl-mgmtvrf. There is a list of sys control settings to make the defaults happen for IPv4 and IPv6. So because of changing the system defaults you will not see any mangle rules associated with this.

Documentation for cl-mgmtvrf

Hope this helps.

Jason
thanks for reply,

however I looked at enable() in /usr/sbin/cl-mgmtvrf, it didn't mention anything about 0xfe marking, which of following mark the pkt?

cumulus@leaf$ cat /etc/sysctl.d/cl-mgmtvrf.conf
net.ipv4.default_iif=2
net.ipv6.route.default_iif=2
net.ipv4.ipv4_pick_iif_based_on_saddr=1
net.ipv6.ipv6_pick_iif_based_on_saddr=1
net.ipv4.icmp_errors_use_inbound_ifaddr=1

what I thought about this rule is, when pkt is marked by firewall as 0xfe, they should be looked in main routing table first, but who is doing marking? if not mangle table.

To be more specific the marking you are asking about is done by the actual application. That is one way that applications that are not able to specifically bind to an interface or IP address are able to use the mgmt routing table. As an example think of Apache using the listen directive you can specify the address to bind the application. If you specify the IP on eth0 it will use the mgmt routing table. If for some reason Apache did not have the ability to bind to an address then we would need the application to mark traffic to use the mgmt routing table.

In newer Linux kernel versions and coming in Cumulus Linux 3.0 we have contributed VRF support into the kernel where the inner workings of this feature will change. The user experience will remain the same but all of these details will change.

Is there a specific use case that you are trying to solve where cl-mgmtvrf is needed or do you need to modify the ip rules? I might be able to answer the use case question better because like I mentioned all of these internal workings are going to change.

make sense now, if this is just a hook for another application to use, thanks.