View source for Tech/Many NICs in the same subnet

I recently had the chance to work on a proof of concept at work and faced an interesting challenge: what does it take to be able to use more than one network interface connected to the same subnet? One might expect that once you've got the ip addresses from your local dhcp server all is good and you can start sending and receiving traffic. I've learned that's not the case.

In this article I want to take notes about what's necessary for such configuration to work. I haven't fully understood (yet) all the pieces involved so his page will end up also being a tracker for my learnings

== Charts ==
[[File:Many-nics-same-subnet.png|center|frame|scenario]]

== Quick and dirty notes ==


Let's say our subnet is has address <code>10.55.0.0/18</code> and our nics these addresses:

* eth0: 10.55.36.33
* eth1: 10.55.8.255
* eth2: 10.55.38.187

Then the necessary commands would be: <syntaxhighlight lang="shell" line="1">
# ==================================================================
# Block 1: tables creation

echo "10 eth0" >> /etc/iproute2/rt_tables
echo "11 eth1" >> /etc/iproute2/rt_tables
echo "12 eth2" >> /etc/iproute2/rt_tables

# ==================================================================
# Block 2: Add rules to tables

ip route add 10.55.0.0/18 dev eth0 src 10.55.36.33 table eth0
ip route add default via 10.55.0.1 dev eth0 table eth0

ip route add 10.55.0.0/18 dev eth1 src 10.55.8.255 table eth1
ip route add default via 10.55.0.1 dev eth1 table eth1

ip route add 10.55.0.0/18 dev eth2 src 10.55.38.187 table eth2
ip route add default via 10.55.0.1 dev eth2 table eth2

# ==================================================================
# Block 3: bind ip addresses to tables

ip rule add from 10.55.36.33/32 table eth0
ip rule add to 10.55.36.33/32 table eth0

ip rule add from 10.55.8.255/32 table eth1
ip rule add to 10.55.8.255/32 table eth1

ip rule add from 10.55.38.187/32 table eth2
ip rule add to 10.55.38.187/32 table eth2
</syntaxhighlight>

== Quick and dirty explanation ==
The block 1 creates three tables. Tables are sorted and consulted by (ascending, if I understood correctly?) order of priority. The priorities here are the 10, 11 and 12 numbers. Tables with low numbers comes first, tables with high priority numbers come later.

To give an idea, the default would be something like this:<syntaxhighlight lang="shell-session" line="1">
manu@astrolabio:~$ ip rule show
0:	from all lookup local
32766:	from all lookup main
32767:	from all lookup default
</syntaxhighlight>So adding tables with priorities 10/11/12 would mean that such tables are consulted after the <code>local</code> table and before the <code>main</code> and <code>default</code> tables.

Then we move to adding rules to tables (block 2)

(to be continued)

== Links and references ==

* [https://serverfault.com/questions/524054/simple-multihomed-linux-server-issue/736047#736047 <nowiki>[ServerFault] Simple Multihomed Linux Server Issue</nowiki>]
* https://lartc.org/howto/ - Linux Advanced Routing & Traffic Control: poorly aged, from a quick skim seems less good than it claims to be
** [https://lartc.org/howto/lartc.rpdb.html Policy routing section]
* [http://www.linuxhorizon.ro/iproute2.html Linux Advanced Routing mini how-to from linuxhorizon.ro]
* https://kindlund.wordpress.com/2007/11/19/configuring-multiple-default-routes-in-linux/
** Another person that had the same issue... Doesn't explain much more than other websites, but has some nice commentary and links
* http://www.policyrouting.org/
** in particular: http://www.policyrouting.org/PolicyRoutingBook/ ("Policy Routing With Linux by Matthew G. Marsh")
* http://www.policyrouting.org/iproute2-toc.html -- iproute2 docs

[[Category:Tech]]