nat sample MAC address not correct

[vincentmli]

Hi Asim

I might may have misconfigured something for the NAT sample, so here again is my test example, let me know if I missed anything :)

client <---->dpdk0 mOS dpdk1 <---->server

client (IP 10.0.0.7, MAC 00:1b:21:50:bc:38 )
mOS (dpdk0 10.0.0.9 MAC a0:36:9f:a1:4d:6c, dpdk1 10.0.1.9 MAC A0:36:9F:A1:4D:6D)
server (IP 10.0.0.8, MAC 00:15:60:0e:3d:0a )

1, nat mos.conf

mos {
forward = 1

    #######################
    ##### I/O OPTIONS #####
    #######################
    # number of memory channels per socket [mandatory for DPDK]
    nb_mem_channels = 4

    # devices used for MOS applications [mandatory]
    netdev {
            dpdk0 0x00FF
            dpdk1 0x00FF
    }

    #######################
    ### LOGGING OPTIONS ###
    #######################
    # NICs to print network statistics per second
    # if enabled, mTCP will print xx Gbps and xx pps for RX and TX
    stat_print = dpdk0 dpdk1

    # A directory contains MOS system log files
    mos_log = logs/

    ########################
    ## NETWORK PARAMETERS ##
    ########################
    # This to configure static arp table
    # (Destination IP address) (Destination MAC address)
    arp_table {
    }

    # This is to configure static routing table
    # (Destination address)/(Prefix) (Device name)
    route_table {
    }

    # This is to configure static bump-in-the-wire NIC forwarding table
    # DEVNIC_A DEVNIC_B ## (e.g. dpdk0 dpdk1)
    nic_forward_table {
            dpdk0 dpdk1
    }

2, add server MAC in client and client MAC in server

client: #arp -i p1p1 -s 10.0.0.8 00:15:60:0e:3d:0a

server:

arp -i eth0 -s 10.0.0.7 00:1b:21:50:bc:38

arp -i eth0 -s 10.0.0.9 A0:36:9F:A1:4D:6D ( Note I had to add dpdk1 MAC for NAT IP 10.0.0.9 on server)

3, run nat

./nat -i 10.0.0.9

4, run curl on client

curl http://10.0.0.8/

client side capture:

09:42:47.809727 00:1b:21:50:bc:38 > 00:15:60:0e:3d:0a, ethertype IPv4 (0x0800), length 74: 10.0.0.7.37680 > 10.0.0.8.80: Flags [S], seq 1791631497.....

09:42:47.809987 a0:36:9f:a1:4d:6c > a0:36:9f:a1:4d:6d, ethertype IPv4 (0x0800), length 74: 10.0.0.8.80 > 10.0.0.7.37680: Flags [S.], seq 3146701557, ack 1791631498, ...

Note the problem here is:

SYN+ACK from mOS has source MAC of dpdk0 and destination MAC of dpdk1, instead of source MAC of dpdk0 and destination MAC of client, thus this result in SYN+ACK dropped by client.

4 server side capture:

09:45:56.065765 a0:36:9f:a1:4d:6d > 00:15:60:0e:3d:0a, ethertype IPv4 (0x0800), length 74: 10.0.0.9.1027 > 10.0.0.8.80: Flags [S], seq 1791631497...........

09:45:56.065865 00:15:60:0e:3d:0a > a0:36:9f:a1:4d:6d, ethertype IPv4 (0x0800), length 74: 10.0.0.8.80 > 10.0.0.9.1027: Flags [S.], seq 3146701557, ack 1791631498.............

server side source MAC and destination MAC looks ok for both SYN and SYN+ACK.

[vincentmli]

I traced the code to ForwardIPPacket in core/src/ip_out.c

it appears line 55 assigned the saved dpdk1 MAC to haddr instead of client MAC. this would work in server side, but it could break in client side, maybe add a server or client side check condition to assign the MAC from pkt_ctx and go to fast_tx ? Am I on the right track?

43 /*----------------------------------------------------------------------------*/
 44 void
 45 ForwardIPPacket(mtcp_manager_t mtcp, struct pkt_ctx *pctx)
 46 {
 47         unsigned char * haddr;
 48         struct iphdr *iph;
 49         uint32_t daddr = 0;
 50
 51         if (g_config.mos->nic_forward_table != NULL) {
 52                 pctx->out_ifidx =
 53                         g_config.mos->nic_forward_table->nic_fwd_table[pctx->in_ifidx];
 54                 if (pctx->out_ifidx != -1) {
 55                         haddr = pctx->p.ethh->h_dest;
 56                         TRACE_INFO("go to fast_tx pctx->out_ifidx %d pctx->in_ifidx %d tcp source %d\n", pctx->out_ifidx, pctx->in_ifidx, pctx->p.tcph->source);
 57                         goto fast_tx;
 58                 }
 59         }

[a-jamshed]

Hi Vincent,

The problem that I see in your setup is that the globally shared (public) IP address of the NAT should be different from the IP address of any of the middlebox IP addresses (nat -i 10.0.0.9 <-- Please pick a different IP address. You can try assigning it. e.g. 10.0.0.10. In this case you should put the static ARP entry arp -i eth0 -s 10.0.0.10 00:1b:21:50:bc:38 in the server).

I think I should come up with a simpler method to set-up inline mode or automate the inline mode setup more. I will think of a way to make this more transparent.

[a-jamshed]

Vincent, I made a small update to the code which lets the mOS middlebox work fine in inline mode even if the interfaces are not assigned any IP addresses (reason mentioned in issue #1). When you run the script setup.sh, you can assign IP address 0.0.0.0 to both dpdk0 and dpdk1. When you run a mOS sample application, both the interfaces would read traffic in promiscuous mode and exchange traffic between each other.

[vincentmli]

excellent! it works. the ifconfig command does not like IP address 0.0.0.0, so the setup.sh script complains error, but I used ip command like ip addr add 0.0.0.0/0 dev dpdk0, it works, maybe setup.sh script should use ip command instead of ifconfig ?

diff --git a/setup.sh b/setup.sh
index cb684a6..aabe899 100755
--- a/setup.sh
+++ b/setup.sh
@@ -516,8 +516,10 @@ setup_iface_dpdk()
            fi
            echo "invalid IP address!" # continue
        done
-       echo "sudo /sbin/ifconfig dpdk$(($iter)) $ip_addr netmask 255.255.255.0 up"
-       sudo /sbin/ifconfig dpdk$(($iter)) $ip_addr netmask 255.255.255.0 up
+       echo "sudo /sbin/ip addr add $ip_addr/24 dev dpdk$(($iter))"
+       echo "sudo /sbin/ip link set dpdk$(($iter)) up"
+       sudo /sbin/ip addr add $ip_addr/24 dev dpdk$(($iter))
+       sudo /sbin/ip link set dpdk$(($iter)) up
        let "iter=$iter + 1"
     done
 }

[a-jamshed]

Thanks! I will make the update in my next commit.

[DragonmingLi]

There I run the epserver and epwget,but those machines can't interactive data. Could you tell me why or how to build the environment.