Openstack, neutron, cannot connect external network
I followed the Icehouse doc for a 3 node environment setup my openstack nodes are built as VMs using libvirt on CentOS
http://docs.openstack.org/icehouse/install-guide/install/yum/content/basics-networking-neutron.html
-controller-node : 10.0.0.11 (management network)
-network-node : 10.0.0.21 (management network), 10.0.1.21 (data network)
-compute-node : 10.0.0.31 (management network), 10.0.1.31 (data network)
-external network : 192.168.125.0/24
-demo network : 172.30.1.0/24
On my network node, the external NIC is configured without an IP address as per the documentation. (PROMISC = "yes")
DEVICE="eth1"
TYPE="Ethernet"
ONBOOT="yes"
PROMISC="yes"
BOOTPROTO="none"
HWADDR="52:54:00:D3:92:E2"
UUID="7f8a9e99-fbd0-4c59-900f-2369c9e8f780"
However, after adding the network service. I cannot ping the outside network in the next step.
1) inside (gateway of the tenant router in the network - node) -> external (external output)
[root@network-node ~]# ping 192.168.125.254
PING 192.168.125.254 (192.168.125.254) 56(84) bytes of data.
From 192.168.125.54 icmp_seq=1 Destination Host Unreachable
From 192.168.125.54 icmp_seq=2 Destination Host Unreachable
From 192.168.125.54 icmp_seq=3 Destination Host Unreachable
2) outside → inside
[root@desktop ~]# ping 192.168.125.150
PING 192.168.125.150 (192.168.125.150) 56(84) bytes of data.
--> receive no response
This is my environment info:
1) controller node
###The status of the port for external is "DOWN"###
[root@controller-node]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:91:4e:06 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.11/24 brd 10.0.0.255 scope global eth0
inet6 fe80::5054:ff:fe91:4e06/64 scope link
valid_lft forever preferred_lft forever
[root@controller-node]#
[root@controller-node]# neutron net-list
+--------------------------------------+----------+-------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+----------+-------------------------------------------------------+
| 9cebb2a6-fd73-4ef7-81d2-188652f57ecd | demo-net | c66648c9-c34b-4806-af39-3c982378a411 172.30.1.0/24 |
| e5f7b93c-475c-4c9d-95e4-8d1cf7728013 | ext-net | a1e1fcc6-d596-4959-8923-9b46d64445af 192.168.125.0/24 |
+--------------------------------------+----------+-------------------------------------------------------+
[root@controller-node]# neutron subnet-list
+--------------------------------------+-------------+------------------+--------------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+-------------+------------------+--------------------------------------------------------+
| a1e1fcc6-d596-4959-8923-9b46d64445af | ext-subnet | 192.168.125.0/24 | {"start": "192.168.125.150", "end": "192.168.125.159"} |
| c66648c9-c34b-4806-af39-3c982378a411 | demo-subnet | 172.30.1.0/24 | {"start": "172.30.1.2", "end": "172.30.1.254"} |
+--------------------------------------+-------------+------------------+--------------------------------------------------------+
[root@controller-node]#
[root@controller-node]# neutron port-list
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
| 9810105a-edf5-41bc-a140-81ccf71f6bc4 | | fa:16:3e:34:fd:cb | {"subnet_id": "a1e1fcc6-d596-4959-8923-9b46d64445af", "ip_address": "192.168.125.150"} |
| 98c762ea-d7f7-4c1d-9b74-73efc9990236 | | fa:16:3e:cb:0c:11 | {"subnet_id": "c66648c9-c34b-4806-af39-3c982378a411", "ip_address": "172.30.1.1"} |
| f5eec840-e629-448b-ba9a-fbcd60501247 | | fa:16:3e:ae:a6:fa | {"subnet_id": "c66648c9-c34b-4806-af39-3c982378a411", "ip_address": "172.30.1.2"} |
+--------------------------------------+------+-------------------+----------------------------------------------------------------------------------------+
[root@controller-node]#
[root@controller-node]# neutron port-show 9810105a-edf5-41bc-a140-81ccf71f6bc4
+-----------------------+----------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+----------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | os-network |
| binding:profile | {} |
| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": true} |
| binding:vif_type | ovs |
| binding:vnic_type | normal |
| device_id | 8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 |
| device_owner | network:router_gateway |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "a1e1fcc6-d596-4959-8923-9b46d64445af", "ip_address": "192.168.125.150"} |
| id | 9810105a-edf5-41bc-a140-81ccf71f6bc4 |
| mac_address | fa:16:3e:34:fd:cb |
| name | |
| network_id | e5f7b93c-475c-4c9d-95e4-8d1cf7728013 |
| security_groups | |
| status | DOWN |
| tenant_id | |
+-----------------------+----------------------------------------------------------------------------------------+
[root@controller-node]#
[root@controller-node]# neutron router-show demo-router
+-----------------------+-----------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------------+
| admin_state_up | True |
| external_gateway_info | {"network_id": "e5f7b93c-475c-4c9d-95e4-8d1cf7728013", "enable_snat": true} |
| id | 8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 |
| name | demo-router |
| routes | |
| status | ACTIVE |
| tenant_id | c94f1dc5870a4d06a8b6ba947e1ac554 |
+-----------------------+-----------------------------------------------------------------------------+
[root@controller-node]#
[root@controller-node]# neutron router-list
+--------------------------------------+-------------+-----------------------------------------------------------------------------+
| id | name | external_gateway_info |
+--------------------------------------+-------------+-----------------------------------------------------------------------------+
| 8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 | demo-router | {"network_id": "e5f7b93c-475c-4c9d-95e4-8d1cf7728013", "enable_snat": true} |
+--------------------------------------+-------------+-----------------------------------------------------------------------------+
[root@controller-node]#
[root@controller-node]#
2) network node
[root@network-node ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:f6:31:07 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.21/24 brd 10.0.0.255 scope global eth0
inet6 fe80::5054:ff:fef6:3107/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:d3:92:e2 brd ff:ff:ff:ff:ff:ff
inet6 fe80::5054:ff:fed3:92e2/64 scope link
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:48:c8:65 brd ff:ff:ff:ff:ff:ff
inet 10.0.1.21/24 brd 10.0.1.255 scope global eth2
inet6 fe80::5054:ff:fe48:c865/64 scope link
valid_lft forever preferred_lft forever
5: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether ea:8e:aa:ad:57:60 brd ff:ff:ff:ff:ff:ff
6: br-ex: <BROADCAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether a2:f1:0b:6b:34:4f brd ff:ff:ff:ff:ff:ff
inet6 fe80::a0f1:bff:fe6b:344f/64 scope link
valid_lft forever preferred_lft forever
9: br-int: <BROADCAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 32:a4:53:15:fc:4f brd ff:ff:ff:ff:ff:ff
inet6 fe80::30a4:53ff:fe15:fc4f/64 scope link
valid_lft forever preferred_lft forever
12: gre0: <NOARP> mtu 1476 qdisc noop state DOWN
link/gre 0.0.0.0 brd 10.0.0.31
13: gretap0: <BROADCAST,MULTICAST> mtu 1476 qdisc noop state DOWN qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
22: tun0@NONE: <POINTOPOINT,NOARP> mtu 1476 qdisc noqueue state DOWN
link/gre 0.0.0.0 peer 10.0.0.31
inet 10.0.1.21 peer 10.0.1.31/32 scope global tun0
24: br-tun: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether de:a8:a4:b1:b1:46 brd ff:ff:ff:ff:ff:ff
inet6 fe80::dc0e:8cff:fe67:d352/64 scope link
valid_lft forever preferred_lft forever
[root@network-node ~]#
[root@network-node ~]# ovs-vsctl show
23804a8f-7c89-4422-9b9f-67bf26a34c51
Bridge br-int
fail_mode: secure
Port br-int
Interface br-int
type: internal
Port "qr-98c762ea-d7"
tag: 1
Interface "qr-98c762ea-d7"
type: internal
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "tapf5eec840-e6"
tag: 1
Interface "tapf5eec840-e6"
type: internal
Bridge br-ex
Port "eth1"
Interface "eth1"
Port br-ex
Interface br-ex
type: internal
Port "qg-9810105a-ed"
Interface "qg-9810105a-ed"
type: internal
Bridge br-tun
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port "gre-0a00011f"
Interface "gre-0a00011f"
type: gre
options: {in_key=flow, local_ip="10.0.1.21", out_key=flow, remote_ip="10.0.1.31"}
Port br-tun
Interface br-tun
type: internal
ovs_version: "1.11.0"
[root@network-node ~]#
[root@network-node ~]# ip netns list
qdhcp-9cebb2a6-fd73-4ef7-81d2-188652f57ecd
qrouter-8ae4b1fa-fb60-4690-bbe2-febbfbcf7555
[root@network-node ~]#
[root@network-node ~]# ip netns exec qrouter-8ae4b1fa-fb60-4690-bbe2-febbfbcf7555 route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.30.1.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-98c762ea-d7
192.168.125.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-9810105a-ed
0.0.0.0 192.168.125.254 0.0.0.0 UG 0 0 0 qg-9810105a-ed
[root@network-node ~]#
Any hints would be much appreciated!
Greetings,
hbseo
+3
source to share
1 answer
I know it's late ... but let me try to answer. The inability to listen to the external network might be due to virtualization and promiscuous behavior, but I think this step shouldn't work. I will try to explain why. For a very good explanation of network setup, you can check out this presentation: https://www.hastexo.com/system/files/neutron_packet_flows-notes-handout.pdf
There are several parallel network spaces here.
First, you have a global or main host routing table. In the case of a node network, you have 3 ethernet interfaces, two with and an IP address and one without.
Second, you have an external network. It is not configured on your network host. If you run
netstat -rn
you will not see any route to your external network. This means that when you send ICMP packets to an external gateway, they will use the default gateway set on the node's network and will not respond. This network lives in the namespace of the neutron router networks.
Third, you have a tenant network that also does not live in your main network space, but in the same neutron router namespace.
As an example, I'll show you what it looks like in my setup. I have everything in one node running inside a Virtualbox machine and running Ubuntu Trusty on it. My interfaces are eth0 for the admin (also my default gateway is connected to the NAT network in Virtualbox), eth1 for the tunnels, eth2 is my external network (also connected to the NAT network with the 172.16.100.0/24 range).
This is what the Ubuntu VM routing table looks like:
root@columbo:~# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.1.1 0.0.0.0 UG 0 0 0 eth0
10.0.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.56.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
So no sign of my external network (172.16.100.1). If I try to ping, I get nothing:
ping 172.16.100.1
PING 172.16.100.1 (172.16.100.1) 56(84) bytes of data.
^C
--- 172.16.100.1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2004ms
Now you can see what network namespaces you have using the ip netns list command
root@columbo:~# ip netns list
qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b
qdhcp-120a6fde-7e2d-4856-90ee-5609a5f3035f
qdhcp-b7ab2080-a71a-44f6-9f66-fde526bb73d3
Inside the namespace, you can run most of your normal commands - see below
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 172.16.100.1 0.0.0.0 UG 0 0 0 qg-c69702a9-ae
10.255.1.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-e706aba1-46
172.16.100.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-c69702a9-ae
and
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b arp -an
? (10.255.1.12) at fa:16:3e:40:50:36 [ether] on qr-e706aba1-46
? (172.16.100.1) at 52:54:00:12:35:00 [ether] on qg-c69702a9-ae
? (10.255.1.14) at fa:16:3e:75:5f:b4 [ether] on qr-e706aba1-46
and now ping will work (inside namespace)
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b ping 172.16.100.1
PING 172.16.100.1 (172.16.100.1) 56(84) bytes of data.
64 bytes from 172.16.100.1: icmp_seq=1 ttl=255 time=2.55 ms
64 bytes from 172.16.100.1: icmp_seq=2 ttl=255 time=0.555 ms
^C
--- 172.16.100.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.555/1.554/2.553/0.999 ms
I can also ping a router that has an ip of 172.16.100.50:
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b ping 172.16.100.50
PING 172.16.100.50 (172.16.100.50) 56(84) bytes of data.
64 bytes from 172.16.100.50: icmp_seq=1 ttl=64 time=0.065 ms
64 bytes from 172.16.100.50: icmp_seq=2 ttl=64 time=0.065 ms
^C
--- 172.16.100.50 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.065/0.065/0.065/0.000 ms
Same for my network of tenants
root@columbo:~# neutron net-list
+--------------------------------------+---------------+------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+---------------+------------------------------------------------------+
| 120a6fde-7e2d-4856-90ee-5609a5f3035f | SecondVlan | 5432f1c9-0bb6-4619-b897-65d301071f72 5.5.5.0/25 |
| f2597437-a005-44ad-9ce2-168fbc331e56 | outside_world | 3fe35e71-53d7-4432-8c82-a06856b79316 172.16.100.0/24 |
| b7ab2080-a71a-44f6-9f66-fde526bb73d3 | SERVER_VLAN_1 | 87d769f1-5cf3-48cf-8741-44a01479ff3e 10.255.1.0/24 |
+--------------------------------------+---------------+------------------------------------------------------+
ping the tenant router in SERVER vlan
root@columbo:~# ip netns exec qrouter-e53979a8-8bab-4da5-9b57-58dba6d5db7b ping 10.255.1.1
PING 10.255.1.1 (10.255.1.1) 56(84) bytes of data.
64 bytes from 10.255.1.1: icmp_seq=1 ttl=64 time=0.050 ms
64 bytes from 10.255.1.1: icmp_seq=2 ttl=64 time=0.064 ms
^C
--- 10.255.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.050/0.057/0.064/0.007 ms
I hope this is helpful for someone and not completely messed up.
+2
source to share