There is a minor flaw in the way libvirt is configured to add Firewall rules to the host firewall when creating virtual networks. Two Virtual Machines in separate virtual networks cannot speak to each other bidirectionally. I am not the only one who has had this problem but it seems that the issue has not caused enough problems for it to be fixed yet. Or, perhaps I am misunderstanding the way libvirt is supposed to be configured.
Before proceeding, I've read the documentation regarding this issue at: https://libvirt.org/firewall.html
And I've requested assistance from people on IRC irc.oftc.net, channel: #virt.
Presently I am following examples in Michael Jang's RHCSA/RHCE study guide, specifically: RHCSA/RHCE Red Hat Linux Certification Study Guide, Seventh Edition (Exams EX200 & EX300). In the first chapters we are instructed in creating a second virtual network (the first being the default.) Virtual machines are created in both of these networks. Here is the basic idea of the networking setup:
Host OS: CentOS 7.3
Libvirt version: libvirt-daemon-2.0.0-10.el7_3.9.x86_64
Network Setup:
---------------------------
Host network: 192.168.0.1/24 (this can vary)
Host IP: 192.168.0.27 (this can also vary - statically assigned)
1st Virtual Network (Default):
Net: 192.168.122.0/24
GW: 192.168.122.1
DHCP: yes
DHCP Range: 192.168.122.2->254 (I use static IPs)
Forward Mode: Nat
2nd Virtual Network (outsider):
Net: 192.168.100.0/24
GW: 192.168.100.1
DHCP: yes
DHCP Range: 192.168.100.128->254 (I use static IPs)
Forward Mode: Nat
VM status in 1st network (Default):
Name: server1
IP: 192.168.122.50
- can ping host fine
- can ping external DNS addresses (like google)
VM status in 2nd network (outsider):
Name: outsider1
IP: 192.168.100.100
- can ping host fine
- can ping external DNS addresses (like google)
PROBLEM: Here's where things fall apart; outsider1 can ping server1 just fine, but server1 cannot ping outsider1.
[name@outsider1 ~]$ ping 192.168.122.50
PING 192.168.122.50 (192.168.122.50) 56(84) bytes of data.
64 bytes from 192.168.122.50: icmp_seq=1 ttl=63 time=0.171 ms
[name@server1 ~]$ ping 192.168.100.100
PING 192.168.100.100 (192.168.100.100) 56(84) bytes of data.
From 192.168.122.1 icmp_seq=1 Destination Port Unreachable
Below, I have the modifications made by libvirt to my firewalld FORWARD chain configuration (Sorry for the tiny fonts - zoom in if necessary):
Base FORWARD chain (prior to having libvirtd configure any networks:
-------------------
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_IN_ZONES_SOURCE all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_IN_ZONES all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_OUT_ZONES_SOURCE all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_OUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0
DROP all -- 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Default Network Added:
----------------------
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
...
outsider Network Added:
-----------------------
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 192.168.100.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.100.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
ACCEPT all -- 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
...
According to an individual I spoke to on IRC, the intent was to have the firewall block communication both ways, meaning that neither VMs should have been able to communicate with each other.
I'm not sure if Michael Jang is aware of this, but it seems that this is not entirely well documented, and not exactly functional either.
Since I want my two VMs to communicate with each other across this network, my temporary solution is to modify the FORWARD chain and remove the two REJECT rules that were added with the 'outside' network. Therefore we will delete the 4th and 5th rules.
From the host system, as root, delete the 4th rule in the FORWARD chain.
[root@hostname ~]# iptables -D FORWARD 4
Since the 4th rule is deleted, the 5th rule now became the 4th, so verify with:
[root@hostname ~]# iptables -L FORWARD
And delete the new 4th rule (formerly 5th).
[root@hostname ~]# iptables -D FORWARD 4
Verify your final configuration and try pinging from both VMs.
[root@hostname ~]# iptables -L FORWARD
NOTE: Of course, this is a temporary rule for testing purposes only and not meant to be used for a production VM host. If that is your purpose, a proper firewall configuration design should be undertaken with all that it entails.
Before proceeding, I've read the documentation regarding this issue at: https://libvirt.org/firewall.html
And I've requested assistance from people on IRC irc.oftc.net, channel: #virt.
Presently I am following examples in Michael Jang's RHCSA/RHCE study guide, specifically: RHCSA/RHCE Red Hat Linux Certification Study Guide, Seventh Edition (Exams EX200 & EX300). In the first chapters we are instructed in creating a second virtual network (the first being the default.) Virtual machines are created in both of these networks. Here is the basic idea of the networking setup:
Host OS: CentOS 7.3
Libvirt version: libvirt-daemon-2.0.0-10.el7_3.9.x86_64
Network Setup:
---------------------------
Host network: 192.168.0.1/24 (this can vary)
Host IP: 192.168.0.27 (this can also vary - statically assigned)
1st Virtual Network (Default):
Net: 192.168.122.0/24
GW: 192.168.122.1
DHCP: yes
DHCP Range: 192.168.122.2->254 (I use static IPs)
Forward Mode: Nat
2nd Virtual Network (outsider):
Net: 192.168.100.0/24
GW: 192.168.100.1
DHCP: yes
DHCP Range: 192.168.100.128->254 (I use static IPs)
Forward Mode: Nat
VM status in 1st network (Default):
Name: server1
IP: 192.168.122.50
- can ping host fine
- can ping external DNS addresses (like google)
VM status in 2nd network (outsider):
Name: outsider1
IP: 192.168.100.100
- can ping host fine
- can ping external DNS addresses (like google)
PROBLEM: Here's where things fall apart; outsider1 can ping server1 just fine, but server1 cannot ping outsider1.
[name@outsider1 ~]$ ping 192.168.122.50
PING 192.168.122.50 (192.168.122.50) 56(84) bytes of data.
64 bytes from 192.168.122.50: icmp_seq=1 ttl=63 time=0.171 ms
[name@server1 ~]$ ping 192.168.100.100
PING 192.168.100.100 (192.168.100.100) 56(84) bytes of data.
From 192.168.122.1 icmp_seq=1 Destination Port Unreachable
Below, I have the modifications made by libvirt to my firewalld FORWARD chain configuration (Sorry for the tiny fonts - zoom in if necessary):
Base FORWARD chain (prior to having libvirtd configure any networks:
-------------------
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_IN_ZONES_SOURCE all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_IN_ZONES all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_OUT_ZONES_SOURCE all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_OUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0
DROP all -- 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Default Network Added:
----------------------
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
...
outsider Network Added:
-----------------------
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 192.168.100.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.100.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
ACCEPT all -- 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
...
According to an individual I spoke to on IRC, the intent was to have the firewall block communication both ways, meaning that neither VMs should have been able to communicate with each other.
I'm not sure if Michael Jang is aware of this, but it seems that this is not entirely well documented, and not exactly functional either.
Since I want my two VMs to communicate with each other across this network, my temporary solution is to modify the FORWARD chain and remove the two REJECT rules that were added with the 'outside' network. Therefore we will delete the 4th and 5th rules.
From the host system, as root, delete the 4th rule in the FORWARD chain.
[root@hostname ~]# iptables -D FORWARD 4
Since the 4th rule is deleted, the 5th rule now became the 4th, so verify with:
[root@hostname ~]# iptables -L FORWARD
And delete the new 4th rule (formerly 5th).
[root@hostname ~]# iptables -D FORWARD 4
Verify your final configuration and try pinging from both VMs.
[root@hostname ~]# iptables -L FORWARD
NOTE: Of course, this is a temporary rule for testing purposes only and not meant to be used for a production VM host. If that is your purpose, a proper firewall configuration design should be undertaken with all that it entails.
No comments:
Post a Comment