通过iptables分析OpenStack 安全组规则
在OpenStack中创建一个实例，同时会生成如下的bridge和port :
· tap-xxx vm用的端口，配置在libvirt配置文件中的
· vnet-x 图中有vnet，实际上是没有的，直接用tap插到了qbr上
· qbr-xxx 虚拟网桥，桥接tap和qvb
· qvb-xxx 连接br-int的veth端口
· qvo-xxx qvb-xxx的另一端
例如：
21:qbr14c032e9-bc: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueuestate UP
22: qvo14c032e9-bc@qvb14c032e9-bc:<BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1450 qdisc pfifo_fastmaster ovs-system state UP qlen 1000
23:qvb14c032e9-bc@qvo14c032e9-bc: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP>mtu 1450 qdisc pfifo_fast master qbr14c032e9-bc state UP qlen 1000
25:tap14c032e9-bc: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdiscpfifo_fast master qbr14c032e9-bc state UNKNOWN qlen 500
查看bridge:
$ brctl show
bridgename bridgeid STPenabled interfaces
qbr14c032e9-bc 8000.b21e7be143d6 no qvb14c032e9-bc tap14c032e9-bc
分析Security Group规则
分析FORWARD链
由于规则是配置在host的，所以进出以上端口的规则都只走forward链
删除security group所有规则后，查看iptables
# iptables--list -v
Chain FORWARD (policyACCEPT 0 packets, 0 bytes)
pkts bytes target protopt in out source destination
348K 19Mneutron-filter-top all -- any any anywhere anywhere
348K 19Mneutron-openvswi-FORWARD all -- any any anywhere anywhere
Chainneutron-openvswi-FORWARD (1 references)
pkts bytes target protopt in out source destination
214 25538neutron-openvswi-sg-chain all -- any any anywhere anywhere PHYSDEV match --physdev-outtapc0a350e0-43 --physdev-is-bridged /* Direct trafficfrom the VM interface to the security group chain. */
248 23854neutron-openvswi-sg-chain all -- any any anywhere anywhere PHYSDEV match --physdev-in tapc0a350e0-43--physdev-is-bridged /* Direct traffic from the VM interface tothe security group chain. */
Chainneutron-openvswi-sg-chain (4 references)
pkts bytes target protopt in out source destination
214 25538neutron-openvswi-ic0a350e0-4 all -- any any anywhere anywhere PHYSDEV match --physdev-out tapc0a350e0-43--physdev-is-bridged /* Jump to the VM specific chain. */
248 23854neutron-openvswi-oc0a350e0-4 all -- any any anywhere anywhere PHYSDEVmatch --physdev-in tapc0a350e0-43--physdev-is-bridged /* Jump to the VM specific chain. */
--physdev-out tapc0a350e0-43是指tapc0a350e0-43即vm发送到bridge的包
匹配到的包跳转到了neutron-openvswi-ic0a350e0-4
Chainneutron-openvswi-ic0a350e0-4 (1 references)
num pkts bytes target prot opt in out source destination
1 169 20508 RETURN all -- any any anywhere anywhere state RELATED,ESTABLISHED /* Directpackets associated with a known session to the RETURN chain. */
2 2 731 RETURN udp -- any any 192.168.1.2 anywhere udpspt:bootps dpt:bootpc
3 0 0 DROP all -- any any anywhere anywhere stateINVALID /* Drop packets that appear related to an existing connection(e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
4 6 1944 neutron-openvswi-sg-fallback all -- any any anywhere anywhere /* Send unmatched traffic tothe fallback chain. */
Chainneutron-openvswi-sg-fallback (4 references)
num pkts bytes target prot opt in out source destination
1 947 71484 DROP all -- any any anywhere anywhere /* Default drop rule for unmatchedtraffic. */
· num:1 放行所有已建立连接的包
· num:2 放行192.168.1.2（dhcp服务器）发过来的udp包
· num:3 丢弃状态异常的tcp包
· num:4 丢弃不匹配以上三条的所有包
--physdev-in tapc0a350e0-43 是指从tapc0a350e0-43即vm发出来的包
匹配到的包跳转到了neutron-openvswi-oc0a350e0-4
Chainneutron-openvswi-oc0a350e0-4 (2 references)
num pkts bytes target prot opt in out source destination
1 2 648 RETURN udp -- any any default 255.255.255.255 udp spt:bootpc dpt:bootps /* Allow DHCP client traffic. */
2 246 23206 neutron-openvswi-sc0a350e0-4 all -- any any anywhere anywhere
3 4 1272 RETURN udp -- any any anywhere anywhere udp spt:bootpc dpt:bootps /* Allow DHCPclient traffic. */
4 0 0 DROP udp -- any any anywhere anywhere udp spt:bootpsdpt:bootpc /* Prevent DHCP Spoofing by VM. */
5 210 19802 RETURN all -- any any anywhere anywhere state RELATED,ESTABLISHED /* Directpackets associated with a known session to the RETURN chain. */
6 0 0 DROP all -- any any anywhere anywhere state INVALID /* Droppackets that appear related to an existing connection (e.g. TCP ACK/FIN) but donot have an entry in conntrack. */
7 0 0 neutron-openvswi-sg-fallback all -- any any anywhere anywhere /* Sendunmatched traffic to the fallback chain. */Chainneutron-openvswi-sc0a350e0-4 (1 references)
num pkts bytes target prot opt in out source destination
1 246 23206 RETURN all -- any any 192.168.1.12 anywhere MAC FA:16:3E:C3:EA:D5 /* Allowtraffic from defined IP/MAC pairs. */
2 0 0 DROP all -- any any anywhere anywhere /* Drop trafficwithout an IP/MAC allow rule. */Chain neutron-openvswi-sg-fallback (4 references)
num pkts bytes target prot opt in out source destination
1 947 71484 DROP all -- any any anywhere anywhere /* Default drop rule for unmatchedtraffic. */
· num1 允许vm发出来的dhcp udp广播包允许源端口是67，目标端口是68端口的数据包通过
· num2 只允许ip地址为192.168.1.12（vm的分配的ip）通过
· num3 允许vm（dhcp客户端）发出来的UDP 单播报文
· num4 禁止vm做dhcp嗅探
· num5 允许通过所有已建立连接的包通过
· num6 丢弃所以异常连接的包
· num7 丢弃不匹配以上任何规则包
说明：
o bootpc 服务器向67端口(bootpc)广播dhcp回应请求
o bootps 客户端向68端口(bootps)广播dhcp请求配置
可以看出，在不匹配security规则的情况下，除了dhcp包可以通过之外，其他数据包全部丢弃
配置securitygroup，新增规则后再查看iptables
增加规则1：允许vm发出的所有数据包
查看neutron-openvswi-oc0a350e0-4链
Chainneutron-openvswi-oc0a350e0-4 (2 references)
num pkts bytes target prot opt in out source destination
1 2 648 RETURN udp -- any any default 255.255.255.255 udp spt:bootpc dpt:bootps /* Allow DHCP client traffic. */
2 246 23206 neutron-openvswi-sc0a350e0-4 all -- any any anywhere anywhere
3 4 1272 RETURN udp -- any any anywhere anywhere udp spt:bootpc dpt:bootps /* Allow DHCPclient traffic. */
4 0 0 DROP udp -- any any anywhere anywhere udp spt:bootpsdpt:bootpc /* Prevent DHCP Spoofing by VM. */
5 210 19802 RETURN all -- any any anywhere anywhere state RELATED,ESTABLISHED /* Directpackets associated with a known session to the RETURN chain. */
6 0 0 RETURN all -- any any anywhere anywhere
7 0 0 DROP all -- any any anywhere anywhere state INVALID /* Droppackets that appear related to an existing connection (e.g. TCP ACK/FIN) but donot have an entry in conntrack. */
8 0 0 neutron-openvswi-sg-fallback all -- any any anywhere anywhere /* Sendunmatched traffic to the fallback chain. */
· num6 为新增的规则，为放行所有包
增加规则2：允许vm发出的icmp协议包通过
查看neutron-openvswi-ic0a350e0-4链
Chainneutron-openvswi-ic0a350e0-4 (1 references)
num pkts bytes target prot opt in out source destination
1 169 20508 RETURN all -- any any anywhere anywhere state RELATED,ESTABLISHED /* Directpackets associated with a known session to the RETURN chain. */
2 2 731 RETURN udp -- any any 192.168.1.2 anywhere udp spt:bootps dpt:bootpc
3 0 0 RETURN icmp -- any any anywhere anywhere
4 0 0 DROP all -- any any anywhere anywhere state INVALID /* Droppackets that appear related to an existing connection (e.g. TCP ACK/FIN) but donot have an entry in conntrack. */
5 6 1944 neutron-openvswi-sg-fallback all -- any any anywhere anywhere /* Send unmatchedtraffic to the fallback chain. */
· num3 为新增的规则，放行所有icmp协议包
可以看到优化前在实时性方面原始的KVM还是会出现毛刺。在优化后基本上能达到很好的结果。
作者介绍：
曹云涛 ，来自诺云信息系统（上海）有限公司。
相关阅读：
加入中国最活跃的KVM技术讨论QQ群，加群主QQ：502207183，并注明城市、行业、技术方向。