Hello,
感谢lixingcong解答我之前的问题,但是今天又遇到一个新的问题,自己尝试解决无果,还请解答一下:
拓扑如下:
A>>>>B>>>C>>>公网>>>D>>>E>>>F
C和D均为路由设备,E和B为OpenWrt路由器,Minivtun均部署在OpenWrt路由器上,F和A均为内网PC,想要通过Minivtun实现两端私网IP互访,也就是A设备和F设备。
由于只有D设备有公网IP,所以D设备做端口映射,把Minivtun的6000端口映射到E设备上。
问题:E和B已经能够互相ping通隧道接口上的IP,A设备也能够访问F设备,但是F设备却不能访问A设备,也就是服务端不能主动访问客户端。
B to E
[root@xRoute ~]# ping 10.37.21.1 -c 5
PING 10.37.21.1 (10.37.21.1) 56(84) bytes of data.
64 bytes from 10.37.21.1: icmp_seq=1 ttl=64 time=13.8 ms
64 bytes from 10.37.21.1: icmp_seq=2 ttl=64 time=17.0 ms
64 bytes from 10.37.21.1: icmp_seq=3 ttl=64 time=9.66 ms
64 bytes from 10.37.21.1: icmp_seq=4 ttl=64 time=8.77 ms
64 bytes from 10.37.21.1: icmp_seq=5 ttl=64 time=9.51 ms
--- 10.37.21.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4016ms
rtt min/avg/max/mdev = 8.773/11.774/17.061/3.193 ms
[root@xRoute ~]# ping 10.37.20.20 -c 5
PING 10.37.20.20 (10.37.20.20) 56(84) bytes of data.
64 bytes from 10.37.20.20: icmp_seq=1 ttl=64 time=10.3 ms
64 bytes from 10.37.20.20: icmp_seq=2 ttl=64 time=7.12 ms
64 bytes from 10.37.20.20: icmp_seq=3 ttl=64 time=10.3 ms
64 bytes from 10.37.20.20: icmp_seq=4 ttl=64 time=10.9 ms
64 bytes from 10.37.20.20: icmp_seq=5 ttl=64 time=12.5 ms
--- 10.37.20.20 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4018ms
rtt min/avg/max/mdev = 7.127/10.268/12.575/1.774 ms
E to B
[root@AdvancedTomato:/root]# ping 10.37.21.2 -c 5
PING 10.37.21.2 (10.37.21.2): 56 data bytes
64 bytes from 10.37.21.2: seq=0 ttl=64 time=10.100 ms
64 bytes from 10.37.21.2: seq=1 ttl=64 time=9.660 ms
64 bytes from 10.37.21.2: seq=2 ttl=64 time=12.781 ms
64 bytes from 10.37.21.2: seq=3 ttl=64 time=10.680 ms
64 bytes from 10.37.21.2: seq=4 ttl=64 time=10.640 ms
--- 10.37.21.2 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 9.660/10.772/12.781 ms
[root@AdvancedTomato:/root]# ping 10.88.16.33 -c 5
PING 10.88.16.33 (10.88.16.33): 56 data bytes
--- 10.88.16.33 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
E(OpenWrt)的Minivtun和路由配置:
minivtun -l 0.0.0.0:6000 -a 10.37.21.1/24 -e hillstone -n tun0 -d
root@AdvancedTomato:/root]# ip route list
default via 10.10.10.1 dev eth0.2 proto static
10.10.10.0/24 dev eth0.2 proto kernel scope link src 10.10.10.10
10.37.20.0/24 dev br-lan proto kernel scope link src 10.37.20.1
10.37.21.0/24 dev tun0 proto kernel scope link src 10.37.21.1
10.88.16.0/24 via 10.37.21.2 dev tun0
[root@AdvancedTomato:/root]# ifconfig
br-lan Link encap:Ethernet HWaddr 8C:AB:8E:73:64:A1
inet addr:10.37.20.1 Bcast:10.37.20.255 Mask:255.255.255.0
inet6 addr: fe80::8eab:8eff:fe73:64a1/64 Scope:Link
inet6 addr: fdda:503a:ecd2::1/60 Scope:Global
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2253271 errors:0 dropped:0 overruns:0 frame:0
TX packets:3153572 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:760875318 (725.6 MiB) TX bytes:3271794394 (3.0 GiB)
eth0 Link encap:Ethernet HWaddr 8C:AB:8E:73:64:A1
inet6 addr: fe80::8eab:8eff:fe73:64a1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3011827 errors:0 dropped:0 overruns:0 frame:0
TX packets:2238092 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3219770298 (2.9 GiB) TX bytes:802390213 (765.2 MiB)
Interrupt:3
eth0.1 Link encap:Ethernet HWaddr 8C:AB:8E:73:64:A1
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:3535 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:535198 (522.6 KiB)
eth0.2 Link encap:Ethernet HWaddr 8C:AB:8E:73:64:A2
inet addr:10.10.10.10 Bcast:10.10.10.255 Mask:255.255.255.0
inet6 addr: fe80::8eab:8eff:fe73:64a2/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3011819 errors:0 dropped:11 overruns:0 frame:0
TX packets:2234547 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3165556476 (2.9 GiB) TX bytes:790979500 (754.3 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:1300 errors:0 dropped:0 overruns:0 frame:0
TX packets:1300 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:137417 (134.1 KiB) TX bytes:137417 (134.1 KiB)
ra0 Link encap:Ethernet HWaddr 8C:AB:8E:73:64:A8
inet6 addr: fe80::8eab:8eff:fe73:64a8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2164637 errors:0 dropped:0 overruns:0 frame:0
TX packets:2862417 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:815299109 (777.5 MiB) TX bytes:2915993154 (2.7 GiB)
Interrupt:4
rai0 Link encap:Ethernet HWaddr 8C:AB:8E:73:64:B0
inet6 addr: fe80::8eab:8eff:fe73:64b0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:174507 errors:0 dropped:0 overruns:0 frame:0
TX packets:205577 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:41867097 (39.9 MiB) TX bytes:266157033 (253.8 MiB)
Interrupt:13
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.37.21.1 P-t-P:10.37.21.1 Mask:255.255.255.0
UP POINTOPOINT RUNNING MTU:1300 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:3443 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:2161628 (2.0 MiB)
B(OpenWrt)的Minivtun和路由配置:
minivtun -r 121.227.184.135:6000 -a 10.37.21.2/24 -e hillstone -d ##121.227.184.135为D设备的公网IP
[root@xRoute home]# ip route list
10.37.21.0/24 dev mv0 proto kernel scope link src 10.37.21.2
10.37.20.0/24 via 10.37.21.1 dev mv0
10.88.16.0/24 dev eth0 proto kernel scope link src 10.88.16.12
169.254.0.0/16 dev eth0 scope link metric 1002
default via 10.88.16.1 dev eth0
[root@xRoute ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:F2:BB:CB
inet addr:10.88.16.12 Bcast:10.88.16.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fef2:bbcb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4732767 errors:0 dropped:0 overruns:0 frame:0
TX packets:1271058 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:818800681 (780.8 MiB) TX bytes:258715718 (246.7 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:29132 errors:0 dropped:0 overruns:0 frame:0
TX packets:29132 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2561091 (2.4 MiB) TX bytes:2561091 (2.4 MiB)
mv0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.37.21.2 P-t-P:10.37.21.2 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1300 Metric:1
RX packets:284493 errors:0 dropped:0 overruns:0 frame:0
TX packets:612379 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:11841288 (11.2 MiB) TX bytes:151415448 (144.4 MiB)
目前怀疑E上的防火墙规则有问题,测试用F设备(10.37.20.20)pingA设备(10.88.16.33)的时候tcpdump如下:
C:\Users\777>ping 10.88.16.33 -n 5 -l 500
正在 Ping 10.88.16.33 具有 500 字节的数据:
请求超时。
请求超时。
请求超时。
请求超时。
请求超时。
10.88.16.33 的 Ping 统计信息:
数据包: 已发送 = 5,已接收 = 0,丢失 = 5 (100% 丢失),
tcpdump tun0口有收到 10.37.20.20发送给10.88.16.33的报文,报文大小508。
[root@AdvancedTomato:/root]# tcpdump -i tun0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 65535 bytes
14:45:45.982865 IP Sony.lan > 10.88.16.33: ICMP echo request, id 3, seq 28172, length 508
14:45:50.611722 IP Sony.lan > 10.88.16.33: ICMP echo request, id 3, seq 28177, length 508
14:45:55.604103 IP Sony.lan > 10.88.16.33: ICMP echo request, id 3, seq 28183, length 508
14:46:00.601663 IP Sony.lan > 10.88.16.33: ICMP echo request, id 3, seq 28189, length 508
14:46:05.601363 IP Sony.lan > 10.88.16.33: ICMP echo request, id 3, seq 28195, length 508
^C
5 packets captured
5 packets received by filter
0 packets dropped by kernel
同时抓E到B设备经过Minivtun封装的报文,结果只抓到48长度大小的keepalive报文,怀疑报文被E设备上的防火墙drop。
[root@AdvancedTomato:/root]# tcpdump host 221.224.33.133 and port 6000
tcpdump: WARNING: eth0.2: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0.2, link-type EN10MB (Ethernet), capture size 65535 bytes
14:45:41.799291 IP 10.10.10.10.x11 > 221.224.33.1333.42908: UDP, length 48
14:45:47.814460 IP 221.224.33.133.42908 > 10.10.10.10.x11: UDP, length 48
14:45:55.604403 IP 10.10.10.10.x11 > 221.224.33.133.42908: UDP, length 48
14:46:01.622071 IP 221.224.33.133.42908 > 10.10.10.10.x11: UDP, length 48
14:46:11.615272 IP 10.10.10.10.x11 > 221.224.33.133.42908: UDP, length 48
^C
5 packets captured
5 packets received by filter
0 packets dropped by kernel
E设备的防火墙配置如下
[root@AdvancedTomato:/root]# cat /etc/config/firewall
config defaults
option output 'ACCEPT'
option drop_invalid '0'
option forward 'ACCEPT'
option input 'ACCEPT'
config include
option path '/etc/firewall.user'
config include 'miniupnpd'
option type 'script'
option path '/usr/share/miniupnpd/firewall.include'
option family 'any'
option reload '1'
config include 'adbyby'
option type 'script'
option path '/usr/share/adbyby/firewall.include'
option reload '1'
config zone
option name 'Lan'
option input 'ACCEPT'
option output 'ACCEPT'
option network 'lan'
option forward 'ACCEPT'
config zone
option name 'Wan'
option output 'ACCEPT'
option masq '1'
option mtu_fix '1'
option network 'wan wan6'
option input 'ACCEPT'
option forward 'ACCEPT'
config zone
option name 'Minivtun'
option output 'ACCEPT'
option network 'Minivtun'
option input 'ACCEPT'
option forward 'ACCEPT'
config rule
option target 'ACCEPT'
option name 'Allow'
option proto 'all'
option src ''
option dest ''
config rule
option enabled '1'
option target 'ACCEPT'
option name 'Allow-Minivtun'
option proto 'all'
option src 'Lan'
option dest 'Minivtun'
config forwarding
option dest 'Wan'
option src 'Minivtun'
config redirect
option target 'DNAT'
option src 'Wan'
option dest 'Lan'
option proto 'tcp'
option src_dport '3389'
option dest_ip '10.37.20.20'
option dest_port '3389'
option name 'RDP'
config forwarding
option dest 'Minivtun'
option src 'Lan'
config forwarding
option dest 'Wan'
option src 'Lan'
config forwarding
option dest 'Lan'
option src 'Minivtun'
[root@AdvancedTomato:/root]# cat /etc/config/network
config interface 'loopback'
option ifname 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fdda:503a:ecd2::/48'
config interface 'Minivtun'
option proto 'none'
option ifname 'tun0'
config interface 'lan'
option type 'bridge'
option ifname 'eth0.1'
option macaddr '8c:ab:8e:73:64:a1'
option proto 'static'
option netmask '255.255.255.0'
option ip6assign '60'
option ipaddr '10.37.20.1'
config device 'lan_dev'
option name 'eth0.1'
option macaddr '8c:ab:8e:73:64:a1'
config interface 'wan'
option ifname 'eth0.2'
option _orig_ifname 'eth0.2'
option _orig_bridge 'false'
option proto 'static'
option ipaddr '10.10.10.10'
option netmask '255.255.255.0'
option gateway '10.10.10.1'
option broadcast '10.10.10.255'
option dns '114.114.114.114 114.114.115.115'
config device 'wan_dev'
option name 'eth0.2'
option macaddr '8c:ab:8e:73:64:a2'
config interface 'wan6'
option ifname 'eth0.2'
option _orig_ifname 'eth0.2'
option _orig_bridge 'false'
option proto 'none'
config switch
option name 'switch0'
option reset '1'
option enable_vlan '1'
config switch_vlan
option device 'switch0'
option vlan '1'
option ports '0 1 2 3 6t 7t'
config switch_vlan
option device 'switch0'
option vlan '2'
option ports '4 6t 7t'
[root@AdvancedTomato:/root]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere ID:66773300
input_rule all -- anywhere anywhere ID:66773300 /* user chain for input */
ACCEPT all -- anywhere anywhere ID:66773300 ctstate RELATED,ESTABLISHED
zone_Lan_input all -- anywhere anywhere ID:66773300
zone_Wan_input all -- anywhere anywhere ID:66773300
zone_Minivtun_input all -- anywhere anywhere ID:66773300
Chain FORWARD (policy ACCEPT)
target prot opt source destination
forwarding_rule all -- anywhere anywhere ID:66773300 /* user chain for forwarding /
ACCEPT all -- anywhere anywhere ID:66773300 ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere ID:66773300 / Allow */
zone_Lan_forward all -- anywhere anywhere ID:66773300
zone_Wan_forward all -- anywhere anywhere ID:66773300
zone_Minivtun_forward all -- anywhere anywhere ID:66773300
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere ID:66773300
output_rule all -- anywhere anywhere ID:66773300 /* user chain for output */
ACCEPT all -- anywhere anywhere ID:66773300 ctstate RELATED,ESTABLISHED
zone_Lan_output all -- anywhere anywhere ID:66773300
zone_Wan_output all -- anywhere anywhere ID:66773300
zone_Minivtun_output all -- anywhere anywhere ID:66773300
Chain MINIUPNPD (1 references)
target prot opt source destination
Chain forwarding_Lan_rule (1 references)
target prot opt source destination
Chain forwarding_Minivtun_rule (1 references)
target prot opt source destination
Chain forwarding_Wan_rule (1 references)
target prot opt source destination
Chain forwarding_rule (1 references)
target prot opt source destination
Chain input_Lan_rule (1 references)
target prot opt source destination
Chain input_Minivtun_rule (1 references)
target prot opt source destination
Chain input_Wan_rule (1 references)
target prot opt source destination
Chain input_rule (1 references)
target prot opt source destination
Chain output_Lan_rule (1 references)
target prot opt source destination
Chain output_Minivtun_rule (1 references)
target prot opt source destination
Chain output_Wan_rule (1 references)
target prot opt source destination
Chain output_rule (1 references)
target prot opt source destination
Chain reject (0 references)
target prot opt source destination
REJECT tcp -- anywhere anywhere ID:66773300 reject-with tcp-reset
REJECT all -- anywhere anywhere ID:66773300 reject-with icmp-port-unreachable
Chain zone_Lan_dest_ACCEPT (3 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Lan_forward (1 references)
target prot opt source destination
forwarding_Lan_rule all -- anywhere anywhere ID:66773300 /* user chain for forwarding /
zone_Minivtun_dest_ACCEPT all -- anywhere anywhere ID:66773300 / Allow-Minivtun /
zone_Minivtun_dest_ACCEPT all -- anywhere anywhere ID:66773300 / forwarding Lan -> Minivtun /
zone_Wan_dest_ACCEPT all -- anywhere anywhere ID:66773300 / forwarding Lan -> Wan /
ACCEPT all -- anywhere anywhere ID:66773300 ctstate DNAT / Accept port forwards */
zone_Lan_dest_ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Lan_input (1 references)
target prot opt source destination
input_Lan_rule all -- anywhere anywhere ID:66773300 /* user chain for input /
ACCEPT all -- anywhere anywhere ID:66773300 ctstate DNAT / Accept port redirections */
zone_Lan_src_ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Lan_output (1 references)
target prot opt source destination
output_Lan_rule all -- anywhere anywhere ID:66773300 /* user chain for output */
zone_Lan_dest_ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Lan_src_ACCEPT (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Minivtun_dest_ACCEPT (4 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Minivtun_forward (1 references)
target prot opt source destination
forwarding_Minivtun_rule all -- anywhere anywhere ID:66773300 /* user chain for forwarding /
zone_Wan_dest_ACCEPT all -- anywhere anywhere ID:66773300 / forwarding Minivtun -> Wan /
zone_Lan_dest_ACCEPT all -- anywhere anywhere ID:66773300 / forwarding Minivtun -> Lan /
ACCEPT all -- anywhere anywhere ID:66773300 ctstate DNAT / Accept port forwards */
zone_Minivtun_dest_ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Minivtun_input (1 references)
target prot opt source destination
input_Minivtun_rule all -- anywhere anywhere ID:66773300 /* user chain for input /
ACCEPT all -- anywhere anywhere ID:66773300 ctstate DNAT / Accept port redirections */
zone_Minivtun_src_ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Minivtun_output (1 references)
target prot opt source destination
output_Minivtun_rule all -- anywhere anywhere ID:66773300 /* user chain for output */
zone_Minivtun_dest_ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Minivtun_src_ACCEPT (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Wan_dest_ACCEPT (4 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Wan_forward (1 references)
target prot opt source destination
MINIUPNPD all -- anywhere anywhere
forwarding_Wan_rule all -- anywhere anywhere ID:66773300 /* user chain for forwarding /
ACCEPT all -- anywhere anywhere ID:66773300 ctstate DNAT / Accept port forwards */
zone_Wan_dest_ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Wan_input (1 references)
target prot opt source destination
input_Wan_rule all -- anywhere anywhere ID:66773300 /* user chain for input /
ACCEPT all -- anywhere anywhere ID:66773300 ctstate DNAT / Accept port redirections */
zone_Wan_src_ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Wan_output (1 references)
target prot opt source destination
output_Wan_rule all -- anywhere anywhere ID:66773300 /* user chain for output */
zone_Wan_dest_ACCEPT all -- anywhere anywhere ID:66773300
Chain zone_Wan_src_ACCEPT (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ID:66773300