Upcoming Maintenance Alert:

The UBNT Community will be upgraded at 5pm MDT on April 25th. During this time the community forums will be set to read-only status.

Learn more

×
Reply
Member
Posts: 201
Registered: ‎10-04-2014
Kudos: 17
Solutions: 2

pixelserv dnat not working

Would someone mind looking-over this config-snippet and tell me why my ad-servers redirect is not working (AFAIK)?

 

I'm trying to redirect http[s] traffic to the servers added to the address group defined (I can list the group by name via ipset list) to my local host that is just serving a 1x1 replacement image.

 

Testing any http call directly to my internal host:50000 works (does not care what page you request).  But if I try to explicitly go to one of the "banned IPs" in the list from my browser it does not return anything - in fact it times-out, but IDK if that is because that actual server is down on the internet or if the dnat is partially working and my redirect is timing out.

 

Happy to provide further details, just ask!

 

Thanks,

AJ

 

    nat {
        rule 1 {
            description "ADServer Redirect"
            destination {
                group {
                    address-group ADS-A
                }
                port 80,443
            }
            inbound-interface eth2
            inside-address {
                address 192.168.1.100
                port 50000
            }
            log disable
            protocol tcp
            source {
                address 192.168.1.0/24
            }
            type destination
        }
...
Senior Member
Posts: 4,160
Registered: ‎03-24-2016
Kudos: 1170
Solutions: 491

Re: pixelserv dnat not working

You're missing a hairpin rule

 

Add masquerade rule , outgoing interface=eth2 , destination  address = 192.168.1.100 dport =50000

Member
Posts: 201
Registered: ‎10-04-2014
Kudos: 17
Solutions: 2

Re: pixelserv dnat not working

Really appreciate the reply...I'm just not sure exactly what you're asking me to do (and why).

 

There is an automatic route for 192.168.1.0/24 for eth2 shown in the route table (and that makes sense).  Are you saying that we won't use the route table once we hit the NAT table, and therefore I have to add an explicit route for that traffic?

 

Would it go something like this?

edit service nat
set rule 6000 description "Hairpin to pixelserv"
set rule 6000 type masquerade
set rule 6000 destination address 192.168.1.100
set rule 6000 destination port 50000
set rule 6000 outgoing interface eth2
commit

Thanks again!

Senior Member
Posts: 4,160
Registered: ‎03-24-2016
Kudos: 1170
Solutions: 491

Re: pixelserv dnat not working

exactly like that.

 

Why:

Assume PC 192.168.1.33 tries to open connection to adserver  1.1.1.1:

packet source  192.168.1.33 port 1024  

packet destination   1.1.1.1 port 80

 

This is d-NATted to:

packet source  192.168.1.33 port 1024  

packet destination   192.168.1.100 port 80

and arrives on your internal adserver

 

adserver responds:

packet source  192.168.1.100 port 80

packet destination   192.168.1.33 port 1024

 

So the PC that sended the request does get a response....but from the wrong source IP , so the PC has no clue what to do with it 

 

Member
Posts: 201
Registered: ‎10-04-2014
Kudos: 17
Solutions: 2

Re: pixelserv dnat not working

OK, that makes a ton of sense now.  Thank you for making me see the flip-side of the traffic.

 

I have to ask though, is the rule I prototyped correct in that case?  Am I telling the ERL to nat the traffic in both directions?  That is, my DNAT is basically redirecting the traffic.  The masq will change the IP back on the return trip to the originating PC?  With the syntax I have?  (It feels like my rule is kind-of backwards, but I can't put my finger on it.)

 

Thanks so much,

AJ

Member
Posts: 201
Registered: ‎10-04-2014
Kudos: 17
Solutions: 2

Re: pixelserv dnat not working

FWIW, I added the following rule, but I'm getting the same functional result.  Browser says server is not responding.

 rule 6000 {
     description "Hairpin to pixelserv"
     destination {
         address 192.168.1.100
         port 50000
     }
     outbound-interface eth2
     protocol tcp
     type masquerade
 }
Senior Member
Posts: 4,160
Registered: ‎03-24-2016
Kudos: 1170
Solutions: 491

Re: pixelserv dnat not working

[ Edited ]

Whats the output of:

sudo tcpdump -i eth2 -n -v host  192.168.1.100 or host w.x.y.z

while accessing an ad site with address w.x.y.z ?

 

Highlighted
Member
Posts: 201
Registered: ‎10-04-2014
Kudos: 17
Solutions: 2

Re: pixelserv dnat not working

Here you go:

tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size 262144 bytes
16:48:31.741150 IP (tos 0x0, ttl 128, id 14112, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.1.52.60391 > 62.26.68.51.80: Flags [S], cksum 0xfe39 (correct), seq 782299610, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:48:31.741915 IP (tos 0x0, ttl 128, id 14113, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.1.52.60392 > 62.26.68.51.80: Flags [S], cksum 0x9a9d (correct), seq 4154166394, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:48:31.992266 IP (tos 0x0, ttl 128, id 14116, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.1.52.60393 > 62.26.68.51.80: Flags [S], cksum 0x9d83 (correct), seq 555114521, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:48:34.743867 IP (tos 0x0, ttl 128, id 14122, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.1.52.60391 > 62.26.68.51.80: Flags [S], cksum 0xfe39 (correct), seq 782299610, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:48:34.746788 IP (tos 0x0, ttl 128, id 14123, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.1.52.60392 > 62.26.68.51.80: Flags [S], cksum 0x9a9d (correct), seq 4154166394, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:48:34.991859 IP (tos 0x0, ttl 128, id 14127, offset 0, flags [DF], proto TCP (6), length 52)
    192.168.1.52.60393 > 62.26.68.51.80: Flags [S], cksum 0x9d83 (correct), seq 555114521, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:48:40.744343 IP (tos 0x0, ttl 128, id 14157, offset 0, flags [DF], proto TCP (6), length 48)
    192.168.1.52.60391 > 62.26.68.51.80: Flags [S], cksum 0x1243 (correct), seq 782299610, win 8192, options [mss 1460,nop,nop,sackOK], length 0
16:48:40.748259 IP (tos 0x0, ttl 128, id 14158, offset 0, flags [DF], proto TCP (6), length 48)
    192.168.1.52.60392 > 62.26.68.51.80: Flags [S], cksum 0xaea6 (correct), seq 4154166394, win 8192, options [mss 1460,nop,nop,sackOK], length 0
16:48:40.990372 IP (tos 0x0, ttl 128, id 14161, offset 0, flags [DF], proto TCP (6), length 48)
    192.168.1.52.60393 > 62.26.68.51.80: Flags [S], cksum 0xb18c (correct), seq 555114521, win 8192, options [mss 1460,nop,nop,sackOK], length 0
Senior Member
Posts: 4,160
Registered: ‎03-24-2016
Kudos: 1170
Solutions: 491

Re: pixelserv dnat not working

nat rule 1 seems disfunctional.  Double check address-group ADS-A

 

Also, command below will show how many times NAT rule has been hit

sudo iptables -t nat -L -v

 

Member
Posts: 201
Registered: ‎10-04-2014
Kudos: 17
Solutions: 2

Re: pixelserv dnat not working

[ Edited ]

I thought that too.  I ran "ipset list ADS-A" and it returned a lot of IP Addresses.  I picked one of those returned for the following testing.

 

Running the recommended command, I got the following...

 

NOTE, I ran this exact same command twice.  1st to get a "baseline".  Then I ran a test with my browser to try and get to one of the servers in the ADS-A list to see which, if any, packet counters would change.  Obviously there is other stuff going on in my network at the same time, but I thought it might be worth a try.  It looks to me that the rule I'm trying to get working is getting triggered.  See below...

 

Baseline:

iptables -t nat -L -v
Chain PREROUTING (policy ACCEPT 687K packets, 86M bytes)
 pkts bytes target     prot opt in     out     source               destination 
 928K  114M MINIUPNPD  all  --  any    any     anywhere             anywhere    
 928K  114M UBNT_PFOR_DNAT_HOOK  all  --  any    any     anywhere             anywhere
 928K  114M VYATTA_PRE_DNAT_HOOK  all  --  any    any     anywhere             anywhere
 1659 99332 DNAT       tcp  --  eth2   any     192.168.1.0/24       anywhere             multiport dports http,https match-set ADS-A dst /* NAT-1 */ to:192.168.1.100:50000
    0     0 DNAT       udp  --  eth0   any     anywhere             192.168.1.200        /* NAT-5 */ to:192.168.2.200:10000-20000
    5   339 DNAT       udp  --  eth1   any     anywhere             anywhere             multiport dports 10000:20000 /* NAT-10 */ to:192.168.1.200
  438 26583 DNAT       tcp  --  eth0   any     anywhere             anywhere             tcp dpt:imaps /* NAT-20 */ to:192.168.1.100:993
   30  1800 DNAT       tcp  --  eth0   any     anywhere             anywhere             tcp dpt:32400 /* NAT-30 tcp_udp */ to:192.168.1.134:32400
    0     0 DNAT       udp  --  eth0   any     anywhere             anywhere             udp dpt:32400 /* NAT-30 tcp_udp */ to:192.168.1.134:32400

Chain INPUT (policy ACCEPT 132K packets, 12M bytes)
 pkts bytes target     prot opt in     out     source               destination 

Chain OUTPUT (policy ACCEPT 460K packets, 33M bytes)
 pkts bytes target     prot opt in     out     source               destination 

Chain POSTROUTING (policy ACCEPT 139K packets, 8203K bytes)
 pkts bytes target     prot opt in     out     source               destination 
 893K   76M UBNT_VPN_IPSEC_SNAT_HOOK  all  --  any    any     anywhere             anywhere
 893K   76M MINIUPNPD-POSTROUTING  all  --  any    any     anywhere             anywhere
 893K   76M UBNT_PFOR_SNAT_HOOK  all  --  any    any     anywhere             anywhere
 452K   44M MASQUERADE  all  --  any    eth0    anywhere             anywhere             /* NAT-5000 */
 248K   19M MASQUERADE  all  --  any    eth1    anywhere             anywhere             /* NAT-5002 */
    0     0 MASQUERADE  tcp  --  any    eth2    anywhere             app2.xyz.com  tcp dpt:50000 /* NAT-6000 */
 188K   11M VYATTA_PRE_SNAT_HOOK  all  --  any    any     anywhere             anywhere

Chain MINIUPNPD (1 references)
 pkts bytes target     prot opt in     out     source               destination 

Chain MINIUPNPD-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination 

Chain UBNT_PFOR_DNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 6585  434K UBNT_PFOR_DNAT_RULES  all  --  eth1   any     anywhere             anywhere             match-set ADDRv4_eth1 dst
    0     0 UBNT_PFOR_DNAT_RULES  all  --  eth2   any     anywhere             anywhere             match-set ADDRv4_eth1 dst

Chain UBNT_PFOR_DNAT_RULES (2 references)
 pkts bytes target     prot opt in     out     source               destination 
   23  1260 DNAT       tcp  --  any    any     anywhere             anywhere             tcp dpt:imaps to:192.168.1.100
    2    80 DNAT       tcp  --  any    any     anywhere             anywhere             tcp dpt:5001 to:192.168.1.254
    0     0 DNAT       udp  --  any    any     anywhere             anywhere             udp dpt:5001 to:192.168.1.254
    6   360 DNAT       tcp  --  any    any     anywhere             anywhere             tcp dpt:32400 to:192.168.1.134
    0     0 DNAT       udp  --  any    any     anywhere             anywhere             udp dpt:32400 to:192.168.1.134

Chain UBNT_PFOR_SNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 5197 1454K UBNT_PFOR_SNAT_RULES  all  --  any    eth2    anywhere             anywhere

Chain UBNT_PFOR_SNAT_RULES (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 4693 1423K MASQUERADE  all  --  any    eth2    anywhere             anywhere             match-set NETv4_eth2 src

Chain UBNT_VPN_IPSEC_SNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 

Chain VYATTA_PRE_DNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 928K  114M RETURN     all  --  any    any     anywhere             anywhere    

Chain VYATTA_PRE_SNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 188K   11M RETURN     all  --  any    any     anywhere             anywhere    

After trying to GET a fake page on one of the IP Addresses listed in the ADS-A group:

iptables -t nat -L -v
Chain PREROUTING (policy ACCEPT 688K packets, 86M bytes)
 pkts bytes target     prot opt in     out     source               destination 
 928K  114M MINIUPNPD  all  --  any    any     anywhere             anywhere    
 928K  114M UBNT_PFOR_DNAT_HOOK  all  --  any    any     anywhere             anywhere
 928K  114M VYATTA_PRE_DNAT_HOOK  all  --  any    any     anywhere             anywhere
 1668 99788 DNAT       tcp  --  eth2   any     192.168.1.0/24       anywhere             multiport dports http,https match-set ADS-A dst /* NAT-1 */ to:192.168.1.100:50000
    0     0 DNAT       udp  --  eth0   any     anywhere             192.168.1.200        /* NAT-5 */ to:192.168.2.200:10000-20000
    5   339 DNAT       udp  --  eth1   any     anywhere             anywhere             multiport dports 10000:20000 /* NAT-10 */ to:192.168.1.200
  438 26583 DNAT       tcp  --  eth0   any     anywhere             anywhere             tcp dpt:imaps /* NAT-20 */ to:192.168.1.100:993
   30  1800 DNAT       tcp  --  eth0   any     anywhere             anywhere             tcp dpt:32400 /* NAT-30 tcp_udp */ to:192.168.1.134:32400
    0     0 DNAT       udp  --  eth0   any     anywhere             anywhere             udp dpt:32400 /* NAT-30 tcp_udp */ to:192.168.1.134:32400

Chain INPUT (policy ACCEPT 132K packets, 12M bytes)
 pkts bytes target     prot opt in     out     source               destination 

Chain OUTPUT (policy ACCEPT 461K packets, 33M bytes)
 pkts bytes target     prot opt in     out     source               destination 

Chain POSTROUTING (policy ACCEPT 139K packets, 8212K bytes)
 pkts bytes target     prot opt in     out     source               destination 
 893K   76M UBNT_VPN_IPSEC_SNAT_HOOK  all  --  any    any     anywhere             anywhere
 893K   76M MINIUPNPD-POSTROUTING  all  --  any    any     anywhere             anywhere
 893K   76M UBNT_PFOR_SNAT_HOOK  all  --  any    any     anywhere             anywhere
 453K   44M MASQUERADE  all  --  any    eth0    anywhere             anywhere             /* NAT-5000 */
 248K   19M MASQUERADE  all  --  any    eth1    anywhere             anywhere             /* NAT-5002 */
    0     0 MASQUERADE  tcp  --  any    eth2    anywhere             app2.xyz.com  tcp dpt:50000 /* NAT-6000 */
 188K   11M VYATTA_PRE_SNAT_HOOK  all  --  any    any     anywhere             anywhere

Chain MINIUPNPD (1 references)
 pkts bytes target     prot opt in     out     source               destination 

Chain MINIUPNPD-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination 

Chain UBNT_PFOR_DNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 6594  434K UBNT_PFOR_DNAT_RULES  all  --  eth1   any     anywhere             anywhere             match-set ADDRv4_eth1 dst
    0     0 UBNT_PFOR_DNAT_RULES  all  --  eth2   any     anywhere             anywhere             match-set ADDRv4_eth1 dst

Chain UBNT_PFOR_DNAT_RULES (2 references)
 pkts bytes target     prot opt in     out     source               destination 
   23  1260 DNAT       tcp  --  any    any     anywhere             anywhere             tcp dpt:imaps to:192.168.1.100
    2    80 DNAT       tcp  --  any    any     anywhere             anywhere             tcp dpt:5001 to:192.168.1.254
    0     0 DNAT       udp  --  any    any     anywhere             anywhere             udp dpt:5001 to:192.168.1.254
    6   360 DNAT       tcp  --  any    any     anywhere             anywhere             tcp dpt:32400 to:192.168.1.134
    0     0 DNAT       udp  --  any    any     anywhere             anywhere             udp dpt:32400 to:192.168.1.134

Chain UBNT_PFOR_SNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 5200 1455K UBNT_PFOR_SNAT_RULES  all  --  any    eth2    anywhere             anywhere

Chain UBNT_PFOR_SNAT_RULES (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 4696 1424K MASQUERADE  all  --  any    eth2    anywhere             anywhere             match-set NETv4_eth2 src

Chain UBNT_VPN_IPSEC_SNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 

Chain VYATTA_PRE_DNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 928K  114M RETURN     all  --  any    any     anywhere             anywhere    

Chain VYATTA_PRE_SNAT_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination 
 188K   11M RETURN     all  --  any    any     anywhere             anywhere    
Member
Posts: 201
Registered: ‎10-04-2014
Kudos: 17
Solutions: 2

Re: pixelserv dnat not working

May be obvious, but just checked and I have access-logging on for the pixelserv service (lighttpd)...no traffic seems to get there in these tests. Direct tests against the server (using the internal address) show on the service, and in the access log.
Senior Member
Posts: 4,160
Registered: ‎03-24-2016
Kudos: 1170
Solutions: 491

Re: pixelserv dnat not working

dnat rule seems to get hit

post your full config.

Member
Posts: 201
Registered: ‎10-04-2014
Kudos: 17
Solutions: 2

Re: pixelserv dnat not working

Here you go.  Too big to fit into a post.

 

config.boot

 

Thanks

Senior Member
Posts: 4,160
Registered: ‎03-24-2016
Kudos: 1170
Solutions: 491

Re: pixelserv dnat not working

Ahh load-balance

 

You don't have a lan2lan exclusion rule , like in

https://help.ubnt.com/hc/en-us/articles/205145990-EdgeRouter-Dual-WAN-Load-Balance-Feature

 

try adding a rule like below:

modify balance {
    rule 5 {
        action modify
        destination {
            group {
                address 192.168.1.0/24
            }
        }
        modify {
            table main
        }
    }

I'm not sure if dNAT takes place before modify.  If so , exchange address 192.168.1.0/24 for address-group ADS-A 

 

Member
Posts: 201
Registered: ‎10-04-2014
Kudos: 17
Solutions: 2

Re: pixelserv dnat not working

I appreciate you hanging in there with me, but that didn't seem to help.  I, in fact, added both modify rules (LAN nets and the ADS-A) to the beginning of the firewall rules and they aren't working.

 

I may need to hang it up and leave it.  It's starting to get pretty crazy to do this, and it seems so darn simple!

 

-AJ

Reply