Scheduled maintenance: Community available only in read-only mode until 6:00 AM (PT)
Emerging Member
Posts: 82
Registered: ‎07-02-2014
Kudos: 26
Solutions: 4

UniFi Security Gateway 3P unstable if it gets load

One of my USG's droppes all network connections as soon as it get a bit load!

Most of the Time all VPN (Site-2-Site) Connections drop, sometimes even the communication to the Cloud Key (local on the same site) droppes.

Screenshot 2018-10-05 at 12.53.05.png

 

The Log is flodded with this:

Oct  5 12:26:05 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:26:05 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:26:08 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:26:08 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:26:10 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:26:12 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:26:14 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:26:14 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:26:14 FRA2PSES02 mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Oct  5 12:26:24 FRA2PSES02 perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Oct  5 12:26:24 FRA2PSES02 perl_wrapper: 22 open retries at /usr/bin/perl_wrapper.pl line 58.
Oct  5 12:26:24 FRA2PSES02 perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Oct  5 12:26:24 FRA2PSES02 perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.
Oct  5 12:29:36 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:29:36 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:29:38 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:29:38 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:29:40 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:29:40 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:29:42 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:29:43 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:29:43 FRA2PSES02 mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Oct  5 12:29:52 FRA2PSES02 perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Oct  5 12:29:52 FRA2PSES02 perl_wrapper: 29 open retries at /usr/bin/perl_wrapper.pl line 58.
Oct  5 12:29:52 FRA2PSES02 perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Oct  5 12:29:52 FRA2PSES02 perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.
Oct  5 12:32:12 FRA2PSES02 mca-monitor: mca-client.service(): Failed sending request to '/tmp/.mcad' - 'Resource temporarily unavailable'
Oct  5 12:33:34  mca-monitor: last message repeated 3 times
Oct  5 12:34:34  mca-monitor: last message repeated 4 times
Oct  5 12:35:14 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:35:14 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:35:15 FRA2PSES02 mca-monitor: mca-client.service(): Failed sending request to '/tmp/.mcad' - 'Resource temporarily unavailable'
Oct  5 12:35:17 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:35:17 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:35:19 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:35:19 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:35:22 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 12:35:22 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 12:35:22 FRA2PSES02 mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Oct  5 12:35:34 FRA2PSES02 perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Oct  5 12:35:35 FRA2PSES02 perl_wrapper: 16 open retries at /usr/bin/perl_wrapper.pl line 58.
Oct  5 12:35:35 FRA2PSES02 perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Oct  5 12:35:35 FRA2PSES02 perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.
Oct  5 12:36:00 FRA2PSES02 mca-monitor: mca-client.service(): Failed sending request to '/tmp/.mcad' - 'Resource temporarily unavailable'
Oct  5 12:36:36  mca-monitor: last message repeated 3 times
Oct  5 12:36:36 FRA2PSES02 syswrapper: kill-mcad. reason: mcad not responding
Oct  5 12:36:51 FRA2PSES02 mcad: ace_reporter.set_default_localtime(): failed to get /usr/lib/build.properties
Oct  5 12:37:05 FRA2PSES02 redirector: redirector.sigint_handler(): receive signal 15
Oct  5 12:38:53 FRA2PSES02 kernel: [sched_delayed] sched: RT throttling activated
Oct  5 12:41:52 FRA2PSES02 mcad: ace_reporter.create_socket(): create_socket(ipv4): SO_LINGER 2:No such file or directory
Oct  5 12:41:52 FRA2PSES02 mcad: mcad-discover.init_mcast_sock(): init_mcast_socket(ipv4): SO_LINGER 2:No such file or directory

Since I saw that problem the first time, things get worst: If I download an ISO (in my example i downloaded the latest Centos ISO) the USG get's very instable!!!

 

Setup:

Model: UniFi Security Gateway 3P
Version: 4.4.29.5124210

CLOUD KEY: UCK.mtk7623.v0.12.0.29a26c9.181001.1444
Controller Version: 5.8.30-11076-1
 
Offload is enabled:
Screenshot 2018-10-05 at 12.56.39.png
 
I also saw, that the speetest returned very stange values from time to time:
Screenshot 2018-10-05 at 12.46.34.pngScreenshot 2018-10-05 at 12.47.00.png
The connection has just 90Mbps/30Mbps, but sometimes to upload speed seems to be above 100Mbps. My Carrier is German Telekom and the VDSL/PPPOE Modem is a Vigor 130.
 
If I use the Vigor as a Router, the connection is very(!) stable and the speedtest doesn't show any of the creapy peaks. But double NAT is something that the USG handels very (very) bad. So I try everything to avoid that.
 
A few more details could be found in the other post I mentioned above.
 
Any idea?
--
Cheers
Josh
Emerging Member
Posts: 82
Registered: ‎07-02-2014
Kudos: 26
Solutions: 4

Re: UniFi Security Gateway 3P unstable if it gets load

Could this be related to IPv6 in any kind?

 

What I saw:

Screenshot 2018-10-05 at 14.34.20.png

The dhcpv6-pd-respo process on this device is consuming a lot of CPU!

The load goes up (over 85 percent) and down (as low as 2 percent). But It's above 40 percent most of the time.

 

And this is with NO load:

Screenshot 2018-10-05 at 15.18.58.png

 

My carrier provides a /64 routing network (The USG gets one within this rage) and an larger /56 network for my network.

My VLAN's use small chunks (several /64 networks) out of the delegated /56 network. The USG is the router for all of them and do the DHCP handling.

 

When I disable IPv6 on the Windows 10 host (internal) the download starts and the USG looks stable. As soon as I enable IPv6 again (The Box gets an address from the USG based DHCP Server) the USG get instable when I start the download.

 

And I just found this in the logs:

Oct  5 14:51:13 FRA2PSES02 mca-monitor: mca-client.service(): Failed sending request to '/tmp/.mcad' - 'Resource temporarily unavailable'
Process 29498 (linkcheck) has crashed (parent 1 (init) signal 10, code 128, addr            (nil)), coredumps disabled
Oct  5 15:07:49 FRA2PSES02 kernel: Process 29498 (linkcheck) has crashed (parent 1 (init) signal 10, code 128, addr            (nil)), coredumps disabled
Oct  5 15:10:43 FRA2PSES02 syswrapper: kill-linkcheck. reason: linkcheck not updating

For your Information: I'm connected remotely and the Windows 10 Box is connected to the USG console Port.

 

--
Cheers
Josh
New Member
Posts: 1
Registered: ‎12-31-2017

Re: UniFi Security Gateway 3P unstable if it gets load

I loaded this firmware upgrade and now my USG cannot grab a dhcp assigned IP address from my ISP. Do you know what could be causing this. The cable modem is set to bridge mode and isp assigns wan ip and gateway via dhcp

Emerging Member
Posts: 82
Registered: ‎07-02-2014
Kudos: 26
Solutions: 4

Re: UniFi Security Gateway 3P unstable if it gets load

What I had to do in the past when the IPv6 did come up:

release dhcpv6-pd interface pppoe0
delete dhcpv6-pd duid
renew dhcpv6-pd interface pppoe0

Not with the latest Firmware, but from time to time the IPv6 connection on the USG seems to cause issue.

That solved at least all my issues.

--
Cheers
Josh
Emerging Member
Posts: 82
Registered: ‎07-02-2014
Kudos: 26
Solutions: 4

Re: UniFi Security Gateway 3P unstable if it gets load

I saw that Firmware v4.4.31dev available for testing.

I'm not a big fan of beta testing on this device, but due to the very unstable condition I tried it.

The load went back to normal, the external interface has an IPv6 address out of the /64 network assigned, but the inetrnal IPv6 doesn't seem to work.

Ping from the USG to external IP adresses worked fine, from any client not.

 

So I did the following (Solved most of my issues with IPv6 before):

release dhcpv6-pd interface pppoe0
release dhcpv6-pd interface eth0
delete dhcpv6-pd duid
renew dhcpv6-pd interface pppoe0

And the load is up to 50 percent and more:

Screenshot 2018-10-05 at 16.16.09.pngScreenshot 2018-10-05 at 16.15.43.png

 

On another Site the USG has a low CPU usage, even with more traffic:

Screenshot 2018-10-05 at 16.19.22.png

Same model, Firmware is on the release version and IPv6 is also enabled.

 

UBNT equipment and IPv6 is strage from time to time Man Sad

 

--
Cheers
Josh
Emerging Member
Posts: 82
Registered: ‎07-02-2014
Kudos: 26
Solutions: 4

Re: UniFi Security Gateway 3P unstable if it gets load

[ Edited ]

I have no idea what to do! The USG is totally instable. As soon as a download starts, the USG dopes all connections locally and remote. All VPN connection get disconnected imidically. 

And I did a few more tests: Only when IPv6 is used! 

 

I triple checked: I did everything according to the Welcome to UniFi IPv6 article from @UBNT-cmb.

I found the IPv6 issues - DHCPv6-PD-Response issues causing high USG CPU entry

That solved the load problem, but the system is still totally instable.

By the end of last week, I also tried a complete reset of the system (factory reset and re-adopted it). That doesn't change a thing.
I would drop IPv6, but I already use it externally to access a couple of services inside of the site. And we have at least one (cable) user who desperately needs IPv6.
 
Screenshot 2018-10-05 at 18.59.11.png
 
I will downgrade the USG to the release now, cause the DEV Build doesn't fix my issue Man Sad
Oct  5 18:00:58 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:00:59 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:01:01 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:01:01 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:01:03 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:01:03 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:01:06 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:01:06 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:01:06 FRA2PSES02 mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Oct  5 18:01:15 FRA2PSES02 perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Oct  5 18:01:15 FRA2PSES02 perl_wrapper: 11 open retries at /usr/bin/perl_wrapper.pl line 58.
Oct  5 18:01:15 FRA2PSES02 perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Oct  5 18:01:15 FRA2PSES02 perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.
Oct  5 18:04:26 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:04:26 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:04:29 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:04:29 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:04:31 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:04:31 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:04:33 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:04:33 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:04:33 FRA2PSES02 mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Oct  5 18:04:43 FRA2PSES02 perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Oct  5 18:04:43 FRA2PSES02 perl_wrapper: 19 open retries at /usr/bin/perl_wrapper.pl line 58.
Oct  5 18:04:43 FRA2PSES02 perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Oct  5 18:04:43 FRA2PSES02 perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.
Oct  5 18:06:51 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:06:51 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:06:53 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:06:53 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:06:55 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:06:55 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:06:58 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:06:58 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:06:58 FRA2PSES02 mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Oct  5 18:07:09 FRA2PSES02 perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Oct  5 18:07:09 FRA2PSES02 perl_wrapper: 16 open retries at /usr/bin/perl_wrapper.pl line 58.
Oct  5 18:07:09 FRA2PSES02 perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Oct  5 18:07:09 FRA2PSES02 perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.
Oct  5 18:10:39 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:10:39 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:10:41 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:10:41 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:10:43 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:10:44 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:10:46 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:10:46 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:10:46 FRA2PSES02 mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Oct  5 18:10:54 FRA2PSES02 perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Oct  5 18:10:55 FRA2PSES02 perl_wrapper: 10 open retries at /usr/bin/perl_wrapper.pl line 58.
Oct  5 18:10:55 FRA2PSES02 perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Oct  5 18:10:55 FRA2PSES02 perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.
Oct  5 18:13:55 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:13:55 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:13:58 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:13:58 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:14:00 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:14:01 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:14:03 FRA2PSES02 mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Oct  5 18:14:03 FRA2PSES02 mcad: perl_wrapper.perl_request_response(): error reading SOM
Oct  5 18:14:03 FRA2PSES02 mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Oct  5 18:14:12 FRA2PSES02 perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Oct  5 18:14:12 FRA2PSES02 perl_wrapper: 16 open retries at /usr/bin/perl_wrapper.pl line 58.
Oct  5 18:14:12 FRA2PSES02 perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Oct  5 18:14:12 FRA2PSES02 perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.
Maybe I need to drop the USGs due to the issues with double NAT (More stable, much fewer functionalities) and the missing IPv6 stuff (like using it within a VPN tunnel, and more).
--
Cheers
Josh
Emerging Member
Posts: 82
Registered: ‎07-02-2014
Kudos: 26
Solutions: 4

Re: UniFi Security Gateway 3P unstable if it gets load

interesting: A reply from me to my own post that contains some forum internal links and a mention of the author was removed by a moderator as Spam!

 

Maybe, but just maybe, because I criticised the IPv6 support?

 

Time to think about an alternative, better than stay with a total unstable solution... Right?

--
Cheers
Josh
Highlighted
Member
Posts: 183
Registered: ‎10-24-2017
Kudos: 22
Solutions: 7

Re: UniFi Security Gateway 3P unstable if it gets load

I had the same logs with USG firmware version 4.4.29.5124210 - AND KILLS MY NETWORK.

 

I was doing an internal large file transfer between VLANs - the USG acting as the Router on a Stick - and during this time my internet got killed for other clients (no DNS resolution) and the USG was failing to report to the Unifi controller. 

 

The file transfer was only hitting ~11MB/s (annoyingly even this seemed slow for a 1000Mb switched home network with no other active clients)

 

ANSWER:

My issue was that IPS is enabled - restricting the USG to 85mbps - which explains the poor transfer speeds and confirms that the USG starts smoking when pushed to 100%

 

FIX:

Disable IPS;

or rate limit switch ports to lower than 80mbps;

or use USB sticks to transfer large files;

or go back to pen and paper

 

Capture.JPG

 

 

My logs:

Thu Nov 08 10:45:35 2018;192.168.254.1; <12>Nov 8 10:45:18 BookerNETGateway mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Thu Nov 08 10:45:38 2018;192.168.254.1; <11>Nov 8 10:45:21 BookerNETGateway mcad: perl_wrapper.perl_request_response(): error reading SOM
Thu Nov 08 10:45:40 2018;192.168.254.1; <12>Nov 8 10:45:23 BookerNETGateway mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Thu Nov 08 10:45:41 2018;192.168.254.1; <11>Nov 8 10:45:23 BookerNETGateway mcad: perl_wrapper.perl_request_response(): error reading SOM
Thu Nov 08 10:45:42 2018;192.168.254.1; <12>Nov 8 10:45:26 BookerNETGateway mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Thu Nov 08 10:45:44 2018;192.168.254.1; <11>Nov 8 10:45:26 BookerNETGateway mcad: perl_wrapper.perl_request_response(): error reading SOM
Thu Nov 08 10:45:45 2018;192.168.254.1; <12>Nov 8 10:45:28 BookerNETGateway mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Thu Nov 08 10:45:46 2018;192.168.254.1; <11>Nov 8 10:45:28 BookerNETGateway mcad: perl_wrapper.perl_request_response(): error reading SOM
Thu Nov 08 10:45:46 2018;192.168.254.1; <11>Nov 8 10:45:28 BookerNETGateway mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Thu Nov 08 10:45:49 2018;192.168.254.1; <4>Nov 8 10:45:32 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=80.82.64.116 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=236 ID=1853 PROTO=TCP SPT=57573 DPT=5743 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:45:50 2018;192.168.254.1; <172>Nov 8 10:45:34 BookerNETGateway perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Thu Nov 08 10:45:54 2018;192.168.254.1; <172>Nov 8 10:45:37 BookerNETGateway perl_wrapper: 20 open retries at /usr/bin/perl_wrapper.pl line 58.
Thu Nov 08 10:45:54 2018;192.168.254.1; <172>Nov 8 10:45:37 BookerNETGateway perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Thu Nov 08 10:45:56 2018;192.168.254.1; <172>Nov 8 10:45:39 BookerNETGateway perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.
Thu Nov 08 10:45:57 2018;192.168.254.1; <4>Nov 8 10:45:40 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=94.102.56.235 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=236 ID=2310 PROTO=TCP SPT=43315 DPT=6807 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:46:03 2018;192.168.254.1; <4>Nov 8 10:45:46 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=94.102.56.235 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=234 ID=17284 PROTO=TCP SPT=43214 DPT=6148 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:46:54 2018;192.168.254.1; <4>Nov 8 10:46:37 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=80.82.64.116 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=234 ID=2935 PROTO=TCP SPT=57525 DPT=5530 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:46:57 2018;192.168.254.1; <4>Nov 8 10:46:39 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=185.143.223.168 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=231 ID=40742 PROTO=TCP SPT=58696 DPT=8080 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:46:59 2018;192.168.254.1; <11>Nov 8 10:46:42 BookerNETGateway mcad: ace_reporter.reporter_fail(): Server Reject (http://192.168.2.254:8080/inform)
Thu Nov 08 10:47:00 2018;192.168.254.1; <11>Nov 8 10:46:42 BookerNETGateway mcad: ace_reporter.reporter_fail(): inform failed #2 (last inform: 367 seconds ago), rc=5
Thu Nov 08 10:47:28 2018;192.168.254.1; <4>Nov 8 10:47:11 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=80.82.64.116 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=236 ID=63550 PROTO=TCP SPT=57597 DPT=5996 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:48:01 2018;192.168.254.1; <4>Nov 8 10:47:44 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=80.82.64.116 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=236 ID=18741 PROTO=TCP SPT=57573 DPT=5721 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:48:11 2018;192.168.254.1; <4>Nov 8 10:47:54 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=89.248.168.107 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=234 ID=6075 PROTO=TCP SPT=53507 DPT=4557 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:48:19 2018;192.168.254.1; <4>Nov 8 10:48:02 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=94.102.56.235 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=234 ID=37757 PROTO=TCP SPT=43230 DPT=6280 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:48:27 2018;192.168.254.1; <4>Nov 8 10:48:10 BookerNETGateway kernel: ALIEN BLOCK: IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=176.119.7.38 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=233 ID=55352 PROTO=TCP SPT=49819 DPT=7985 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:48:56 2018;192.168.254.1; <4>Nov 8 10:48:39 BookerNETGateway kernel: [WAN_LOCAL-4000-D]IN=eth0 OUT= MAC=f0:9f:c2:c5:49:ca:70:1f:53:cb:78:19:08:00 src=185.143.223.153 DST=110.23.42.245 LEN=40 TOS=0x00 PREC=0x00 TTL=233 ID=48611 PROTO=TCP SPT=57473 DPT=40004 WINDOW=1024 RES=0x00 SYN URGP=0
Thu Nov 08 10:49:25 2018;192.168.254.1; <12>Nov 8 10:49:08 BookerNETGateway mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Thu Nov 08 10:49:27 2018;192.168.254.1; <11>Nov 8 10:49:08 BookerNETGateway mcad: perl_wrapper.perl_request_response(): error reading SOM
Thu Nov 08 10:49:28 2018;192.168.254.1; <12>Nov 8 10:49:11 BookerNETGateway mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Thu Nov 08 10:49:30 2018;192.168.254.1; <11>Nov 8 10:49:11 BookerNETGateway mcad: perl_wrapper.perl_request_response(): error reading SOM
Thu Nov 08 10:49:31 2018;192.168.254.1; <12>Nov 8 10:49:14 BookerNETGateway mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Thu Nov 08 10:49:31 2018;192.168.254.1; <11>Nov 8 10:49:14 BookerNETGateway mcad: perl_wrapper.perl_request_response(): error reading SOM
Thu Nov 08 10:49:33 2018;192.168.254.1; <12>Nov 8 10:49:16 BookerNETGateway mcad: perl_wrapper.read_with_timeout(): timeout on select() for response pipe
Thu Nov 08 10:49:35 2018;192.168.254.1; <11>Nov 8 10:49:16 BookerNETGateway mcad: perl_wrapper.perl_request_response(): error reading SOM
Thu Nov 08 10:49:36 2018;192.168.254.1; <11>Nov 8 10:49:16 BookerNETGateway mcad: mca-edgemax.egdemax_stats_vpn_status(): error getting VPN status
Thu Nov 08 10:49:51 2018;192.168.254.1; <172>Nov 8 10:49:35 BookerNETGateway perl_wrapper: Timeout opening /var/run/perl_wrapper_resp_fifo at /usr/bin/perl_wrapper.pl line 51.
Thu Nov 08 10:49:52 2018;192.168.254.1; <172>Nov 8 10:49:35 BookerNETGateway perl_wrapper: 18 open retries at /usr/bin/perl_wrapper.pl line 58.
Thu Nov 08 10:49:52 2018;192.168.254.1; <172>Nov 8 10:49:35 BookerNETGateway perl_wrapper: timeout opening response FIFO for write at /usr/bin/perl_wrapper.pl line 209.
Thu Nov 08 10:49:53 2018;192.168.254.1; <172>Nov 8 10:49:35 BookerNETGateway perl_wrapper: request_response returned error at /usr/bin/perl_wrapper.pl line 301.

 

New Member
Posts: 9
Registered: ‎04-25-2018

Re: UniFi Security Gateway 3P unstable if it gets load

Did anyone get a reason for the "mcad: perl_wrapper.perl_request_response(): error reading SOM" messages??
Emerging Member
Posts: 82
Registered: ‎07-02-2014
Kudos: 26
Solutions: 4

Re: UniFi Security Gateway 3P unstable if it gets load

Hi @edwarj3,

 

nope! That happens with the latest firmware as well. Much better, but it still happens from time to time.

What I saw it happens most of the time, when the system get a lot of load (Traffic to handle).

 

Not using the the automated (scheduled) Speedtest feature mitigated that a lot on my USG. But it still happens from time to time, then all the connections are dropped most of the time.

--
Cheers
Josh
New Member
Posts: 9
Registered: ‎04-25-2018

Re: UniFi Security Gateway 3P unstable if it gets load

Thanks Josh --- for me these messages seem to be occuring randomly abd not when the network is under any particular or high load. Also they may occur once or several at a time. I have tried to correlate them with other events (I have set up a Syslog server to capture all logs from most of my devices into the same log) but there is nothing which correlates.

 

There are some posts which suggest that this may be connected with IPV6 and also the MTU setting between the USG and the modem. What MTU setting do you have and do you have any IPV6 traffic which could be causing this?

 

I have a Vigor 130 modem connected to BT in the UK with MTU set at 1492. I have read in some postings that 1500 is best --- and in fact looking at the messages at boot-up for the USG it seems to change it to 1492 from 1500.... so I may change it to 1500 and see what happens. I do not have IPV6 configured, but there may be a device which requests it from time to time...

 

Any thoughts on this ? Do you think it could be connected?

Emerging Member
Posts: 82
Registered: ‎07-02-2014
Kudos: 26
Solutions: 4

Re: UniFi Security Gateway 3P unstable if it gets load

Hi @edwarj3,

 

I changed the MTU to 1492 very early. Documented in my blog.

I see the message from time to time (with and without load). Sometimes, when the load is high, then the USG gets unstable and drops all running connections.

 

I didn’t found the root cause. But: I have a WSUS Server running in the office. With the last patch day, I saw the USG unstable again. I use a remote logging (syslog and Graylog) and around 30 seconds after the message the termination of both permanent VPN connection was logged.

 

With the latest firmware, things changed: Better! Much better, but still not fixed.

--
Cheers
Josh