We’ve been troubleshooting this problem for 4 months now and wish some concepts at this level.
Concern: ADP Time clock units at 2 of 4 areas are having delays stating “Ready for server” between 5 seconds to a number of minutes inflicting delays in worker clock-ins for the day and clock-outs for lunch and finish of shift.
Abstract: We’ve 4 areas, we’ll name A, B, C and D. Areas A and B have the problem, areas C and D don’t. All areas have time clocks for workers to punch out and in of (bought from ADP, InTouch 9000 units). The time clock units attain out to 2 WAN IP’s of ADP’s servers over 443.
Similarities:
- All areas have Sophos Firewalls operating the identical firmware, Web site-to-Web site connectivity
- All areas have Dell Switches linked to the Sophos firewall
Setup
Location A has 3 time clock units, all having the problem:
Sophos Firewall <-> N1548P Dell change (‘Server Swap’ firmware v 6.8.1.5 and 1 Timeclock patched in to this change) <-> N1548P Dell change (‘Workplace Swap’ firmware v 6.8.1.5 and 1 Timeclock patched in to this change) <-> N1524P Dell change (‘Shed Swap’ firmware v 6.8.1.5 and 1 Timeclock patched in to this change)
To reiterate, the Shed change is linked to the Workplace change, which is linked to the Server change, which is linked to the firewall. Every of those switches all have the identical firmware, and 1 timeclock patched in.
Location B has 1 time clock machine having the problem:
Sophos Firewall <-> N1108EP-ON Dell change (change firmware v 6.8.1.5 and 1 Timeclock patched in to this change)
Areas C and D have 1 time clock at every web site. The switches they’re plugged into are a Dell N1548P and a Dell N2024P. The N1548P is operating firmware 6.8.1.5, and the N2024P is operating firmware 6.7.1.4.
Here’s what I’ve gathered thus far:
- The problem appeared to have began on the identical time (not 100% positive although) for the two websites and the 4 units, however the problem by no means occurred on the different 2 websites. The problem was supposedly began in March.
- All Sophos firewalls are operating the identical firmware, SFOS 20.0.0 GA-Build222 in the meanwhile. With the problem beginning round March, and the final firewall replace carried out was again in December or January, I used to be questioning if the replace was delayed in inflicting points (perhaps Sophos firewalls do not play good natively with Dell switches on a regular basis). A firmware replace for the Dell switches was launched lately, so when pushed, abruptly, the problem went away for two weeks. Then, our engineers needed to push a firewall replace to SFOS 20.0.0.1, and all of a sudden they began reporting the problem was again. We did a reversion on the firewalls to see if this was certainly a bug, and now it is again to SFOS 20.0.0 and the problem persists. Perhaps we weren’t getting good information from the on-site workforce through the time it was ‘working’.
- The switches at one web site have the identical firmware and fashions as Location A, so I am pondering it isn’t mannequin or firmware particular at this level.
- I’ve run 2 completely different packet captures, one from the firewall and one doing port mirroring on the Server change at Location A to our probe. The firewall’s findings present the message “Couldn’t affiliate packet to any connection” and in keeping with Sophos help, it means one thing downstream precipitated a delay within the packet to be despatched up. The packet seize from the mirrored port reveals a variety of resets and retransmissions from the time clock machine to the change.
- We had been hoping it was a tool problem, however all 4 units taking place on the identical time appeared too large a coincidence to be machine and even cable problem. We had ADP change one of many time clocks as nicely, to no avail. Similar problem stored occurring.
My subsequent steps:
- On-Web site useful resource to straight join time clock machine to Sophos firewall to see if the problem persists
Aside from that, I am curious what else we should always troubleshoot right here or different packet captures to attempt to take a look at with.