Internet Goes Down Every Day - Not sure what else to try

[BadBoy House]

We’ve got a really strange problem with our own in-house broadband connection. I’ve been fighting it for the last few weeks without any real success. Would appreciate the opinion of fellow forum members on this.

The problem is as follows:-

Every day we lose internet connectivity and we can’t access websites. This occurs first thing in the morning. I’ve received reports that websites cannot be accessed at around 7.30AM when there’s only been one or two staff members in the building.

Restarting the router brings the connection back online.

There is one key point that I need to mention: When the problem occurs we can ALWAYS access the website of the company who supplies our main line of business financial application (we are an accountancy firm for whom I am the IT manager). This is always uncached.

Other than their website we can’t access any other websites until we’ve restarted the router.

I’ve reached out to the IT support manager of the line of business software to find out if he’s seen this before.

There are a few other points I should mention:-

- When the problem occurs we can ping websites by name and IP address.

- When the problem occurs we’re still unable to access websites even if we just have a laptop connected into the router and nothing else. Until we restart the router we can only access our software supplier’s website.

- We've tried various different known working routers - different makes/models and had the same problem.

- Our ISP - BT have performed many line tests all of which have come back as ok. They have also reset the exchange hardware.

My gut feeling is that this has to be something to do with the line of business application somewhere along the line - I can't figure out though why we can access their site as if there isn't a problem yet can't access any other sites.

Appreciate any opinions or suggestions on this.

Cheers.

 

[glricht]

Since the problem occurs with different routers, it's *probably* not a router problem. But if it occurs usually at around 7:30am, I'd check the AC power circuit that the router is on. Maybe it's on the same circuit as a coffee pot or microwave oven, which when turned on, causes voltage problems?

Just a quick thought.

 

[FremontPC]

Can you access the router's admin interface?

Is the site in question accessing your network after hours?

 

[YeOldeStonecat]

What type of modem/gateway is in front of your router? Is it a pure bridged modem? Or is it actually a combo modem/router...(gateway)? If so...is your router setup properly behind it so that there is NO double NAT?

What is being used for DNS for your network? Is there a Windows server running as a DC...therefore of course it's doing DNS for the workstations? Or is it just a peer to peer network...so you're stuck relying on the router for your DNS?

I know you said you can ping by name and IP....but the key is did you get healthy replies? Everyone can ping something....but the clue is in if you get replies or not.

If you're just using your router..what is it using for its DNS? Typically most people accept the default ISP DNS service...let the router do "obtain auto" on its WAN interface. But many ISPs DNS services suck (surprise surprise). I hardly ever use them. As as test...try manually setting the routers WAN interface to use a safe DNS service like OpenDNS's servers. ( over here in the US I use 208.67.222.222)...not sure if they balance those same IPs over your side of the pond or if they have more local ones. But I'd be curious to change those out as a test.

What is happening at night? Any daily uploads or downloads happening that might swamp your routers state tables with tons of traffic and sessions?

 

[SAG]

Lots of good stuff here already, but might I suggest one further test.

I had a client who had a similar issue. They could ping successfully but not access any web pages besides their ISP.

I have a small server I host for testing client connectivity and the like, we'll call it xyz.com

They could ftp to xyz.com, but not http to it. Even a telnet to port 80 of xyz.com would fail.
Turned out to be a corrupt router firmaware in the gateway supplied by the ISP.

See if they can ftp when http is failing. And see if you can telnet to port 80 of a known good site. Corrupt firmware can cause all sort of things that can not be explained.

 

[brandonkick]

So when this problem occurs at 7:30 AM, and you reset the router, your good to go for the rest of the day? Are you sure it happens in a short and specific time frame?

Alternatively, what happens if you plug directly into the modem / gateway? As stonecat said, make sure if it's a modem in bridge mode or if it's a gateway (modem and router built into one).

Has this problem just cropped up? How long were things working okay before that (if at all)? What has changed since the problem started (if anything)?

 

[Cambridge PC Support]

You don't happen to have a cleaner working at about 7.30 by any chance?

 

[BadBoy House]

Thanks for all the replies. Much appreciated.


Just to let you know where we are at the minute with this problem, I called BT (our ISP) first thing Monday as I previously reported the problem to them last week.

They told me on Monday that fault had been fixed by them "resetting equipment at the exchange" and that I should reboot the router in order to re-sync it.

Funnily enough the problem hasn't occurred since but I'm skeptical about it. All along I thought it was a problem at their end but I can't help thinking it's a bit of a conincidence that we've not had problems since.

Anyway...

The problem usually occurs when there is either nobody in the building or just a couple of people.

I can access the router admin interface fine when the problem occurs and the line of busiess app's supplier site doesn't access the network at all.

The network diagram looks like this:

ISP
|
Router
|
Untangle Server
|
LAN Switch
|
LAN Computers/Servers


When the problem occurs, if I disconnect the router from the Untangle server and just have a laptop in the router I still can't access any websites other than the line of business supplier's website. If I restart the router it's fine again.

Also, when the problem occurs we can ping websites by name and IP and we get good response times.

As the problem hasn't occurred since I've not been able to try FTP or telnet etc but I will do if/when the problem occurs again.

After the problem occurs first thing and I reboot the router it's fine for the rest of the day. Weird eh?

The problem started out of the blue about 4 weeks ago. Nothing changed at all our end.

 

[YeOldeStonecat]

The network diagram looks like this:

ISP
|
Router
|
Untangle Server
|
LAN Switch
|
LAN Computers/Servers

.

What is the "router"?
And I take it Untangle is running in bridged mode? And not doing NAT itself?
Why not take the router out of the mix and let Untangle run in traditional gateway mode?

 

[BadBoy House]

Hi guys.

The problem occurred again today, however it didn't happen yesterday or Tuesday.

I spoke to the IT manager at the line of business app whose website we can always access when the problem occurs. They're stumped of course.

Today whilst the problem occurred, which was early on as usual, I performed the following tests:

Websites - As before I can only access the website of the supplier of our line of business accountancy software package. No other websites accessible.

Telnet - I was able to Telnet out to multiple external destinations.

FTP - FTP connects to the remote hosts but fails to display the directory listing.

Ping - I was able as before to ping websites fine with good response times.

Outlook Web Access - I was able to connect to Outlook Web Access on our server from an external connection (My iPhone).

Incoming Emails - Incoming emails were not coming in whilst the problem was occuring.

Outgoing Emails - Outgoing emails send out fine whilst the problem occurs.

The router is a BT Business Hub 3 although we've tried other routers.

Untangle runs in Router/Gateway mode.

 

[YeOldeStonecat]

The router is a BT Business Hub 3 although we've tried other routers.

Untangle runs in Router/Gateway mode.

So....you've got a router at the edge...and then another router behind it.....thus double NAT. Good grief WHY? Can you disable the NAT or do the public IP subnet passthrough and get Untangle pulling the public IPs on its WAN interface?

 

[BadBoy House]

...public IP subnet passthrough and get Untangle pulling the public IPs on its WAN interface?

It's configured like this.

 

[YeOldeStonecat]

It's configured like this.

Still need more info to help....can't offer much advice without much greater detail of the setup. I'm not familiar with those BT units....but I know over here we have similar units from the phone co for DSL/Fiber...and from the cable co for cable. There are right ways to set them up, and I have come across "other ways" to set them up which might lead to funky issues.

I guess I'd want to know what steps were done on the BT unit to configure a router behind it..and what steps were done on Untangles WAN interface to configure that behind the BT box.

You don't have the WAN balancer or WAN Failover modules loaded in Untangle do you?

I forget if you mentioned this before in the thread...but when the problem occurs....have you just rebooted the Untangle box to see if the problem goes away?

 

[BadBoy House]

No don't have the WAN Balancer or WAN Failover modules in Untangle.

The routers are simply set up with A) the broadband login username and password and B) the firewall/port forwards are configured to allow/forward ports 25 and 443 to Untangle. No other configuration.

Each router that we've tried (different makes) has been set up like this - the first router before we changed it when this problem started to occur had been set up for years in the same way.

On your point about rebooting the Untangle box, I have tried that, however when the problem occurs if I disconnect the Untangle server from the router and just have a laptop into the router and nothing else, I still can't access any other websites other than the site of the LOB provider.

i.e. Laptop -- Router -- Phone Line

Something must be happening to the router, early each morning to send it into it's problem state.

A colleague who opens up the office each morning at around 6.30AM told me that the internet is often down when he arrives. Although as I've mentioned before it doesn't happen every day.

 

[FremontPC]

Is logging enabled in the router? If so, do the logs reveal anything? How about any info recorded in the modem? Could it be a firmware issue in the modem, despite BT checking on it?

 

[Maniaman]

I'm not sure why you need a router between the untangle box and the modem, as Untangle is a router itself. I would try to eliminate that router all together and let Untangle handle everything. Also make sure you don't have an IP address conflict somewhere in that mix. I had an issue at a client site where someone hooked up a router so they could have wifi in their office which conveniently had the same IP as their Untangle box. That caused all kinds of random and strange routing//connectivity issues until we found it.

 

[BadBoy House]

Untangle is the router but it has to physically connect to the modem which in turn connects to the phone line.

It's deffinitely something that happens over night - I get reports from early starters that the internet is down at 5AM. It's ok all the previous day and evening.

It occurs most days but not every day.

My next job is to review the event logs on the router. I'm thinking it might possibly be something externally that's hitting the router and knocking it off. Quite why we can still access our line of business providers website is a mystery.

Additionally, whilst restarting the ISP router brings the connection back online, I've also found that disconnecting from the ADSL from within the router web interface and then reconnecting also brings the connection back online.

 

[creapure]

Is there any access restrictions in place overnight that are not being lifted cleanly in the morning?

 

[SteveSi]

I am confused about what equipment you have. There was no 'modem' in your equipment list earlier (I thought it was in the BT business hub)? Now there is one??? Have you tried replacing this modem?
Can you give exact and precise list of equipment (make/model/age) starting with your ADSL socket and going back to a PC?

 

[knc]

Some sort of screwed up proxy host?