Probelm beyond my scope

[Jore]

Hello!

I have a problem that I cant even understand more less solve.
The scenario is following:
I have made an asp.net site running on IIS and asp.net 2.0 a couple of years ago. The site has worked well until recently. What happened before the site started behaving oddly is that the customer complained the server the site is on produced some additional traffic to the local network they had it in. Well, they removed the site from that network to some other less used network and (i guess) solved the problem. It had something to do with looping addresses on that network. I dont know what they did exactly to fix the problem. During this time the site had different ip. Then it was moved back to its original ip. And I changed the ip back. Thats all that I touched the code, only in web.config connection strings. All was well again.. No.
After moving the site back to its original ip and network the problems with my site started. People can log in and move around through links, but trying to load pages by form submits do nothing. Browser seems to do some loading, but nothing gets actually loaded.

Now we have debugged the problem a day and here are some weird results:
- When I use this site from my vista pc with my ie7/FF3/Opera it works fine, all the time.
- When customer uses the site, it works fine with ie6 from their VPN but not from local network. From vpn it seems to work all the time and with all browsers. Also from local network FF3 and IE7 seem to work just fine.
- When I try to access the site form a test server with ie6 it worked fine in the beginning, but after I tested some IE security settings, the same kind of problematic behavior appeared. Even if I change the settings to low. I installed FF3 on the server but the same behavior appears with it.

We also have another customer using an identical site on another server. It has not been moved or tampered in any way. when that site was tested the results were following:
- working fine from my vista pc with all browsers
- customer couldnt get it to work from their local network with ie6 but it worked fine from vpn
- working fine from my test server with ie6/FF3 also.


Short site description:
A .net based space reservation system that requires logging running on windows 2003 server. Site uses mssql 2005 database.


First it seemed that it was an ie6 problem. But when my test servers FF3 also had the same problems it cant be the browser. Can it be the site? The site worked fine with the customers ie6 before the ip-switching. Can it be the network? The identical site that is not on their network did not work with ie6 from their network but worked from vpn.

Have anyone had any similar problems or does anyone have any clue of what this could be about? I would really appreciate any answers that can point me to right direction.

I know that there are a lot of "moving parts" in this equation, but I cant start listing all of them here.

- Jore

 

[Jore]

Now that there is close to no traffic, in the customer web server where the site is, I could check iis log and task manager network or local area connection activity while loading pages on the site.

It seems that from browsers that the site works rows to iis log seem to appear and also local area connection shows activity.
But when I try from a browser that jams on form submits, nothing comes to the web server iis log nor does the local area connection draw anything but straight line.

So it seems that the requests dont even reach the web server.


- Jore

 

[Nate Bross]

Has the server been rebooted? (when it doubt, reboot).

Are the network settings on the ASP.NET Server such that it has access to the SQL Server?

Is there any client side data access?

 

[Jore]

Thanks for a quick response.

Yes, the server and iis service has both been rebooted several times.

Yes, the server can access the sql-server. Otherwise users would not be able to even log in.

What do you mean by client side data access?
Form data is fetched from and stored to the database with dynamic sql requests.

By the way in addition to my previous post. It seems that the request goes to the server after all and from the logs I can see that the server response is 400 (Bad Request). This points to bad code, but only ip addresses has been touched in the code close to the beginning of the strange behavior. So still quite lost with this one.

-Jore

 

[Nate Bross]

I was thinking JavaScript and Dynamic Data Services, but it doesnt sound like you are using that. If that were the case, it would be possible that different methods of access (VPN, intranet, internet) could yeild different DNS responses causing JavaScript/Dynamic Data Services to give similar results to what you are seeing because DNS may be differet based on access method.

Is anyone able to consistently access the system from their intranet (not VPN)? It doesnt sound like a browser issue, it sounds like a networking issue to me.

 

[PlausiblyDamp]

What ip addresses are contained in the code / config files? Can all these addresses be pinged from the web server? Are there any firewalls or similar between the web server / database etc and have these all had their rules correctly configured since the ip address problems?

 

[Jore]

The asp.net controls (like datagrids and autopostback ddls) produce javascript to the site, so there is some JavaScript. And if you mean this: http://en.wikipedia.org/wiki/ASP.NET_Dynamic_Data by dynamic data, then there is no dynamic data.

I quess no one can use the system normally from the intranet. Everyone can log in, but inside the site no form submit seems to work. Links work fine.

I also think that it is a problem caused by their network. But they dont seem to think so.

 

[Jore]

The sql-server and the site are on the same server. I dont know if the sql-requests from the site go out side the server (to some router or dns-server or something) and come back to the sql, or do they travel "inside" the server only?

I now changed all the ip:s in the code to localhost, but it didnt seem to have any affect at all.

I can ping the local ip of that server form the server, but I cant ping the public ip of the server, probably its blocked.

 

[Jore]

Could viewstate cause this kind of problems?
Why mostly IE6 is affected, but it works through vpn?
Do you know of any tool that would give better diagnostics than the browsers?

Here is a request captured by the customer from a workstation in their local network with netmon when he used a calendar control on the site that for one causes the problems. Can someone decipher it and tell if it can cause http 400 error?


GMBU

 

[Jore]

I now tried to set enableviewstate to false. and the pages started to work. So it seems to be the viewstate that causes this.
Other form submits seem to work after disabling the viewstate, but asp.net calendar control partly still has the same problem. It now loads the page when changing the month but still gets stuck when you select a day by clicking on a day number.

I dont know do I really need viewstate or is it really necessary. And now Im still left with the problem with the calendar control..

- Jore

 

[Jore]

I have a lot of code that rely on viewstate it seems. Things like ddl.selectedvalue and asp.net validation control. At leas they all stopped working when I disabled viewstate. So disabling viewstate isnt a solution to this problem. But what is it in viewstate that causes even the smallest pages not to load anything on form submit?

- Jore

 

[Jore]

Problem solved.

It was at customers end and a it was a network problem. Customer has an external provider for IT-stuff like servers, workstations and networks and they had made some configuration changes simultaneously to the moving of the web server back and forth which led to confusement of how and where the strange behavior had started. So apparently they had messed up some configuration settings while adjusting some switch.

So wrong configuration settings in a data communications switch can result to inconsistently ie6 packets containing viewstate data not going entirely through and in some cases to blocking other browsers packets too, IIS returnin http400 and in some cases all browsers working just fine.

- Jore

 

[Nate Bross]

Very strange, but Im glad you got the issue resolved!