Terminal Server lockups

  • Thread starter Thread starter Ray234
  • Start date Start date
R

Ray234

Guest
We are running Windows Server 2003 Terminal Services. We are getting random
lockups where the taskbar is missing on the server, and the only way to get
it back is to reboot. Also, users will get performance issues including
lockups and slow performance. Sometimes their taskbar is missing. When this
happens, either the backup will fail and/or the Symantec Anti-Virus will stop
getting the definition updates. Rebooting will fix both. We have ran
multiple virus and spyware scans and checked startup items with Hijack This.
I have removed and re-installed Symantec and the backup client. I have
updated Windows to SP2, updated several device drivers, but can't figure out
why this keeps locking up. I have changed the virtual memory settings to
higher amounts and also to System Managed.

Any ideas would be appreciated.
 
Re: Terminal Server lockups

Hi,

This could be caused by many things.

As a start how about *completely* removing the anti-virus,
backup, and any other security/anti-malware/anti-spyware/etc.
type of software from the TS server. After this is done restart
the server and have your users work as normal and report any
problems to you.

Are there any errors in the System and Application logs?

When you have performance issues what numbers do you
see in task manager for CPU, Commit Charge Total, and
Physical Memory Total? If CPU utilization is high, which
processes are using the bulk of the CPU?

Thanks in advance for answering my questions.

-TP

Ray234 wrote:
> We are running Windows Server 2003 Terminal Services. We are getting
> random lockups where the taskbar is missing on the server, and the
> only way to get it back is to reboot. Also, users will get
> performance issues including lockups and slow performance. Sometimes
> their taskbar is missing. When this happens, either the backup will
> fail and/or the Symantec Anti-Virus will stop getting the definition
> updates. Rebooting will fix both. We have ran multiple virus and
> spyware scans and checked startup items with Hijack This. I have
> removed and re-installed Symantec and the backup client. I have
> updated Windows to SP2, updated several device drivers, but can't
> figure out why this keeps locking up. I have changed the virtual
> memory settings to higher amounts and also to System Managed.
>
> Any ideas would be appreciated.
 
Re: Terminal Server lockups

I'll have to look into removing Symantec from this server. I don't like
leaving 25+ users on a server with no A/V protection.

As far as the logs go, the System log had:
1) Dcom service failed (5-6 entries).
2) "Timeout waiting for a transaction response from the Symantec Antivirus
service" several times.
The Application log had:
1) Application Hang errors - Excel, IE, Outlook, Acrobat.
2) Fault Bucket.

The server is currently 2 days behind in definitions, and the backup failed
last night with a "TCP Reconnect timeout".
I did an 'arp -a', but didn't see the server that does the backup and
Symantec updates (same server for both). I did a ping of that server, and
then it showed up. I cleared the arp cache before that. Didn't hear from
any users of any other issues, but most likely they'll show up tomorrow, if
it follows the trend.

As far as performance, the CPU was about 3-5%, with spikes of 10-15%. The
Physical memory is: Total-2096492, Available-92240, System Cache-414632. The
Commit Charge is: Total-2825012, Limit-4043560, Peak-2890136. They Kernel
Memory is: Total-264136, Paged-206216, Nonpaged-57920.

Hope that helps.

Ray




"TP" wrote:

> Hi,
>
> This could be caused by many things.
>
> As a start how about *completely* removing the anti-virus,
> backup, and any other security/anti-malware/anti-spyware/etc.
> type of software from the TS server. After this is done restart
> the server and have your users work as normal and report any
> problems to you.
>
> Are there any errors in the System and Application logs?
>
> When you have performance issues what numbers do you
> see in task manager for CPU, Commit Charge Total, and
> Physical Memory Total? If CPU utilization is high, which
> processes are using the bulk of the CPU?
>
> Thanks in advance for answering my questions.
>
> -TP
>
> Ray234 wrote:
> > We are running Windows Server 2003 Terminal Services. We are getting
> > random lockups where the taskbar is missing on the server, and the
> > only way to get it back is to reboot. Also, users will get
> > performance issues including lockups and slow performance. Sometimes
> > their taskbar is missing. When this happens, either the backup will
> > fail and/or the Symantec Anti-Virus will stop getting the definition
> > updates. Rebooting will fix both. We have ran multiple virus and
> > spyware scans and checked startup items with Hijack This. I have
> > removed and re-installed Symantec and the backup client. I have
> > updated Windows to SP2, updated several device drivers, but can't
> > figure out why this keeps locking up. I have changed the virtual
> > memory settings to higher amounts and also to System Managed.
> >
> > Any ideas would be appreciated.

>
 
Re: Terminal Server lockups

Hi Ray,

All of your regular users should have limited rights to the server.
They should not be a member of any of the built-in local groups
besides Users and Remote Desktop Users. The TS should be
set to Full Security in Terminal Services Configuration. With that
in mind the likelihood of them doing serious damage to the server
is greatly reduced (not eliminated).

Do you allow your users to browse the Internet while on your
TS? Are they allowed to download files?

Are you stripping dangerous attachments from email messages
before they reach the mailboxes? Do you have Antivirus on
your email server scanning each message?

Do you have your security zones configured so that users are
unable to run programs from unapproved network locations?

Are you using Software Restriction Policies to limit which
programs each user can run?

Do you have Group Policies configured for your TS to limit
what each user can do while logged on to the TS?

My suggestion to remove the Antivirus is a temporary measure.
I would recommend that you consider the above questions and
then decide whether it is an acceptable risk or not to remove
the Antivirus. Antivirus software can help reduce but by no
means eliminate the risk of infection/data loss.

What other third-party applications are installed that integrate
with the shell or IE? For example, one sign (there are many)
of an application that integrates with the shell is that when you
right-click on a file/folder an extra menu will be automatically
added (like "Scan for Viruses"). The default shell is explorer.exe;
this is what provides the taskbar, desktop icons, etc.

It appears that you don't have enough RAM in your server for
the load. I recommend increasing to 4GB. This will *not*
normally cause the crashing you are seeing, but will cause
slowness and temporary pauses.

Has this server always had crashes/hangs like this?

If there are lots of issues it *may* be better to simply restore
the last known working image of the server. If you do not have
that then perhaps a complete reinstall may be called for.

You may have too many different variables in play on your
server for me to help you via a newsgroup post. Removing
variables is helpful because it will allow us to drill down to
the real cause which is often not obvious.

-TP

Ray234 wrote:
> I'll have to look into removing Symantec from this server. I don't
> like leaving 25+ users on a server with no A/V protection.
>
> As far as the logs go, the System log had:
> 1) Dcom service failed (5-6 entries).
> 2) "Timeout waiting for a transaction response from the Symantec
> Antivirus service" several times.
> The Application log had:
> 1) Application Hang errors - Excel, IE, Outlook, Acrobat.
> 2) Fault Bucket.
>
> The server is currently 2 days behind in definitions, and the backup
> failed last night with a "TCP Reconnect timeout".
> I did an 'arp -a', but didn't see the server that does the backup and
> Symantec updates (same server for both). I did a ping of that
> server, and then it showed up. I cleared the arp cache before that.
> Didn't hear from any users of any other issues, but most likely
> they'll show up tomorrow, if it follows the trend.
>
> As far as performance, the CPU was about 3-5%, with spikes of 10-15%.
> The Physical memory is: Total-2096492, Available-92240, System
> Cache-414632. The Commit Charge is: Total-2825012, Limit-4043560,
> Peak-2890136. They Kernel Memory is: Total-264136, Paged-206216,
> Nonpaged-57920.
>
> Hope that helps.
>
> Ray
 
Re: Terminal Server lockups

Users do have limited rights to the server. They don't have the ability to
install programs, and are limited to what is on their desktop. They can
browse the Internet and download files, but can't install anything. We do
have AV for the Exchange server. Security zones are set to not allow running
of programs from unapproved locations. We have Group Policies locked down as
per MS and a third-party recommendations. I don't see any third-party
programs coming up when right-clicking.

We have ordered 2GB of RAM to install on this server, which should help. I
did some analysis, and even though we only had 19 users on the server, the
memory usage was about 1.8GB according to my calculations. I checked 7 users
in Task Manager, and they had 850MB alone.

This server has not always crashed and behaved like this. It started March
or April of this year. I have tried several things since then to correct.
This includes virtual memory changes, driver updates, program updates,
spyware and virus scans and network changes (Winsock, etc.).


"TP" wrote:

> Hi Ray,
>
> All of your regular users should have limited rights to the server.
> They should not be a member of any of the built-in local groups
> besides Users and Remote Desktop Users. The TS should be
> set to Full Security in Terminal Services Configuration. With that
> in mind the likelihood of them doing serious damage to the server
> is greatly reduced (not eliminated).
>
> Do you allow your users to browse the Internet while on your
> TS? Are they allowed to download files?
>
> Are you stripping dangerous attachments from email messages
> before they reach the mailboxes? Do you have Antivirus on
> your email server scanning each message?
>
> Do you have your security zones configured so that users are
> unable to run programs from unapproved network locations?
>
> Are you using Software Restriction Policies to limit which
> programs each user can run?
>
> Do you have Group Policies configured for your TS to limit
> what each user can do while logged on to the TS?
>
> My suggestion to remove the Antivirus is a temporary measure.
> I would recommend that you consider the above questions and
> then decide whether it is an acceptable risk or not to remove
> the Antivirus. Antivirus software can help reduce but by no
> means eliminate the risk of infection/data loss.
>
> What other third-party applications are installed that integrate
> with the shell or IE? For example, one sign (there are many)
> of an application that integrates with the shell is that when you
> right-click on a file/folder an extra menu will be automatically
> added (like "Scan for Viruses"). The default shell is explorer.exe;
> this is what provides the taskbar, desktop icons, etc.
>
> It appears that you don't have enough RAM in your server for
> the load. I recommend increasing to 4GB. This will *not*
> normally cause the crashing you are seeing, but will cause
> slowness and temporary pauses.
>
> Has this server always had crashes/hangs like this?
>
> If there are lots of issues it *may* be better to simply restore
> the last known working image of the server. If you do not have
> that then perhaps a complete reinstall may be called for.
>
> You may have too many different variables in play on your
> server for me to help you via a newsgroup post. Removing
> variables is helpful because it will allow us to drill down to
> the real cause which is often not obvious.
>
> -TP
>
> Ray234 wrote:
> > I'll have to look into removing Symantec from this server. I don't
> > like leaving 25+ users on a server with no A/V protection.
> >
> > As far as the logs go, the System log had:
> > 1) Dcom service failed (5-6 entries).
> > 2) "Timeout waiting for a transaction response from the Symantec
> > Antivirus service" several times.
> > The Application log had:
> > 1) Application Hang errors - Excel, IE, Outlook, Acrobat.
> > 2) Fault Bucket.
> >
> > The server is currently 2 days behind in definitions, and the backup
> > failed last night with a "TCP Reconnect timeout".
> > I did an 'arp -a', but didn't see the server that does the backup and
> > Symantec updates (same server for both). I did a ping of that
> > server, and then it showed up. I cleared the arp cache before that.
> > Didn't hear from any users of any other issues, but most likely
> > they'll show up tomorrow, if it follows the trend.
> >
> > As far as performance, the CPU was about 3-5%, with spikes of 10-15%.
> > The Physical memory is: Total-2096492, Available-92240, System
> > Cache-414632. The Commit Charge is: Total-2825012, Limit-4043560,
> > Peak-2890136. They Kernel Memory is: Total-264136, Paged-206216,
> > Nonpaged-57920.
> >
> > Hope that helps.
> >
> > Ray

>
 
Back
Top