 |
AlertSiteOverviewMonitoring Plans
Details
Help
Reports API What's New DéjàClick Recorder |
ServerAgent Systems Monitor1. Overview: What is ServerAgent?ServerAgent is AlertSite’s systems monitoring agent. It resides on a Windows or Linux/Unix server, collecting system health statistics, such as CPU load and disk utilization, and reporting them back to the AlertSite monitoring service on a user-specified interval. Thresholds can be set for both warning and error conditions, providing instant notification through AlertSite notification mechanisms.2. ServerAgent FeaturesServerAgent collects, reports and alerts on system health and connectivity:
alertsite.com within the expected time.
To activate notifications for a monitored metric, the Notify and Enabled check-boxes for that metric must be enabled on the Manage Servers screen accessed by logging in to your account and selecting ServerAgents from the Configuration Control Menu; in the Configuration: ServerAgents screen, click on the name of the ServerAgent you wish to configure. Note that it will be necessary to open the View that holds your ServerAgent. If you have not yet created a View, new ServerAgents are placed in the AlertSite Default View following the first publication of collected metrics.
Current device status is reported in the AlertSite monitoring console and historical details are available in the Report Center. Daily, weekly and monthly email summaries are provided as well.
Configuration is centrally managed through the AlertSite Monitoring Console. To configure a single ServerAgent, or a group of ServerAgents assigned to a View, log in to your account and navigate to Configuration: ServerAgents. From there, select the particular agent or View by clicking on the link in the Name column.
2.1 Key Enhancements of ServerAgent v2.x
2.2 Benefits
2.3 Supported Platforms
2.4 ServerAgent Status CodesThere are 3 error status codes returned for problems detected by ServerAgent:
3. Downloading ServerAgentTo download the ServerAgent software for your platform, select Downloads from the Support dropdown of your AlertSite Control Menu. The Linux/Unix version is provided as a bash script installation package. The Windows version is a .EXE installer program. Click on the download appropriate for your platform and save it to your desktop.4. Installation InstructionsIf upgrading from ServerAgent 1.x (aka:Nemo), the ServerAgent v2.x installer will detect any previous installation and present you with the option to apply the Nemo settings to your new installation.
Note: Ensure that the Nemo Linux/Unix daemon or Win32 service appropriate for your deployment environment is not set to run automatically, as it is not advised that both collection environments run concurrently on the same system.
4.1 Configure ServerAgent DeviceNote: If you have a Usage Based Monitoring account, you can skip the Purchase Credits step4.1.1 Purchase Credits for ServerAgentIf you have a standard account and have not purchased a credit for your ServerAgent device, follow the procedure below:
† A 15-Minute ServerAgent plan can be configured for reporting intervals every 15, 20, 30, 60 and 120 minutes 4.1.2 Add ServerAgent DeviceTo add a new ServerAgent device in the AlertSite Console, perform the following steps:
Back to top 4.2 Installation instructions for Microsoft WindowsInstallation directory:C:\Program Files\AlertSite\ServerAgent
Note: If upgrading to a new version of ServerAgent, you must uninstall the existing version:
4.3 Installation for Linux/Unix/CygwinInstallation directory:/usr/local/alertsite
# sh ./server-agent-installer.sh
Note: If you're trying to install ServerAgent on a Linux server such as Suse or Redhat, and get the error: The ServerAgent software may not be compatible with your compiler and/or hardware. this indicates that the ServerAgent Installer is looking for g++ libraries on the system, and they are not installed. Please install the g++ libraries for the installation to complete.
# cp /usr/local/alertsite/init.d/server-agent /etc/init.d
# /etc/init.d/server-agent start
# ps -ef | grep mgr
You should see:
# ps -ef | grep mgr root 5613 1 2 Oct22 ? 00:35:15 /usr/local/alertsite/bin/commgr root 5631 1 1 Oct22 ? 00:21:35 /usr/local/alertsite/bin/agentmgr This completes the Linux/Unix/Cygwin installation. In a few minutes, you should see information related to your server device displayed on the AlertSite console. Back to top 5. Configuring a ServerAgent Device in the ConsoleFrom the Configuration dropdown in the Control Menu on the AlertSite Console, select ServerAgents to bring up the Configuration: ServerAgents screen, then click on the name of the ServerAgent you wish to configure.5.1 ServerAgent ViewsIf you have multiple ServerAgents, they can be grouped in Views for more controlled monitoring. For example, you can group ServerAgents monitoring systems in IT in one view and ServerAgents monitoring machines in Accounting in another view. To create a view, select ServerAgent from the Configuration dropdown in the Control Menu, and click the Add a View button:
The Add a View window displays a ServerAgent selection box with the names of ServerAgents configure in the account. To move the ServerAgent into the selected view, highlight the name (<CTRL>-Click to select multiple names), and click the Add button: 
The names will appear in the ServerAgents in View box. Click Submit to save the changes. The Configuration: ServerAgents screen will display the view name. Click the '+' to the left of the name to display the ServerAgent(s) listed in the view: 
Back to top 5.2 ServerAgent ConfigurationThere are two main sections of the Manage Servers screen. The top portion is the Server Information Section describing the device and contains monitoring selection criteria. The bottom section, Agent Configuration, is comprised of a set of tabs - CPU, Disk, Process, User Scripts, Connection, and Distributed Collection - where the data thresholds and process information are configured. The Manage Servers screen, displayed by clicking on the ServerAgent name in Configuration: ServerAgents, appears as:
Note: the Agent Configuration section takes some time to populate with the tabs after initial installation. Allow 15-20 minutes for the screen to update. Refresh the browser to check. After the Agent Configuration section has been populated, you may then select individual metrics in the CPU, Disk, Process, User Scripts, and Connection tabs for which to establish thresholds. The Thresholds Available column of the Configuration: ServerAgents screen will display No for ServerAgent devices with no thresholds configured, Yes if you have established any thresholds. 5.2.1 Server Information SectionThe upper portion of the Manage Servers page contains a set of fields that describe the common elements of the ServerAgent, some of which were completed when the device was added as previously described. The table below discusses these fields:
5.2.2 Agent Configuration SectionThe lower portion of the Manage Servers screen provides a means of configuring specific settings for each collection agent, separated into tabs:
The Enable checkbox found below the category tab name enables monitoring of the selected metrics by the ServerAgent. Each collection agent also has a checkbox to enable specific settings and alert notification. The following sections illustrate the settings for the each collection agent. See 9. Description of Errors below for more information. 5.2.2.1 CPUThe metrics collected by the CPU agent are determined by the operating system where the ServerAgent is running. Therefore, the only settings that the user may affect are theWarning and Threshold Values:
Linux/Unix:
Windows: 
5.2.2.2 DiskAs with the CPU monitor, the metrics collected by the Disk agent are determined by your system. Therefore, the only settings that the user may affect are those forWarning and Threshold Values:
Linux/Unix:
Windows: 
5.2.2.2 ProcessYou can monitor specific processes, e.g., the syslog daemon or print spooler, and configure ServerAgent to notify you if the process has stopped running. The initial screen appears as follows:
You can add more processes to the list by clicking on the '+' on the right side of the window under “Last Reported”. To determine the appropriate value to enter in the Name field for Process monitoring, use the following methods: Windows:
$ ps -e -o comm
syslogd and sshd:
Notice that the Enable Process Monitor, Enabled, and Notify checkboxes have all been checked. This enables monitoring of both processes and will alert all assigned notifiers if either of these processes is no longer detected (Threshold Value < 1). 5.2.2.4 User ScriptsThe ServerAgent software considers the presence of user scripts staged by the user to perform monitoring functions beyond the categories offered by ServerAgent. Details of the User Script functionality and supported configuration options may be found under the section Monitoring User-Defined Scripts found later in this document.5.2.2.5 ConnectionThe Connection monitor enables a user to monitor specific network services running on a designated host. The syntax for adding a new connection to monitor ishost:port, for example:
Values returned by the Network Monitor are 0 or 1 for failed to connect and connect success, respectively.
5.2.2.6 Distributed CollectionThe ServerAgent software may be instructed to proxy for other ServerAgents that do not have visibility to the public Internet. The Distributed Collection tab provides a means of specifying which ServerAgents will be subordinate to this ServerAgent. In the example below, ServerAgent T-blue will be the proxy server for T-aqua, T-green, T-lavender, and T-scarlet:
Click on the ServerAgent name in the Available ServerAgents box to select the ServerAgent to act as proxy for the current ServerAgent and click the Add>> button. The selected ServerAgent will appear in the Proxied ServerAgents box. Select multiple ServerAgents with <CTRL>-click. Note: This is valid only for accounts with two or more ServerAgents configured. Back to top 5.3 Monitoring User-Defined ScriptsServerAgent can monitor processes, CPU, and basic disk utilization natively. Many customers, however, want to integrate resource- or environment-specific monitoring with ServerAgent. This is done by adding user-defined monitoring scripts to ServerAgent. Almost any executable script,cmd file or program that outputs a single integral value to standard out can be integrated into ServerAgent.
How it Works
ServerAgent will scan for any user scripts both at startup and during run-time. For any scripts found, ServerAgent will attempt to run them on each monitoring pass (typically every minute).
It will read the first line of output generated by the script, expecting an integral value, and use that value to test for
error/warning conditions configured in your AlertSite account.
Any other text produced by the script will be saved as additional information for your own use. For example, if you have a script that searched for error conditions in your system log files, you could add the actual error messages as output to the script after the numeric value used to indicate the error/warning condition.
Installing Scripts
Windows
.cmd file that checks for the presence of a file:
testfile.cmd:
@ECHO OFF IF EXIST c:\tmp.txt ( echo 1 echo File Found ) ELSE ( echo 0 echo File Not Found )Note: If using a .bat or .cmd file and echo is used to generate output for ServerAgent, be sure to provide the '@' character prior to ECHO KEYWORD.
Sample Unix shell script that checks for the presence of a file:
chkfile.sh:
#!/bin/sh
# returns "1" if file exists
if [ -f /tmp/somefile.txt ]
then
echo "1"
echo “File Found”
fi
Limiting User Scripts
Under normal conditions, the ServerAgent on your machine will run your user scripts approximately once per minute. Data is sent to AlertSite based on your ServerAgent plan, usually at 5- or 15-minute intervals, or as often as once a minute for Usage Based Monitoring accounts. However, if an error threshold is raised on your machine, the error condition is transmitted immediately to AlertSite so you can be notified. After an error has been reported, no more data is sent to AlertSite until the next monitoring interval.
There could be reasons why you want to limit how often a particular script runs on your machine, including:
:<N>, where N is the run interval, to the name of the user script.
For example, consider that you have a 5-minute ServerAgent collection interval and want to limit the shell script http_stat.sh to run only on the actual 5-minute interval. You also have a perl script, db_check.pl, that you want to run once per hour (that is, every 12th interval).
To do this, add the script names with the appended interval limit in the Name field, then select the Compare type, Warning Value and Threshold Value, check Enabled and Notify as required, then submit the changes by clicking on the Submit button in the upper right of the page. To illustrate:
Note: Once the information has been saved, it will be necessary to delete and re-add the script in order to add or modify :<N>. Any other configured scripts without :<N> will be run on the standard once-a-minute schedule.
Back to top
6. ServerAgent-to-AlertSite CommunicationsThe client software opens an outgoing socket to our server on Port 4892. Both the client side check-in packet and the server side response are encrypted and encoded ASCII packets. If you are monitoring InSite by AlertSite with ServerAgent, the firewall setting for TCP OUTBOUND Port 4892 has to be open. Here's a sample from one of our servers, decoded and unencrypted... Incoming from client:<REPORT><Function>MONITOR</Function><ServerID>8f2264198cb88f896c88fe15182ebbd4</ServerID> <ClientVersion>2.0</ClientVersion><Hostname>serveragent</Hostname><DISK_INFO> <DRIVE_root_BLOCKS>8</DRIVE_root_BLOCKS><DRIVE_dev_shm_BLOCKS>0</DRIVE_dev_shm_BLOCKS></DISK_INFO> <PROCESS><httpd>0</httpd><bash>2</bash><sshd>7</sshd><ftpd>0</ftpd></PROCESS><CPU_USAGE> <LOAD_1_MIN>0</LOAD_1_MIN><LOAD_5_MIN>0</LOAD_5_MIN><LOAD_15_MIN>0</LOAD_15_MIN> <PROCS>61</PROCS><UPTIME>2220055</UPTIME><RAM>57</RAM><SWAP>0</SWAP> <NOTIFY_ON_REBOOT>0</NOTIFY_ON_REBOOT></CPU_USAGE></Report>Outgoing from server <Response><TIME>1211373241</TIME><SEED>4</SEED><STATUS>0</STATUS><LogLevel>info</LogLevel> <ServerList>blue.acme.com:4882,green.acme.com:4882</ServerList><Thresholds>THRESHOLDS=0</Thresholds> <VERSION>2.0</VERSION><INTERVAL>5</INTERVAL></Response>This is a "typical" handshake between the ServerAgent and the AlertSite server. 7. Log filesServerAgent currently keeps a 7-day rolling history of log files. Naming convention:server-agent.YYYYMMDD Location:
c:\Program Files\ServerAgent\logs
/usr/local/alertsite/bin/logs/
A successfully functioning ServerAgent will have entries in the log file that show its present configuration:
Sample:
02/24/2008@11:57:07 (info ) commgr -- ready 02/24/2008@11:57:07 (info ) agentmgr -- starting cpu metrics collector 02/24/2008@11:57:08 (info ) agentmgr -- starting disk metrics collector 02/24/2008@11:57:08 (info ) agentmgr -- starting task metrics collector 02/24/2008@11:57:08 (info ) agentmgr -- starting network metrics collector 02/24/2008@11:57:08 (info ) agentmgr -- starting user metrics collectorBack to top 8. TroubleshootingEnsure the service or daemon is running Under Linux/Unix, a simpleps (process status) command will reveal if the daemons are running as seen in the following example:
$ ps -ea | grep mgr
or
$ ps ax | grep mgr
Ensure that the processes commgr and agentmgr appear in the output.
Under Windows, go to Control Panel->Administrative Tools and double-click on Services. It may be necessary to expand the Name column to view the full text of the service names. In the list, find "AlertSite ServerAgent Manager v2.x" and "AlertSite ServerAgent Monitor v2.x" and ensure their status is shown as "Started".
Verify connectivity to the assigned monitoring server
If ServerAgent cannot connect to its assigned monitoring server, error messages will be displayed in the log files that look like this:
01/31/2008@09:11:06 (error ) commgr -- failed to connect to myserver.acme.com:4892 (connection refused): will retry 2 more times 01/31/2008@09:11:14 (info ) agentmgr -- waiting for collector (localhost:8000) 01/31/2008@09:11:14 (error ) commgr -- failed to connect to myserver.acme.com:4892 (connection refused): will retry 1 more timesCheck if the server running ServerAgent has outbound connectivity to the appropriate AlertSite monitoring location. To verify that there is appropriate connectivity: # telnet myserver.acme.com 4892
A connection should be established, and upon hitting <ENTER>, the “Connection closed by foreign host” should be displayed.
Also, the communication manager will accept browser connection to its default listener port (2525). Point your browser to where the ServerAgent is running, e.g., http://localhost:2525 and you will be able to view the data pending publication as displayed in the example below:
Debugging ServerAgent can be configured to add debugging information to your log files for network problems. Review the earlier section in this document regarding configuring a ServerAgent.
9. Description of Metrics
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||