ZenPacks

Web-Based Synthetic Transactions ZenPack

The ZenWebTx ZenPack allows you to test the availability and performance of Web sites by performing some of the same activities performed by your user community.

Support

This ZenPack is included with commercial versions of Zenoss and enterprise support for this ZenPack is provided to Zenoss customers with an active subscription.

Releases

Version 3.0.1 Download
Released on 2017/04/25
Compatible with Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.x.x
Version 3.0.0 Download
Released on 2016/01/15
Compatible with Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.x.x

Background

The ZenPacks.zenoss.ZenWebTx ZenPack adds the zenwebtx daemon, which enables availability and performance monitoring of web sites through synthetic HTTP transactions.


Synthetic transactions perform some of the same activities performed by your user community. You create one or more tests that mimic user actions in a Web browser. Zenoss platform then performs these tests periodically, creating events when a test fails or exceeds a time threshold.

Additionally, Zenoss platform can record data for each test run, such as:

  • Time required for the test to execute
  • Time taken for any portion of the test to complete
  • Values extracted from Web pages during the test

This ZenPack uses a scripting language called twill to describe the steps of a test. These steps include actions such as:

  • Clicking a link
  • Completing form fields
  • Assertions, which check for the presence or absence of text on a page. In addition, you can extract data from the Web page and record the numeric values that are a part of these patterns
  • Descriptions of data to collect during the test

You can write twill commands manually. You also can use a Firefox add-on called TestGen4Web to record a browser session that this ZenPack then translates into twill commands. The zenwebtx daemon processes the twill commands periodically, recording data and creating events as appropriate.

Data Points

Data produced by any Zenoss platform data source are called data points. WebTx data sources contain two default data points:

  • totalTime – Number of seconds taken to complete the entire transaction.
  • success – Returns 1 (success) or 0 (failure), depending on whether or not the transaction succeeded.

You can create other data points by using the extract and printTimer twill commands, which output data values when the twill commands are run. You must create new data points with the same name you used in those commands to bring that data into Zenoss platform.

The zenwebtx daemon supports XPath queries to extract data from XML documents.

Event Generation

There are several situations in which this ZenPack creates events in Zenoss platform. These events use the component and event class specified on the Data Source tab. These situations are:

  • The zenwebtx daemon is unable to retrieve a page during the transaction.
  • One of the twill commands fails, such as finding text that does not exist or following a link that does not exist.
  • The timeout (specified on the Data Source tab) is exceeded.
  • A threshold defined for one of the data points in this data source is exceeded. Thresholds are defined in the monitoring template that contains the data source.

Enable Monitoring

To create a WebTx data source:

  1. From the data sources area, click Add Data Source.
  2. In the Create Data Source dialog, enter the name of the new data source, and then select the data source type WebTx.
  3. Click Submit.
  4. Select the data source to edit it. Enter information or make selections to specify how and when this data source's Web transactions are performed, and which data should be collected:
    WebTx Data Source Options
    OptionDescription
    NameDisplays the name of the data source that you specified in the Create Data Source dialog. This name is used in thresholds and graph definitions to refer to the data collected by this data source.
    Source TypeSet to WebTx, indicating that this is a synthetic Web transaction data source. You cannot edit this selection.
    EnabledSet to True (the default) to collect information from this data source. You may want to set this value to False to disable data sources when developing the data source, or when making changes to the Web application being tested.
    ComponentAny time the Web transaction fails, Zenoss platform generates an event. Use this field to set the Component field of the generated event.
    Event ClassSelect the event class of the event generated by this data source. Normally, this is set to /Status/Web (according to the value set on the data source).
    TimeoutSpecify the number of seconds that zenwebtx will attempt to execute this data source's commands before it generates an error event.
    Cycle TimeSpecify the number of seconds that zenwebtx will wait between the start of one test run and the start of the next.
    User AgentSpecify the text that zenwebtx will present to target Web sites to identify itself.

  5. Click Save to save the specified settings.
  6. Select Script. From here, you will specify the details of the transaction. Information here also helps you debug twill commands when setting up the data source.
  7. Note: If you provide values for Initial User, Initial Password, and Initial Authentication Realm, Zenoss platform will use these credentials before accessing the URL specified for Initial URL. All three (Initial User, Initial Password, and Initial Authentication Realm) must be present; otherwise, the values are ignored. Enter information or make selections:
    WebTx Script Settings
    OptionDescription
    Initial URLSpecify the URL of the page where the transaction will start. This field frequently contains a TALES expression to refer to a device's ID or IP address, such as http://${dev/id} or http://${dev/manageIp}.
    Initial UserSpecify the user name for authentication.
    Initial PasswordSpecify the user password for authentication.
    Initial Authentication RealmSpecify the basic HTTP authentication realm.
    TestDeviceUse this field to test and debug twill commands. Enter the ID of a device, and then click Test Twill Commands to execute the twill commands against the device. If you do not specify a device, then Zenoss platform will select a device for you.
    Upload RecordingUpload a Web session recording generated by the Firefox TestGen4Web add-on. Enter or browse to the recording location.

    If you specify a file here, and then click Save, Zenoss platform translates the file to twill commands and replaces the contents of the Twill Commands field with the newly translated commands.

    Twill CommandsSpecify the number of seconds that zenwebtx will wait between the start of one test run and the start of the next.

    Enter twill commands that Zenoss platform will execute to produce values and events for the data source.

    If you select this action, then the current contents of the Twill Commands field is completely replaced. Zenoss platform does not save the replaced information.

  8. Click Save to save the data source.

Creating twill Commands

This ZenPack uses twill to specify the steps of a Web test. Each WebTx data source has a field that contains the twill commands that describe a Web transaction. You can create this list of twill commands manually, or you can record a session in a browser and use that as the basis for your data source.

Some twill commands specify an action, such as following a specific link on a page or entering data in a form field. Other twill commands specify a test, such as searching for specific text on a page or making sure the title does not contain specific text.

Creating twill Commands from TestGen4Web

The TestGen4Web Firefox add-on allows you to record browser sessions. This ZenPack can take these sessions and convert them to twill, creating a starting point for developing WebTx data sources.


Follow these general steps to record and convert a TestGen4Web session:

  1. From the TestGen4Web toolbar in Firefox, use the Record and Stop buttons to record a session.
  2. Use the Save button in the toolbar to save the session to a file.
  3. From the Script page of a WebTx data source in Zenoss platform, browse to and select your saved session.
  4. Click Save to convert the TestGen4Web session to twill. The newly converted commands appear in the Twill Commands field on the page, replacing any previous twill commands in that area.

Creating twill Commands Manually

Even if you use TestGen4Web to initially create twill commands, you will frequently want to edit these commands manually to add data points or additional content checks. The Test Twill Commands button on the Script page is helpful when testing twill commands as you create or edit them.

You also can execute twill commands interactively by using the twill-sh program from the command line. This program lets you enter commands one at a time and then inspect the pages that come back.

With Zenoss 5.x you will want to do this from inside the zenwebtx container, on the specific collector if it applies, and switch to the zenoss user with su - zenoss.

Invoke twill-sh with:

PYTHONPATH=$PYTHONPATH:$ZENHOME/ZenPacks/ZenPacks.zenoss.ZenWebTx-3.0.0-py2.7.egg/ZenPacks/zenoss/ZenWebTx/lib
$ZENHOME/ZenPacks/ZenPacks.zenoss.ZenWebTx-3.0.0-py2.7.egg/ZenPacks/zenoss/ZenWebTx/bin/twill-sh

Within twill-sh, use the help command to list available commands and see a command descriptions. Of particular interest are these commands:

  • showforms – Lists the forms on the page and the fields within each.
  • showlinks – Lists the links on the page.
  • show – Lists the source HTML from the page.
  • exit – Quits the twill-sh program.

Often the most convenient way to use twill-sh is to create a text file that contains your twill commands. You can then specify that file on the command line when you invoke twill-sh. This lets you analyze problems that occur.

Invoke twill-sh with a text file as such:

PYTHONPATH=$PYTHONPATH:$ZENHOME/ZenPacks/ZenPacks.zenoss.ZenWebTx-3.0.0-py2.7.egg/ZenPacks/zenoss/ZenWebTx/lib
$ZENHOME/ZenPacks/ZenPacks.zenoss.ZenWebTx-3.0.0-py2.7.egg/ZenPacks/zenoss/ZenWebTx/bin/twill-sh -i myTwillCommands.txt

The -i option instructs twill-sh to stay in the twill shell rather than exiting when it finishes running the commands in the myTwillCommands.txt file.

Monitoring through Proxy Servers

The zenwebtx daemon can access Web servers through HTTP proxy servers and non-authenticating HTTPS proxy servers.


To configure the zenwebtx daemon to use a proxy, you must define the http_proxy and https_proxy environment variables.

  1. Open the ~zenoss/.bashrc file.
  2. Add the following lines:
    export http_proxy=http://Address:Port/
    export https_proxy=http://Address:Port/

    where Address is the address of your HTTP or HTTPS proxy server, and Port is the port on which your proxy server listens.

Example Proxy Setup

HTTP and HTTPS proxies frequently listen on port 3128. If your proxy server is "my.proxyserver.loc" and it uses port 3128, then add these two lines to the ~zenoss/.bashrc file:

export http_proxy=http://my.proxyserver.loc:3128/
export https_proxy=http://my.proxyserver.loc:3128/

Testing the Proxy Setup

You can test the proxy setup by using twill-sh, the twill interpreter shell.


After setting up the proxy information in the ~zenoss/.bashrc file, follow these steps to test your setup:

  1. Make sure http_proxy and https_proxy are defined in your current shell:
    $ source ~zenoss/.bashrc
  2. Launch the twill shell:
    PYTHONPATH=$PYTHONPATH:$ZENHOME/ZenPacks/ZenPacks.zenoss.ZenWebTx/ZenPacks/zenoss/ZenWebTx/lib
    $ZENHOME/ZenPacks/ZenPacks.zenoss.ZenWebTx/ZenPacks/zenoss/ZenWebTx/bin/twill-sh
  3. Try to retrieve a URL through HTTP or HTTPS. For example, to retrieve the Zenoss platform home page, enter:
    go http://www.zenoss.com


    You should see a message similar to this:

    current page: http://www.zenoss.com

    If an error message appears, then your proxy may not be correctly configured in the ~zenoss/.bashrc file.

  4. Exit the twill shell:
    exit

Changes

3.0.1
  • Initial password is not displayed in ZenWebTx datasource. (ZPS-1289)
3.0.0
  • Added compatibility with Zenoss 5.x for metric publishing

Installation

Normal Installation (packaged egg)

  1. Download the appropriate egg file for the version of Zenoss you are running.
  2. Ensure you are logged in as the zenoss user:
    $ sudo su - zenoss
  3. Install the ZenPack:
    $ zenpack --install ZenPacks.zenoss.ZenWebTx-*.egg
  4. Restart these services:
    $ zenoss restart

Commercial

This ZenPack is developed and supported by Zenoss Inc. Commercial ZenPacks are available to Zenoss commercial customers only. Contact Zenoss to request more information regarding this or any other ZenPacks. Click here to view all available Zenoss Commercial ZenPacks.

randomness