Ian Sharpe - Technical Notes - HTTP Proxy Stats Generator


Introduction

This note briefly describes the Proxy Stats Perl library. The library consists of two classes (ProxyStats and NTLProxyStats), two timer support classes (UnixTimer and WindowsTimer) and two example programs, ntlinktest and tracehttp. The package is available as ProxyTool.tar.gz.

The code was originally written to investigate the characteristics of the Inktomi Traffic Master proxies used on the NTL network, however the facilities provided by the classes may be of wider interest.

In addition to the basic Perl classes, use is made of the World-Wide Web library for Perl (LWP). Timing requires the presence of Time::HiRes for Unix systems or Win32 for MS-Windows. Note that Perl 5.6 is required for these modules.

Installation

There is no standard installer as yet: unpack in a suitable directory.

Modules

To Do.

Demo Programs

ntlinktest

Description

The primary function of ntlinktest is to determine the max, min and average times taken to transfer a page from a given site. The request can be made through one or more proxies.

Command Line

ntlinktest <url> {<method> <proxy> <clientip> <timeout> <count>}

where:

url
The URL to fetch. This parameter is required.
method
The HTTP method to use. Currently GET, HEAD, OPTIONS and TRACE are supported. The default is TRACE.
proxy
Either a comma-separated list of proxies or a regular expression that selects from the NTL Inktomi names. For example, "gui" will select the three Guilford Inktomis. Two useful special cases are ".*", meaning all of the NTL proxies, and "", which ignores all of the proxies. In the case of NTL this means that the transparent proxy will be used. Default is "" (i.e. no proxy / transparent proxy).
clientip
If specified, this should be the client ip in dotted form, in which case the TRACE response will be checked for a matching Client-Id line. Default (an empty string) is not to do this check.
timeout
Timeout to use for the request. Default is 1 second.
count
Number of replications to perform. Note that requests are sent to each proxy in turn (i.e. the inner loop is over the array of proxies, and the outer loop is over the number of repetitions. Default is 10.

Output

When running, '*' is output at the beginning of each replication and '.' is output for each page that is fetched.

The substantive output is in the form of a comma-separated set of lines, one per proxy.

TRACE
  1. Name of proxy
  2. Max time taken
  3. Min time taken
  4. Average time taken
  5. Number of successful requests
  6. Inktomi id string
  7. HTTP version in response
  8. List of errors or discrepancies found. The iteration number is prepended.

HEAD, GET, OPTIONS
  1. Name of proxy
  2. Max time taken
  3. Min time taken
  4. Average time taken
  5. Number of successful requests
  6. Inktomi id string
  7. Inktomi status string
  8. List of errors or discrepancies found. The iteration number is prepended.

tracehttp

Description

tracehttp uses the Max-Forwards header along with the TRACE command to gather timings for each step in a chain of HTTP proxies. If it receives identical Via lines for consecutive Max-Forwards values, it assumes the target server has been reached and terminates.

Command Line

tracehttp <url> {<proxy> <depth> <timeout> <iterations> <method>}

where:

url
The URL to fetch. This is a required parameter.
proxy
The name of an explicit proxy. Default is to not use an explicit proxy; this should catch the use of transparent proxies. An empty proxy name will have the same effect, otherwise the name should be of the form http://<hostname>:<port>/.
depth
Maximum number of hops to the target server. Places an upper bound on the automatic target detection described above. Default is 3.
timeout
Timeout to use for the request. Default is 1 second.
iterations
Number of replications to perform to generate statistics. Default is 4.
method
The HTTP method to use. Currently GET, HEAD, OPTIONS and TRACE are allowed, but servers are only required to support the Max-Forwards functionality for TRACE and OPTIONS. The default is TRACE.

Output

Output is a tab-delimited set of values, one per hop:
  1. Hop number
  2. Max time
  3. Min time
  4. Average time

Original at http://www.sharpe-practice.co.uk/isharpe/technote/proxystats.htm. $Id: proxystats.htm,v 1.3 2002/06/05 10:42:23 isharpe Exp $