CRAWDAD dataset dartmouth/campus (v.2007-02-08)

The traceset of polling every AP every five minutes, during Fall term 2001, Spring term 2002, Fall term 2003 and Winter term 2004. The Fall 2001 data was used for [MobiCom 2002 paper]. The 2003/4 data was used for [MobiCom 2004 paper].

We recommend only using the 2003/4 data. See this important note about problems with the 2001/2002 SNMP dataset. Any questions about the 2001/2 data will go into our LBE (Less than Best Effort) queue, i.e., they may not be answered... please use the 2003/4 data instead.

release date

2004-11-09

date/time of measurement start

2001-09-14

date/time of measurement end

2004-02-28

methodology

We used the Simple Network Management Protocol (SNMP) to poll each AP every five minutes, querying AP and client-specific counters. AP-specific variables included inbound/outbound bytes, packets and errors, and the clients associated with a given AP. Client-specific variables included MAC and IP addresses, signal strength and quality.

sanitization

To sanitize the data, we randomly (but consistently) mapped the MAC address field into a randomly chosen MAC of the same vendor, and we mapped all IP addresses using a prefix-preserving sanitizer.

disruptions to data collection

There were unfortunate gaps in the data collection, generally caused by power failures.

limitation

We have no way to distinguish periods with no clients from periods when the AP was off or unreachable. We also have no way to detect an AP reboot or reset, which reset all of the per-client counters reported here. Thus it is critical to take care when interpreting the per-client byte counts... a counter can be reset or roll over between two polls, and the delta can thus appear to be negative (or, in unsigned math, a very large number).

error

In 2001/2 data, the perl scripts that performed the SNMP queries suffered from some problems, in that they queried inappropriate SNMP values, or misunderstood the meaning of other values. This data was also used in a subsequent analysis. The same scripts were used to collect data for a subsequent study of another wireless network. See http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/

the dartmouth/campus/snmp/fall01 trace

download the fall01.tar.gz file: from a CRAWDAD mirror: US UK AU
type="tar.gz" size="221 MB"

last modified

2008-09-12

reason for most recent change

SNMP trace is resanitized.

short description

Records of SNMP polling during Fall term 2001.

description

The trace of SNMP polling every AP during Fall term 2001.

derived

false

release date

2004-11-09

date/time of measurement start

2001-09-14

date/time of measurement end

2002-01-10

format of trace data

There are two types of file.  The first type of file has one file for each access point. 
Each file gives information about each interface for each poll.  Most of the interfaces 
are irrelevant; you only care about the wireless interface (the one with ifSpeed=11000000).
Here is an example: V1.0ap
timestamp,ap,ifIndex,ifInOctets,ifOutOctets,ifSpeed,ifInErrors,ifOutErrors
Notice the periodic restatment of the file format version and the description of the MIB 
variable names used in the lines that follow.
The second type of file occurs once for each AP and each date. The file name conveys 
the blinded name of the AP, e.g. AcadBldg10AP3 where this indicates the third AP in the tenth 
building of type 'Academic.' The building types are Adm (Admin), Ath (Athletic), Lib (Library), 
Oth (Other - mainly sysadmin test APs), Res (Residential) and Soc (Social). Each data file 
contains all of the data for that access point on that date.  Below is a sample of 
the top of one file.  The top two lines are documentation; the first indicates the file format 
version (always V1.0).  The second identifies the column headers for each of the data lines.

After the timestamp (standard Unix time in seconds), the remaining fields are MIB variables 
from the AWC MIB (Aironet Wireless Communications is the name of the company that developed 
our access points; Aironet was bought by Cisco who then branded and sold the APs under their name).
V1.0
timestamp,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr,awcTpFdbDdpProdDevID,awcTpFdbDdpRadioDevID,awcDot11TpFdbAID,awcDot11TpFdbTxShortRetries,awcDot11TpFdbLatestRxSignalQuality,awcDot11TpFdbCapabilities
1001908847,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,-15,unknown,state2,73,73
1001909056,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,generic80211Client,unknown,state2,73,73
1001909266,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,-15,unknown,state2,73,73
1001909476,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,broadcast,-16,state2,73,73
1001909683,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,broadcast,-16,state2,73,73
1001909892,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,generic80211Client,unknown,state2,73,73
1001910102,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,generic80211Client,unknown,state2,73,73
1001910311,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,ethernetAP,34,state2,73,73

configuration

The job of polling all access points was divided among two machines: agentnews and klebb. Each of those hosts has a directory here. In each directory there is a number of subdirectories, one for each day, and in each of those subdirectories there are a number of SNMP log files.

error

During the fall there were some access points with old, buggy firmware that sometimes filled our SNMP data files with garbage entries. Any code you write to parse the data must be very robust. Unfortunately the code that I used was hacked on by two or three different students and is not currently in presentable form. See http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/.

the dartmouth/campus/snmp/spring02 trace

download the spring02.tar.gz file: from a CRAWDAD mirror: US UK AU
type="tar.gz" size="633 MB"

last modified

2008-09-12

reason for most recent change

SNMP trace is resanitized.

short description

Records of SNMP polling during Spring term 2002.

description

The trace of SNMP polling every AP during Spring term 2002.

derived

false

release date

2004-11-09

date/time of measurement start

2002-03-25

date/time of measurement end

2002-06-09

format of trace data

The first five lines are comments.  The first gives basic information:
V2.1: file format version 2.1, timestamp of file creation, AP name, and date code YYMMDD
All timestamps are standard Unix timestamps (seconds since 1970). The other four comment 
lines describe the format of lines that occur later in the file.  Other than the timestamp 
and AP name, the rest of these fields are MIB variable names.
After the five comment lines comes a series of polls.  Each poll consists of one ''sys'' line, 
one ''if'' line describing stats of the the wireless interface, and zero or more pairs of 
''c1'' and ''c2'' lines, each pair describing a currently connected client.  The c1 and c2 
lines are a collection of MIB variables from the AWC MIB (Aironet Wireless Communications is 
the name of the company that developed our access points; Aironet was bought by Cisco 
who then branded and sold the APs under their name).
V2.1,1018929767,AdmBldg27AP2,020416
sys,timestamp,AP,sysUpTime
if,timestamp,AP,ifIndex,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards
c1,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality
c2,timestamp,AP,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr

configuration

The job of polling all access points was divided among three machines: agentnews, klebb, and molari. I used molari only briefly until I was able to accomplish the same thing on one of the other two (access points cummings-* were missed for a week, because they are behind a firewall, and eventually I set up molari inside the firewall, and eventually I was able to add a hole in the firewall so that agentnews and klebb could poll through the firewall). Each polling machine has a directory. Each such directory has a subdirectory for each date. In each date, there is a file for each access point polled by that machine on that date. In the file are several types of lines. Below is a sample from the top of one file. Notice that it uses a comma-separated format suitable for import into spreadsheets, or easy parsing with AWK or perl

error

PLEASE NOTE THAT THERE ARE ERRORS in dartmouth/campus/snmp/fall01 and dartmouth/campus/snmp/spring02 data sets. See http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/ .

the dartmouth/campus/snmp/fall0304 trace

download the fall03.tar.gz file: from a CRAWDAD mirror: US UK AU
type="tar.gz" size="1.1 GB"

last modified

2006-11-14

reason for most recent change

SNMP trace is resanitized.

short description

Records of SNMP polling during Fall term 2003 and Winter term 2004.

description

The trace of SNMP polling every AP during Fall term 2003 and Winter term 2004.

derived

false

release date

2004-11-09

date/time of measurement start

2003-11-01

date/time of measurement end

2004-02-28

format of trace data

We folded both the ''c1'' and ''c2'' client-specific lines into one ''cl'' line 
(this made the parser code easier to maintain). To identify whether a given SNMP log is IOS 
or VxWorks (see the configuration), look at the ''sys'' line in a V3.1 log. 
The fifth field of this line is a formatted ''sysDescr'' indicating the OS version of the AP. 

An example VxWorks file
#V3.1,1073710881,ResBldg48AP1,040110
#sys,timestamp,AP,sysUpTime,sysDescr
#if,timestamp,AP,ifIndex,ifDescr,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards,awcDot11AssociatedStationCount,awcDot11AuthenticatedStationCount,awcDot11ReassociatedStationCount,awcDot11RoamedStationCount,awcDot11DeauthenticateCount,awcDot11DisassociateCount,awcFtClientSTASelf,awcFtBridgeSelf,awcFtRepeaterSelf
#cl,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbAID,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr,awcTpFdbSrcPktsImmed,awcTpFdbDestPktsImmed,awcTpFdbSrcErrorPktsImmed,awcTpFdbDestErrorPktsImmed

An example IOS file:
#V3.1,1075784465,ResBldg47AP1,040203
#sys,timestamp,AP,sysUpTime,sysDescr
#if,timestamp,AP,ifIndex,ifDescr,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards,cDot11AssStatsAssociated,cDot11AssStatsAuthenticated,cDot11AssStatsRoamedIn,cDot11AssStatsRoamedAway,cDot11AssStatsDeauthenticated,cDot11AssStatsDisassociated,cDot11ActiveWirelessClients,cDot11ActiveBridges,cDot11ActiveRepeaters
#cl,timestamp,AP,cDot11ClientAddress,cDot11ClientRoleClassType,cDot11ClientPowerSaveMode,cDot11ClientAid,cDot11ClientAssociationState,cDot11ClientIpAddress,cDot11ClientUpTime,cDot11ClientSignalStrength,cDot11ClientSigQuality,cDot11ClientBytesSent,cDot11ClientBytesReceived,cDot11ClientPacketsSent,cDot11ClientPacketsReceived,cDot11ClientDuplicates,cDot11ClientMsduRetries,cDot11ClientMsduFails

configuration

We only used one machine for SNMP polls in this period. The SNMP poller was rewritten to be more robust and more efficient, and so we only needed one machine to poll all ~ 560 APs on campus. We queried more counters in this trace. The variables are listed at the beginning of each file. For more details see the MIBS: ftp://ftp-sj.cisco.com/pub/mibs/v2/AWCVX-MIB.my ftp://ftp-sj.cisco.com/pub/mibs/v2/IEEE802dot11-MIB.my At the time of this data collection, Dartmouth mainly used Cisco 340 and 350 APs. These used to run the VxWorks operating system. During December 2003 to May 2004, our 350 APs migrated from running VxWorks to the Cisco IOS (the APs didn't originally run IOS as they were made by Aironet, a company that was later bought by Cisco). IOS uses completely different SNMP MIBs to VxWorks, and so the variable names and their order are slightly different. When the upgrades started taking place, we incremented the log version number to "V3.1" (the first line of each log) to indicate the new variables being queried. We also folded both the "c1" and "c2" client-specific lines into one "cl" line (this made the parser code easier to maintain).

how to cite this traceset