David Harkness, Paul Lindstrom and Tom Dubois, Nielsen Media Research
(dave_harkness@tvratings.com)
The Nielsen/ CommerceNet Survey was done in two parts, a RDD telephone based survey and a Website based survey. The methodology for the phone survey will be reviewed and the results of the phone survey and Website survey will be compared.
Slides of the PresentationExecutive Summary of Nielsen Internet Survey
Donald Campbell and Scott Finer, FIND/SVP's Emerging Technologies
Research Group
(dcampbell@findsvp.com)
Some 9.5 million Americans now use the Internet , including 8.4 million adults and 1.1 million children under 18, who tap into it from the workplace, school and homes, according to findings from the American Internet User Survey, a just completed study by FIND/SVP. The study culminates a year and a half of intensive research and development. The American Internet User Survey was fielded by telephone in November and December to 1,000 randomly dialed Internet users. It was preceded by focus groups and a series of online surveys, and represents the fourth time FIND/SVP has studied Internet users by telephone in the past 18 months. This makes FIND/SVP the only firm with historical random sampled data, enabling Internet usage behavioral tracking since 1994.
Roberta M. McConochie and Tom White, Arbitron NewMedia
(RBRTAMCARB@aol.com)
Approximately one year ago, the demographics and other descriptors of Internet accessors portrayed the "classic" innovators and early adopters of diffusion of innovation research. Yet our survey results point to important differences in the specific ways in which consumers use the Internet and other PC utilities. We discuss the forecasts of PC penetration, based on our survey data, and we also consider possibilities for further distinctive patterns of PC use as PCs proliferate. The analysis also recaps consumers' media technology motivations from Arbitron NewMedia's segmentation based on a nationally projectable survey of over 4,000 U.S. consumers. We also consider the question of how much consumers might spend on new media technologies and services based on a combination of Pathfinder research results and consumer expenditure data.
Bruce MacEvoy, SRI International
(bruce_macevoy@qm.sri.com)
I present demographic and psychographic information from a longitudinal survey of over 15,000 World Wide Web users administered by SRI International in 1995. Comparisons to other Internet surveys, utilizing research procedures similar to or different from ours, provide a unique, integrative look at the characteristics of the Web population. This integration will be used to evaluate the different research methodologies and to suggest new techniques, such as snowball sampling, for Internet research and measurement.
James Pitkow and Colleen Kehoe, Graphics, Visualization and Usability Center, Georgia
Institute of Technology
(pitkow@cc.gatech.edu)
The GVU Center's World Wide Web User Surveys ask questions on: General Demographics, Authors & Information Providers, Web & Internet Usage, Web Service Providers, and Consumer Surveys. Four surveys have been conducted, approximately every six months, since January 1994. The last survey, conducted October 10, 1995, received over 23,000 responses. We have used diversified exposure in an effort to minimize any systematic effect introduced via the sampling method. Specifically, we designed the Third & Fourth Surveys to enable us to determine how the respondents found out about the survey. This allows us to group respondents accordingly and look for significant differences between these user populations. In the Fourth Survey, we note that there are more differences between groups, notably between male and female users. Yet despite these differences in the method of participation, the Fourth Survey's ratios for gender and other core demographic characteristics like income, marriage, etc. are almost exactly those reported by North American based random sampling surveys. This decreases the reliance we formerly placed upon method of entry as an indicator of bias and increases our confidence in the overall robustness of the Fourth Survey's results.
Arnold B. Urken, SmartChoice Technologies Corporation
(aurken@smartchoice.com)
Choice Processor(tm) technology, based on collective decision analysis methodology, provides a survey tool for setting up Web surveys, for collecting granular data (including information about gradations of preference and judgment as well as comments), and for providing instant analysis with graphics. This technology, embedded in Sm@rtPoll(tm), enables analysts to make sensitive interpretations of survey data. Longitudinal survey data can be collected to complement hit rate analysis or can be derived from polls that are generated randomly or, non-randomly, on-the-fly to reflect what users have seen or done. This type of analysis provides insight into respondent preference or judgment structures and enables identification of key background variables. Such insight also provides a basis for understanding target populations and constructing rigorous sampling procedures for interactive polling.
Roger Hurwitz, Mark Bonchek and John Mallery, MIT Artificial
Intelligence Laboratory
(rhhu@ai.mit.edu; bonchek@ai.mit.edu; jcma@ai.mit.edu
The presentation covers the methodology and preliminary results for a current survey of users of the White House's on-line documents service and its World Wide Web site. The purpose of the survey is to map how and to whom White House information is distributed, how recipients use the information, the effect of their receiving it through computer networks and ways of improving White House on-line service. Since part of the survey replicates a survey done two years ago, the data collected will support some longitudinal as well as correlational analysis. The system implements hierarchical, adaptive survey methodologies (question trees), with respondents automatically selected according to their answers in a top level survey and sent follow up surveys specific to their subgroups. In the present survey, follow ups are sent for different types of document end users, and (re)distributors, as identified by the top level survey. The survey is intended to identify distribution patterns of White House documents and provide demographic, behavioral and attitudinal data on their users. One social research issue of interest is whether computer network access of the documents is chiefly by politically experienced, computer literate users and compliments their other political information seeking (the main finding of the earlier survey). Or, does on-line access also attract the politically inexperienced, increase their interest in participation and decrease the perceived social distance between themselves and government?
Paul M. Grand, NetCount LLC
(pgrand@netcount.com)
NetCount is a third-party Website tracking and verification service with products that have shipped commercially since October 1995. NetCount's mission is to provide accurate measurement of the World Wide Web, including user counts and demographic collection, without violating user privacy. NetCount is the technology leader in Web measurement using true census-based tracking techniques, auditable systems and timely reporting practices. NetCount's HeadCount is the first transparent and anonymous user authentication system which reports demographics without sacrificing user privacy. NetCount has a suite of products and services designed to meet the measurement needs of Websites, Media Research Professionals, Advertisers and Media buyers.
Through a transparent and anonymous user-authentication system, NetCount HeadCount is able to provide a definitive accounting of individual Web Site users and independently verifiable user demographics. Users can be tracked completely anonymously and identified over repeat visits and multiple sessions. Once a user has been to any NetCount HeadCounted Website, they can be "recognized" persistently by other NetCount HeadCounted Websites without having to stop to fill in a user ID or password -- authentication without registration. The ID's are purposefully weakened to prohibit the ability to track a user by name or e-mail address. ID's generated by one Website cannot be read by a Website in another domain, prohibiting sites from sharing user data. If presented with a subpoena, NetCount has no "user" information in its database and would have nothing to provide, thus ensuring full protection of user privacy.
Dave Olivier, Clickshare Corporation
(Dave@Newshare.com)
The Clickshare(sm) service is a distributed user access management and tracking system which supports audience measurement as one of its suite of services. Clickshare(sm) allows individual Web sites to maintain their own user base yet cooperate with other Web sites to share users for financial gain. Clickshare(sm) can track an individual's usage across multiple unrelated Web sites in real time, provide information about that individual's general usage preferences, group users into "service classes," and periodically provide reports of multi-site usage for each individual user. All sensitive personal and financial information about individuals resides at one Web site the user selects as "most trusted" and never crosses the Internet without the user's explicit permission. All access is tracked by the individual's globally-unique (but anonymous) user ID, and no long-lived central database of user information is maintained by Clickshare(sm). Clickshare(sm) has three components: a Clickshare(sm) Web Server that runs at each Web site subscribing to the Clickshare(sm) Service; the Clickshare(sm) Authentication Facility which provides real-time multi-site user authentication, and the Clickshare(sm) Logging Facility which gathers and sorts in real time multi-site user access information for later use in audience measurement and micro-transaction domains.
Robert Spielvogel, Han-hua Chang and Margaret Honey, Education
Development Center
(rspiel@edc.org)
FootPrints is a system of evaluation instruments that collect and interpret information in the context of a project (guided by research and ethical issues) on student use of the Internet and how that use evolves over time. These standardized instruments and techniques allow even small projects to compare and contrast results with regional and national data as well as add findings to a larger context while preserving the unique character of their work. FootPrints includes:
FootPrints can be used in a variety of assessment and evaluation contexts, short and long-term projects, and certain types of cross-project analysis.
Ariel Poler, I/PRO
(ariel@profiles.ipro.com)
The presentation will address our beliefs of "why" it is important to measure the web, and "what" it is meaningful to measure. I will outline our view of how traditional media measurement compares with the present and future of Web measurement, and what the key challenges are in Web measurement. I will also describe I/PRO's principles towards analyzing traffic at specific sites, and our approach towards learning about consumers. Finally, I will explain our view of how organizations and consumers can develop a win-win scenario that delivers information to the sites and value to the individuals.
Neil Smith, HENSA Unix
(ngs@sesame.hensa.ac.uk)
The collection of server log information is a key tool in the analysis of the population of the Internet . It is a non-invasive survey methodology that can be used with no effect on the behavior of the population studied. Unfortunately, a survey relying purely on the logs collected from a small collection of servers will not accurately represent the whole population, merely that part of it accessing the servers chosen. Collecting access information from the client end of the communication model does not help matters as, the choice of clients will have a great bearing on the results obtained. Collecting the information at some intermediate point in the communication model involves no choice of servers or clients to analyze and, consequently, does not suffer from the same independence problems. The HENSA Unix public Web proxy is such a point and acts as a focus for all communication between tens of thousands of clients and thousands of servers. Currently the service responds to more than one million requests every day and could provide a wealth of useful data to any project investigating the population of the Web.
Kaenan Hertz Macro International), Patrick W. O'Carroll (Center
for Disease Control), Andrew Friede (CDC), Eric Zook (Macro) and Kira
Chittwood (Macro)
(hertz@macroint.com)
We will present the methodologies used to study the WONDER/PC on-line. This system was designed to enhance the ability of distally-located public health employees to access important information for use in public health decision making and practice. It provides easy access to CDC data sets, public health reports and guidelines, and electronic mail. Part of this evaluation is taking place entirely on the system. 1000 randomly selected users were emailed a survey and asked to respond via email. In addition, 250 individuals were contacted by telephone. The methodologies and results of this research project are presented in three sections: purpose of the research and sample selection, information on response rates and related issues, and comparisons of email results to telephone surveys.
Marc Abrams and Stephen Williams, Computer Science Department, Virginia Tech
(abrams@vt.edu)
Thus far, World Wide Web usage surveys have been conducted using traditional techniques. We propose use of an alternative and complementary method: automated network monitoring (ANM). In ANM, a machine, attached to the network monitors packets, decodes the HTTP protocol headers, and produces a log of HTTP network traffic. The software which we have implemented using this method produces a "common log file" format log, augmented with additional fields, so that it can be analyzed by existing analysis tools. ANM has advantages traditional online surveying. First, ANM collects a complete record of all URL requests that appear on the network, so it is not subject to the sampling and self-selection problems. Second, the accuracy of the information is guaranteed because an independent computer is used to monitor and log data.
Jeffrey E. Danes, California Polytechnic State University and
Marketruler Corporation
(network@marketruler.com)
The paper presents a model-based method of descriptive inference for volunteer responses to electronic surveys. Probability weighting is used as a quasi-randomization method for mean imputation for ignorable nonresponse. A Bayesian model for predicting the distribution of the sample mean of nonrespondents is used to test for sensitivity to nonresponse. Self-administered, computerized interviewing is used for data collection. Descriptive inference to the sample mean for nonrespondents is made by calculating a posterior probability distribution on unknown values in the population given the known values in the population and an a priori specified model. Respondents are self-selected participants in a survey with data collected by automated interviewing machines. An example is provided.
John C. Mallery, MIT Artificial Intelligence Laboratory and Political
Science Department
(jcma@ai.mit.edu)
This presentation explains the design of the automatic survey facility which is part of COMLINK, a larger system for electronic publication and wide-area collaboration. The survey facility builds on the following components:
Sunil Gupta, Hermes Project, University of Michigan Business School
(sunil_gupta@ccmail.bus.umich)
Focusing on the need to find a match between data, methodologies, and decision making objectives, this presentation will examine two major issues:
Tim Stehle, Knight-Ridder Inc.
(tstehle@geis.knightridder.com)
A brief taxonomy of the issues and clarification of terminology. It seems to me that issues could be addressed and parsed and problems solved more readily by understanding more precisely what "layer" is being dealt with. There are multiple layers (rows) of issues being addressed, and two broad categories or tables. They should be separated and addressed individually. For each layer of issue, we need to look at several areas (columns): precisely what it is; the goal or function we are trying to achieve; the method(s) of achieving it; issues and tradeoffs involved. The basic framework differentiates user and usage information, on one hand, from user demographics and user technographics (e.g., browser types, services received), on the other.
The policy issues include privacy, usage and user information exchange for cached sites, on-line surveys and other research, secure transactions and the like. There are obvious linkages between the policy issues in one layer to those in another, but the policy making may be different for each.
Bill Harvey, Next Century Media, Inc.
(bhncm@ix.netcom.com)
The presentation overviews the promise of Interactive Media to advertisers and marketers, its unique opportunities for measurement and the obstacles, like statelessness, caching and privacy, to realizing these fully. We note positive developments on the horizon, such as this conference and what it might lead to in the way of new Internet protocols. We suggest joint industry and academia action to provide better clickstream measurement for cyberspace by means of new Internet protocols plus communications programs to educate practitioners. The anticipated benefits of better measurement include more advertiser revenues into the new medium, leading to higher quality content supported by sponsorship, and advertisers sponsorship of individuals with a proven ability to create web content interesting to others.
Development in these directions might likely require initiatives for additional CASIE standards. These would include a Hard Visit Code, a Persistent User ID, cacheback of data, X rated invisibility and universal registrations. All these proposals are likely to be controversial and will stimulate discussion and the generation of refined ideas for the accomplishment of the overall common aims.