Tanya L. Cheyne and Frank E. Ritter
Department of Psychology, University of Nottingham, Nottingham, UK NG7 2RD
< lpyktlc ritter > @psychology.nottingham.ac.uk
Phone +44 (0) 115 951-5292;
Fax +44 (0) 115 951-5324
Running head: Targeting respondents on the Internet
We are interested here in using this media to perform psychology and market research. Using the Internet to conduct a survey such as this requires a method of targeting appropriate respondents. We report here an initial exploration of how this can be done, particularly how to target respondents, and what is good practice in this new medium.
The users of the Internet are an interesting group. The Graphics, Visualization and Usability Centre (GVU) at the Georgia Institute of Technology, have been carrying out surveys of Internet users for almost four years. Their survey is thought to be the most extensive of the population of WWW users. Although considerable changes have been seen in the user population, the two most recent surveys have seen a stabilisation in the core demographics. The seventh GVU survey ran for one month during April and May 1997, attracting a total of 19,970 respondents. High level demographics showed that the average age was 35.2 years, 31.3% of users were female (European users are still predominately male) and 80.0% were US residents. Their results are consistent with other surveys (Pitkow & Kehoe 1996). From these results, we can see that Internet users are not typical of the general population; they skew towards male Americans in their thirties. However, it is worth remembering that there are a significant number of Internet users who do not fall into this category. There are people on the Internet with the characteristics to fulfil most market research requirements; so a suitable targeting method needs to be found.
Another advantage of Internet surveys is that they utilise new technology and so arouse interest in respondents. As the medium becomes increasingly exploited, it may be that people will become less likely to complete surveys as it is no longer new and different. However, as the number of Internet users continues to grow, new Internet users will continue to exist who will not have experienced online surveys before.
The anonymity of the Internet can produce an advantage where sensitive enquiry subjects are concerned. People are more likely to be open about sensitive subjects when the researcher does not know who they are; the social desirability effect may be reduced or eliminated. The respondent does not have to enter any information that will identify them personally, so feels freer in telling the truth about subject areas that they would not discuss under normal circumstances. Kiesler & Sproull (1996), randomly assigned respondents to either a paper or an electronic survey. They included five questions from the Marlowe-Crown Need for Approval Scale and found significant social desirability effects between the groups; respondents replying electronically gave less socially desirable answers than those taking the paper and pencil questionnaire.
Several lessons from Computer-Assisted Personal Interviewing (CAPI) are applicable to web-based surveys. In CAPI the respondent views questions that are presented on a computer monitor and types in answers via a keyboard, as happens in an Internet survey. A review of the CAPI literature (Strauss 1996) suggests that sensitive questions yield more frank and complete answers when presented by computer. The novelty of computer-aided research results in higher respondent interest and thus higher response rates, but respondents using a computer keyboard are more likely to select inaccurate answers on multiple choice questions than with paper questionnaires.
Krantz, Scher & Ballard (1996), performed an experiment to explore the differences between results collected via the Internet and those collected in a traditional laboratory. They were interested in this because external validity can be enhanced using the web, as it provides access to people around the world. They found that the Internet could provide a larger and more demographically diverse sample. In the laboratory study, available respondents tend to be in the 18 to 22 age range, whereas their Internet survey obtained responses from people aged 18 to over 50. The Internet respondents were also much more varied in terms of race and continent of origin. In the laboratory experiment, an even number of males and females were surveyed, this was uncontrolled in the Internet experiment where a much higher proportion of males (self-reported) responded. Many of the data-sets received from the Internet were incomplete, as Internet respondents must be entirely self-motivated. However, Keisler & Sproull (1996) investigated the use of open-ended questions, and found no difference between electronic and paper groups.
Problems endemic to more traditional survey methods also exist on-line. People whose sole motive in completing a survey is to obtain an incentive may not take the necessary time to complete a survey correctly or may select responses at random. McDaniel & Rao (1981) concluded that respondent anonymity does not have a clearly defined effect on response quality. They showed that respondents asked to identify themselves on a mailed questionnaire may be more motivated to do a more accurate job of filling out the form, although not necessarily a more thorough job. However, many people appreciate the fact that they are completely anonymous when on the Internet. Requesting any identifying or demographic information might therefore be difficult, and conflicts with offering an incentive to those who identify themselves.
Multiple submissions can be a problem as they invalidate data. GVU put email address screening in their second user survey and found that 3.8% of submissions were multiple submissions from the same address (Pitkow & Recker 1995).
There are numerous advantages to conducting a survey over the Internet. There remain problems, but most of them are not unique to the Internet. The considerable advantages suggest that it is worth trying.
Newsgroups are forums for people to conduct discussions about various topics. They act like a kind of serial bulletin board. There are thousands of newsgroups. Engineering, politics, and films all have specialised groups. Counting them and their readers is difficult--most sites do not carry every newsgroup, and there is an uncountable number of local newsgroups available only at a single site or distributed in a limited area. People will read newsgroups that match their interests, and thus could be targeted using this method. Postings can be made freely on newsgroups. This can be a simple way to get to the people whose opinions you need. However, people who use newsgroups tend to react strongly to "off-topic" postings, so care must be taken in use of this method.
Websites offer a similar targeting opportunity to newsgroups. They are a way to provide hypermedia to users outside (and within) a site. An underlying assumption of the web is that websites with a clear subject matter are likely to be read by people with an interest in that subject matter. A banner advertisement placed on these websites could attract people with these interests to visit a survey page. Listing a site on search engines might also generate general traffic to the survey. Search engines act as indexes to the web, typically with an advertising banner at the top of their display.
Electronic mail is another method that could be used to target individuals. Unlike surface mail, where only the sender pays, with email the recipient also has to pay. With email the payment is made through time to load the message onto the user's computer, which when using a slow modem may impose a small but noticeable charge upon the recipient. Broad, completely unselected and unsolicited mailing where most users have little interest in the material is extremely frowned upon; it usually contradicts usage agreements, and users are regularly thrown off systems for doing it. Marketers who have used the net in this way have lost their accounts and sometimes have had their site shut down temporarily or permanently. Email used in this way can easily generate large amounts of ill-will. We find such mail offensive ourselves. Unless you have an existing relationship with the respondant, it would also be bad practice, for the survey results will most likely to be filled out by an angry public. Where this has been done to a highly selected audience and is highly interesting to the audience (such as conference announcements, positions vacant, and surveys of existing users of a software package), this has gone unnoticed on the net, and may even be welcomed. We did not have an existing mailing list of people interested in mobile phones and that we knew, so we did not use this approach.
In an attempt to attract the required respondents, the current survey was advertised in newsgroups and on web pages where our target groups were most likely to be found. This choice of distribution is not just to fit into the online culture, it is also good marketing. Directing your efforts to your target audience means not annoying non-respondents and obtaining the desired respondent demographics. We hypothesised that most visitors to these sites would be members of the target groups. We were then able to compare the effectiveness of newsgroups and advertisements on websites as a means of targeting respondents.
Other online surveys were examined to ascertain how the survey should be written to optimise the number of responses. Conclusions concerned the minimisation of open-ended questions requiring text entry, keeping the survey short, and minimising the number and size of graphics on pages, so reducing the amount of time taken for the pages to download. Speed is the number one problem for Internet users (Pitkow & Kehoe 1996); so this is a key factor in maintaining respondent interest. The online medium does not lend itself to long complex surveys. Respondents tend to lose interest after 25 to 30 questions, any more and the respondent tends to abandon the entire survey (Krasilovsky 1996).
So the aims of the research were to explore some of the issues involved in conducting a piece of research over the Internet; with main consideration given to the possibility of targeting respondents. A comparison will be made between the characteristics of respondents reaching the site through the two different methods, websites and newsgroups, along with the relative success of targeting respondents in these two ways. The use of incentives was also explored.
Newsgroups were chosen using a newsgroup directory with a search mechanism. Care was taken in the choice of newsgroups, trying to be within the topics of the newsgroup discussion and to keep readership numbers as similar as possible. Where this was not completely possible, for example, targeting females in a newsgroup about mobile phones, posts were constructed as carefully as possible to cause the least amount of offence. The wording of the newsgroup postings is shown in Table 2. For each target group the incentive was mentioned on one newsgroup and not on the other. These posts were engineered to enable a direct comparison between rate of response generated by promise of an incentive versus no incentive.
Relevant websites were found using web advertising space directories. This was verified by checking the audience demographics that are available on most rate-cards. The banner advertisement in Figure 1 was placed on the chosen pages. The details between sites varied to fit within the time scale for the survey and with the traffic on each website.
To target females, "Your Weekly Kiss" web site was chosen. It is a site devoted to romance novels. The audience of the site was stated as almost exclusively female. The advertisement was on the site's home page for 14 days, in which time the banner was expected to receive 800 viewings.
"GSMag International" was used to target mobile phone users. GSM is a type of radio technology for mobile phones, which means that most people with any interest in mobile phones would be aware of GSM and may have an interest in this site. The banner was placed on this site's home page until it received 5,000 impressions (or the survey period closed).
The UK Hyperbanner was used to target respondents from the UK. It is a link exchange mechanism available to UK related sites. As all members of the mechanism own UK-related sites, the majority of readers is said to come primarily from the UK. Link exchanges do not usually cost any money, but work through a basis of reciprocal advertising between member sites. As well as being used to target people from the UK, this mechanism was used to explore the use of link exchanges to advertise such websites. The reciprocal advertisement was placed on the concluding page of the survey. This meant that everybody who completed a questionnaire saw the advert once they had completed the survey. Placing the advert on any other page may have attracted potential respondents away to other sites before they had completed the survey. The advertisement was on the exchange for 30 days.
As mentioned earlier, search engines were also used to generate general traffic to the site. The URL of the site was submitted to Yahoo!, Alta Vista, Infoseek, WebCrawler, Excite, Lycos and OpenText. By the end of the thirty days that the survey remained on the site, it had only been listed on Alta Vista, Infoseek, and Lycos. The site description passed for entry into the database of each search engine was "WIN a Mobile Phone: Complete a short survey and WIN a high quality digital mobile phone".
No matter how they entered the survey site, all respondents were directed to an "entry" page. This page introduced potential respondents to the purposes of the survey and gave them instructions as to how they should proceed. Respondents were asked to classify themselves as Users, Buyers, or Nons. The prize draw incentive was mentioned to all the groups, and respondents were informed that only fully completed surveys would be entered into the competition. Based upon the GVU experiences mentioned above, it was thought noting this early would increase the number of fully completed surveys. As this was a variable which was held constant across groups, no conclusions will be drawn to support this.
When respondents submitted their survey responses, by pressing a button on the page, they reached a page that thanked them for taking part. This page also reminded respondents that only one survey per respondent would be entered into the competition to discourage them from re-submitting the same answers. This "thanks" page also served to inform respondents that the survey process was complete.
Target Audience Newsgroups Banner Ads
Incentive Incentive not
mentioned mentioned
Females alt.fashion soc.women Your Weekly Kiss
UK Residents uk.misc uk.d-i-y UK Hyperbanner
Mobile Users alt.cellular.gsm alt.dcom.teleco GSMag Int.
m
General Search Engines
Table
2 Survey announcement posted to newsgroups.
I am running a survey about mobile phones with the sponsorship of a telecommunications company:
http://xxxx.xxxxxxx.xx.xx/xxxxx/xxx/ver1.html
Everybody who completes a questionnaire will be entered into a prize draw to win a high quality digital mobile phone.
Your help is appreciated, Thanks, Tanya
I am running a survey about mobile phones with the sponsorship of a telecommunications company:
http://xxxx.xxxxxxx.xx.xx/xxxxx/xxx/ver2.html
Everybody can complete a questionnaire, but I am particularly interested in people who <live in the UK/ are female/ use a mobile phone.>
[The corresponding phrase in angle brackets was used for each newsgroup.]
Your help is appreciated, Thanks, Tanya
Figure 1 Banner Ad placed on websites. The top main line was in orange and the bottom main line was in green. "Click here" appeared in white.
Table 3 Responses to the various approaches.
Form of contact Responses Newsgroups 39 Banner ads 225 Search engines 750Figure 2 Responses per day from the initial posting on 9 June 1997 to survey termination on 8 July 1997.
Count Count Count
USA 737 France 2 Slovenia 1
UK 24 Poland 2 Spain 1
Italy 14 Sweden 2 Switzerland 1
Canada 11 Hong Kong 2 Turkey 1
Germany 9 India 2 Philippines 1
Finland 8 Indonesia 2 Singapore 1
Belgium 7 Malaysia 2 Taiwan 1
Croatia 6 Bulgaria 1 Thailand 1
Netherlands 6 Cyprus 1 Anguilla 1
Australia 6 Denmark 1 Israel 1
Russia 5 Greece 1 Korea 1
New Zealand 4 Hungary 1 Lebanon 1
Romania 3 Ireland 1 Sultanate of 1
Oman
South Africa 3 San Marino 1
The "Mobile Phone Users" newsgroups yielded a slightly better response rate, but here responses did not come exclusively from mobile phone users. The remainder of the respondents, however, formed a related group, self-stated "buyers", which would also be a useful group to survey when researching mobile phones, as was attempted here.
Table 5 Responses to newsgroup postings by target group. "Hits" signifies the percentage of responses received from this source, which fell into the targeted category.
Target Total Hits (%)*
Responses
UK 5 100
Mobile Phone Users 33 91
Females 1 100
Computing
the response rate for newsgroups is difficult. It is easy to see how many and
how much people post to a newsgroup by counting the messages posted. However,
the number of people posting on a newsgroup does not reflect the number of
people reading the newsgroup. While we have used the best data available, we
have had to make several assumptions to make these comparisons.We computed newsgroup readership in three ways: a previous survey of newsgroups, estimates from an online archive, and local readership of the six groups. Brian Reid, of Digital's Network Systems Lab in Palo Alto estimated readership statistics on the various newsgroups as a community service to the net up until July 1995 (Reid 1995). These numbers, shown in Table 6, were based on running an 'Arbitron' program at selected, volunteer sites, and collecting their local, summary statistics each month. While these readership numbers are admittedly not absolutely accurate, they are useful for comparing relative readership changes across time and across groups, not absolute numbers, which would have grown as well since this survey.
We used several search engines (e.g., http://altavista.telia.com) to look for other sources of readership statistics. The Usenet Information Center (http://sunsite.unc.edu/usenet-i/) provides readership numbers that are slightly different from Reid's. They appear to be from early 1995, and may have come from an earlier, unpublished, survey of readership, or from earlier versions of Reid's survey.
We could not get readership numbers at the U. of Nottingham's general computer service, but we could get the number of articles read in July 1997. These numbers reflect the local audience, with all the biases that you might expect.
What is interesting is that these partial pictures of readership are relatively consistent. The uk groups are smaller; the alt groups are larger; and the estimates, if they are to be believed, indicate that these newsgroups represent relatively large populations of 20,000 to 200,000 readers.
Where numbers are not available we have entered a readership estimate based on the response rate of its paired audience group. Our estimates do not add any information, but comparing the number of responses in this way suggests that the response rate from the UK newsgroups was higher than the than from the female target newsgroups. We have also added an estimate of the readership of the alt.cellular.gsm newsgroup unadjusted for the incentive. If there had been no incentive and the readers of this group replied at the rate of the alt.dcom.telecom newsgroup, we would expect the readership to be 313,200. This group probably does not have this many readers.
The ratio of responses received from newsgroups where the incentive was mentioned to those where the incentive was not mentioned was 3:1. This statistic is slightly difficult to interpret, however, as there are no absolute measures of the number of readers of each newsgroup. But if the measures in Table 6 are consistent and the readership is roughly the same across these groups, it suggests that an incentive increases the response rate by about a factor of 3 to 5 in people who might be particularly interested in the incentive. We can also see that being 'on-topic', having a message closely related to the topic of the newsgroup, can also increase the response rate. The number of responses was greatest from the most relevant newsgroup, and we do not have any reason to believe that its readership is significantly higher than the other newsgroups. Overall though, the response rates in Table 6 are a bit disappointing, being far less than those solicited by more traditional methods.
These results were found by passing one survey request to a limited number of groups. Different survey topics, with different requests for assistance, at a different time on different newgroups may yield different results. The expense to the surveyor and to the community are large enough that these results must be accumulated over time.
With these large numbers of readers, the financial cost per potential respondent is low. However, the loss of goodwill possible, from those who read the request for participation and dislike it, can also be quite high per response. Internet users typically resistant being used for the practice of completely self-serving commercial activity, including performing market research. This seems stronger than in everyday life, as users of the Internet tend to see it as a non-commercial space. In addition to completed forms, there were three additional responses suggesting that the request was inappropriate for the given group. This is about 8% of all the responses. It is interesting to note that all the "negative" responses came from uk.* newsgroups; perhaps suggesting that people in the UK are more sensitive to these unwritten rules being breached, or felt they had more influence over an author from a UK address, or the survey request was particularly off-topic for these groups.
Table 6 The response rate by target groups indicating the possible effect of incentives. Estimates are put in brackets and explained in the text. The total number of replies is 38 because the newsgroup identity of one of the mobile user respondants was lost.
Target Group Newsgroups Response Readership Article Replies Response
Incentive s read Rate
?
Reid UIC U of N
Females alt.fashion Yes 41,223 20,000 46 0 0.0000 %
soc.women no 83,187 53,000 9 1 0.0015 %
UK Residents uk.misc Yes 16,637 1994 2 0.0120 %
uk.d-i-y no [24,955 1336 3
]
Mobile Users alt.cellular Yes [313,20 222 27
.gsm 0]
alt.dcom.tel no 58,000 0 5 0.0086 %
ecom
Advertising on GSMag International yielded significantly more responses than any other of the banner advertisements. The click-through rate was high, approximately 4.4%. Most of the responses received from this source were from Europeans. As noted in Table 7, most of these responses, 182, were received from mobile phone users, the target group. The remainder of responses, 37 out of 40, came almost exclusively from potential buyers, another useful group to receive responses from when performing a survey about mobile phones.
Overall both newsgroups and banner ads, targeting of the defined respondent groups was achieved. Only 6% of the responses failed to fall into their relevant target group. There was a significant relationship between whether a respondent was a "User", "Buyer", or "Non" depending on the path used to reach the survey ([[chi]]2(4),=137.9, p<0.001).
Table 7 Responses to banner advertisements by target group.
Target Group Responses Target CTR* Cost/reply($
Group )
Females (Your weekly Kiss) 2 100% 0.25 7.98
%
UK residents (UK 1 100 % 0.21 0**
Hyperbanner) %
Mobile Users (GSMag 222+ 82 % 4.4 % 1.13
International)
*
CTR = Click-through rate - the percentage of responses received from viewings
of the banner. ** Cost per reply was 0 because advertising was reciprocal.
+ The survey terminated before the advert had received its full quota of 5,000 viewings.
The proportion of "users" arriving via search engines, where no targeting was employed, was significantly lower than the proportion arriving via other methods. This result could be biased due to the relatively high response rates to the advertisements targeting mobile phone users on newsgroups and web sites.
Based on these findings, it would appear that if a target population was North American including the site and its keywords in search engine databases would be a fruitful and inexpensive method to employ. This method would enable you to quickly and easily reach a large number of respondents.
Table 8 Responses to from search engines by target group.
Target Group Responses Females 97 UK residents 7 Mobile Users or buyers 541
Nearly all of the respondents (97%) requested entry to the prize draw by adding their name and address to their survey form. Asking respondents to enter their name and address had several advantages for the interpretation of results. Many respondents failed to enter their country of residence when asked for it, but the majority of these people did add their country to their address. The same could be inferred for people who did not fill in their gender but did fill in their name.
Incentives had an impact, we believe, on response rates from every distribution approach. While the readership numbers are not clear, if the measures are in relative numbers, offering an incentive had a large impact on the response rate when listed in search engines' data bases, and had a large differential effect on newsgroups and general site banner ads.
No attempt was made to target respondents on the search engines. It is questionable whether this would be possible. There are some who have attempted to target respondents through search engines; one example is by using the title "Attention Women" in their search engine listing. The problem with such strategies, is that the target group is explicitly stated. There is nothing to prevent people who are not in the target group from "disguising" themselves as someone from the target group; this could be especially prevalent when there is an incentive at stake. By explicitly stating that you are looking for respondents from groups with particular characteristics, there may be a tendency for bona-fide members of these groups to answer the survey in accordance with how they think that this group would behave, as opposed to how they personally would behave. We already suspect that search engines are using what is being searched for as a way to target groups for their banner ads. Targeting via search engines holds the potential to generate a large response from a desired target group because they find the survey as part of their search and are interested.
Posting on newsgroups proved a good way to reach small numbers of targeted respondents. This would seem a good method to recruit participants for online focus groups, where a large numbers of responses are not required, but the characteristics of respondents are important to the research. Newsgroup postings, can be used to quite easily recruit participants on the basis of country, possibly by occupation (e.g. uk.education.teachers), specific gender based issues (with corresponding newsgroups), and hobbies. However, anything more complex can become difficult, and requests that are too general are inappropriate to the culture. Currently, inappropriate posts can generate quite a backlash against the poster. Reputable firms will keep this firmly in mind, for the recipient must often pay to receive what to them is junk mail. This could lead to serious public relations problems. Peers and non-profit organisations working towards their public goals are probably safe in this region. Because of the relatively low response rate and these potential difficulties, we do not recommend using general newsgroups to solicit responses.
It appears that it is worthwhile providing an incentive for respondents both to encourage them to visit a site and to complete a survey. The 3:1 response ratio, between newsgroups where the incentive was mentioned vs. where it was not, could be wrong. The readership of newsgroups remains difficult to quantify. While the readership numbers may not be accurate in an absolute sense, it is unlikely that their relative values are biased.
Where banners ads appear is important. The low response rate yielded from "Your Weekly Kiss" could be attributed to the placement of the advert on their page; at the top of a very long, slow-loading page. It is likely that many people would have scrolled down the page before the banner had loaded up, thus did not see the advert. The same was possibly true of the UK Hyperbanner link exchange. Link exchange mechanisms are very difficult to monitor. You cannot check where your banner will be displayed on all potential pages, so you have no idea whether each time your banner is displayed, it will be of any use. Reciprocal banners take a long time to load because the mechanism's database has to be contacted. It is possible that most readers would have moved on before the banner was displayed. If contemplating using a link exchange, it would be advisable to explore some member sites to see where banners are displayed, the length of time banners take to load, and the time to reach the page that the banner represents.
The low rates of response of two of the banners can be compared to the impressive click-through rate offered by GSMag International. The banner was displayed on the home page of the site which consisted of less than a screenful of information, with only six buttons available to click on. This meant that all visitors to the site saw the banner, and would thus be more likely to follow it to the survey site.
Dong, X., & Su, L. T. (1997). Search engines on the world wide web and information retreival from the internet: A review and evaluation. Online & CDROM Review, 21(2), 67-81.
Keisler, S. & Sproull, L. (1996). Response effects in the electronic survey. Public Opinion Quarterly, 50, 402-413.
Krantz, J.H., Ballard, J. & Scher, J. (1997). Comparing the results of laboratory and World-Wide Web sample on the determinants of female attractiveness, Behavior Research Methods Instruments and Computers,29(2), 264-269.
Krasilovsky, P. (1996). Surveys in Cyberspace, Nov/Dec 96, Marketing Tools http://WWW.marketingtools.com/Publications/MT/96_MT/9611_MT/9611M18.HTM (visited 6 March 98).
McDaniel, S. W. & Rao, C. P. (1981). An investigation on respondent anonymity's effect on mailed questionnaire response rate and quality. Journal of the Market Research Society, 23(3), 150-160.
Mehta, R. & Sivadas, E. (1995). Comparing response rates and response content in mail versus electronic mail surveys, Journal of the Market Research Society, 37, 429-439.
Pitkow, J. E., & Kehoe, C. M. (1996a). Emerging trends in the WWW user population. Communications of the ACM, 39(6), 106-107.
Pitkow, J. E. & Kehoe, C. M. (1996b). GVU's 6th WWW User Survey, http://www.gvu.gatech.edu/user_surveys/survey-10-1996/ (visited 6 March 1998).
Pitkow, J., & Recker, M. (1995). Using the Web as a survey tool: Results from the Second WWW User Survey. http://www.igd.fhg.de/www/www95/papers/79/survey/ survey_2_paper.html (visited 6 March 1998).
Press, L. (1995). Resources for networks in less industrialized nations. IEEE Computer, 28(6), 66-71.
Reid, B. (1995). USENET Readership summary for Jul 1995. Posted to news.groups on 6 Aug. 1995, archived at http://www.tlsoft.com/arbitron/jul95/arbitron.summary.txt (visited 6 March 1998).
Strauss, J. ( 1996). Early survey research on the Internet: Review, illustration and evaluation, American Marketing Association 1996 Winter Educators' Conference: Marketing Theory and Applications http://WWWsb.ccsu.ctstateu.edu/xroads/strauss_paper.HTML (visited 6 March 1998)