GEMS OR FAKES? USING LOCATION AWARE TWEETS TO ASSESS ONLINE REVIEW-RELIABILITY

다국어 초록 (Multilingual Abstract)

Online review sites have become both popular and indispensable for many industries that have recognized the importance of word-of-mouth as advertising tools. Hotels and restaurants that are rated highly by travel site “Trip advisor” proudly put a sticker outside their business locations demonstrating their popularity. The review site logos, and the business scores on stickers and badges regularly serve as seals of approval and symbols of reliability. This has given rise to a cottage industry that misuse the trust. While some businesses post flattering reviews as advertising, competitors sometimes falsely slander reputation of competitors.
There has been some research which explores the issue of reliability of online reviews, for example, Luca and Zervas, (2015)* identify different restaurant characteristics that cause them to use fake reviews. Ney (2013)* identifies factors consumers use to assess credibility of online reviews. The problem of unreliable reviews creates an interesting set of issues that we attempt to address in this paper. First, if there is a way to confirm whether the reviews are reliable without engaging in primary data collection. Second, what explains the underreporting or over reporting of the quality of a place? To answer the above questions, in this paper the authors extract emotions embedded in location-based tweets emerging from restaurant locations to verify the reliabilities of their online review scores on Yelp. Due to the real-time nature of the feedback, location based tweet content is free of certain survey response biases like social desirability bias.
In order to collect location based tweets, we mined data from consumers checking-in via Foursquare (a location based social network application) at restaurants, across six regions in USA. These regions were chosen because of the high volume of check-ins emanating from them on foursquare. Using this data set we were able to extract specifics such as the name of the restaurant, the content of the tweet and related temporal variables impacting the consumer’s experience in a particular business location. Over twenty five thousand tweets were analyzed which were posted by approximately 14000 users. Further, we developed a scale measuring emotions embedded in the tweets with the help of University of Florida’s Affective Norms for English Words (ANEW) scale. Each of the tweets were divided into its constituent’s words and the words were checked against the Anew scale items. When a word was identified, we allotted a numerical pleasure value to that word. At the end of the processing we had an average numerical pleasure score for each tweet.
Using the tweet pleasure score and the Yelp score, an index was computed that could reveal whether Yelp overrated or underrated the restaurant. Further analysis led to preliminary findings that demonstrated how underrated or overrated a restaurant was varied with the type of cuisine served in the restaurant. Among all restaurants, over 75% of the restaurants were classified as overvalued. In other words, based on tweet emotion content, most Yelp ratings appear positively biased. Asian restaurants were the most overvalued (100%) followed by Latin restaurants, which were 88% overvalued. One interesting initial finding was that American category restaurants were the most undervalued. 43% of the restaurants were undervalued on yelp as compared to their pleasure ratings.