Social media analysis finds happiness in Indianapolis and not so much in Newark
Half of the people on the planet live in cities, so we decided to find out whether urban residents are happy and whether some cities are more desirable than others. We found some of the answers in an unconventional place—your Twitter feed.
Every time somebody posted something about their life, their dreams or simply what they ate for dinner, we analyzed it for sentiment—how many positive or negative words tweeters used to describe their feelings. Over two years we looked at millions of social media posts in eight U.S. cities for our book Urban Social Listening: Potential and Pitfalls for Using Microblogging Data in Studying Cities.
Tweets are the digital versions of those fabled crumbs that Hansel and Gretel dropped in the forest. They are a rich source of new insights into a range of social phenomena. Big companies already have mined this new frontier of social listening to market their products and guide their decision-making and investments. Twitter and other microblog data have been used to aid rescue efforts after earthquakes, tweak public transit routes and monitor heart disease rates in rural areas.
For government agencies and other organizations looking to address the challenges facing urban America, this kind of digital eavesdropping can be transformative. Our own work seems to contradict conventional wisdom that residents in cities experiencing population decline must surely be unhappy.
For our research, we created rankings from a vocabulary-based analysis of all words in a tweet containing some kind of sentiment. For instance, the word “good” generated a positive score, while “terrible” led to a negative score. After totaling up the scores, we were able to rank the happiness quotient in the eight cities based on whether residents expressed positive or negative sentiments. We compared our happiness scores with data from the U.S. census, like income and educational attainment—the conventional way of assessing the well-being of a population.
The winner was Indianapolis, Indiana, which came out on top for both positive tweets and U.S. census rankings. Atlanta took third place among the eight cities, both in the census rankings and in our batch of tweets. Newark, New Jersey, which ranked worst in census demographics like income and education, also had some of the lower levels of positive tweets. We also rated tweets in Houston, New Orleans, Providence, Rhode Island, and Washington, D.C.
We then looked at how population changes in each city between 1970 and 2010 could have affected the tweeters living there. We discovered that there was no correlation between population shift and our tweet scores, belying the conventional notion that residents living in cities that are losing population are generally unhappy. For example, St. Louis, Missouri, had the highest rate of population decline, but its tweet scores were second best among the eight cities. It turns out that urban dwellers may be satisfied with the neighborhoods in which they live, but maybe not with the city as a whole—at least according to what they share on social media.
Digital crumbs are an urban planner’s gold mine. They provide extraordinary opportunities to gain fresh, fast and novel insights into what residents care about and what they’re talking about—so maybe that’s the best way to determine the location for that next light rail stop or new housing or a city park.
Justin B. Hollander, A96, is an associate professor of urban and environmental policy and planning at Tufts. Andrew Wiley, G15, works at the national nonprofit the Fund for the Public Interest. They are among the co-authors of Urban Social Listening: Potential and Pitfalls for Using Microblogging Data in Studying Cities, released last year by Palgrave.
