Please Note: This project is not currently active. The content on this site is provided for reference and is not actively maintained.

Why Weather Mood for Pulse?

by February 14, 2011

As we developed our interactive platform for the analysis of dialogue in the social media, we needed to identify a topic to start with. Specifically, we needed to identify a topic that would have a high volume of chatter on Twitter, be of general interest, and present a decent challenge for our research team. Plus, we wanted a topic that was likely to vary geographically, because Pulse is fundamentally a platform for examining trends in dialogue across geographies. We had undertaken early work looking at the 140-character posts on Twitter (“tweets”) to see if we could infer public opinion on global warming (look for a post summarizing this work soon). We’ll be returning to that topic as we fill out our portfolio of topics on Pulse.

The idea of tracking “weather mood” caught on soon after the idea came up. Here’s why.

Photos used under Creative Commons from: K. Burkett, gr33n3gg, B. Leon, and F. Marini.

Everyone experiences the weather day-to-day. It’s a constant in our lives, and it evokes all sorts of emotions. There’s excitement for the first snow in some places, while at the same time cities unaccustomed to snow can be thrown into turmoil. We lament the effect lousy weather will have on a planned outing. We delight in exceptional weather that lines up with special days. The list is endless.

Our research team began by using Twitter’s advanced search tools to see what types of statements people make about the weather. Yep, there is certainly lots of chatter on Twitter about the weather.

To characterize the chatter, the team’s immediate task was to develop a list of keywords that would enable us to grab mostly relevant tweets from the millions that are posted each day.  Some of the questions we were asking included:

  • Are the keywords found in a lot of unrelated tweets? For example, “slush” gets tweets about “slush funds,” whereas “slushy” is much better at getting weather-related tweets. We still wanted the weather-related tweets with “slush” in them, so we needed a way to exclude those that also had “fund” in them.
  • For the tweets that are related to weather condition, are people expressing emotions?
  • Is there a decent distribution of tweets from across the US (we’re only focusing on the US right now)?
  • Does the keyword consistently return relevant tweets over time?

We soon realized that the list of keywords was growing long and we need more advanced tools to evaluate the merits of various keywords. This led us to fast-track a forecasting tool we had put on hold back in the summer of 2010. We’ll do another post soon to describe how we use the Forecasting App to develop keyword lists and extract sample batches of tweets.

This post will be followed by one that will describe how we developed a keyword list and survey for extracting weather mood from the Twittersphere.

Leave a Reply