Please Note: This project is not currently active. The content on this site is provided for reference and is not actively maintained.

Training the Cloud with the Crowd: Training A Google Prediction API Model Using CrowdFlower’s Workforce

by February 29, 2012

 

kcocco_twitter_data_google_prediction_api

Can a machine be taught to determine the sentiment of a Twitter message about weather?  With the data from over 1 million crowd sourced human judgements the goal was to use this data to train a predictive model and use this machine learning system to make judgements.  Below are the highlights from the research and development of a machine learning model in the cloud that predicts the sentiment of text regarding the weather.  The following are the major technologies used in this research:  Google Prediction APICrowdFlowerTwitter,  Google Maps.

The only person that can really determine the true sentiment of a tweet is the person who wrote it.  When the human crowd worker makes tweet sentiment judgements only 44% of the time do all 5 humans make the same judgement.  CrowdFlower’s crowd sourcing processes are great for managing the art and science of sentiment analysis.  You can scale up CrowdFlower’s number of crowd workers per record to increase accuracy, of course at a scaled up cost.

The results of this study show that when all 5 crowd workers agree on the sentiment of tweet the predictive model makes the same judgement 90% of the time.  When you take all tweets the CrowdFlower and Predictive model return the same judgement 71% of the time.  Both CrowdFlower and Google Predictions supplement rather than substitute each other.  As shown in this study, CrowdFlower can successfully be used to build a domain/niche specific data set to train a Google Prediciton model.  I see the power of integrating machine learning into  crowd sourcing systems like CrowdFlower.  CrowdFlower users could have the option of automatically training a predictive model as the crowd workers make their judgements.  CrowdFlower could continually monitor the models trending accuracy and then progressively include machine workers into the worker pool.  Once the model hit X accuracy you could have a majority of data stream routed to predictive judgments while continuing to feed a small percentage of data the crowd to refresh current topics and continually validate accuracy.  MTurk hits may only be pennies but Google Prediction ‘hits’ cost even less.

(more…)

More »

How Do Greenhouse Gases Trap Heat in the Atmosphere?

by February 26, 2012

If you would like to see a high-quality infographic developed for this question, please share it with your network using the sharing icons above—these actions will increase the question’s priority. Please look at the example answers here. Also, if you would value a high-quality answer being available on the Web, please consider a small donation ($0.99) to support our research team.

More »

What Causes Oxygen to be Used Up In Deep Waters?

by February 26, 2012

ANSWER (CURRENTLY UNDER REVIEW): Bacteria use up the oxygen in the process of breaking down organic matter, like the waste from tiny animals and dead algae. This answer is focused on coastal ocean waters, but generally applies to deep waters in estuaries, ponds, lakes, and slow-moving stretches of rivers.
 

Q001 how oxygen is used up in deep waters v04

More »

EarthQ Core Business Model: Where Is the Sweet Spot?

by February 1, 2012

Recently, we have been working to launch a project dubbed EarthQ. The goal has been to shepherd good questions and cultivate good answers by bringing the right mix of experts to Quora, as is described in this post about bringing the carbon cycle to Quora. This work continues, but so far, my attempts to draw in a few experts to help with answers have come up dry. The Quora team thinks it is okay for those who pose a question to answer it, so that will likely be my next step. Perhaps with a solid answer to the first question, What are the compartments in which carbon is stored on Earth?, I can then start to draw in some colleagues to endorse the answer by up-voting it… (more…)

More »

Tracking the Mood About Gas Prices on Twitter: A Case Study

by January 25, 2012

As another test of our strategy for teasing out public opinion from social media, we explored measuring mood about gas prices on Twitter. This post summarizes the findings from this case study. Incidentally, we are set up to measure mood from Twitter on an ongoing basis, although we would need to find a partner to help defray the ongoing costs of crowdsourcing the sentiment judgments. (See this post to read more about our decision to examine the discussion about gas prices on Twitter.)

The sentiment we mapped was culled from tweets gathered from four weeks’ worth of data starting on May 22nd, 2011. This time period was chosen to coincide with Memorial Day, a holiday during which many Americans travel by car. Our team was curious to see whether there would be an uptick in either the volume of tweets about gas prices during this period or a noticeable change in sentiment about these prices. (more…)

More »

MinutePhysics: Short Videos Demystify Physics

by October 25, 2011

If you haven’t seen the videos produced by Henry at MinutePhysics, they are definitely worth a look. Here’s his latest, tackling two very different questions: Why does light speed up after leaving glass or water? What does this have to do with the President of the US?

Incidentally, Henry placed second in our energy video contest, with his “Uncle Sam’s Dinner” video, where the guy playing Uncle Sam literally eats his way through the U.S. energy diet to demonstrate our various sources of energy.

More »

So You’re On Quora, Now What?

by October 16, 2011

As we gear up to tackle some basic questions and answers on Quora about the carbon cycle as part of our EarthQ project, with a goal of soon being able to tackle issues like ocean acidification and global warming, it seemed that a roadmap to Quora would be key for collaborators who might never have hear of it. Hence, the idea of a Quora Infographic emerged late this past week. Here’s version 0.1 that I’m throwing out for feedback. What do you think? Helpful? Silly? Gaping holes?

infographic-sketch-v02_small

UPDATE: Now showing version 0.2, thanks to a comment from friend Mike Troiano to highlight the start arrow. Keep the comments coming!

Version 0.1
infographic-sketch-v01_small

See this also on my post on Quora. I’d appreciate you up-voting it there if you like the idea.

Creative Commons License
So you’re on Quora, now what? by Dialogue Earth is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Permissions beyond the scope of this license should be submitted here.

More »

Building A Knowledge Base About Ocean Acidification on Quora

by October 13, 2011

Some months ago, we did a proof-of-concept video contest on the subject of ocean acidification. Here’s the winning video, about a snail that doesn’t have a shell, goes to learn why, and then forms a band to spread the word about this phenomenon.

Our overall vision is that we’d only initiate video, or other multimedia, projects once we’d nailed down the “science behind the story.” Furthermore, to us, nailing down the story means that we’ve ferreted out those key points for which we can get widespread agreement from experts drawn from across society–those from industry, NGOs, government, as well as academics.

To be blunt, we skipped this (huge) step for ocean acidification for two reasons. We were trying to proof out a crowd-based process for creating videos that conveyed science points fairly. Also, I had a good bit of background on the issue from my graduate work in oceanography.

102__0x0_quora-picture Now that we’re aiming to cultivate Q&As on Quora on topics related to the environment, it seems only logical to revisit the topic of ocean acidification. We’re dubbing this Quora-based project EarthQ (see also this post about putting candidates on the spot about global warming). For reference, see this post I did on the science behind the story of ocean acidification.

The goal of EarthQ will be to identify the top questions about various issues people are likely to ask about ocean acidification–assuming we can shepherd solid answers that are endorsed by a wide spectrum of experts. Taking this approach ties into Anticipating Questions People Will Ask About a Topic. Of course, if we’re successful, we will have created a great network of experts ready to jump to assistance as new questions emerge organically on Quora.

I’m a big advocate of working with building blocks on an issue. Having given this a good deal of thought recently, it is clear to me that step one will be to develop Q&A that fully explain the carbon cycle. Then, we can move on to the topic of ocean acidification. Then we should have worked out enough of the kinks to take on the topic of global warming. Onward!

More »

Putting Candidates and Others on the Spot About Global Warming

by September 29, 2011

This opinion piece, written in response to a recent Star Tribune op-ed, was published on Yale’s Forum for Climate Change and the Media on September 29, 2011.

The presence, or perhaps more so the absence, of serious climate change dialogue in the run-up to the presidential elections could make the issue a volatile unknown: Few politicians in the national spotlight want to be caught holding strong opinions in favor of aggressive policies to slow, curb, or reverse anticipated climate change.

James Lenfestey, a former Star Tribune editorial writer on climate and education, recently argued in Minneapolis Star Tribune (op-ed 9/17) that journalists should drill presidential candidates on six questions related to climate change, presumably to highlight those who are silent, or openly hostile, toward taking action in the face of this global threat.

Commentary

The media clearly can do a better job in raising serious issues like global warming with those leading, or aspiring to lead, our country. BUT putting candidates on the spot is unlikely to yield what our country desperately needs: a serious, ongoing dialogue about this global issue, which has connections to nearly every aspect of our society. Such a dialogue MUST be built on a foundation of trust and respect for political and ideological differences, as well as different scientific backgrounds.

Lenfestey’s six questions can be boiled down to these two: Do you believe global warming is occurring? And, do you believe we should have aggressive policies to counter this threat?

Lumping together opinions about how persuaded one is by the underlying science with questions about the appropriateness of a policy response is a recipe for an unproductive dialogue. Having spent a number of years working with experts drawn from across society to describe the condition and use of our ecosystems, I learned that it is essential to separate these aspects of the conversation—especially for the more contentious issues. In other words, we collectively need to find some common ground before arguing about appropriate policy responses.

We need to build an initiative that is viewed as trustworthy to all of us, no matter our ideological and political positions, or our understanding of science. WE can do that. We can run a transparent process—open to full scrutiny and input from anyone—that develops a series of questions and broadly accepted answers relevant to the issue of climate change. Not 100 questions, but rather more like 10. Each answer would be endorsed by experts drawn from across society and viewed as trustworthy by citizens and politicians from across the spectrum, both politically and ideologically.

As an example, these answers would undoubtedly include:

  • Carbon dioxide, a colorless gas present in tiny concentrations in the atmosphere, can trap radiation from the Earth that would otherwise dissipate into space.
  • There is strong evidence of a marked increase in carbon dioxide directly linked with the growth of industrialized societies across the globe.
  • Our “fingerprints” are detectable in the altered chemical nature of atmospheric carbon dioxide.
  • Long-term temperature trends indicate significant increases in recent decades.

These would be the first building blocks to a serious national dialogue.

Let’s take on this challenge. Let’s develop a robust, multimedia Web presence that brings to life the science behind the questions-and-answers for general audiences. Ideally, this effort would have its foundation on a public Q&A site, like Quora (http://quora.com).

Circling back to Lenfestey’s questions, let’s ask each candidate if he or she would support such an open, transparent process to create robust common ground across society. Global warming, which has the potential to irreversibly alter our world, demands a serious dialogue, and now is the time for to initiate it. Time is short, and there is a lot to do. Let’s make it happen.

Kent Cavender-Bares is an environmental scientist and the founder of Dialogue Earth, a nonprofit media project that has received major seed funding from the University of Minnesota’s Institute on the Environment and the Foundation for Environmental Research. The views represented are his alone and do not represent those of the University of Minnesota.

More »

Capturing Mood About Daily Weather From Twitter Posts

by September 29, 2011

After considerable preparation, we’ve just launched a version of our interactive tool, Pulse. Using Pulse, users can explore feelings about the weather as expressed on Twitter.

We began the process by choosing a topic that would yield a substantial volume of discussion on Twitter as well as be of general interest. Once we settled on weather, we wrote a survey designed to gauge Twitter users’ sentiments about the topic. With the help of workers from the “crowd” accessed through CrowdFlower, we had tens of thousands of relevant tweets coded as to the expressed emotion about the weather. These results were then used to create an “instance” of the Pulse tool, which manifests as a map of the United States that at a glance reveals Twitter users’ sentiments about the weather in their region on a given day. (You can read more about the coding process here and our choice of weather as a topic here.)

For our launch of Pulse for weather, we chose to feature tweets published over a month beginning in late April, 2011, a period in which many extreme weather events occurred—the devastating tornado in Joplin, MO; widespread drought throughout the South; and flooding of the Mississippi River, among others. The image below is from May 25, three days following the Joplin tornado (jump to the interactive map here).

may-25-pulse

We gathered tweets from all 50 states as well as for about 50 metro areas. Here you can see a zoom up on several states centered on Missouri.

zoom-may-25-pulse

The interactive map tells part of the story, namely a state’s or city’s overall sentiment about the weather, while the content under the “Analysis” and “Events” tabs reveal some of the “why” behind this sentiment: what were some of the most notable weather events occurring on a given day? [Note: our "events" feature has a bug in it and is currently turned off. In the future, icons will show up on the map to highlight out-of-the-ordinary weather events, like outbreaks of tornadoes, persistent flooding or drought, etc.] To what extent did the weather deviate from normal conditions? Why were tweets from, say, the South, uniformly negative during a certain time? What was happening when we saw a single positive state amidst a region that was otherwise negative?

We hope that weather is just the beginning. We envision using the Pulse tool to visualize nationwide sentiments about more complex, nuanced topics in the future—a sample of emotions about gas prices is just around the corner, and see our preliminary work on opinions about global warming. For now, you can explore the Pulse tool here, and let us know what you think!

More »

Message From the Bubble: A Concept for Periodic Videos About the Environment From A Roving Earth Bubble

by September 23, 2011

Brace yourselves, this is not high-production value material. Rather, this is a video that I produced in about 15 minutes, start-of-script-to-exporting-video-from-iMovie. It is the product of a brainstorming session during a great workshop put on by Liz Neeley of the COMPASS team at the U of MN’s Institute on the Environment.

Here’s the concept. I build an “Earth Bubble” that would be a moveable “studio.” Each week, or so, I plan and shoot a new video of the flavor of the one below. Aside from hopefully being a fun and engaging style, the final bit would be to zoom out and show where the Bubble is this week. Hanging from a tree, by a river… You get it.

Thoughts?

More »

Could A Social Q&A Site Like Quora Serve As Home On the Web for Information About Global Warming?

by September 21, 2011

quora-picture Over the past few weeks, I have been giving a good bit of thought to this question. Quora, began as a vibrant Q&A site for the start-up community in Silicon Valley. Its creators, who were key technologists with Facebook, know how to build a good interface. Their goal is not modest:

“Quora is a continually improving collection of questions and answers created, edited, and organized by everyone who uses it. The most important thing is to have each question page become the best possible resource for someone who wants to know about the question.”

Some months back, I posed a still-unanswered question: Should Quora be used to gather and distill expert answers on polarizing topics? It is not at all clear if the answer should be “yes” or “no.” Perhaps, as a Quora admin put it in a response to another question of mine, the right strategy is to follow Wikipedia’s “be bold” policy, which is summed up in three words “Just do it.” In other words, it may be necessary just to start doing it and figure out a way to make it work. (more…)

More »

What’s Going To Happen To Your Vehicle’s Lithium Battery At the End Of Its Life?

by September 12, 2011

Energy consumed in transportation has been the focus of a number of recent posts here, including one contemplating the use of smiley faces to help drivers understand the impact of their behavioral choices, and one exploring why the cost-benefit balance is tipped against the purchase of a hybrid car based on fuel savings alone. Another big issue to consider with either a hybrid or an all-electric vehicle is the battery, which necessarily needs to pack a lot of charge, both by being large and, increasingly, by using metals like lithium.

5029455937_5ff2964379_b

How a lithium-ion battery works: This illustration shows the inner workings of a lithium-ion battery. When delivering energy to a device, the lithium ion moves from the anode to the cathode. The ion moves in reverse when recharging. Compared to other rechargeable batteries, lithium-ion batteries can store more energy in smaller, lighter packages. This unsurpassed energy-to-weight ratio make them the battery of choice for consumer electronics like cell phones and laptops, but also a great fit for electrified vehicles. Illustration and text courtesy Argonne National Laboratory and was accessed on Flickr.

A recent post about the prevalence of rare earth metals highlighted how much there is to know about the components used to make current vehicles based on new technologies, like hybrid drive trains. Massive supplies of elements like lithium are going to be key to permit scale-up of hybrid and all-electric vehicles requiring lithium batteries. Supply can come from mining operations, as well as recycling. An article a few days ago in the NY Times Business section highlighted the reality that there is no consensus on how electric car batteries should be recycled or reused. (more…)

More »

How Would Smiley Faces Change My Driving Behavior?

by August 12, 2011

xcelenergyreport1 Having just spent a bunch of time working on a pre-proposal for the National Science Foundation, I’m excited about an idea to create more feedback for drivers about their energy use. This project concept is motivated by recent research that many of us have a tough time judging the relative energy savings from a variety of behavior changes.

For example, the researchers found that many Americans estimate that a central air conditioning system uses about the same amount of energy as a window unit, when in fact a window unit typically uses several times less energy (link to abstract of article by Shahzeen Z. Attari and colleagues in the Proceedings of the National Academy of Sciences here).

Perhaps you are an Xcel Energy customer, or you’re with another utility that is taking advantage of OPOWER’s innovative approach to giving feedback to utility customers. As a residential customer of Xcel’s, we receive a monthly statement showing our use of energy compared to similarly-sized homes in our neighborhood. Thus, we receive a chart with our use of electricity and natural gas over the past 12 months, comparing our usage to that of our “efficient neighbors” and “all neighbors.” Here is our household usage over the past year from Xcel’s Energy Report website, which provides all of the data digitally after registering. (more…)

More »

Which Energy Questions To Answer?

by July 28, 2011

2540266946_df7114d8fa The memory of our frustration during our last car purchase, more than 10 years ago, is still strong. Fuel economy was central to our decision, and we ended up opting for a turbo diesel Volkswagon. Soon after our purchase, I certainly engaged in some second guessing. This was the older generation turbo diesel, which means the dirtier version. We were in Washington, D.C., so we had to carry with us the reality that we were contributing to lowering the air quality. A more nuanced issue—and one that I’ve not fully researched—is that fuel mileage for a diesel is not directly comparable to that of a gasoline powered car, especially if one is interested in their transportation carbon foot print. I’d forgotten about this for a while, but it is very relevant now that we at Dialogue Earth are working on a project proposal to supply information to support consumer decisions that are connected to energy use.

Over the coming weeks and months, we’ll be exploring how people get their information about energy use when they’re pondering a purchase, be it a new car or a clothes dryer. Is there the proverbial “low hanging fruit” in terms of information that would be relatively easy to provide, but is not readily accessible currently? No doubt consumers will span the spectrum from knowing very little about energy to having a deep knowledge. What do consumers with different energy knowledge want to know?

Have you thought about your personal energy use recently? Did you seek out new information? If so, was it easy to find? What sources did you tap? If not, why?

More »

Sentiment Analysis Milestone: More Than One Million Human Judgments

by June 27, 2011

judgment-shot We have developed a process, dubbed Pulse, to extract nuanced sentiment from social media, like Twitter. We recognized early on that tools weren’t available to adequately answer specific questions, such as: “What’s the mood about today’s weather?” or “What portion of Twitter authors who discuss global warming believe that it is occurring?” or “Did Apple or Google have a more favorable buzz during this year’s South-by-Southwest Interactive?” Specifically, we concluded that it was necessary to get humans involved in the process—especially for Twitter posts, or tweets, which are often cryptic and have meaning that might be missed by a computer algorithm.

So, we turned to crowdsourcing.

However, successfully leveraging the power of the crowd for our sentiment analyses required cultivating the crowd, which we have achieved by working with partner CrowdFlower. In short, CrowdFlower offers an approach where we can access various work channels (we have relied mostly on Amazon’s Mechanical Turk), yet do so by layering on a quality control filter. Specifically, we intersperse within jobs what CrowdFlower terms “gold” units—in our case, tweets for which we already know the sentiment.  Workers build trustworthiness scores by getting the gold units correct. If they miss a gold unit, they get some feedback from us that has been tailored to that unit, such as “This person is happy that their garden is getting rain, so this should be marked as a positive emotion about the weather.”

We have been running a lot of jobs through CrowdFlower, but only recently did I step back and add up the tweets processed. For more than 200,000 individual tweets, we have received more than 1,000,000 trusted, human judgments from the CrowdFlower workforce! I know our research team, who had to do a bunch of judgments early on as we worked out a viable strategy, are grateful that we could get help from the crowd.

cf-jobs More »

Teasing Out Opinions About Global Warming From Twitter

by June 24, 2011

snapshot-ca A couple of months ago, we posted results from a quick sampling of mood about global warming in the Twittersphere that was featured in Momentum, the publication of the University of Minnesota’s Institute on the Environment. Along with our work on weather mood and mood about gas prices, we are on the verge of releasing a more in-depth analysis of sentiment about global warming. Here, we explain the method behind our sentiment analysis related to global warming, building off an earlier post that presented some of the details of our methodology on studying global warming chatter. (more…)

More »

The Periodic Table of Videos From University of Nottingham

by June 15, 2011

periodic-table

Periodic Table, with colors representing numbers of views for each element's videos. Red is more than 200K, blue less than 10K.

I have a chronic problem of letting my issues of Science and Nature pile up. Having picked up an issue essentially at random from the pile, I experienced that reminder that I’m missing a lot by not keeping the pile under control.

Video journalist Brady Haran and chemist Martyn Poliakoff authored an article in the 27 May 2011 issue about their amazingly successful project to create videos for each element on the periodic table.

Having just watched the video for hydrogen, I am really impressed. It is no wonder that they have amassed a loyal following and over 15 million views in total. Lots to learn from this impressive science communication endeavor!

More »

Tom Toles Weighs In on Explaining versus Persuading

by June 15, 2011

A few days ago I wrote a post on the importance of of nonpersuasive communication. Scanning the Washington Post site just now, I noticed that Tom Toles, my favorite political cartoonists, weighed in on the topic two days ago in a post entitled “Explaining and Persuading.” His statement that “[t]hese are two things that seem like they ought to go together, but somehow rarely do” illuminates the issue nicely within a different domain: inside the Beltway.

This reminds me of advice of a PR exec this week at the Google Science Communication workshop, which I had the honor of attending: scientists should “stay in their lane.” It can be a very tough pill to swallow when one feels that decisions do not line up with the latest understanding from the science community, but as Baruch Fischhoff stated in an ES&T piece discussed in my previous post:

Scientists faced with others’ advocacy may feel compelled to respond in kind. However, they can also try to become the trusted source for credible, relevant, comprehensible information by doing the best job possible of nonpersuasive communication. With long-term problems, like climate change, communication is a multiple-play game. Those who resort to advocacy might lose credibility that they will need in future rounds.

I sure wish Toles would pen a cartoon on this topic!

More »

To Persuade, or Not to Persuade? That’s A Key Question in Science Communication

by June 11, 2011

The philosophy behind Dialogue Earth is deeply rooted in a sense that our provision of information should be advocacy-free in order to maximize trustworthiness (see our history and strategy sections). That is, providing information about an issue while also urging the public to take a particular action based on this information has the real potential to erode trustworthiness, especially among those audience segments who are likely the most important: those people who are skeptical about the credibility of the information and the intentions of its source. This becomes all the more true for issues that are controversial, where I believe advocacy-driven information campaigns can deepen the societal polarization (here’s a short video I recorded explaining more about how I feel we can cut through the polarization, based on an op-ed that I penned for the Pioneer Press in April).

I recently discussed this issue in the context of an edgy hip-hop video featuring climate scientists that has become a YouTube phenomenon over the last month. In that piece, I referenced a 2010 letter to the editor in the journal Science by Bowman and colleagues that advocates for a new initiative to spread information about climate change, and that this initiative should be rooted in a nonpersuasive approach.

Digging into this issue further, it is clear that there is a debate centered on the goal of science communications. To boil things down into a few words: those calling for advocacy and persuasion by scientists cite growing disregard for science in public discourse and within policymaking, while those who urge caution suggest that mixing advocacy with the provision of science-based information jeopardizes the trust in the source of such information. By the way, a great book to read as background on the role of science in American society is “Unscientific America” by Chris Mooney and Sheril Kirschenbaum. (more…)

More »