Please Note: This project is not currently active. The content on this site is provided for reference and is not actively maintained.

Soliciting Questions from the Crowd, Looking for the Crowd to Rank Questions

by April 2, 2012

process-sketch-v01 Over the past few weeks, we’ve been preparing to launch the EarthQ project. Modeled after Spot.us, which was recently acquired by American Public Media, we’re soliciting questions for which people would like to see high-quality, evergreen content developed (see some example answers here). In order for a question to move up the queue, or have its EarthQ rank increased, it needs to be shared a lot through Twitter, Facebook, and other social media. We’re doing this because it will be a substantial research effort to develop the high-quality answers, and we want to ensure that the question is of interest broadly.

Then, just like Spot.us raises funds to support the reporting on a particular topic, we’ll be looking for donations to support our research team. We’ll be aiming to raise micro-donations in 99 cent increments.

Head over to the Questions page to share (rank) questions and feel free to click the Amazon donate button to throw your 99 cents behind a question.

Over the past few days, these questions have been submitted via the website:

We welcome your feedback and ideas on this new project.

Training the Cloud with the Crowd: Training A Google Prediction API Model Using CrowdFlower’s Workforce

by February 29, 2012

 

kcocco_twitter_data_google_prediction_api

Can a machine be taught to determine the sentiment of a Twitter message about weather?  With the data from over 1 million crowd sourced human judgements the goal was to use this data to train a predictive model and use this machine learning system to make judgements.  Below are the highlights from the research and development of a machine learning model in the cloud that predicts the sentiment of text regarding the weather.  The following are the major technologies used in this research:  Google Prediction APICrowdFlowerTwitter,  Google Maps.

The only person that can really determine the true sentiment of a tweet is the person who wrote it.  When the human crowd worker makes tweet sentiment judgements only 44% of the time do all 5 humans make the same judgement.  CrowdFlower’s crowd sourcing processes are great for managing the art and science of sentiment analysis.  You can scale up CrowdFlower’s number of crowd workers per record to increase accuracy, of course at a scaled up cost.

The results of this study show that when all 5 crowd workers agree on the sentiment of tweet the predictive model makes the same judgement 90% of the time.  When you take all tweets the CrowdFlower and Predictive model return the same judgement 71% of the time.  Both CrowdFlower and Google Predictions supplement rather than substitute each other.  As shown in this study, CrowdFlower can successfully be used to build a domain/niche specific data set to train a Google Prediciton model.  I see the power of integrating machine learning into  crowd sourcing systems like CrowdFlower.  CrowdFlower users could have the option of automatically training a predictive model as the crowd workers make their judgements.  CrowdFlower could continually monitor the models trending accuracy and then progressively include machine workers into the worker pool.  Once the model hit X accuracy you could have a majority of data stream routed to predictive judgments while continuing to feed a small percentage of data the crowd to refresh current topics and continually validate accuracy.  MTurk hits may only be pennies but Google Prediction ‘hits’ cost even less.

More »

From Toilet to Treatment to Treatment to Tap in San Diego

by February 10, 2012

dog-drinking-from-toilet_sm2 My first reaction to hearing at the breakfast table about today’s piece in the NY Times about water reuse in San Diego was that it isn’t all that different from what we have been doing for years: discharging treated waste water into streams and rivers and then drawing out drinking water downstream, counting on bacterial decomposition, dilution, and other processes to treat further the discharged water. I was pleased to see that this point was discussed in the article.

Drinking water that recently was flushed down a neighbor’s drain is a tough concept, pardon the pun, to swallow. However, as the piece in the Times correctly points out, we are headed into times in which resources like water are likely to be scarcer. To my mind, this community is a shining example of people—who rightly had very strong feelings on an issue—being willing to accept what the science community had to offer. Jerry Sanders, the mayor of this San Diego community, put it this way: “If science is behind you and you can prove that, I think people are willing to listen.” Here, here!

Image in post from Climate Watch. Thumbnail image on home page from the NY Times article.