10,000 Monkeys - Harnessing the Power of Typing Monkeys

Microsoft's PhizzPop Design Competition Finals Wrap-up

I've had this post in my blog drafts for a bit, but I hadn't had a chance to finish.  I haven't seen much out there on teh Internets about PhizzPop so I thought I'd give a recap of the event.  First, I'd like to congratulate Cyngergy on the win.  Obviously that means we didn't win which sucks :( but they have a great team and created some cool stuff which is always good for the industry :) 

The design problem for the finals is here.  For those of you too lazy to read the the whole problem is based around Citizenship in the 21st century and designing a system to create a more efficient democracy.  Really cool, but challenging.  We had about 7 weeks to complete the problem with our team of three including me (developer, UX storyboarding) Erik Klimczak (designer / developer / integrator )from Clarity and Dale Jones (designer).  Due to other project / job duties we didn't really start working on this until the Friday before the competition.  It made for a really hectic weekend, but it's always fun to work under pressure and it made the finals similar to the regional competition.  (Big thanks to Paul Treichler for providing us with 2 cases of Red Bull which makes staying awake for 72hrs possible)

As per my last post on PhizzPop I'll cover what we built / wanted to build in about 5K words to give you a complete inside look at our process / ideas.  It's kind of long and boring but I'd like to at least capture it somewhere.  If it's not on the internet, it never happened.  If you're not interested in all of the details you can skip to Overview of contest / what we presented.

Initial Storyboard / Wireframes

Our initial brainstorming sessions were a bit tougher because we had a lot of different ideas of the entire system we wanted to build.  We met via Live meeting with Erik in Chicago, me in SF and Dale in Seattle about a week or two before the contest.  Live meeting is pretty handy for the remote meetings because of the whiteboard, screen sharing, video/voice and shared notes.  After the brainstorming session, I put together some wireframes / story overview in my tool of choice - PowerPoint. I love me some PowerPoint.

Problems

•Sheer volume of news & content is overwhelming - time consuming to shift through all the noise out there.

•Difficult to asses reliability of news sources and find a balanced perspective

•Current news outlets don’t handle discussions well and allow people to engage in civil discourse

•Lack of connection with politicians and decision making process

•Hard to find local vs. international news across a variety of topics

•Difficult to manage multiple roles in privacy i.e. what can my friends see vs. my co-workers

Vision

•Automatically aggregate news and self-moderate quality

•Provide summarized facts and insights about news independent of underlying source reliability to create an informed populace

•Facilitate enaging real-time discussion of news closer to the in-person experience

•Integrate existing social networks while protecting privacy across various social groups

Main Screen Wireframe for Codename: "Convenient Truth"

1.Daily Truth – RSS feed of news articles including summarized facts and insights. Rather than subscribing to dozens of news feeds or reading multiple papers, you receive one item per news story with a snippet from the most reliable source, list of common facts, and quality community generated opinion. For people that are busy like Jennifer, this give you a quick overview and discussion points that you can consume on any device with an RSS reader such as smartphones.

2. News Scale Slider – The slider allows you to change the scope of news being viewed from Global to Local. At the global level news is related to international events and at the local level news is filtered down to your specific neighborhood – i.e new about a school board referendum or a local crime report which are often hard to find without going to multiple local sources. As opposed to just having a local section or global section, it’s easy to dynamically adjust. For someone like Fred or David, they can easily find local news.

3.Location filter – Scale of the map changes in conjunction with the local – global slider. If you switch to more local new, the map will display center on your current location. You can click inside a neighborhood to find any articles pertaining to housing prices in that neighborhood. At the global level you can select countries to change which news is displayed. In conjunction the news scale slider, you can pinpoint content to any level from local to global around the world.

4.Graphical display of news frequency by day. Allows you to view articles from more active news day. i.e there was a debate two days ago so you can just select that day.

5.Source truthiness – Based on our system self-moderation. As sites provide more articles that are listed as factual sources, their truthiness rating increases. This gives people a quick view to evaluate overall source reliability. Like metacritic for news.

6.Quick list of categories to filter articles by subject area. Articles are auto-categorized by the system by looking at Urls, RSS tags, and keywords in the article

7.Each news article is aggregated into one item. If the same news item appears from 10 sources, it appears in one item. Pulls from a variety of sources – TV, Radio, Print, Online, Local news – automatically – parses geographic area of impact and category. By feeding from a variety if mainstream and non-traditional news sources, it provides a better cross-section of information and is not dependent on user submissions. A short snippet is provided from the source that has the highest truthiness rating. You can also quickly see how much activity and discussion there is for a news item, as well as the geographic location. If there are currently live discussions about the article, there is an icon so you can join in. Finally, if there are any upcoming online votes that this article is relevant to, you can click the link to view those.

The Location, Scale Slider, Pub Date, Sources and Categories all help uses filter content to their specific news diet.

8. Popup notifications similar to XBOX Live system. Uses existing social networks to notify you that your friends are currently using Convenient Truth so you can easily discuss news.

Demo Script

-Show news sources being aggregated into one item

-Show adjusting local – global slider

-Show clicking map changes where the stories pertain to

-Should periodically show friends joining online

Article Drill-down for Codename: "Convenient Truth"

1.Common ground is pieces of user submitted evidence that have been voted highly by the moderators. They could be items from one of the sources reporting on this article or from outside sources like Wikipedia, NGO reports, etc. The idea is to give readers a quick summary of the facts related to this new item without regardless of the underlying credibility of the news source. The facts are listed with a link to the original source so you can view further info. Like all site links in the application , it is accompanied by a truthiness rating so you have a perspective on the overall community opinion of that source’s reliability.

2.Insights are comments highly rated by the moderators. When reading sights like Digg or Slashdot there is a lot of good opinion that adds value to the article and also a lot of junk. Rather than having to sift through all of the comments, this section displays the ones that add the most value to discussion of the article, the best of the best. The overall idea is that an article is presented with a snippet from the most reliable news source so you can read more. Without reading the article though you can quickly view the facts that pertain to this article as well as the most insightful points / counterpoints from the community. Essentially the goal is to be an emergent journalist providing a balanced perspective.

3.Live Discussions allow users like Stephanie or Thomas that want to engage with other users that are interested in a more in depth discussion about the current news item. The live discussions use Live Meeting so users can chat with video / voice in real-time, share articles, notes, etc. Users can propose times for a discussion for others to indicate if they’d like to join. The live discussions can be recorded and archived so other can view them in the future. While not as ideal as in-person “fire-side” chat, it does allow people to have a similar experience regardless of geographic location. The live discussions are more or less an open forum.

Each person’s name is always accompanied by a presence indicator [presence info controlled through privacy settings], so if you’d like just discuss with a few people, you can schedule a discussion with just them.  The presence info also allows a variety of options for engaging with actual people. You can IM/email them or if online, you can video/voice chat. Communication is handled through office communication server. For someone like Fred, this makes it simple to engage with others and have a shared reading experience.

1.Sources Reporting is similar to the view on the main page. It shows every outlet that is reporting the same item along with the site’s overall truthiness rating.

2.Displays a list of previous items in convenient truth that are related.

Demo:

-Join online discussion

-Show Chat or voice call with someone using presence info

Article Drill-down discussion for Codename: "Convenient Truth"

So how do items get populated into the fact / insights section? They are selected from the highest rated comments/facts submitted by users as voted on by the moderators. Who does the moderating? The moderation is done using a system similar to Slashdot and outlined in Emergence. As registered user you will be occasionally notified that you have moderator status. Moderator status allows you to rate a comment, but you can only rate a finite number of comments. If your contributions are highly rated you receive karma and with karma your comments start with an initial higher rating. So moderators rate posts and those ratings are used to select future moderators. Moderators cannot comment on the article if they moderated it. There are two differences. One is the change that Johnson outlined in Emergence. Karma is determined by those who have the greatest variance in scores i.e. the most +5 and -1. This tweak helps ensure diversity of opinions while still filtering out low quality comments. The 2nd difference we added is that karma is category specific. You might get greater karma because of insight on Science articles but that doesn’t necessarily mean you should be extended the same privilege on Political articles. This helps to establish experts in certain areas.

You can still view all the comments if you’d like by adjusting the quality filter, but the facts / insights are the comments most consistently rated as +5. This moderation system allows the community to self moderate for quality as well as extracting the best info for the article summary.

The emergent vForum allows for video commenting. Seeing someone actually talk vs. just reading gives you many of the non-verbal queues that are important in everyday conversations, but are missing in typical web discussions. Webcams are becoming more prevalent as computer companies like Apple include them on each machine and most smartphones have cameras capable of video.

Anytime you comment you can choose the privacy settings for the comment. So you can allow the public to see your comment or limit it to only friends.

When you view a video comment you can also respond to a specific point in the video. Just move the slider to the point in the video that you’d like to respond to and record your own video comment. When other users look at the comment thread they can choose to “watch discussion”. Watching a discussion plays the comment videos back spliced together. This allows you to see the comments in more logical structure that mimics an actual conversation where people inject comments at a certain point rather than chronological structure of threaded comments. Since you can always adjust the quality filter, it’s easy to watch higher quality discussions.

Video comments are also automatically transcribed to text so you can just read rather than watching.

When commenting you can also include some factual evidence from a linked source. Items posted as a factual basis for you comment serve to populate the Common Ground in the article summary above. It’s part of the moderators role to rate posts on the quality of the comment and it’s factual basis. As news sources are cited more for factual evidence, that feeds into their truthiness rating and determines what sources are automatically aggregated.

The combination of video commenting and the ability to interact with people in real-time over IM or Video chats brings discussion to a new level of engagement.

Demo:

-Adjusting quality filter

-Adding a video comment

-Respond to a video comment

-Watch discussion

-Show moderator role

Video Comment Watching for Codename: "Convenient Truth"

Video Comment Responding for Codename: "Convenient Truth"

Presence & Live Discussions for Codename: "Convenient Truth"

Privacy Page

Privacy is key as users are contributing comments about potential controversial topics that essentially go on public record on the internet.

We have several privacy features built into the site.

1.Users can adjust there current presence status. If you want to appear offline you can do that. The presence info serves to control how people can interact with you so you can adjust those settings. If you don’t want to chat, set yourself to away.

2.Rather than rebuilding a social network, Convenient Truth ties into your existing one. You can view all of your friends across various social networks.

3.The Circle of trust is used to group your friends / contacts in order to control privacy setting across multi people. You can add a new circle and call it friends. Then you can drag people from Facebook or Myspace into that circle. You can define a separate circle for co-workers and drag others into that. Some people have overlap so like a Venn diagram you can have overlapping circles. Some people might be both a friend and a co-worker.

4.For each circle you can set privacy settings. By default people can just view your comments and not any presence or profile info. You can either allow the general public to view your presence info or you can just grant that permission to the friends circle.

The circles are there to help you visually identify your groups of friends. The goal is to allow you flexibility in configuring privacy settings. Obviously you can block certain users from viewing any comments by you or interacting with you in any way. When commenting on the site you can always adjust who can see that specific comment and override the privacy settings on a per item basis.

The final portion was a 2nd application that was loosely connected called Suffragium for online voting.  The vision of Suffragium was:

•Educate voters on relevant topics & discussion related to the issue being voted upon

•Provide incentives or at least remove disincentives from voting

•Simplify voting across a wide range of decisions

Suffragium is the latin word for “the right to vote” or more generally an expression of opinion, assent or approval.

One of the problems with voting on current issues is the lack of education on issues among the voting population. Suffragium is an attempt to provide relevant news articles and related discussions about the issue at hand. Voters can easily find reliable facts and insights vetted through the community alongside the vote choices to help the decision process.

A second issue with voting, particularly on political elections, is the lack of incentive. While ideally people should feel that voting is an important duty many do not. We believe that this stems from people evaluating the impact of their vote vs. the time / effort tradeoff of doing research, going down to a voter station, waiting in line and ultimately voting. On-line voting allows people to vote in a more convenient manner with the added benefit of providing contextual research. Young adults have been raised with technology and online voting will tap what they are more familiar with.

Finally, this system can be used to provide a consistent interface across a wide range of voting purposes such as deciding a stance for a political action committee, local referendums or in the far, far future, national elections. While we believe that online voting for something like national elections is far off, it could start off with smaller goals. Maybe a school district uses it to vote on new educational policies. Or a state representative pledges to voted based on the outcome of online voting in his district. If elected officials made decisions based more on the direct input of voters, we believe less of them would feel estranged from government.

It might be a ways off, but people can become accustomed to online voting through gradual steps and slowly breakdown mistrust or misconceptions.

Initial UI Mockup

So there we have the original storyboard with some mockups which translated into this first UI prototype on Friday afternoon- 2.5 days before the contest. I really liked the design but ultimately we ended up changing the look and feel.

From Design to Development

So we had a story and some mockups, it was time to build. We we finally met in person we had some more discussions on how the application should work.  We decided to make it more issue-centric rather than news based.  We also had an alternate set of mockups in Fireworks.  Dale had a nice workflow for exporting from Fireworks design to XAML (markup language for creating Silverlight/WPF apps).  The intent was to create a system where people could easily find an overview of something like Global Warming regardless if there was a specific news story that day about Global Warming.  We decided to use WPF instead of Silverlight because we had some webcam functionality that wasn't going to work in Silverlight - at least not within the time remaining. Thus began the weekend coding / designing spree.

Overview of contest / what we presented

Most of this night is pretty fuzzy for me due to the lack of sleep.  We were still actually coding at 6pm when we arrived at Maggie Mae's for the final presentations which were scheduled to start at 7pm.  Luckily the start got pushed back to 8pm which gave us some more time to actually put together a presentation and add some more code.  Coding in a bar is not easy to say the least. Barcamp sure, actual bar, no.  I was sitting on some patio table and the DJ was blasting techno in a speaker right behind me.  The table surface + Red Bull jitters made using the mouse like trying to play Operation.  I think I leeched Wi-Fi off some neighbors apartment so I could get to our source control server (note to self - get everyone's code on the presentation laptop before you leave) .

So 8:00pm rolls around and the teams are starting to present.  We were scheduled to go 4th.  I drew the short straw and was selected to present.  Awesome. No sleep, nothing prepared and I may just pass out from anxiety.  I spend the next hour glancing at the presentations and jotting down notes in my little Moleskin notebook (trying to fit in with the cool design kids).  AKQA went first.  There was some technical difficulties with the A/V equipment.  I couldn't really hear what they were talking about, but I think ours was cooler.  Second team, Create The, goes, I only briefly paid attention since I was still coming up with a presentation, but again I think ours is better.  Starting to feel Ok because at least we won't be terrible relative to everyone else.  Next team up is Cynergy.  Huge excitement from the crowd.  Their app looks awesome and they have a great presentation.  I my mind I think maybe we can beat them.  Then they call Barack Obama via VOIP in a Facebook app.  The crowd goes nuts.  In my mind I say "Fu*ck".  Or maybe I said that out loud.  Who knows.  Regardless, there is no way we can top that.  Game over.  I go to the bar, down a double shot of whisky and go on stage to follow a ridiculous presentation and bring the crowd off their emotional high.

Really nervous, I start off with a joke - "We just finished coding this so I'll being seeing this for the first time along with you" - no laughs.  Fu*k. 

My presentation went something like:

Really exciting topic based on the current elections.  More excitement about this election than in previous years, but overall participation is still relatively low and more of today's youth feel disconnected.  I think the founding fathers would be disappointed to see this current sate.  I don't think they ever would have imagined a time where people didn't care about voting.  They gave us this great tool for change where everyone can vote and every vote counts (well in most cases) yet the majority of citizens don't care.  Not only do people care less they are also less informed.  If you watch cable news political pundits you've probably seen this first hand.  Being informed is essential to democracy and to exercise the power of voting.  Like Uncle Ben says "With great power comes great responsibility"  That great responsibility is being informed as a citizen.  Looking at the politics, the biggest advance in technology has been around campaign finance and collecting donations.  16yr olds can contribute money online to politicians but can't vote - that just shows were our politicians choose to focus their use of technology.

So what are the problems to stay informed? - sheer volume of content, finding balanced perspectives, lack of connection to politicians, finding local vs. international news, and a lack of incentive.

Our vision is to provide a balanced perspective of issues based on community opinion and semantic processing, facilitate real-time interactions online closer to in-person experiences and to connect people with issues to build an informed populace. 

The general structure of our user experience is based on "Connect, Learn, Discuss, Act".  There was a recent study at Stanford where people were separated into two groups.  Both groups were given a survey about issues.  The first group was the control group and they were given a follow-up survey 6 weeks later.  The people in that group didn't change their views over 6 weeks.  The 2nd group was divided into small discussion groups and given a collection of unbiased research.  They met weekly over the next few weeks to discuss and were given the same survey again.  The people in that group changed their opinions significantly based on things they had learned and discussions with others.  Our application is designed to mimic the setup of the 2nd group.  We want to change peoples' views based on unbiased research and discussions with others to ultimately cause them to act.

The entry point to the application, Cityzens, allows people to connect to issues.  They can filter issues based on the connection to others in their social networks or based on politicians.  (Clicking the little chain icon adjusts the connection filtering).  You can also adjust the issue perspective using the slider at the bottom which filters from global to local.  It's a double ended slider so you can also limit the amount found. 

Changing the global / local perspective also changes the background to give you a visual indication of the geographic region of the issue.  Above you can see a world map in the background for global.  Below, you can see how the background changes to show a more local perspective.  That slider works across all areas of the application when viewing issues to provide different perspectives and filtering.

Each person's name also includes a presence indicator.  Our application used Office Communication Server for presence in the demo, but the idea is that you can see if people are online at any given time and communicate with them via IM, Video chats, email, etc.  The presence info can be controlled via the privacy screen as well as who can communicate with you and how.

Politicians are also linked to information provided by the Sunlight Foundation similar to popup politicians.  This allows you to have quick access to politicians voting records and campaign finance history.  The purpose is to provide better transparency when viewing politicians related to a specific issue.

The next screen allows you to drill into a list of issues for a certain category to learn more.

Drilling into a specific issue like nuclear power gives you a detailed overview.  The first set of information on an issue is the background.  In our demo the information in the background was hard-coded.  The intent here was to use a system to similar to Microsoft Blews.  BLEWS uses political blogs to categorize news stories to according to their reception to conservatives and liberals.  It uses natural language processing to determine emotional charge.  Using a similar system we could extract data from news articles that are reference by both conservatives and liberals and generally agreed upon.  This information can then be used to provide a relatively unbiased background.

The second part of our issue overview is links to relevant government docs / bills from the executive and legislative brances.  Again, the Sunlight Foundation provides some great info in the LOUIS project.  This allows people to quickly see government information regarding a specific issues to provide more research and transparency.

The last part of our issues overview is the commenting section.  Users can comment directly via Cityzens or via outside sources like Twitter or Facebook.  Comments from outside sources are clearly indicated.  Again, each user has the presence icon so you can quickly connect with that person to further discuss.  Traditional forums or discussions are very asynchronous and little follow-up occurs.  We want to make that experience closer to real-time interactions.  The other feature of our commenting system is video commenting.  Video is becoming more prevalent as more phones/computers allow video recording.  Commenting via video has a few benefits - it's faster than typing and seeing someone speak gives you the non-verbal / language queues that you miss in written comments.  Ultimately I think more commenting systems in the future will include video.  I have seen one or two out there but we made a few innovations that I haven't seen done before.  First, video comments in Cityzens are processed via a speech engine to transcribe the comments.  This allows for better searching and allows users on devices that can't play video to view the comments.   A second feature is that you can watch someone's video comment and then respond with your own comment at any point in the video.   This allows users to actually watch a thread of comments with the videos spliced in their natural place rather than chronological.  It's more like watching a real discussion.  

All of the comments are rated via a system similar to slashdot which I discussed above.  The ratings can be used to filter the comment view so for instance you can watch a thread of only highly rated comments.  The other purpose of the ratings is to provide people incentive to participate.  Our vision is that people would earn tax incentives based on their contributions.  As users build a higher reputation for contributing quality insight, that rating is used to provide something similar to educational tax credits.  People can take tax deductions for educational classes so in a similar manner that could earn tax deductions for educating themselves and other via participating in this system.  Obviously that has some complications, but it's something that I'd like to see happen in the future - the government creating better incentives for being informed.

Here is a screenshot of watching a video comment.

When you choose to respond while watching a comment, the player does a cool 3d flip to the comment recording side. 

After posting your comment, it appears in the thread with the transcription.  This functionality all works via some DirectX APIs and the Speech engine in .NET.  Transcription isn't perfect but I only had a few hours to get it working.  Using a combination of maybe Speech Server to offload processing and perhaps something like Amazon's Mechanical Turk to provide manual correction, the transcriptions could be made almost perfect.

Next up is Discuss.

This section allows people to engage in small group discussions similar to the format in the Stanford study.  Discussions with other informed people is critical to shaping a person's view on an issues.  It helps cross-pollinate views and bring new perspectives.  Technology can help enable these discussions via something like Live Meeting.  Live Meeting allows people to share screens, whiteboard, takes notes and actually see / chat with others in real-time.  It's almost as good as being there in person.  Almost better because a moderator can silence people to give others a chance to speak, notes are easily shared, there is a question queue and you can record for others to view.  Plus with online meetings you aren't tied to discussing with people in your geographic area.  The person who maybe the most insightful person to to talk to may be across the country.  With Live Meetings in Cityzens, geographic boundaries are removed for real-time civil discourse.

Live Meeting

Next we have Act.

Creating an informed populace is one step, but it's important to turn that into action.  This section allows people to quickly find upcoming related elections and events pertaining to issues.

The final piece which we didn't build out due to time was the privacy settings which are obviously really important based on the nature of the real-time interactions and type of information being shared.  The privacy page we wanted to build was what I outlined earlier here.  During the presentation I just spoke to some of those ideas which probably wasn't super effective without any visualization.

So that's about it.  Overall I think we designed something pretty cool in 3 days that was a fairly good solution to the problem statement.  I really like a few of the concepts like the "Learn Discuss Act" organization and the user experience around changing between local / global perspectives and video commenting.  Unfortunately I don't think the judges really liked it.  The wrap-up comments were something like "Clean UI and good use of real data" - Kind of depressing to say the least.   If we spent more time I think it could have been better, but Cynergy put up a great performance that still would have been tough to beat.  I'd like to think we finished second although Thirteen23 had some really cool ideas and as always, a really beautiful UI. So maybe we finished 3rd.  Or last.  Who knows.

My general feedback for the event is:

• Logistics were great
• Design problems were really interesting to work on
• Great selection of competitors.  I was really excited to compete against what I view as most of the top forms in this space
• Having a 2nd/3rd place winner or runner up would help give at least some recognition to the firms that didn't win it all
• Live stream the event.  I have a lot of friends / co-workers who couldn't attend so it would be cool if they could have watched me lose
• If no live streaming, at least put up a recording afterwards so I can watch myself lose over and over again
• There was a ton going on at SXSW, but most people I talked to afterward never knew the event took place
• Invite more bloggers and give them a nice section to live blog the event
• 7 weeks is a long time to build something - I know it's fair since everyone has the same timeframe, but realistically it's hard to commit to more than a few days for one event.

Thanks to Chris Bernard, Will Tschumy, Sean Seibel  & Paul Treichler from Microsoft for organizing this event.  Also to the judges - Bull Scott, Peter Merholz, Dan Burkhart and Silona Bonewald.  It would have been cool to talk to the judges later and get some more detailed feedback, but oh well.  I really hope there is a PhizzPop II and that we are invited to compete.  I want a rematch with Cynergy :)