Kevin Marshall's Epic Work Blog for Awesome People

Kevin the Coder is a maverick. Almost like a team of mavericks.
in
Beginning Multi-touch on Windows 7 & basic gesture recognition

Doing my part to spark the economy I recently purchased a HP TouchSmart IQ506 PC.

 

HP IQ506 TouchSmart Desktop PC (2.16 GHz Intel Core 2 Duo Processor T5850, 4GB RAM, 500GB Hard Drive, DVD Drive, Vista Premium)

 

Fairly nice looking computer that my stable of Apple machines aren't embarrassed to be seen next to. The best part is that the screen is capable to multi-touch.  It isn't exactly surface on the cheap since it only picks up two contact points with the current driver, but it's fairly accurate. My friend Kurt Brockett at our distinguished competition Identity Mine posted about getting the TouchSmart up and running on Windows 7. I wanted to write that first and call it "Kevin on 7" but he beat me to it. Although he skipped out on giving me credit for telling him to download the original drivers from HP and use WinRar to extract the .exes instead of hunting down the Vista drivers for each item. In retaliation, I will not tell him what are the drivers for his two unknown devices.

 

Next step is to actually build something using Multi-touch.  There isn't a ton of information out there at the moment. You can watch this session from PDC - Windows 7: Developing Multi-touch Applications.  The take away is that WPF 4.0 will have multi-touch support which looked cool, the WPF and Surface teams are working together so at some point there will be common interface for building multi-touch apps on Surface & Win 7 and right now you can use the real-time stylus APIs or unmanaged code for handling multi-touch. Using unmanaged code you can either handle WM_TOUCH or WM_GESTURE windows messages. If you handle the WM_TOUCH messages then you aren't able to use their gesture recognition.

 

Since unmanaged code makes Kevin the Coder cower in fear, I found some sample code in one of the PDC labs that handles creating the necessary structs and pInvoking the right functions. It's just a simple multi-touch line drawing app.  To use this in your own code you can just add the WMTouchForm.cs class and inherit your form from that. You can then add handlers for TouchUp, TouchDown, and TouchMove.

 

Using the two contact points doesn't really get you much without some gestures. Andy Wilson and Yang Li created a great "$1 Gesture Recognizer" that works in JS and C#. To start out I downloaded the C# version, added the WMTouchForm as a base class for the main form and rewired the mouse event to correspond to the multi-touch events with the new location data.  You can download the modified version here.

 

Here are some samples of gestures I made:

 

One Finger Swipe

onefinger

 

Two Finger Swipe

twofinger

 

Pinch

pinch

 

Two Finger Swipe Rotated

twoFingerVertical

 

The $1 recognizer works pretty good considering I only loaded 1 sample of each of the four gestures. It's also great at recognizing regardless of the rotation of the gesture. I'm going to work on adding in support for my own scaling and rotation gestures next. I looked at porting over the scatter view control from the surface sdk, but that quickly got out of hand. I think I'd be better off starting with a simple control with similar properties.

 

Technorati Tags: ,,
Clarity on teh Internets

We've set up a view new things to start sharing more content. First we have our video channels on YouTube and Vimeo courtesy of our new Director of Videography.  The quality difference in HD between YouTube and Vimeo is pretty huge. Vimeo videos look sharp and I like the UI. Plus no popup ads since we are high rollers and have a pro account. But since my grandma is more likely to see the video if we upload them to YouTube, they'll be in both places for the time being.  Grandma loves some Surface videos.  We've also set up a friendfeed channel at http://friendfeed.com/clarityconsulting  Now you can follow along with all our videos, blogs, Upcoming events, photos, tweets and Yelp reviews of our competitors all in one place.

Visual Studio Wish List: Item #0001 - Managing Project types, SDKs, Plug-ins

Tonight is my first entry in my new column, Visual Studio Wish List. Here is my scenario - I just set up a new machine to try out some CTPs for Windows Azure, Live Framework, SQL Data Services & .NET services. After signing up on a series of connect sites and entering some invite codes (Am I the only person who thought it was a little confusing to navigate the Microsoft connect sites?) I get the following instructions:

image

 

Not a huge deal, but annoying that I have to go download all that stuff. Plus all the stuff I needed for SL, Azure & SDS.  Maybe like 200 clicks to download 20 things and keep clicking Next on some installers.  Not hard, but annoying especially when all these products are from one company.  What I'd love is to have some sort of feature manager in VS when I can click "Add Feature" and select "Live Framework Development" or "Silverlight Mobile" so I don't have to hunt for Silverlight Mobile SP1 CTP Tools for Visual Studio to make it work. Clicking that would go out to the appropriate sites and download the dependencies and install everything. Even better if when I startup VS it can alert me of updates like "Silverlight Mobile SP1 CTP December Preview Tools for Visual Studio is available, would you like to upgrade?" Done. So much easier to get up and running. Most of the project types could just be add-on features so I can enable Windows Mobile development if I need it rather than having to go through the VS install process again which with it's normal speed takes about 4hrs on a quad core 3.2ghz machine.

 

Eclipse has something very similar to this already - Here is an example of the flow at http://www.comeclipse.com/updatesite.html

 

COM Plugin features

Screenshot from Eclipse's update manger

 

Even a command line tool similar to mac ports would be a huge help.  Like on my mac if I want to do something in Django, I can just "port install django" and I get all the stuff I need to start building an app and can easily update when new versions are released.

And of course allow 3rd parties to plug-in to a directory service so I can download new unit testing frameworks or whatever.

Technorati Tags: ,
Microsoft PDC 2008 Day 2 Keynote Live Blog
Microsoft PDC 2008 Keynote Live Blog
Microsoft's PhizzPop Design Competition Finals Wrap-up

I've had this post in my blog drafts for a bit, but I hadn't had a chance to finish.  I haven't seen much out there on teh Internets about PhizzPop so I thought I'd give a recap of the event.  First, I'd like to congratulate Cyngergy on the win.  Obviously that means we didn't win which sucks :( but they have a great team and created some cool stuff which is always good for the industry :) 

 

The design problem for the finals is here.  For those of you too lazy to read the the whole problem is based around Citizenship in the 21st century and designing a system to create a more efficient democracy.  Really cool, but challenging.  We had about 7 weeks to complete the problem with our team of three including me (developer, UX storyboarding) Erik Klimczak (designer / developer / integrator )from Clarity and Dale Jones (designer).  Due to other project / job duties we didn't really start working on this until the Friday before the competition.  It made for a really hectic weekend, but it's always fun to work under pressure and it made the finals similar to the regional competition.  (Big thanks to Paul Treichler for providing us with 2 cases of Red Bull which makes staying awake for 72hrs possible)

 

As per my last post on PhizzPop I'll cover what we built / wanted to build in about 5K words to give you a complete inside look at our process / ideas.  It's kind of long and boring but I'd like to at least capture it somewhere.  If it's not on the internet, it never happened.  If you're not interested in all of the details you can skip to Overview of contest / what we presented.

 

Initial Storyboard / Wireframes

Our initial brainstorming sessions were a bit tougher because we had a lot of different ideas of the entire system we wanted to build.  We met via Live meeting with Erik in Chicago, me in SF and Dale in Seattle about a week or two before the contest.  Live meeting is pretty handy for the remote meetings because of the whiteboard, screen sharing, video/voice and shared notes.  After the brainstorming session, I put together some wireframes / story overview in my tool of choice - PowerPoint. I love me some PowerPoint.

 

Problems

•Sheer volume of news & content is overwhelming - time consuming to shift through all the noise out there.

•Difficult to asses reliability of news sources and find a balanced perspective

•Current news outlets don’t handle discussions well and allow people to engage in civil discourse

•Lack of connection with politicians and decision making process

•Hard to find local vs. international news across a variety of topics

•Difficult to manage multiple roles in privacy i.e. what can my friends see vs. my co-workers

 

Vision

•Automatically aggregate news and self-moderate quality

•Provide summarized facts and insights about news independent of underlying source reliability to create an informed populace

•Facilitate enaging real-time discussion of news closer to the in-person experience

•Integrate existing social networks while protecting privacy across various social groups

 

Main Screen Wireframe for Codename: "Convenient Truth"

image

 

1.Daily Truth – RSS feed of news articles including summarized facts and insights. Rather than subscribing to dozens of news feeds or reading multiple papers, you receive one item per news story with a snippet from the most reliable source, list of common facts, and quality community generated opinion. For people that are busy like Jennifer, this give you a quick overview and discussion points that you can consume on any device with an RSS reader such as smartphones.

2. News Scale Slider – The slider allows you to change the scope of news being viewed from Global to Local. At the global level news is related to international events and at the local level news is filtered down to your specific neighborhood – i.e new about a school board referendum or a local crime report which are often hard to find without going to multiple local sources. As opposed to just having a local section or global section, it’s easy to dynamically adjust. For someone like Fred or David, they can easily find local news.

3.Location filter – Scale of the map changes in conjunction with the local – global slider. If you switch to more local new, the map will display center on your current location. You can click inside a neighborhood to find any articles pertaining to housing prices in that neighborhood. At the global level you can select countries to change which news is displayed. In conjunction the news scale slider, you can pinpoint content to any level from local to global around the world.

4.Graphical display of news frequency by day. Allows you to view articles from more active news day. i.e there was a debate two days ago so you can just select that day.

5.Source truthiness – Based on our system self-moderation. As sites provide more articles that are listed as factual sources, their truthiness rating increases. This gives people a quick view to evaluate overall source reliability. Like metacritic for news.

6.Quick list of categories to filter articles by subject area. Articles are auto-categorized by the system by looking at Urls, RSS tags, and keywords in the article

7.Each news article is aggregated into one item. If the same news item appears from 10 sources, it appears in one item. Pulls from a variety of sources – TV, Radio, Print, Online, Local news – automatically – parses geographic area of impact and category. By feeding from a variety if mainstream and non-traditional news sources, it provides a better cross-section of information and is not dependent on user submissions. A short snippet is provided from the source that has the highest truthiness rating. You can also quickly see how much activity and discussion there is for a news item, as well as the geographic location. If there are currently live discussions about the article, there is an icon so you can join in. Finally, if there are any upcoming online votes that this article is relevant to, you can click the link to view those.

The Location, Scale Slider, Pub Date, Sources and Categories all help uses filter content to their specific news diet.

8. Popup notifications similar to XBOX Live system. Uses existing social networks to notify you that your friends are currently using Convenient Truth so you can easily discuss news.

Demo Script

-Show news sources being aggregated into one item

-Show adjusting local – global slider

-Show clicking map changes where the stories pertain to

-Should periodically show friends joining online

 

 

Article Drill-down for Codename: "Convenient Truth"

image

 

1.Common ground is pieces of user submitted evidence that have been voted highly by the moderators. They could be items from one of the sources reporting on this article or from outside sources like Wikipedia, NGO reports, etc. The idea is to give readers a quick summary of the facts related to this new item without regardless of the underlying credibility of the news source. The facts are listed with a link to the original source so you can view further info. Like all site links in the application , it is accompanied by a truthiness rating so you have a perspective on the overall community opinion of that source’s reliability.

2.Insights are comments highly rated by the moderators. When reading sights like Digg or Slashdot there is a lot of good opinion that adds value to the article and also a lot of junk. Rather than having to sift through all of the comments, this section displays the ones that add the most value to discussion of the article, the best of the best. The overall idea is that an article is presented with a snippet from the most reliable news source so you can read more. Without reading the article though you can quickly view the facts that pertain to this article as well as the most insightful points / counterpoints from the community. Essentially the goal is to be an emergent journalist providing a balanced perspective.

3.Live Discussions allow users like Stephanie or Thomas that want to engage with other users that are interested in a more in depth discussion about the current news item. The live discussions use Live Meeting so users can chat with video / voice in real-time, share articles, notes, etc. Users can propose times for a discussion for others to indicate if they’d like to join. The live discussions can be recorded and archived so other can view them in the future. While not as ideal as in-person “fire-side” chat, it does allow people to have a similar experience regardless of geographic location. The live discussions are more or less an open forum.

Each person’s name is always accompanied by a presence indicator [presence info controlled through privacy settings], so if you’d like just discuss with a few people, you can schedule a discussion with just them.  The presence info also allows a variety of options for engaging with actual people. You can IM/email them or if online, you can video/voice chat. Communication is handled through office communication server. For someone like Fred, this makes it simple to engage with others and have a shared reading experience.

1.Sources Reporting is similar to the view on the main page. It shows every outlet that is reporting the same item along with the site’s overall truthiness rating.

2.Displays a list of previous items in convenient truth that are related.

Demo:

-Join online discussion

-Show Chat or voice call with someone using presence info

 

Article Drill-down discussion for Codename: "Convenient Truth"

image

So how do items get populated into the fact / insights section? They are selected from the highest rated comments/facts submitted by users as voted on by the moderators. Who does the moderating? The moderation is done using a system similar to Slashdot and outlined in Emergence. As registered user you will be occasionally notified that you have moderator status. Moderator status allows you to rate a comment, but you can only rate a finite number of comments. If your contributions are highly rated you receive karma and with karma your comments start with an initial higher rating. So moderators rate posts and those ratings are used to select future moderators. Moderators cannot comment on the article if they moderated it. There are two differences. One is the change that Johnson outlined in Emergence. Karma is determined by those who have the greatest variance in scores i.e. the most +5 and -1. This tweak helps ensure diversity of opinions while still filtering out low quality comments. The 2nd difference we added is that karma is category specific. You might get greater karma because of insight on Science articles but that doesn’t necessarily mean you should be extended the same privilege on Political articles. This helps to establish experts in certain areas.

You can still view all the comments if you’d like by adjusting the quality filter, but the facts / insights are the comments most consistently rated as +5. This moderation system allows the community to self moderate for quality as well as extracting the best info for the article summary.

The emergent vForum allows for video commenting. Seeing someone actually talk vs. just reading gives you many of the non-verbal queues that are important in everyday conversations, but are missing in typical web discussions. Webcams are becoming more prevalent as computer companies like Apple include them on each machine and most smartphones have cameras capable of video.

Anytime you comment you can choose the privacy settings for the comment. So you can allow the public to see your comment or limit it to only friends.

When you view a video comment you can also respond to a specific point in the video. Just move the slider to the point in the video that you’d like to respond to and record your own video comment. When other users look at the comment thread they can choose to “watch discussion”. Watching a discussion plays the comment videos back spliced together. This allows you to see the comments in more logical structure that mimics an actual conversation where people inject comments at a certain point rather than chronological structure of threaded comments. Since you can always adjust the quality filter, it’s easy to watch higher quality discussions.

Video comments are also automatically transcribed to text so you can just read rather than watching.

When commenting you can also include some factual evidence from a linked source. Items posted as a factual basis for you comment serve to populate the Common Ground in the article summary above. It’s part of the moderators role to rate posts on the quality of the comment and it’s factual basis. As news sources are cited more for factual evidence, that feeds into their truthiness rating and determines what sources are automatically aggregated.

The combination of video commenting and the ability to interact with people in real-time over IM or Video chats brings discussion to a new level of engagement.

Demo:

-Adjusting quality filter

-Adding a video comment

-Respond to a video comment

-Watch discussion

-Show moderator role

 

Video Comment Watching for Codename: "Convenient Truth"

image

 

Video Comment Responding for Codename: "Convenient Truth"

image

 

Presence & Live Discussions for Codename: "Convenient Truth"

image

 

Privacy Page

image

 

Privacy is key as users are contributing comments about potential controversial topics that essentially go on public record on the internet.

We have several privacy features built into the site.

1.Users can adjust there current presence status. If you want to appear offline you can do that. The presence info serves to control how people can interact with you so you can adjust those settings. If you don’t want to chat, set yourself to away.

2.Rather than rebuilding a social network, Convenient Truth ties into your existing one. You can view all of your friends across various social networks.

3.The Circle of trust is used to group your friends / contacts in order to control privacy setting across multi people. You can add a new circle and call it friends. Then you can drag people from Facebook or Myspace into that circle. You can define a separate circle for co-workers and drag others into that. Some people have overlap so like a Venn diagram you can have overlapping circles. Some people might be both a friend and a co-worker.

4.For each circle you can set privacy settings. By default people can just view your comments and not any presence or profile info. You can either allow the general public to view your presence info or you can just grant that permission to the friends circle.

The circles are there to help you visually identify your groups of friends. The goal is to allow you flexibility in configuring privacy settings. Obviously you can block certain users from viewing any comments by you or interacting with you in any way. When commenting on the site you can always adjust who can see that specific comment and override the privacy settings on a per item basis.

 

The final portion was a 2nd application that was loosely connected called Suffragium for online voting.  The vision of Suffragium was:

•Educate voters on relevant topics & discussion related to the issue being voted upon

•Provide incentives or at least remove disincentives from voting

•Simplify voting across a wide range of decisions

 

Suffragium is the latin word for “the right to vote” or more generally an expression of opinion, assent or approval.

One of the problems with voting on current issues is the lack of education on issues among the voting population. Suffragium is an attempt to provide relevant news articles and related discussions about the issue at hand. Voters can easily find reliable facts and insights vetted through the community alongside the vote choices to help the decision process.

A second issue with voting, particularly on political elections, is the lack of incentive. While ideally people should feel that voting is an important duty many do not. We believe that this stems from people evaluating the impact of their vote vs. the time / effort tradeoff of doing research, going down to a voter station, waiting in line and ultimately voting. On-line voting allows people to vote in a more convenient manner with the added benefit of providing contextual research. Young adults have been raised with technology and online voting will tap what they are more familiar with.

Finally, this system can be used to provide a consistent interface across a wide range of voting purposes such as deciding a stance for a political action committee, local referendums or in the far, far future, national elections. While we believe that online voting for something like national elections is far off, it could start off with smaller goals. Maybe a school district uses it to vote on new educational policies. Or a state representative pledges to voted based on the outcome of online voting in his district. If elected officials made decisions based more on the direct input of voters, we believe less of them would feel estranged from government.

It might be a ways off, but people can become accustomed to online voting through gradual steps and slowly breakdown mistrust or misconceptions.

 

Initial UI Mockup

So there we have the original storyboard with some mockups which translated into this first UI prototype on Friday afternoon- 2.5 days before the contest. I really liked the design but ultimately we ended up changing the look and feel.

citizen

 

From Design to Development

So we had a story and some mockups, it was time to build. We we finally met in person we had some more discussions on how the application should work.  We decided to make it more issue-centric rather than news based.  We also had an alternate set of mockups in Fireworks.  Dale had a nice workflow for exporting from Fireworks design to XAML (markup language for creating Silverlight/WPF apps).  The intent was to create a system where people could easily find an overview of something like Global Warming regardless if there was a specific news story that day about Global Warming.  We decided to use WPF instead of Silverlight because we had some webcam functionality that wasn't going to work in Silverlight - at least not within the time remaining. Thus began the weekend coding / designing spree.

 

Overview of contest / what we presented

Most of this night is pretty fuzzy for me due to the lack of sleep.  We were still actually coding at 6pm when we arrived at Maggie Mae's for the final presentations which were scheduled to start at 7pm.  Luckily the start got pushed back to 8pm which gave us some more time to actually put together a presentation and add some more code.  Coding in a bar is not easy to say the least. Barcamp sure, actual bar, no.  I was sitting on some patio table and the DJ was blasting techno in a speaker right behind me.  The table surface + Red Bull jitters made using the mouse like trying to play Operation.  I think I leeched Wi-Fi off some neighbors apartment so I could get to our source control server (note to self - get everyone's code on the presentation laptop before you leave) .

So 8:00pm rolls around and the teams are starting to present.  We were scheduled to go 4th.  I drew the short straw and was selected to present.  Awesome. No sleep, nothing prepared and I may just pass out from anxiety.  I spend the next hour glancing at the presentations and jotting down notes in my little Moleskin notebook (trying to fit in with the cool design kids).  AKQA went first.  There was some technical difficulties with the A/V equipment.  I couldn't really hear what they were talking about, but I think ours was cooler.  Second team, Create The, goes, I only briefly paid attention since I was still coming up with a presentation, but again I think ours is better.  Starting to feel Ok because at least we won't be terrible relative to everyone else.  Next team up is Cynergy.  Huge excitement from the crowd.  Their app looks awesome and they have a great presentation.  I my mind I think maybe we can beat them.  Then they call Barack Obama via VOIP in a Facebook app.  The crowd goes nuts.  In my mind I say "Fu*ck".  Or maybe I said that out loud.  Who knows.  Regardless, there is no way we can top that.  Game over.  I go to the bar, down a double shot of whisky and go on stage to follow a ridiculous presentation and bring the crowd off their emotional high.

Really nervous, I start off with a joke - "We just finished coding this so I'll being seeing this for the first time along with you" - no laughs.  Fu*k. 

My presentation went something like:

Really exciting topic based on the current elections.  More excitement about this election than in previous years, but overall participation is still relatively low and more of today's youth feel disconnected.  I think the founding fathers would be disappointed to see this current sate.  I don't think they ever would have imagined a time where people didn't care about voting.  They gave us this great tool for change where everyone can vote and every vote counts (well in most cases) yet the majority of citizens don't care.  Not only do people care less they are also less informed.  If you watch cable news political pundits you've probably seen this first hand.  Being informed is essential to democracy and to exercise the power of voting.  Like Uncle Ben says "With great power comes great responsibility"  That great responsibility is being informed as a citizen.  Looking at the politics, the biggest advance in technology has been around campaign finance and collecting donations.  16yr olds can contribute money online to politicians but can't vote - that just shows were our politicians choose to focus their use of technology.

So what are the problems to stay informed? - sheer volume of content, finding balanced perspectives, lack of connection to politicians, finding local vs. international news, and a lack of incentive.

Our vision is to provide a balanced perspective of issues based on community opinion and semantic processing, facilitate real-time interactions online closer to in-person experiences and to connect people with issues to build an informed populace. 

The general structure of our user experience is based on "Connect, Learn, Discuss, Act".  There was a recent study at Stanford where people were separated into two groups.  Both groups were given a survey about issues.  The first group was the control group and they were given a follow-up survey 6 weeks later.  The people in that group didn't change their views over 6 weeks.  The 2nd group was divided into small discussion groups and given a collection of unbiased research.  They met weekly over the next few weeks to discuss and were given the same survey again.  The people in that group changed their opinions significantly based on things they had learned and discussions with others.  Our application is designed to mimic the setup of the 2nd group.  We want to change peoples' views based on unbiased research and discussions with others to ultimately cause them to act.

The entry point to the application, Cityzens, allows people to connect to issues.  They can filter issues based on the connection to others in their social networks or based on politicians.  (Clicking the little chain icon adjusts the connection filtering).  You can also adjust the issue perspective using the slider at the bottom which filters from global to local.  It's a double ended slider so you can also limit the amount found. 

 

image

Changing the global / local perspective also changes the background to give you a visual indication of the geographic region of the issue.  Above you can see a world map in the background for global.  Below, you can see how the background changes to show a more local perspective.  That slider works across all areas of the application when viewing issues to provide different perspectives and filtering.

 

image

Each person's name also includes a presence indicator.  Our application used Office Communication Server for presence in the demo, but the idea is that you can see if people are online at any given time and communicate with them via IM, Video chats, email, etc.  The presence info can be controlled via the privacy screen as well as who can communicate with you and how.

 

image

 

Politicians are also linked to information provided by the Sunlight Foundation similar to popup politicians.  This allows you to have quick access to politicians voting records and campaign finance history.  The purpose is to provide better transparency when viewing politicians related to a specific issue.

 

The next screen allows you to drill into a list of issues for a certain category to learn more.

image

 

Drilling into a specific issue like nuclear power gives you a detailed overview.  The first set of information on an issue is the background.  In our demo the information in the background was hard-coded.  The intent here was to use a system to similar to Microsoft Blews.  BLEWS uses political blogs to categorize news stories to according to their reception to conservatives and liberals.  It uses natural language processing to determine emotional charge.  Using a similar system we could extract data from news articles that are reference by both conservatives and liberals and generally agreed upon.  This information can then be used to provide a relatively unbiased background.

 

image

 

The second part of our issue overview is links to relevant government docs / bills from the executive and legislative brances.  Again, the Sunlight Foundation provides some great info in the LOUIS project.  This allows people to quickly see government information regarding a specific issues to provide more research and transparency.

The last part of our issues overview is the commenting section.  Users can comment directly via Cityzens or via outside sources like Twitter or Facebook.  Comments from outside sources are clearly indicated.  Again, each user has the presence icon so you can quickly connect with that person to further discuss.  Traditional forums or discussions are very asynchronous and little follow-up occurs.  We want to make that experience closer to real-time interactions.  The other feature of our commenting system is video commenting.  Video is becoming more prevalent as more phones/computers allow video recording.  Commenting via video has a few benefits - it's faster than typing and seeing someone speak gives you the non-verbal / language queues that you miss in written comments.  Ultimately I think more commenting systems in the future will include video.  I have seen one or two out there but we made a few innovations that I haven't seen done before.  First, video comments in Cityzens are processed via a speech engine to transcribe the comments.  This allows for better searching and allows users on devices that can't play video to view the comments.   A second feature is that you can watch someone's video comment and then respond with your own comment at any point in the video.   This allows users to actually watch a thread of comments with the videos spliced in their natural place rather than chronological.  It's more like watching a real discussion.  

All of the comments are rated via a system similar to slashdot which I discussed above.  The ratings can be used to filter the comment view so for instance you can watch a thread of only highly rated comments.  The other purpose of the ratings is to provide people incentive to participate.  Our vision is that people would earn tax incentives based on their contributions.  As users build a higher reputation for contributing quality insight, that rating is used to provide something similar to educational tax credits.  People can take tax deductions for educational classes so in a similar manner that could earn tax deductions for educating themselves and other via participating in this system.  Obviously that has some complications, but it's something that I'd like to see happen in the future - the government creating better incentives for being informed.

 

image

Here is a screenshot of watching a video comment.

clip_image002

When you choose to respond while watching a comment, the player does a cool 3d flip to the comment recording side. 

 

clip_image002[5]

After posting your comment, it appears in the thread with the transcription.  This functionality all works via some DirectX APIs and the Speech engine in .NET.  Transcription isn't perfect but I only had a few hours to get it working.  Using a combination of maybe Speech Server to offload processing and perhaps something like Amazon's Mechanical Turk to provide manual correction, the transcriptions could be made almost perfect.

 

Next up is Discuss.

This section allows people to engage in small group discussions similar to the format in the Stanford study.  Discussions with other informed people is critical to shaping a person's view on an issues.  It helps cross-pollinate views and bring new perspectives.  Technology can help enable these discussions via something like Live Meeting.  Live Meeting allows people to share screens, whiteboard, takes notes and actually see / chat with others in real-time.  It's almost as good as being there in person.  Almost better because a moderator can silence people to give others a chance to speak, notes are easily shared, there is a question queue and you can record for others to view.  Plus with online meetings you aren't tied to discussing with people in your geographic area.  The person who maybe the most insightful person to to talk to may be across the country.  With Live Meetings in Cityzens, geographic boundaries are removed for real-time civil discourse.

 

image

 

Live Meeting

image

 

Next we have Act.

Creating an informed populace is one step, but it's important to turn that into action.  This section allows people to quickly find upcoming related elections and events pertaining to issues.

image

 

The final piece which we didn't build out due to time was the privacy settings which are obviously really important based on the nature of the real-time interactions and type of information being shared.  The privacy page we wanted to build was what I outlined earlier here.  During the presentation I just spoke to some of those ideas which probably wasn't super effective without any visualization.

So that's about it.  Overall I think we designed something pretty cool in 3 days that was a fairly good solution to the problem statement.  I really like a few of the concepts like the "Learn Discuss Act" organization and the user experience around changing between local / global perspectives and video commenting.  Unfortunately I don't think the judges really liked it.  The wrap-up comments were something like "Clean UI and good use of real data" - Kind of depressing to say the least.   If we spent more time I think it could have been better, but Cynergy put up a great performance that still would have been tough to beat.  I'd like to think we finished second although Thirteen23 had some really cool ideas and as always, a really beautiful UI. So maybe we finished 3rd.  Or last.  Who knows.

 

My general feedback for the event is:

  • Logistics were great
  • Design problems were really interesting to work on
  • Great selection of competitors.  I was really excited to compete against what I view as most of the top forms in this space
  • Having a 2nd/3rd place winner or runner up would help give at least some recognition to the firms that didn't win it all
  • Live stream the event.  I have a lot of friends / co-workers who couldn't attend so it would be cool if they could have watched me lose
  • If no live streaming, at least put up a recording afterwards so I can watch myself lose over and over again
  • There was a ton going on at SXSW, but most people I talked to afterward never knew the event took place
  • Invite more bloggers and give them a nice section to live blog the event
  • 7 weeks is a long time to build something - I know it's fair since everyone has the same timeframe, but realistically it's hard to commit to more than a few days for one event.

 

Thanks to Chris Bernard, Will Tschumy, Sean Seibel  & Paul Treichler from Microsoft for organizing this event.  Also to the judges - Bull Scott, Peter Merholz, Dan Burkhart and Silona Bonewald.  It would have been cool to talk to the judges later and get some more detailed feedback, but oh well.  I really hope there is a PhizzPop II and that we are invited to compete.  I want a rematch with Cynergy :)

 

Google AJAX Language API & Office Communicator Custom Translation Tabs

While waiting for Lost to start, I was catching up on some RSS feeds.  I just dropped TechCrunch & anything mentioning Scoble in my recent news feed pruning which has decreased my news volume nicely.  One noteworthy item today was the launch of Google's AJAX Language API.  I think it's the only translation web service out there.  One downside is that it's JavaScript only.  It would be nicer if there was a REST web service to call to make it easier to integrate into desktop apps or to bulk convert text in a service.

A cool use of it I think would be to integrate into existing IM clients.  I have a few French & Spanish speaking friends so it would be helpful if my IM client could auto-translate since my foreign language skills are at a kindergarten level. 

Lately I've been using Office Communicator since we have a new kick a$$ VOIP setup..(so nice to be able to transfer an IM conversation to a phone call and chat via my USB speaker phone.)  Anyway, I was looking for something to try out the Google translation API and I started looking at the communicator API.  I didn't realize you could just add a new tab of HTML via a quick registry setting.  In about 15 minutes I was able to create this:

image

It's probably not super useful, but it's always fun to try out new stuff. 

Adding a New Tab to Communicator

This TechNet article explains how to add a new tab in communicator.  Basically you just need to create a new registry string value under HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Communicator.

I set the value to "file:///C:/communicatorTab.xml"

In the XML file I put:

<?xml version="1.0" ?>
<tabdata>
   <tab>
     <image>file:///c:/communicatorTab.png</image>
      <accessibility>both</accessibility>
     <userid>false</userid>
     <name>the name to display on the tab</name>
     <tooltip>the tooltip to show</tooltip>
     <contenturl>http://localhost/communicatorTab.htm</contenturl>
     <contactid>false</contactid>
     <method>get</method>
     <tabid>1</tabid>
   </tab>
  </tabdata>

For the contentUrl, you need to point to an HTML file which will be displayed in the tab.  I wasn't able to get it working via "file:///c:/communicatorTab.htm", but according to the docs that should work.  I'm running IIS so I just put it on my web server.  If you are sharing this, you'd probably want to put it on a web server anyway.

Calling the Translation API

Here is a copy of the HTML file I used.  I made the UI simple so it didn't take up much space.  You just enter some text in one of the supported languages and select a language to translate to.  Clicking "Translate" replaces the text you entered with the translated text.  It doesn't support every language translation pair.  So you can't to Chinese to Arabic yet.  The supported pairs are here.  The nice thing about the API is that it auto detects the source text language so you don't have to select "Spanish to English"

Invoking the Google API is really simple.  Just call translate with the source, leave the source language blank to auto-detect, the target language code and the return function.

google.load("language", "1");

function translate() {
  var source = document.getElementById("source");
  var target = document.getElementById("target");
  google.language.translate(source.innerHTML, "", target.value, function(result) {
    if (!result.error) {
      var source = document.getElementById("source");
      source.innerHTML = result.translation;
    }
    else {
      alert(result.message);
    }
  });
}

 

Now you can pretend to be a cultured international business man/woman and easily throw a few foreign words into the conversation.  Tres Cool.

In the future I might try integrating the API into the Office Communicator Web client because I think I'd be able to do real-time translation of conversations pretty easily.  It's currently #11 on my side-project list.

 

From the 10K Monkey's Vault: Coding4Fun

Writing blog posts just keeps jumping down the list. I think I need to realize that regular blogging is just not in me.  I have some posts in the works that will be finished at some point.  In the meantime I'll just rehash some of my previous literary gems.

Visual Foos 2005

http://blogs.msdn.com/coding4fun/archive/2006/10/31/910302.aspx

Sadly this is probably the most innovative thing I've ever built.  My masterpiece.  While others have built things like Facebook to change the course of social networking, I built a scoreboard for foosball. Amazing what drunken brainstorming will do for you.  Easily my favorite article not so much content-wise, I just love that foosball table.  I'm like a proud papa when I give a tour of the office and I get to show them the foosball table <tear>.  I don't think I can ever top the Clarity SkyCam - I mean it's instant replay of any goal scored.  

 

image

Since the article was written I've made a bunch of enhancements. 

Web Interface

View Game Log of every game played.  You can also drill into any game and see every goal scored and watch a replay of that goal.

image

Ability to upload sound files which are randomly played when you score a goal.

image

Full list of sortable stats - tracks everything from RPI to Offensive Value of Replacement Player (OVORP) to win streaks.  It's a pretty comprehensive measure of your foos skill.

image

The Swami

The Swami is my massive stored proc that makes pre-game predictions.  Currently it predicts the winner 65% of the time.  Not superb, but in the Swami's defense, that takes into account random guest players.  It gets the team score correct 15% of the time which I think is pretty solid.  28% of the time it gets the final score within +/- 1 goal.  It's eerie when the Swami announces a 12-10 upset before the game starts and it's right. 

image

In-Game Crowd Noise

Based on the flow of the game different crowd noises will play in the background.  If the game has a scoring draught the crowd gets progressively more belligerent.  Some "Bull$hit" chants.   Or maybe some rally music from the organ when a team starts to come from behind.

In-Game Commentary

The virtual announcers will do some talking in the game.  Like pointing out how a team has won their past 8 games or their win percentage in a certain formation.  Right now it's still somewhat simple commentary.  We have SQL Analysis Services running on the database to provide more in-depth analysis like "Team A wins 80% of the time when they take a 5-3 lead"  This isn't fully implemented yet but it'll be there in Visual Foos 2009. Or maybe 2010.  It generates the data, the announcers just need to be configured to translate the results into speech.

Player Awards

When the game starts it'll announce special awards like Brick Wall for best overall defensive player or a Juggernaut for best overall offensive player.  If your lucky enough to play with the best, you'll know by the special icons that appear next to their names. 

Rally Monkey / Shutout Monkey

Two different animated monkeys appear if you are on the verge of a shutout or in the process of a massive comeback.

Bio-metric Authentication

Partially implemented.  Uses a finger-print scanner to select your player.  Still need to build the application for everyone to actually register their fingerprints.

Email Pregame Setup

The idea is that you can schedule a meeting with the table at a certain time and it'll log in for all those players so the game is ready to play when you get there.  It's still in dev and may not make it until VF 2009.

 

At some point I'd like to rebuild a whole new version, but then again Michelangelo never made another David :)

 

WPF Dartboard Scoring Application

http://blogs.msdn.com/coding4fun/archive/2007/06/20/3425360.aspx

Technically I didn't write the article, but I did build the app.  This article completed by ".NET Game Scoreboard" phase.  It was my first attempt at WPF development.  Code is kind of scary.  I should have read Adam Nathan's WPF book before writing the app.  Some people write "Hello World" apps, I build apps to track scores and stats for barroom games.  The bullseye animation with synchronized sound and particle physics is probably one of my favorite things I've ever built.

 

 

Collecting Outlook 2007 Stats Using VSTO 2005

http://blogs.msdn.com/coding4fun/archive/2006/11/20/1111248.aspx

Part of my career called the "Office Add-in" phase.  Much like Picasso's Blue Period it was short lived while I explored the world of managed code in Office.  Anyone seen Xobni?  Yeah, you probably could have taken this code 1.5 years ago and built Xobni and gotten a few $$ in VC funding.  Maybe combine it with the Facebook Outlook Add-in below and build a Xobni killer.  Not that Xobni needs a Xobni killer - it does a pretty good job of killing itself when it pounds Outlook into Application Not Responding submission. 

 

 

Facebook Outlook Add-in

http://blogs.msdn.com/coding4fun/archive/2007/08/30/4660479.aspx

I actually thought this was cool.  Most of the development community didn't seem to share my opinion :) Not one of my most read Coding4Fun posts. My co-workers article about Silverlight 8-ball had about 10x the read count.  I mean what's not to love - the power of Outlook and Facebook merged into one Frankenstein-ish app.   Effectively ended the "Office Add-in" phase.

 

 

If anyone has any future app ideas they'd like to see on Coding4Fun, I'm open to suggestions.  I have some partially finished projects like a Conference Room sign that pulls meeting schedules from Exchange, responds to voice commands to display other items like the weather forecast, company photos, headlines and random lunch options.  That needs a WPF UI face-lift.  I've also thought about a virtual aquarium in WPF.  Or maybe an automated drink delivery system using a Roomba. 

 

Faux Giant Application Icon with Extended Aero Glass in WPF

Maybe not the best title to describe the post.  Anyway, I was looking at some of the apps the other teams did for PhizzPop.  Kudos to all the regional winners.  Great stuff.  In particular, thirteen23 had this cool effect.

image

 

I'm not entirely sure how they get the chair to look like that.  It's almost like magic.  My initial thought was that WPF allowed you to set a ClipToBounds = false on a window and place a control partially outside the bounds of the window.  I've never heard of that and it doesn't look like you can do it either.  Then I thought it might be some type of custom border like this or some use of the DWM functions. That's probably more pinvoking than I'd like to do in one evening.  There has to be an easy way to do that effect.  I'm looking for about 5 mins of coding.  Then I realized I could just add a second window on top of the first, relatively position it, set the window style to none, allow transparency, and make the background transparent.  Done.  Here is how mine looks.

image

There is probably a better way.  Not sure.  A magician never reveals his secrets. 

So here's how I did it. Make a new form in WPF and set the WindowStyle to none .  To get the extended aero glass for the form, I used Adam Nathan's GlassHelper class.  Then make a second window and set

WindowStyle="None" AllowsTransparency="True" Background="Transparent" IsEnabled="False" ShowInTaskbar="False"

Next add an image that you want to serve as your giant faux application icon.  I chose a nice Vista coffee icon I found here.

 

In code behind I set the owner of the 2nd window  to the main window.  I also positioned it accordingly.  Something new I learned today was how to listen to dependency property changed events from the DependencyProprtyDescriptor.  Awesome.  I had never tried to do that before.  I listen for the Top and Left property changes of the main window so I can keep the 2nd window in the same relative position.  That way the illusion works even as you drag the main window around.  It's not perfect as sometimes you notice a slight delay before it moves.  At least the coffee cup isn't off floating on monitor #1 while the rest of the application has migrated to monitor #4.  I also added some simple drag code since there isn't a border at the top to grab.

 

   1:          private void Window_Loaded(object sender, RoutedEventArgs e)
   2:          {
   3:              window2 = new Window2();
   4:              window2.Top = this.Top - 40;
   5:              window2.Left = this.Left + 30; ;
   6:              window2.Owner = this;
   7:              window2.Show();
   8:   
   9:   
  10:              DependencyPropertyDescriptor dpd = DependencyPropertyDescriptor.FromProperty(Window.TopProperty, typeof(Window));
  11:   
  12:              if (dpd != null)
  13:              {
  14:                  dpd.AddValueChanged(this, delegate
  15:                  {
  16:                      window2.Top = this.Top - 50;
  17:                  });
  18:   
  19:                  dpd.AddValueChanged(this, delegate
  20:                  {
  21:                      window2.Left = this.Left + 30; ;
  22:                  });
  23:              }
  24:              
  25:          }
  26:   
  27:          private void Window_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
  28:          {
  29:              this.DragMove(); 
  30:          }

 

 

The full source code is here.

Like I said earlier, probably not the most elegant solution, but I think it looks cool.  I'd be curious to see how thirteen23 did it.

Oh, I also uploaded a video of our PhizzPop application to Clarity TV.

 

Sharepoint Search from Tafiti

I read a post on Angus Logan's blog about using Tafiti to provide a UX on top of Enterprise Search.  I've never used the SharePoint Search SDK before so I thought I'd give it a try.

The code is pretty hacked together since I just wanted to see how it worked. 

Screenshot of Tafiti running in a page viewer web part against my dev machine.  The code is using impersonation to query our MOSS server.

 

image

 

Screenshot of the classic tree view.

image

 

To get started you need to download the Tafiti source from CodePlex and follow the setup guide.

Open the solution in VS 2008.  Although solution didn't work for me.  I had to remove the projects and re-add them.  I also changed the target framework type to 3.0.

Add a web reference to you sharepoint search web service at <server_name>/_vti_bin/search.asmx

 

image

As per Angus's blog post, you really only have to alter code in the Search.aspx.cs page to change the search engine to SharePoint.

 

   1:  private void SoapSearch(SourceType sourceType, string query, int first, int count)
   2:      {
   3:          try
   4:          {
   5:              SourceRequest[] sr = new SourceRequest[1];
   6:              sr[0] = new SourceRequest();
   7:              sr[0].Source = sourceType;
   8:              sr[0].Offset = first;
   9:              sr[0].Count = count;
  10:              sr[0].ResultFields = GetResultFieldMask(sourceType);
  11:   
  12:              //SearchRequest request = new SearchRequest();
  13:              //request.Query = query;
  14:              //request.Requests = sr;
  15:              //request.SafeSearch = SafeSearchOptions.Strict;
  16:              //request.AppID = SettingsWrapper.LiveSearchAppID;
  17:              //request.CultureInfo = "en-US";
  18:   
  19:              //if (sourceType == SourceType.PhoneBook)
  20:              //{
  21:              //    // Using Redmond as the search location.
  22:              //    request.Location = new Location();
  23:              //    request.Location.Longitude = Double.Parse("-122.33482360839846");
  24:              //    request.Location.Latitude = Double.Parse("47.6082462871061");
  25:              //    request.Location.Radius = Double.Parse("5");
  26:              //}
  27:   
  28:              //MSNSearchService searchService = new MSNSearchService();
  29:              //SearchResponse searchResponse;
  30:              //searchResponse = searchService.Search(request);
  31:   
  32:              //---------SharePoint Search Code------
  33:   
  34:              string qXMLString = "<QueryPacket xmlns='urn:Microsoft.Search.Query'>" +
  35:                  "<Query><SupportedFormats><Format revision='1'>" +
  36:                  "urn:Microsoft.Search.Response.Document:Document</Format>" +
  37:                  "</SupportedFormats><Context><QueryText language='en-US' type='STRING'>" +
  38:                  query +
  39:                  "</QueryText></Context></Query></QueryPacket>";
  40:   
  41:   
  42:              System.Security.Principal.WindowsImpersonationContext impersonationContext;
  43:              impersonationContext =
  44:                  ((System.Security.Principal.WindowsIdentity)User.Identity).Impersonate();
  45:   
  46:              SPSearch.QueryService queryService = new SPSearch.QueryService();
  47:              queryService.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;
  48:              System.Data.DataSet queryResults = queryService.QueryEx(qXMLString);
  49:   
  50:              impersonationContext.Undo();
  51:   
  52:              //--------------------------------
  53:   
  54:              LiveXmlSearchResults result = CreateLiveXmlFromMossSearchResults(queryResults);
  55:   
  56:              JavaScriptSerializer serializer = new JavaScriptSerializer();
  57:              string json = serializer.Serialize(result);
  58:              byte[] jsonUtf8 = System.Text.Encoding.UTF8.GetBytes(json);
  59:   
  60:              Response.StatusCode = 200;
  61:              Response.ContentType = "text/javascript";
  62:              Response.OutputStream.Write(jsonUtf8, 0, jsonUtf8.Length);
  63:          }
  64:          catch (SoapException e)
  65:          {
  66:              throw new HttpException((int)HttpStatusCode.InternalServerError, "Internal Server Error", e);
  67:          }
  68:          catch (WebException e)
  69:          {
  70:              throw new HttpException((int)HttpStatusCode.InternalServerError, "Internal Server Error", e);
  71:          }
  72:      }

 

To make a search against sharepoint you just need to call the QueryEx method off of the QueryService from your web reference.  MSDN has an overview of the QueryEx method here.

The next step is to reformat the dataset returned from MOSS into the Live search results format.  Like Angus mentioned, converting the result into the LiveXmlSearchResults allows you to reuse the JavaScriptSerializer and existing JavaScript code in Silverlight.

 

These functions are a little messy :) Not all of the results had a description so I pulled the HitHighlightedSummary from the dataset and removed some of the extra tags it added.

 

   1:      private LiveXmlSearchResults CreateLiveXmlFromMossSearchResults(System.Data.DataSet mossSearchResults)
   2:      {
   3:          LiveXmlSearchResults searchResults = new LiveXmlSearchResults();
   4:          searchResults.searchresult.documentset._source = "FEDERATOR_MONARCH";
   5:          searchResults.searchresult.documentset._count = mossSearchResults.Tables[0].Rows.Count.ToString();
   6:          searchResults.searchresult.documentset._start = "0";
   7:          searchResults.searchresult.documentset._total = mossSearchResults.Tables[0].Rows.Count.ToString();
   8:   
   9:          LiveXmlResult[] results = ConvertMOSSWebResults(mossSearchResults.Tables[0]);
  10:   
  11:          if (results != null)
  12:              searchResults.searchresult.documentset.document = (results.Length > 1) ? (object)results : (object)results[0];
  13:   
  14:          return searchResults;
  15:      }
  16:   
  17:      private static LiveXmlResult[] ConvertMOSSWebResults(System.Data.DataTable mossSearchResultsTable)
  18:      {
  19:          LiveXmlWebResult[] results = new LiveXmlWebResult[mossSearchResultsTable.Rows.Count];
  20:   
  21:          for (int i = 0; i < mossSearchResultsTable.Rows.Count; i++)
  22:          {
  23:              LiveXmlWebResult result = new LiveXmlWebResult();
  24:              string description = string.Empty;
  25:              description = 
  26:                   GetDescription(mossSearchResultsTable.RowsIdea [I]["Description"].ToString(), mossSearchResultsTable.RowsIdea [I]["HitHighlightedSummary"].ToString());
  27:              result.title = mossSearchResultsTable.RowsIdea [I]["Title"].ToString();
  28:              result.desc  = description;
  29:              result.url = mossSearchResultsTable.RowsIdea [I]["Path"].ToString();
  30:              resultsIdea [I] = result;
  31:          }
  32:          return results;
  33:      }
  34:   
  35:      private static string GetDescription(string description, string hitHighlightedSummary)
  36:      {
  37:          string strippedText = string.Empty;
  38:   
  39:          if (string.IsNullOrEmpty(description))
  40:          {
  41:              Regex regEx = new Regex("<[^>]*>", RegexOptions.IgnoreCase);
  42:              strippedText = regEx.Replace(hitHighlightedSummary, "...");
  43:              return strippedText;
  44:          }
  45:          else
  46:              return description;
  47:      }

 

Next I think I'll try integrating it into MOSS to be an actual page in the search center.  I'd also like to change the News/Phonebook/Web search option carousal to have it do some different searches like People, Office Docs, etc.   Also remove the Windows Live functionality and misc. extra hyperlinks. Oh and fix the range / start offset so it pages properly.  When I started this an hour ago I didn't really realize the difference between Query and QueryEx [Maybe I should have read those docs I linked to :) ].  Looks like I might have to use QueryEx so get some of the extra info like total number of records.  Oh maybe add in the relevance / thumbnail icons for filetypes.  Well it's a prototype so I'll probably just start over anyway.

 

PhizzPop Design Challenge Chicago Wrap-up

In possibly what was one of the biggest surprises of my career, I was part of the team that won the Chicago PhizzPop Design Challenge. Normally I'm rather confident that I'm the greatest developer ever in the heavyweight division, but I was pretty down on our chances the day of the competition. In 48 hrs I had gone from "We are going to dominate this problem" to "This is the worst f*cking application ever designed by anyone, anywhere".

A week later, I can still hardly believe Carlos Segura, one of the judges and a ridiculously talented designer said "Clarity is the clear winner". I thought he was going to say "I can't believe you had the audacity to put this $hit in front of me"

So what is the PhizzPop Design Challenge?  It's described as "The PhizzPop Design Challenge pits top interactive, Web, and design agencies against one another to push the limits of technology and creativity in a battle royale. Think Mad Max for design."  Basically we had 2.5 days to build an application that best demonstrated an innovative design to solve a problem. The problem statement for Chicago can be viewed here.

Powered by several cases of Red Bull and a new coffee machine, I was ready to dominate.  Larry Clarkin mentioned we had sleeping bags.  Actually it was Aero beds.  We roll high class in our office.  Although we didn't really sleep for the 2.5 days.  Once you drink your body weight in Red Bull you can pretty much work 72 hrs straight.  Granted I did start to hallucinate around the 48hr mark - I swear the monkeys were trying copy my designs.

So what did we want to build / demo?

After our initial brainstorming which we had about 100 ideas we wanted to build and we grouped them down to two areas.  The in-flight entertainment experience and the reservation / ticketing process. The main themes that we tried to infuse in both parts were connected, interactive and personalized.

Basically we saw the current flying experience as very impersonal.  To most airlines you are probably just a frequent flyer number at best.  With the exception of Southwest, most airlines know exactly who is in what seat so why not customize the entertainment experience for that seat to a specific person.  Starting with a simple "Welcome Kevin Marshall" to having a selection of movies and reading materials that are selected based on my preferences.  We also wanted it to be interactive.  Like allowing you to actually interact with the map showing the planes current location and zoom in on specific part.  Finally it should be connected to services that you currently use like Netflix for movies, Amazon to get reading material or having knowledge about hotel / car reservations that are related to your flight.

Some ideas that we dropped off due to time constraints were:

  • New rewards program similar to XBOX Live achievements. The idea was that rather than just miles you'd get points based on distance flown plus other achievements like flying to all 7 continents or South American Tour (fly to 3 countries in South America) or Old World Tour (Fly to the UK, France, Spain, Germany).  Plus you'd have a nifty badge so you can add another piece of flair to your blog.  I really wanted to do this but we had to cut some things and it didn't seem like it would demo well or really hit any of the profiles.
  • Kinko's In-flight office.  Like I mentioned earlier, we wanted to tie the in-flight experience into other services that passengers are already using.  The idea was that you could print stuff and it would be waiting for you to pick up when you landed.  Or some in-flight fax machine for the people who still actually send faxes.  Although it's funny that one of the judges said he was looking for someone to build exactly that.  Sucks that I was the one out of the group that decided to cut that feature.
  • Upload Photos to Flickr/Facebook/Windows Live by plugging you camera into the USB port in the seat.  We wanted to have a simple way to upload photos without having to try to set up your laptop / camera in the .5 cubic feet of passenger space in modern planes.  I'd love if this actually existed since uploading photos is one of those things I always want to do when I get home after a trip, but it somehow keeps falling down the ToDo list.
  • In-Flight chat.  A lot of the other teams had this feature so I'm kind of glad we didn't build that.  The only thing that we wanted to do that might have been slightly different is to integrate with Facebook and our game system.  Like you could click on a person and and have options such as "Add as friend [on Facebook" or "Invite to game"  None of the firms that did this addressed privacy concerns which was something that we struggled with when brainstorming.  Although it would cool if you were actually friends with someone on Facebook that it would alert you they were also on the plane.  Or maybe if you're really friends then you should know they are on the plane. 
  • Further developing the website portion.  We actually built a website in silverlight, but we cut it from the final presentation because we thought there was a strict 6-8 min presentation length time limit.  8 mins goes by pretty fast when you are trying to describe a whole story and show off 10 screens. Plus it did need some work to add a "wow" factor and we ran out of time. Basically we were still coding it at Underground so it didn't seem like a good idea to show something we hadn't practiced talking about.The website showed more of the setup type stuff for your reservation like linking to Netflix to pick movies from your queue to load onto the plane, choosing your meal by dragging and dropping food choices from Wolfgang Puck and adding in hotel / car reservations. 

So what did we actually demo?

The iPhone.  It still brings me a sense of "child-like wonder" every time I use it.  I wanted to build something for the iPhone as soon as I read the problem statement.  I ended up building our iPhone reservation / virtual ticket system about an hour before we left the office.  I think this won it for us so that worked out well ;)

The scenario for this application that we talked about is that you have a flight booked today along with a hotel reservation and a car rental.  The flight gets canceled so the airline automatically reschedules you for another flight tomorrow.  You receive a text message about this with a link to view your updated itinerary.

iPhone

The screen allows you to accept this rescheduled flight and optionally you can notify the connected hotel and car rental arrangements that your flight doesn't arrive until tomorrow.  You can also notify any friends or co-workers that you have designated in the reservation.  The idea is simplify the flight rescheduling process so you don't have to worry about notifying all of the additional parties involved.

The iPhone app layout is based on the template in iUI library.  I really liked the native iPhone app look and feel rather than creating a new layout.  I think it's far easier for people to use something that's a familiar interaction.

image

After accepting the updated flight you can also get a virtual boarding pass.  Rather than waiting inline at a kiosk to check-in or having to print it from your computer, you can get your boarding pass right on your phone.  This makes the process easier and faster without the waste of a paper ticket that's discarded an hour after you print it out.

image

 

A full video of the iPhone demo can be found here.

The next portion of the of our demo focused on the in-flight experience.  When a passenger sits down they are greeted by a "Welcome, <your name here>" screen.

1

We tried to create a menu with stylish look & feel with some glass-like touch panels for different parts of the app.  The menu includes options for Socializing (in-flight chat, facebook friends), navigation, Games (online games between passengers or download DS demo games via WiFi), Media, Service (order drinks, meal options, call for assistance) and Notifications.  You can't see it, but we had a 3D flip transition where all the panels combined to form each of the following screens.  It took me way too much time to code that and it only lasted about .8 seconds.

 2

Above is the Media screen.  There are options for streaming music from iTunes, viewing photos from Flickr, watching movies from Netflix and reading books / magazines from Amazon.  We discussed how the system is connected to your Netflix account so before the flight it would pull down the next few movies from your queue or you could select which movies to pre-load onto the plane.  The focus here is that the entertainment is personalized to what you'd like to watch.  It also connects to Amazon to bring down recommended reading materials.

3

Using the awesome text rendering controls in WPF you can scroll through magazines and resize the text for your viewing pleasure.  You can also interactively browse and select new reading material.  If your like me and you like to run to gate as they are closing the plane doors, you might not have the time to pick up that magazine that you saw at the newsstand while you were sprinting past (not that I sprint, it's more of a clydesdale like trot).  This way you can still read the latest issue of Guns and Ammo or O on the plane.

5

The navigation screen is connected to Windows Live maps.  Rather than the typical plane GPS view you can follow along on the map and zoom in via the satellite or street view.  Flying over Vegas?  You can zoom in on the satellite view of the Bellagio to check it out in more detail.  It also displays Flickr photos geo-tagged to the current map location.  I've never been to Lake Tahoe, but maybe I'd see some cool photos of it while flying over and decide I'd like to fly there next.

4

The final screen we showed is the in-flight counterpart to the iPhone reservation app.  Using this screen you can adjust hotel or car reservations while in route.  You can also schedule a limo pickup that will notify the limo when your plane actually lands.  The notification screen is connected to a variety of existing services that are personalized to your reservation.

After looking at the screenshots, it seems like we could have built more but 2.5 days goes by quickly when you start from nothing.  Next time I definitely need to time box my 3D screen transition animations.

Big thanks to all of the judges, Carlos Segura, Jeremy Alexis and Scott Kinkaid. I'm truly honored that you chose our design over the other teams who all did an outstanding job. And thanks to the folks who planned this event and helped answer questions like Chris Bernard, Larry Clarkin, Dave Bost, Don Burnett along with Dale Jones and Jared Potter from Identity Mine. 

Also gracias to all the Clarity people that showed up the night of the competition.  Winning is one thing, but having all your co-worker friends there with you is awesome.  Kind of like when a 15 seed upsets Kansas in the NCAA tournament and everyone storms the court.  And obviously big thanks to my teammates Marlon Smith and Erik Klimzack for putting up with me for 2+ days in a confined space without sleep.  Most of you probably haven't enjoyed working with "The Kevin Marshall Experience", but think Matt Foley.

image

Only way more profanity. I get pretty fired up in a competition.

 

Other coverage of the event:

  • David Armano delivered a great keynote speech to the event and wrote a follow-up post 
  • Shellie Hall has some great video of the event here. [ummm....disregard Erik's comments about Silverlight.  He hadn't slept in 3 days :) ]

 

So what's next? All 6 regional winners will compete at the South by Southwest Interactive Conference in Austin, TX on Monday, March 10th.  The PhizzPop site says "They will have approximately 90 days to complete their solutions.." I don't know what that's about.  72 hrs is abut my attention span. If that isn't a typo then I'm definitely going to need more Red Bull.

I talked to a lot of the other contestants during the competition but I didn't get everyone's contact info since it was kind of hectic.  I'd like to keep in touch so you can find me on Facebook or email me at kevin dot marshall AT claritycon dot com. It would be great to meet up again if any of you are going to Mix '08 or SxSW.

One other note.  I don't know if all the apps will be made available somewhere but Chris, Manny and Jared from Tribal DDB did some amazing stuff with Silverlight in 2 days.  Especially considering they were new to most of those tools. That spinning movie animation was sick.

Oh one last note. We're also looking to further develop our creative design ability as firm so if you or someone you know wants to help us design some innovative user experiences, contact me.  Did I mention you get to work with me :)

 

From the 10K Monkey Idea Lab: Yelp + Twitter = Yelpitter

I've started to use Twitter more and more since it's far easier to keep up with people when they distill their thoughts to under 140 characters. Chris Messina and Jeff Atwood are two of may favorite Twitterers.  It's harder to read every blog posts, but it only takes a second to read their latest Twitter when it pops up in Growl via Twitterific.

Anyway regarding Yelp / Twitter.  I was thinking about my old school IRC days in college and chatbots. I think there could be two fun ways to use Twitter and Yelp together and incorporate some chatbot functionality.. 

 

Yelp Review via Twitter

What I think would be cool is to have a new Twitter user, let's call it Yelp. 1

When reviewing a restaurant, you would just add Yelp as a Twitter friend and direct message them the name of the restaurant.  Yelp/Twitter would auto respond back to ask what you thought of the restaurant.  You can then respond back to Yelp/Twitter with the review and it's added to Yelp.

(I know the direct message work a little different in the Twitter UI, but this is just for illustrative purposes)

image

image

image 

Then on Yelp:

image

Sure I could just go to the Twitter site on my iPhone and add a review, but it takes far longer in the Yelp UI.  Plus, I'm already using Twitter and I like the forced 140 character review.  Just seems quicker.  I love the simple interaction model of Twitter.

 

Dinnerlog via Twitter

The next idea is two use the growing "#" tagging convention to track messages about the a restaurant on the Yelp review page.  My friend Dave Bost likes to Twitter his meals, Mmmmmmm.....Cup of Dungeonous Crab & Corn Bisque soup.  It would be great if Yelp would pick up Twitter messaged tagged by restaurant like "#Annie's Bistro, just ate the world's smallest omelette for $11.99" .

 

The dinnerlog would be displayed on the page like so:

 image

 

Dinnerlog shows the last 5 posts tagged with the restaurant name.

image

Unfortunately the Yelp API doesn't seem to support adding reviews so I can really prototype this out.  Although my Hawaiian friend/Fellow U of Illinois alum/Guitar Hero virtuoso Ryan Kanno has recently started a side project creating a local review site called WeGoEat2  I'm going to work with him to try to integrate this functionality into WeGoEat.  If anyone from Yelp is reading this, I'd love to discuss integrating something like this into your site :)

I'll post updates as we get this working in WeGoEat. 

 

1It appears someone has squatted on the Yelp twitter name.  Has Twitter squatting become the newest form of cybersquatting?  Thanks for trying to ruin Twitter Tom@blendah.com

2He's convinced only true Hawaii natives no the best places to eat and he's harnessing their collective gastronomique insight. Supposedly their is some advanced algorithm to filter out non-hawaiians.  Maybe it asks surfing trivia or something.

 

Technorati Tags: , , ,
Mozilla Prism - Splitting Apart Generally Accepted Ideas

I read this post from Mozilla labs a few days ago.  I can't sleep so why not blog.

What is Prism?

"Prism is an application that lets users split web applications out of their browser and run them directly on their desktop."

Well I can't disagree with that. 

"Unlike Adobe AIR and Microsoft Silverlight, we’re not building a proprietary platform to replace the web."

I can disagree with that.

Let's start with AIR.  If AIR is a platform then FireFox is a platform.  AIR has the webkit engine to render html/css/javascript and load swf files.  Same as FireFox.  How is it replacing the web?  AIR solely exists to display web content on a desktop.  Maybe I'm ignorant on the internal working of AIR, but it seems to me to be essentially a web browser that allows some offline storage.

I could be wrong, but Silverlight is not a proprietary platform replacing the web.  There is the open source version, Moonlight which makes it non-proprietary.  Secondly, it extends the web platform not replaces. How is Silverlight a web replacing platform but Javascript is not?  Both are programming languages on the web that run on the client to create richer experiences.  The only difference is that Silverlight has to be added as a plug-in, but browser developers have already built-in support for Javascript.

"Prism lets users add their favorite web apps to their desktop environment.  When invoked, these applications run in their own window"

Maybe I'm over simplifying this but this is basically opening a new browser window for each app and removing the browser chrome?  And I need to download a separate runtime for that innovation? If I download a runtime isn't this a new platform?

Granted it's still new so they have other things planned:

"Prism isn’t a new platform, it’s simply the web platform integrated into the desktop experience. Web developers don’t have to target it separately, because any application that can run in a modern standards-compliant web browser can run in Prism....And while Prism focuses on how web apps can integrate into the desktop experience, we’re also working to increase the capabilities of those apps by adding functionality to the Web itself, such as providing support for offline data storage and access to 3D graphics hardware."

I'm curious to see how offline storage is implemented without developers targeting that in any way.  Same with 3D acceleration.  That clearly won't involve and code changes.

After you add those changes it seems like it really is the same as AIR, but "Unlike Adobe AIR [..] we’re not building a proprietary platform to replace the web."

A few things bothered me about that post:

  1. It seems the authors either misunderstand Prism or Silverlight/AIR
  2. The general insinuation that MS/Adobe are trying to F**k up the web with proprietary platforms.  "we’re not building a proprietary platform to replace the web. We think the web is a powerful and open platform". 
  3. "Ideally you shouldn’t even have to download Prism, it should just be built into your browser." - Yeah, so should Silverlight and Flash. I don't want to have to install those either.

I am curious to see Silverlight running in Prism though.  Actually I would much prefer to see Silverlight running on AIR.  It's like a platform within a platform within a platform.  The mind can barely fathom.

Long live proprietary platforms that replace the web!

 

Technorati Tags: , , , ,
MOSSCamp Update: Event T-Shirts and the Collaborative Keynote

MOSSCamp is less than 2 weeks away.  Don't forget to register now.

All attendees get a MOSSCamp t-shirt designed by our Sr. T-Shirt Designer, Erik Klimczak.

mosscamp_front copy

mosscamp_back copy

 

A couple thoughts on the event keynote.  Does anyone attending have ideas?  One thought I had was to make it a Collaborative Keynote -- harnessing the collective wisdom of the audience.  I don't know about most of you, but I can't listen to someone lecture for more than 5 minutes without zoning out.  What I propose is to have someone be the keynote moderator. 

The keynote moderator aka Camp Counselor can start off by asking "What can SharePoint do for you"?  As people in the audience answer, we can build a discussion around the typical information that would be delivered in a keynote SharePoint overview.  At most events, people attend, eat pizza, listen and leave.  One of the primary goals of this event is to foster group ideas and build connections between SharePoint developers that spread beyond this event.  This is your chance to share knowledge and build a community around fellow SharePoint enthusiasts. I think a collaborative keynote would be the best way to get people talking to each other and set the tone of group participation for the rest of the day.  Between everyone in attendance, we should be able to build a complete overview of SharePoint.

During the keynote discussion, we can then develop a list of breakout sessions.  If someone starts talking about SharePoint Wikis and you want to learn more, it's your chance to volunteer to organize a breakout sessions on SharePoint Wikis.

I'd also encourage everyone to use the wiki to take notes to help spread the ideas we discuss.

Any thoughts?  Collaborative Keynote or 60 minute PowerPoint presentation?

Also, if you're blogging about MOSSCamp, be sure to tag your posts as "MOSSCamp"

Here are some people that have been covering MOSSCamp:

http://timheuer.com/blog/archive/2007/10/26/sharepoint-devcamp-chicago.aspx

http://larryclarkin.com/MarkYourCalendarsMOSSCampOnNovember9th.aspx

http://blogs.msdn.com/angelab/archive/2007/10/22/mosscamp-in-chicago-november-9th.aspx

http://phacker.wordpress.com/2007/10/25/chicago-mosscamp/

http://davebost.com/blog/?p=326

 

Big thanks to Larry Clarkin, Dave Bost, George Durzi, Chris Kadel and Joel Wurfel who are helping organize this event. 

 

Re: WTF is WPF - WPF Re-branding Suggestions

My friend Kurt Brockett at IdentityMine started blogging again (at least somewhat more regularly for the moment)  I was reading some of his older posts and I found one post entitled  WTF is WPF.  He talks about the wow factor created by re-branding WPF/E to Silverlight and competing against cooler named products like AIR, Flash, Carbon, Cocoa, etc.  Well, you can just read the post.

Unfortunately he didn't have any name ideas.  Kurt, I'm a little disappointed in you.  I thought your business card said Director of Product Management?  This should be in your wheel house.

Here are my ideas:

 

Name Description Rating (1-5 Monkeys)
RIALITY Acronym for "Rich Interactive Applications,  Like Totally.  Yeah!"


Should play well with the pivotal teenage silicon valley girl developer demographic.

Otherwise, not so good.

1 Monkey
WPF Ultimate

WPF Home Premium

WPF Home Basic

WPF Business

WPF Enterprise
Part of the problem with WPF is that it's a vague, all-in-one framework.  What if I use my WPF for home entertainment?  Or business productivity?  I need an edition of WPF that's right for me.  Some type of comparison table would help. Like if I don't care about dependency properties, I don't need WPF Ultimate.

WPF Home Basic
Ideal for the casual developer who only needs to display photos.  Easy to develop and maintain.  XAML?  What's that?1  Don't worry, WPF Home Basic is all wizard based.  No glass for you!

WPF Business
WPF Business is the first edition of WPF designed specifically to meet the needs of small businesses. You'll spend less time on technology related issues like coding-so you can spend more time making your business application look pretty.

WPF Enterprise
Designed to significantly lower development costs and risks, WPF Enterprise meets the needs of large, global organizations with complex 2D animation and text beveling effect needs.

WPF Home Premium
WPF Home Premium is the preferred WPF edition for home desktop and mobile PC applications. It provides a breakthrough unified programming model that brings your UI into sharper focus while delivering the Storyboards, StackPanels, and Routed Events you need from your PC application at home or on the go.

WPF Ultimate
WPF Ultimate is the choice for the developers who want to have it all. Easily shift between the worlds of productivity and play with the most complete edition of WPF. Ultimate provides the DockPanel, StackPanel, and Grid controls needed for work, and all the video brushes  that you want for fun.

Specifically, WPF Ultimate offers all of the features found in WPF Home Premium, including Labels, StackPanels, and Routed Events. It also offers all of the features found in WPF Business, including DockPanels, Resource Dictionaries, and Buttons. And WPF Ultimate has all of the new Dependency Properties and Data Templates that help take WPF to a whole new level of programmability.

In addition, WPF Ultimate includes support for new animations, including inverted negative 3G dive, counter-clockwise swirl, the shark and other new, theoretically impossible mind altering effects.

Exclusive to WPF Ultimate are WPF Ultimate Extras. WPF Ultimate Extras are add-ons that extend certain controls in your application or just make programming  more fun. WPF Ultimate Extras currently planned include:

  • Really Big Button
  • Bouncing Textbox
  • Möbius Label
  • Spinny Thing

Has the potential to create confusion in the development community.

Success of WPF Ultimate is also dependent on the release of the highly anticipated WPF Ultimate Extras. 


.5 - 3 Monkeys depending on the edition
Heavymetal WPF/E -> Silverlight
WPF -> Heavymetal

Get it?
Yeah, I didn't think it was too funny either.  It was Kurt's idea.

2.5 Monkeys
Nothing I always thought the Silverlight name was somehow related to "Flash". 

What does a flash look like?  A silver light.

What does AIR look like?  Nothing*. 

*Unless you live in Mexico city, then it's a black ominous cloud of pollution.  But Black Ominous Cloud of Pollution doesn't have a nice ring to it.
Tough call.  When someone asks what you are working on you can say "Nothing" - Short, easy, and probably an accurate description for most corporate developers.

3 Monkeys
EyeCandy From Wikipedia:

"Eye candy
is something that is most remarkable for its visual appeal."


Hmmmm... not bad. WPF does allow you to create rich graphics with holographic 3D inverted color changing rotations all while displaying a nice Apple-like reflection.
From Wikipedia:

"In computer software, especially in operating systems, too much eye candy is often criticized."

Oh.

2 Monkeys
Seven

I overheard this conversation the other day:

GEORGE: Oh, no no no. Course not. I got a great name for our kids. A Real original. You wanna hear what it is? Huh, you ready?

SUSAN: Yeah.

George uses his finger to draw a number 7 in the air, accompanying the

Strokes of his digit with a two-tone whistle.

SUSAN: What is that? Sign language?

GEORGE: No, Seven.

SUSAN: Seven Costanza? You're serious?

GEORGE: Yeah. It's a beautiful name for a boy or a girl...




It's a beautiful name for a boy or a girl.. or a unified programming model for building rich Windows smart client user experiences that incorporate UI, media, and documents.

Seven is also the number of syllables in "Presentation Foundation".  Coincidence?  I think not.

 

4 Monkeys

 

Seven is not bad.  But I have a better one.  I'm going to call the WPF team tomorrow.  It'll go a little like this:

Me: Awright, I tell you what. You look like nice people, I'm gonna help you out. You want a beautiful name? Soda.

WPF Team: What?

Me: Soda. S-O-D-A. Soda.

WPF Team: I don't know, it sounds a little strange.

Me: All names sound strange the first time you hear 'em. What, you telling me people loved the name Silverlight the first time they heard it?

WPF Team: Yeah, but uh... Soda?

Me: Yeah, that's right. It's working.

WPF Team: We'll put it on the list.

Me: I solve problems. That's just what I do.

 

Plus, developers love Soda.  WPF is now Soda, the most advanced desktop RUCCA

 

image

 

1Adam Kinney a.k.a the Silverlight Surfer explained to me that XAML was like XML with more tags.  Cal Schrotenboer said that comparing XAML to XML was like comparing a lawn mower or a bike to a car.  Lawn mowers and cars do both have 4 wheels so they seem similar.  Plus you can race both of them.  Confused, I went to Cal's blog at http://www.wpflearningexperience.com/

Ooooopps.

image

Thank goodness Kaspersky's got my back.

 

More Posts Next page »