Today is a big day, I’m pleased to announce that “Design for Software: A Playbook for Developers” is finally available to the public! It was a long journey taking nearly 14 months to complete, but it’s here and I couldn’t be more excited. Check it out at designforsoftware.com
A little background
Over the past ten years I’ve worked closely with designers and developers in the emerging technology industry. During that time I’ve realized that designing software isn’t like any other type of design. Designing software is unique. It combines many fields of design into a single medium – Motion, typography, color theory, interaction design, information architecture, ethnography, and more. And for many reasons it deserves its own design process.
That is the premiss for this book – a process for designing software. My goal is to make this unique discipline more accessible for those that don’t have expertise in UI/UX design. Throughout the book I attempt to break down design theory concepts and present them in a way that makes sense for application design.
This book is written for non-designers and tech-savvy artists looking to create best in class software. I believe anyone can design great software, regardless of background. It doesn’t matter if you have a design degree (although it might help) or a history of being a hard core developer. Many design principles can be broken down into frameworks that help take the guesswork out of an otherwise abstract field.
By the end of the book you’ll have effectively learned all my design “secrets”. I hope that the design process I’ve written about is as useful to you as it has been for me.
It’s pretty difficult to write about interactive design in a static medium..go figure. During the writing process there were many times that static images didn’t do the content justice, especially when trying to describe motion. So I created a bunch of complementary web content.
You don’t have to buy the book to reap the benefits of the extras, but you should. They will make more sense…and it will make my dog happy (not my actual dog).
Where can I buy it?
The publisher (Wiley) has done an amazing job making the book available from a variety of retailers and in a ton formats. Of course I would encourage you to pick up the physical book, but the digital versions are good too. As of now, you can purchase the book from the following retailers:
We produce a ton of artwork for Windows 8 and when we present ideas to our clients I like the artwork to look as good as possible.
One way we do this is by placing the designs within a device frame. This gives the artwork some context, and a more premium positioning.
Mocking up artwork in a device frame is pretty commonplace for iOS and Android design, so I figured the Win8 community would benefit from these. Unfortunately, many of the MS Surface templates floating around the interwebs appear to be slightly inaccurate. They look as if a Samsung Series 7 and an iPad had an awkward kid. So I was compelled to have a legitimate go at making an accurate template. I’d like to think mine are at least 98.9% accurate.
Get the Goods
If you have Photoshop or Illustrator installed these should come in handy. And if you don’t…you can hand them off to your neighborhood friendly designer and I’m sure they’ll be pleased as punch…
Following up on my last post, I mentioned that we are creating a fun installation that involves tracking small fast-moving objects. So without further ado I give you Rally Keeper:
Before I get into how it works, I want to mention some of the inspiration for the project. A while back I saw this cool product concept for an interactive ping pong table. Inspired by the idea, the team and I spent some time pondering how we could take this “future” concept and turn it now concept.
Researching similar projects we examined a variety of approaches including audio triggers, triangulating sound waves, and color tracking. Then the ‘aHa’ moment came – we only needed to track the ball at a certain depth to capture it’s coordinates. Knowing that we could easily track depth with the Kinect, we quickly put together a prototype.
The diagram above shows our approach..it’s actually quite simple:
Position the Kinect far enough away to capture the whole table
Slice Kinect’s depth image creating an “invisible” layer directly above the table surface
When the ball enters the tracking layer, use OpenCV to grab it’s coordinates
Store and paint the coordinates back onto the UI layer
Most of the tech details of how to do blob tracking can be found in my last post.
The UI Concept
When we overcame the major tech hurdles, it was time to give this thing a face-lift. The aesthetics were mostly inspired by the original concept. And the UI was implemented using WPF.
Aside from a being purely a cool visualization, we also wanted to add some utility. After a typical battle session and a brutal defeat, we often exclaim “I can’t believe you beat me! How’d you do it!?” Convinced, that if we could see the ball patterns we could calculate a better strategy for the next bout. Additionally, analyzing data is cool. With the ball data we can calculate percentages, make predications, and see trends in our games.
After messing around with this for a week we came up with dozens of additional ideas we’d love to explore…time permitting. Some of the highlights include:
Authenticating purely with Kinect gestures
Leverage the Kinect’s microphone array for automated scoring
Pushing game results up to a website for comparison and analysis
Use projection mapping to draw the visualization onto the table in real-time.
Use multiple Kinect cameras for even better tracking
Anybody else out there with any ideas? I’d love to hear them…
With the latest SDK release, the Kinect platform has become pretty robust – skeleton tracking, gesture support, and access to the raw infrared stream all come “out-of-the-box”. Which is great, because if you’ve ever used any of the opensource equivalents, things can get pretty hairy.
Hands and skeletons are cool and all, but unfortunately there isn’t an *easy* way to do blob or generic object detection.. Right now, we’re working on a fun project and ultra fast blob detection is one of the major tech hurdles we’re working through.
After googlin’ around for blob-tracking with Kinect I came up empty handed. So I decided to look into OpenCV for doing blob detection. Many curse words later, I finally got things working the way I wanted. Along the way I noticed a handful of forums with people looking to do similar object tracking with Kinect, so I figured I would share how I cobbled things together.
Demo Overview & Source Code
The demo below is showing a couple of key things:
Consuming the raw Depth image from the Kinect sdk
Slicing the Depth image to a particular range
Using OpenCV to perform object/blob tracking
Rendering OpenCV output with WPF
Before We Get Started
Be sure to download the latest Kinect SDK. There are also a bunch of handy samples that come with the toolkit. You can get it here
EMGU (.Net Wrapper for OpencV)
Emgu is a .net wrapper for OpenCV. It is very complete and does a good job of providing .net hooks into the common OpenCV functions. Unfortunately, getting it to work with WPF is a little tricky (More below ). Also, I wasn’t able to get the lastest (2.4) version of the code to work with .net 4.5 and Visual Studio 2012, but the 2.3 version works well. Get it here
This codeplex project is similar, but it’s using another computer vision library called AForge and only compiles with a beta version of the Kinect sdk. Regardless, there are some handy tid-bits of code in there. You won’t need it to build my sample but its a good resource to look through.
Ok, now that we have the dependencies installed, its time to start setting up our WPF project. There are similar instructions on the Emgu site, but I found them a little confusing, so I’ll give you the cliffnotes version here.
After you’ve installed Emgu you need to add some .dlls to your project. This is the standard Solution->right-click->Add Reference and then browse to the Emgu installation path to add the following .dlls
The next step is a little odd. You need to add 3 .dlls to the root of your project. Right-click the Project fill and select “add existing item”, then navigate to the Emgu installation folder and add the following:
The last thing, and this is important, is to set the build action to “copy always”. This will dump those dlls into your build folder when you build the project. When you’ve got all that done you’re solution should look something like this:
Getting OpenCV to Play Nice with WPF
Ok, now that we have all the references in place we need to make openCV (Emgu) play nice with WPF. The biggest thing to note is that OpenCV uses it’s own type of image objects (IImage) which don’t work by default with the native WPF image controls.
Luckily, there is a little helper class that comes with Emgu that will convert the IIMage to a BimapSource, which works nicely with the WPF image control. By adding this helper method, we now have a convenient way to plug an OpenCV image into a WPF image. See the helper class here
The Fun Part
Now we can get to the good stuff. When you do blob tracking it is very important to have a “clean” grayscale image that has a high contrast between the object you are tracking and the background. This is one of the best parts about Kinect – it provides a grayscale depth image that we can analyze and extract out the regions of interest relatively easily. Let’s looking at the key pieces of the solution.
Depth Image Slicing
Like I mentioned above, when you are doing generic object detection you want to have high contrast between your object and the background. In other words, you want to isolate the object the best you can and ignore everything else. By doing so, you can ensure the vision library, like OpenCV, does a reliable job of finding the blobs.
The beauty of using the Kinect for blob tracking is that it gives us a depth image that contains all of the information we need to “slice” the image into the parts that we’re interested in. While I was trying to figure out the best way to slice the depth image I came upon a Kinect depth data video that really opened my eyes to working directly with the depth byte stream coming from Kinect. Check it out here
By simply ignoring (coloring pixels black) data that is out of the defined depth range, I was able to get a nice high-contrast image of the objects the depth camera was picking up. You can imagine this being something like an invisible plane that excludes everything in front and in back of it, which coincidentally creates a nice interaction space where we can track objects. You can see the slicing code here. This is the output:
OpenCV Contour Finding
The last thing to do is to get the blob tracking working. At this point we have a very clean image that we can feed into OpenCV’s “Find Contours” method. The FindContours method analyzes a grayscale image and returns the number of blobs as well as their bounding rectangles. It was a little tricky getting this to work in WPF, you can see the main piece of the code here.
Once I was able to figure out some of the cryptic tracking mechanisms in OpenCV everything came together. The cool thing about OpenCV is that not only can you get blobs, but you can blobs within blobs..in a tree. Crazy..I know. And lastly, we use our handy helper method from above (ToBitmapSource(OpenCVImg)) and plug it into the Image control in the main window xaml. Which ends up looking something like this:
The last thing I added were a few sliders to the UI to adjust the slicing parameters and the max/min Blob size. The image below shows the finished application.
All said and done, I am happy with the performance and excited about the opportunities to use this slew of technologies in some upcoming installation work.
Ok, so what?
I know what you’re thinking…so you have some blobs, big deal. And you might be wondering what else you can do with blobs. Well, let me tell you. By using generic blob detection you can not only detect when objects are present, you can also track their size, shape and rotation. For example, its possible to track an open hand transitioning to a closed fist for a grab gesture, or a user holding a wand or prop. Or in our case, we want to very efficiently capture the position of small, fast moving objects that penetrate an “invisible” IR plane. There are a ton of uses for blob detection including surveillance, card counting, and even real-time projection mapping.
While I’m pretty happy with the performance of this solution, I’m curious if it would be better/faster if I rewrote everything with the C++ flavor of the Kinect sdk and coupled it with DirectWrite or XAML to render the UI. Anybody out there do anything like this yet? If so, shoot me a tweet or email.
We’re currently rolling this stuff into a fun side project that I’ll post about shortly…stay tuned.
Thanks to everyone who attended my design talk at That Conference. It’s always great to see the dev community genuinely interested in learning about design. That Conference was awesome, the support of the attendees and overall energy at the conference was some of the best I’ve ever seen. Water park after party?! Nuff’ said.
New Content Highlights
The slides from the talk are available below, but before you go jumping in face-first, there is some new hotness worth noting. I posted this deck about a year ago, when I originally gave the talk at DevSum. And since then I’ve spent about a year turning the talk into a BOOK. And over the course of the year I’ve generated quite a bit of new content. Along the way I’ve created a few design tools that help make some mundane design/dev tasks a little easier.
The first of the two tools I showed during the talk is called GridHelper. If time allows, my plan is to write up a more detailed post on how to use it. But here is the gist:
If you’re a designer or a creative coder you’ve probably needed a grid at some point. Whether you are aligning elements on a page, or creating a layout from scratch – grids are the layout power-tool.
If you’re like me, you probably hate the meticulous process of adding a bunch of guides to your photoshop or illustrator files. Or you hate writing code to generate a grid to overlay in your projects.
What’s more, most grid calculators are targeted at web design. What about apps?! iPad, iPhone, Windows Phone, Android, Large format touch screens, kiosks….we all need grids!
So that’s why I built the grid helper. Quickly make vector or pixel-based grids to drop in your design files or add to your dev projects.
GET IT from the Chrome Webstore or just hit this link. *works best in Chorome”
Rhythm & Scale Calculator
The other tool I demoed during the talk is a typography widget. Its called the Rhythm & Scale Calculator and it helps with two key typography taks:
Creating a Typography Scale / Ramp
Ensuring a baseline rhythm
I will defer to my detailed blog post about how to use the widget.
The widget is also available in the Chrome Webstore or hit the link HERE. And for you mac users you can download the dashboard widget for OSX.
I hope these slides get people excited about integrating design into their own software projects. Coming Soon – Stay tuned for the book coming soon!
Last week Clarity was invited to the Windows 8 Developer Launch event in here in Chicago. Matt Hidinger and I were asked by some good friends at Microsoft to give our 2-cents on why we’re excited about Window 8. It was a packed event, and we had a great time. Below are some pictures, slides and a short recap on our talk. Pictures courtesy of Bob Laskey
Win8 : A Business Perspective
Here at Clarity we’re excited about Win8 for a number of reasons. One of the wins we see from a business perspective is reusability in three main areas: Design, Code, and Skillset.
Design – We can now share design assets and create a consistent brand experience across Microsoft’s three major screens: Xbox, Windows Phone, and now Windows 8.
Code – We primarily use C# and XAML to build our Windows Phone apps. Xbox, and Windows 8 also give us a framework to write C# and XAML. While the code isn’t necessarily copy & paste, there is a ton of overlap between the platforms.
Skillset – This is sort of a nice side-effect of having similar coding platforms. We are now sharing talent across our dev projects. In other words, our mobile guys are building Win8 apps and our XBOX guys are able to make the transition to Win8 or Win Phone just as easily.
Win8 : A Design Perspective
I’ve been building and designing software for a long time. And as a ‘design guy’ I tend to focus primarily on the UX aspects of software. With Windows 8, UX and aesthetics have finally been promoted to a first class citizen. This is great news for both designers and developers. This is especially exciting from a development perspective because it makes good design a hellava lot more accessible to the non-designer.
Metro makes Design More Accessible. . I like to think of Metro as a ‘developers kind of design’. It gets to the point, and promotes functionality over flash. Its about solving problems, visually, which is something we can all get on board with.
That last statement might seem a little broad. Specifically, the Windows 8 platform makes a handful of tedious design scenarios very easy for developers :
Touch First Controls – MS has finessed the crap out of these touch controls. They are accurate, and have a ton of user research driving their functionality. This is something that takes a lot of precision and therefore time to nail. Now you get it for ‘free’.
Animation – I’ve spend a good portion of my career figuring out the ‘perfect’ easing functions and timings to create snappy elegant motion in applications. MS has spent a bunch of time making the animation framework super easy to use for everyone. Its really good. Use it.
Built-in Styles – Margins, spacing, alignment, font-sizes, type-casing, font-weight. These are things that make most developers cringe. Well guess what, MS has been awesome enough to provide you with templates that have all of the base styling built-it so you don’t need to fret about those teeny-tiny pixels.
Win8 : A Development Perspective
Our own Matt Hindinger did an awesome job talking about the great new development opportunities that come with the Win8 Metro platform. I’ll make a feeble attempt to recap the highlights of his portion of the talk.
C++ XAML / DirectX – We think this platform is best used when you have a need for performance and precision. C++ and DirectX give you full control of every pixel on the screen and its super fast.
C# / XAML – This is our platform of choice. It offers a very reliable strongly typed coding environment that couples speed of development and a robust control set. If you come from a .net background, this is the one for you.
Below is our slides from the event. Check em’ out and share them with your friends. We had a lot of fun embracing the ‘trenches’ theme .
If you have any questions or thoughts you’d like to share with us don’t hesitate to hit us up on twitter.
Erik Klimczak | e = firstname.lastname@example.org | t = eklimcz
Windows 8 is coming in hot! And here at Clarity we just couldn’t wait for MS to publish the official design guidelines. Our clients want to see their brand on Windows 8 now. So we meticulously picked apart screen shots and some XAML to create our own Photoshop templates. Get the unofficial Windows 8 photoshop templates below to kickstart your next Win8 project. Free of charge from your friends at Clarity Consulting
Whoa..8? I just got 7…
Yeah dude, the Windows 8 beta is right around the corner and the excitement is almost palpable. Over the past 6 months Microsoft has been slowly revealing the next evolution of Windows. The first glimpses of the platform came at BUILD where the developer preview was released along with a ton of sample apps and code snippets. Shortly after, MS publicly announced their plans for the Windows 8 Store. The Store announcement boasts some great opportunities for app makers and showcased some very nice apps.
The new Windows 8 platform has adopted the “Metro” design language that can be seen in Zune, Xbox, and Windows Phone products. The Metro styling has its roots in traditional Bauhaus design principles which, for this designer, is a breath of fresh air. Yet, for a platform driven by rich design concepts, guidelines and templates are sparse. Right now, the best documentation is scattered throughout the MSDN library and doesn’t give much in the way of practical design resources.
Why do I need this?
A few times now I have gone through the rigamarole of designing a Windows 8 app. And every time I go through the same process – take screenshots of my Window 8 tablet; creating guides and font sizes; start adding in content. In efforts to speed up our design process we made some template time-savers that I’d like to share.
Thanks to our senior photoshop template maker Brittany Travitz we now have some basic templates that should help you:
Set up proper margins and spacing
Find consistent Font Sizes & Weights
Create a basic application flow
What do I get?
The templates are primarly based on the Expression Blend windows 8 default project template. But we’ve also baked in some of the fine print found in these articles
There are four files included in the download. These files represent a bare-bones Metro application flow (see below). The four files include:
Home – This is the basic landing page or Hub of an application where you surface featured and curated content.
Category – One level below the home page is the category view. This is where you would present the full list of items for a given category.
Detail – This is pretty self explanatory. Details of a product, article, video, image, etc, go here.
Snap View – All Metro apps in Win8 require to support a snapped view state that takes up 1/3 of the screen. Similar to supporting CSS Media queries in a responsive website design.
I’m sure MS will be releasing official design files and resources any day now. But until then, hopefully these files will save you some time as you spin up your next Win8 Project.
** BONUS **
We stumbled upon some really nice free icons for Metro app design. Check them out at Icons8
About 5 months ago I gave a talk on Design for Software for DevSum11. It was a great event and I was pleasantly surprised at positive attitude the community had towards design. Shortly there after I was contacted by Wiley to see if I would be interested in elaborating the talk into a book. After a many fist pumps, I calmly proceeded to pursue the engagement.
Design For Software: A Playbook for Developers
As a formally educated ‘design guy’ I’ve had the unique opportunity to work along-side some rather brainy engineers here at Clarity. Even though I’ve sort of been the ‘odd man out’ I’ve gained some unique insights into the deep dark trenches of building software.
And over the course of my career I’ve hand-picked a creative process that won’t break the precious budget and cause timelines to get bogged down. That is the basis of the book, and what I’d like to share with both the tech-savvy artists and the creative-minded technologists of the world.
SO WHAT’S IT ALL ABOUT?
In my opinion well-designed software consists of three key ingredients:
Cognitive Psychology – Understanding and anticipating how users will use the software
Aesthetics – The fit n’ finish, bells & whistles, glitz & glam
Solid Code – The way it works and how it ‘feels’
Techy people, Arsty people, you are one in the same, with different priorities, but one great ability in common – Solving Problems.
However, designers and developers are typically viewed as opposing forces, left-brain vs. right-brain, if you will. This implies there is nothing ‘creative’ about developing software, which, most developers would tell is complete bull dooky. Perhaps its because we designers tend to gravitate towards the gray-area and developers have a natural propensity towards the black & white.
Regardless, some design principles in the context of software can be made black & white. And that is precisely what I’m talking about. Design is no more than solving problems, visually. And developers are already great at solving problems, so just because you didn’t go to design school doesn’t mean you don’t have the ability to make your software shine.
WHO IS IT FOR?
‘Design for Software’ is meant to connect the dots between software and design. Its targeted at tech-minded folks that have a taste for design, but aren’t sure what it takes to get there. Or the tech-savvy artists looking to fill the gap between art theory and cold hard code.
WHEN CAN I GET IT?
Unfortunately, I juuuust started the writing process and it will be a few months before anything is available to buy. My plan is to create a blog series that coincides with the high-level concepts that will be elaborated in each chapter of the book. I’ll key you posted as I make progress!
The Original Talk
Below is the original talk I gave at DevSum. It is more-or-less a high-level outline of what I’ll be writing about in the book.