Peter Miller

Paper Zero

2008-08-18T00:26:30Z

Recently I mentioned Inbox Zero, a method of keeping your email inbox organized and uncluttered. Another useful organizational method I practice is what I call “Paper Zero”. Paper Zero is not using zero paper; instead it is a commitment to:

Organize the papers you have
Throw away transitory papers (recycle if possible)
Transcribe important papers into an electronic format (and then throw them away)

I use a good amount of paper at work while doing such tasks as taking notes during meetings, jotting down mental reminders while debugging or marking up project schedules. So I am not anti-paper; until we all have take anywhere, instant on tablet computers with the visceral feedback of pen on paper, I will not abandon my notepad and trusty Pilot G-2 Gel pen (size 07 usually). Sometimes ad-hoc notes, sketches and diagrams are a necessity. In addition, for the sake of my eyes, I don’t like reading long documents on the screen. So printouts can be handy as well.

There are however, many problems with paper. First, its physical dimensions. Papers quickly pile up, cluttering your desk and closing your workspace in around you. Second, paper is inherently volatile. Papers can be torn, ripped, spilled on and just plain lost. Third and perhaps most importantly, papers are inherently information silos when compared to electronic documents. Papers are hard to organize, hard to search, hard to relate to other documents and for the most part single user experiences.

Digitizing your notes is not a revelatory suggestion and the benefits of having your information in an electronic format should be pretty apparent. A typical objection to applying this idea to notes at work is that as notes they are not important enough to spend the time transcribing into digital documents. My response is that they if they are so unimportant, why don’t you toss them and save yourself the desk/filing space? If they are important enough not to toss, then they are important enough that someone else should be able to know about them. Based on my handwriting at least, that implies electronic format.

Another way of looking at that question is to ask could someone pick up on your tasks where you left off from if you were unexpectedly sick or out of the office? No such transition is ever perfect, but it would certainly be helped if the other person doesn’t have to try and decipher your hand writing and filing system. If you are deliberately keeping important information to yourself to be more powerful or save your job, well then you need more than Paper Zero to help you. At least in my field, we value problem solving and effective information sharing, not hoarding.

The final benefit of transcribing important notes into an electronic format is not immediately obvious. When you transcribe notes, which were written while you were thinking about a certain situation with the entire context already loaded into your mind, you are forced to lay out your thoughts logically, exposing your assumptions and gaps in knowledge.

So even if you don’t care about anybody else, by transcribing your notes you can be more effective and have a better grasp of your subject material. In fact, this technique was frequently suggested to me by various college professors as a way to better understand and retain material prior to an exam. I didn’t always follow this advice, but the concept of it stuck with me and has proved useful outside of the classroom, in the quotation mark surrounded “real world” of the workplace.

Does It All Lead Back to MS Project?

2008-08-05T04:15:41Z

If you’ve worked on a software development project, there is a good chance you’ve either used or done everything to avoid using Microsoft Project. I avoid MS Project when I can and have developed my own Excel-centric task management system. However, as my system grows to meet the needs of my projects it is eerily converging on Project itself. Is there any escape or I am bound to run so far away from Project that I run right back into it again?

A complete project in MS Project is in many ways a thing of beauty to behold. Tasks are joined to resources, timelines and other tasks as dependencies. You can adjust someone’s allocation and watch the timeline warp and wriggle in response. You also have the comfort of being able to produce solid, comforting diagrams which show your progress and plans to your non-Project blessed co-workers. Then your project moves along just as you planned, with satisfying milestones along the way to the anticipated successful solution.

With these happy sentiments in mind, why did I spin up my own system for task management and project planning? There are 2 major problems I see with Project, the “buy-in” cost and adaptability. Buy-in cost is the amount of time and effort it takes to get your project into MS Project. Project planning is important, but the mechanics of entering your tasks, setting up resources and linking up dependencies is quite clumsy. By itself however, the buy-in cost would be a reasonable investment if it were not for the second major problem with Project, adaptability.

Changing out resources, redefining tasks and dependencies, adding or deleting entire chains of tasks happens all the time during projects, yet represent great pain points in Project. I remember a specific project I was on where a failed attempt to change a few dependencies cost my project manager literally days to recover from.

Beyond changes to the project itself, Project is restrained in the meta and supporting information you can easily attach to tasks. If you really dig into Project, you can set a lot of this up, but again, based on past experiences, Project does not end up being the definitive source of information about a task. So you have other documents, spreadsheets, etc. which all then try to refer to the tasks in Project, leading to synchronization head-aches and disconnects.

One possible answer to these problems is to spend the time to really learn Project, to deep dive into it and hope to mediate the pain points with your newfound guru status. However, at this point you are likely to be stuck (consciously or not) running your actual project in such a way that matches up with the constraints (and capabilities) of your project management tool.

To avoid this inversion of priorities, I have been experimenting with my own system of task management and project planning. The foundation of my system is Excel and so far it is quite primitive in comparison with Project. Nevertheless, I have enjoyed growing my system to create capabilities I find lacking, rather than growing my project to use capabilities I received by default.

I haven’t had to test how my system scales to a large project yet, but even with a small to medium sized project, I do look at parts of Project with envy. I have to laugh when I see my spreadsheet layout inching closer and closer to a Project style look. While MS Project is over weight for my current purposes, it was not developed out of the blue, but in response to certain scenarios that occur while on a project. The problem lies in trying to solve every one of these at once and in the process making solving any of them individually that much more difficult.

Versatilists

2008-07-28T04:21:24Z

I’ve been reading an excellent book, Head First Design Patterns, which covers many of the Gang of Four’s famous design patterns for software engineering from the Head First series own irreverent and creative perspective.

Design patterns are great; they are a useful shared vocabulary for discussing software design and a source of inspiration for your day to day programming. However, as the GoF and the Head First book point out, design patterns are not invented, they are discovered by working with real world applications whose unfettered complexity has lead to unexpected problems. Or more succinctly, they emerge out of encountering and solving pain points.

The danger with design patterns is that you learn them, apply them liberally and then complain about the extra overhead and up front design time. This danger lessens the more and more you’ve actually experienced the pain of clunky and obtuse code. As with most of life, experience is the best and most bitter teacher.

I was further reminded of this when I was recently talking to a friend who mentioned GTD or getting things done, a time management philosophy that encourages creating and executing to-do lists, among other productivity tips. I am not a strict adherent to GTD, but I discovered I did follow one of its most well known off shoots, the idea of a clean email inbox or “Inbox Zero.”

As the name suggests, the idea behind Inbox Zero is to have no emails sitting in your Inbox. This does not mean automatically deleting every email or anything that dramatic, but instead it means expediently applying one of the following actions to every email that comes in:

Delete
Delegate
Respond
Defer
Do

There are plenty of resources available online to more fully explain this, but it boils down to making a decision about every email that comes in (even if the decision is to defer your decision) so you never get overloaded with too many emails.

I began following my own form of Inbox Zero after going through a period of time in a project where I got at least 100 emails a day and was going crazy trying to keep up with them all, alternating between frantically trying to respond to each one immediately and a kind of exhausted “benign neglect” of any email without at least a few exclamation points or priority flags.

So, like a design pattern, out of a pain point emerged a general approach to me organizing my inbox. Also like a design pattern, the point of a clean inbox is not to follow Inbox Zero techniques for its own sake, it is help manage complexity and reduce pain points. Process does not exist for its sake, but to alleviate a certain problem.

In some ways it would be much easier if this were not the case. If we could create great software solutions every time by using the Decorator and Factory patterns or never get behind in a project by sticking to Inbox Zero, our jobs would be a lot easier.

However, we certainly cannot. Our primary job is to identify problems and then develop solutions for them, whatever the techniques along the way. This is why we sometimes struggle to identify a clear methodology that we use or a strict definition of what we are looking for in an employee. On the flip side, this is also what makes our work so interesting and rewarding.

Good Job Code Camp

2008-07-16T05:39:59Z

A few weeks ago, I attended 2 sessions of the SoCal Rock & Roll Code Camp at the USCD Extension campus. This is an event I’ve attended before and always enjoyed. This year was no exception, as the event was well organized, the schedule was well communicated and the two presenters I saw were good.

The two sessions I attended were Integrating Silverlight 2.0 into an Existing Web Application and Using ASP.NET MVC to Build a Blogging Engine in 60 Minutes or Less. The Silverlight presentation covered some general ideas on how to take an web application you already have and add Silverlight content to it. In particular, I appreciated that the presenter, Robert Atland, emphasized thinking carefully about where to use Silverlight so it is not just a gimmick, but actually enhances the user experience.

The ASP.NET MVC presentation was more of a mixed bag. I didn’t start my programming career in .NET, instead I started in Java where MVC is a much more common paradigm. So MVC as a concept is not new or difficult for me, especially after spending some time recently with Ruby and Python web frameworks. For ASP.NET MVC, Scott Gu provides a brief overview here.

However, a lot of the attendees were new to MVC and used to .NET’s postback model. I think this tripped up Adnan Masood, the presenter, and he spent most of the session valiantly trying to explain the Model View Controller architecture, as well as defend the idea that anything could be better than the postback model. Nevertheless, I thought he did a good job and the ASP.NET MVC framework is worth checking out.

More recently, somewhat inspired by this positive experience, I attended a user group meeting for a non .NET language and had quite a different experience. There is no point in me going into great detail, but the group, although full of what seemed to be very nice people, was incredibly disjointed and vague in it is direction. Almost no content was presented and no particular agenda was followed.

I was more excited about the content I thought would be presented at that user group than the content at the code camp, but the code camp was a far more satisfying experience. So kudos to the folks at the SoCal code camp; it is tough to carve out time outside of work for activities like this and they made me feel like my effort was more than reciprocated.

Tongue Out of Cheek: A Follow Up To Learning Python and Getting Distracted

2008-06-10T05:58:13Z

My previous post, How To Get Really Distracted and Not Learn Any Python, generated enough interest (or at least comments) that I wanted to follow up with some comments of my own.

I have been learning Python in my spare time for quite a while and I am no expert, but I am at least fairly comfortable in it by now. I was reflecting back on what I saw as some missteps I took along the way. The mistake I made was to try and take on too much at once. Instead of just trying to learn Python, I was mixing in learning Linux, a new IDE, a new source control system, etc. All of these are certainly good things to learn, but learning them was not essential to my main task of learning Python.

For the curious, I found the following to be useful activities for actually learning Python:

Read the official Python Tutorial – I’m not kidding, the official docs are good
Study Core Python by Wesley Chun – The weight of this book may damage your wrist, but the quality of the information is superb. I read a chapter at a time, then reread the chapter and make notes
Download and install Django, complete the first tutorial and then try to trace the flow of the code so you can see where the default Django page comes from – I used Wing IDE Personal edition to set up breakpoints and step through
Read and try to understand the code in a Python standard module – for example, if you are looking at Django or other web frameworks, take a look at the HTTP/TCP/SocketServer family of classes

And MOST importantly, write some code! Writing code is best because it is so humbling; you are not just nodding along to an example in a book, you have to be a creative participant. Sometimes though, it can be tough to know what code to write. To avoid “coders block”, I need to have a goal to work towards. Things like the Django tutorials are a good place to start if you have no ideas. However, things really started clicking for me when I came up with a project idea: write some Python code to read barcodes from books/DVDs via a webcam and then use the resulting barcode to look up information about the book and store that information somewhere; basically a Delicious Library knock off. Kind of silly, but motivating.

And speaking of silly, but motivating (this is begging for a sequel), I present (for the second time) the following Erlang video: on Google video (with longer introduction) and YouTube (slimmed down version). The pinnacle of the driest humor or just unintentionally hilarious? You decide.*

* In anticipation of angry comments from Erlang partisans, I would note that for the last paragraph of this post tongue was back in cheek.

How To Get Really Distracted And Not Learn Any Python

2008-06-06T04:45:00Z

The following is a graphical breakdown of how I spent the first 40 hours or so “learning” Python, plus an excuse to show off the Google Visualization API:

As you can see, based on these extremely scientific measurements, I spent the majority of this time not writing Python code, reading about it or watching screencasts, but instead fiddling around the edges. These are important edges; as a .NET programmer by trade, I know well the importance of a good IDE. Source Control is a given, while Test Driven Development (TDD) is a staple of many non-corporate programming environments. And of course you cannot write any code if your environment is not set up correctly.

So why do I call this time spent “fiddling”? Because of the way I approached it. I took having a good IDE to mean finding a Visual Studio for Python. This meant installs and brief trials of Wing IDE, IronPython Studio, Eclipse and E-Text Editor. TDD was slow sledding when the test was really writing anything that worked. Environment setup became an brief foray into Linux followed by endless tweaks of Windows settings to come up with the perfect Python environment.

I approached getting started with Python as I would setting up a new machine for .NET development, except I have years of experience with .NET and know exactly what I want, whereas I was grasping in the dark with Python. Once I got past these obstacles and got rolling (in Vista, with SVN and Eclipse for the curious), I actually began making some progress. While I learned a lot from the process, it was also needlessly frustrating. If learning Python was my number one goal it would have been better to just take others’ advice, get set up by following a laundry list of instructions and try to end up with a more distraction free allocation like this:

Of course, far be it for me to take my own advice, and I have not been able to resist taking some time to brush up on my JavaScript, which lead me to the Google Visualization API, which is responsible for the oh so orange graphs in this post. The Visualization API is one of many JavaScript libraries which Google hosts on its servers, allowing you to load them onto your own page hassle free. Not all the libraries are Google-made, the impressive jQuery is another library you can import off of Google’s servers. I did not do much more with the Visualization API then you can see in the “Hello World” examples, ~~but the magic is there in the source of this page if you are curious.~~ <Updated post to display better in RSS readers>

The Netflix Player: The Tivo of Streaming Content?

2008-05-25T00:39:17Z

(This is not a gadget blog, but occasionally a new piece of technology is interesting enough that I want to write about it.)

The Netflix Player is Roku's 5" deep by 5" wide by 2" tall set top device that allows Netflix subscribers to easily watch any "Instant Watch" content on their TV. I mentioned the Instant Watch feature before, and I have been enjoying it, watching shows and movies on my laptop or occasionally hooking my laptop up to my TV.

I was enjoying it so much that I was about to buy a wireless keyboard and mouse to make using the hooked up laptop easier when I saw that the Netflix Player had come out. For $100 that's not all that much more than a decent wireless keyboard and mouse, so I decided to give it a try.

I've only had this device for a few days, but my initial impression of it is that this is a breakthrough device, with the possibility to do what Tivo did for DVRs, but for streaming content.

The initial setup is easy; the Player has built in wireless or you can use a wired connection. The Player has most any audio/video connections you would want; I am using HDMI for the picture and regular old RCA cables for the audio.

Once you've the Player hooked up to your TV and plug it in, it boots up and asks for a 5 digit activation code that you can get from the Netflix web site. Enter the code and the device jumps right into a nice display of the contents of your Instant Watch Queue. Using the small remote I had no trouble moving through what was available and selecting what to play. I had to tweak the display output to be widescreen in the configuration menu (up arrow from the main menu), but beyond that I was up and running in under 10 minutes.

Once something has started playing, you can fast forward and rewind or skip around to different parts of what you're watching. Because it is streaming, Roku came up with an interesting "chapter" system where you can skip around in 5 second intervals, demarcated by a screenshot of the action at that point in time. TV show episodes of the same series are grouped together logically so you can easily watch through a series in sequence (tested officially via season 1 of 30 Rock.) To summarize, navigation is easy and usability is high: it passed the "non-geek user" test with flying colors as my wife needed no help from me at all when she tried it out.

2 limitations of the Player are that it only shows you content that you have placed in your Instant Queue, via the Netflix web site and it currently has no HD content, although that is on its way. The playback quality is not outstanding, but it is definitely better than regular TV and around DVD quality.

These limitations do not detract from the appeal of the Netflix Player. In fact, based on the easy setup, relatively low cost and excellent usability, the Player is in good position to do what Tivo did for DVRs, which is to not be the first to do something (in this case streaming content), but the first to do it in a way that is intuitive and easy for mass consumption. Netflix's millions of subscribers have a real winner on their hands.

OS Chasing: The Usability Gap

2008-05-02T15:55:38Z

I've written many times before about my trial runs of various Linux distributions and I feel like I've said most of what I have to say about the matter. However, I recently saw a post entitled, Is Ubuntu Useable Enough For My Girlfriend? and I decided to have one more go at explaining my ambivalent feelings towards Linux.

First, a brief summary of the post for the link-impaired; relatively experienced Linux user sets up his girlfriend (Windows user, not super technical) with an install of the latest version of Ubuntu, gives her a set of mundane tasks (watch a YouTube video, Google something, do some photo editing) to achieve while he takes notes and hilarity ensues.

I tend to cringe when highly technical users try to predict what a non-technical user will find easy or difficult; there is just so much background knowledge that someone who is passionate about computers takes for granted, that I think we are uniquely ill suited for the task. So, I try to rely more on colloquial evidence, such as this post.

His girlfriend is able to accomplish many of the tasks, but is stymied by a few, mostly due to poor naming or lack of clear instructions on how to install the necessary application or plug in. The conclusion is that Ubuntu is heading in the right direction, but is still not ideal for your "average" non-power user.

This line of argument has been around since Linux on the desktop became a reality, so at first glance it can seem old, stale, whiny or just plain obvious. If you get beyond that initial reaction, it is an amazing commentary on the state of Linux on the desktop. How is it possible that so many talented individuals have labored so long and yet still been unable to produce a desktop system on which these type of basic tasks are as straightforward as they are on Windows or Mac?

I'll answer that question with another question; could you see Ubuntu declaring a one year new feature freeze, refusing any new graphical bells and whistles, new processor extensions or file system upgrades and instead focusing the next release of Ubuntu entirely on making it as simple as possible for non-technical users to accomplish simple general computing tasks? Imagine the brain power of that many talented contributors all working just towards usability. I think you'd have your killer desktop in way under that one year timeline.

However, my question is flawed. What is Ubuntu? There is an organization around it for sure, but it is not a monolithic one that can give orders to the entire Linux development community. Also, while usability is a stated goal of Ubuntu, it is not necessarily a goal that is shared by many of the elite programmers in the Linux development community. Or more accurately stated, many elite Linux developers are quite comfortable in Linux and don't feel excited about spending time making it "easier".

Quite to the contrary, many technical people are attracted to Linux due to the rapid pace of technological advancement. So in essence, you'd be asking the very people who innovated so much to get Linux to where it is now to stop innovating and focus on the unglamorous task of noob babysitting. Or as the author of the post puts it, "The main issue with the desktop experience is that the geeky programmers and designers assume too much from the average user."

In fairness there are plenty of developers who buck this stereotype and pour effort into usability. Otherwise we wouldn't have distros like Ubuntu at all. Recognizing the good work that the Ubuntu (and other distros such as PCLinuxOS and Xandros) have done with usability, what's left to explain the persistent gap?

I think the often overlooked part of this discussion is that Windows and Mac (OS X) set the norms. Whether or not it is fair or just, most people are exposed first and most often to Windows or OS X and from that initial experience, their expectations are set for how interacting with a computer should take place.

This behavior can be maddening and seem foolish, but is not surprising, at least as explained as Dan Ariely describes in his book Predictably Irrational. Ariely calls this type of behavior "anchoring" and anchoring can explain such oddities as how black pearls suddenly became sought after (they were once relatively unknown and then were introduced first only in high end jewelry stores for high end prices) and how we can feel like we are getting a great deal on a car when we buy it for significantly below the MSRP, despite the MSRP being a somewhat arbitrary number.

So, to come back on topic, even if desktop Linux were easier to use based on some kind of scientific figuring, as long as it remains close enough to our anchor ideal of computer interaction without quite being the same, it will seem inferior and less desirable.

The baseline has to be that either desktop Linux does at least everything as easily as Windows/OS X does plus more or that it presents such a different and satisfying experience that a new anchor is established. A concrete example of this is the iPhone; by offering such a tightly knit and innovative take on the interface of a cell phone it was able to define a new standard for what cell phone UI's should be.

There is movement within the Linux community to develop new desktop interface experiences such as Symphony One and mainstream distros using KDE and Gnome are inching ever closer to matching the out of the box Windows/OS X experience while still offering the power and variety that make Linux fascinating. I am not sure which approach if either will ultimately win out because the final elephant in the room is that Microsoft and Apple are not just standing still, they are trying to take the best bits they see from their competitors and improve their own products.

Notes and notes...

2008-04-24T06:26:04Z

After 2 months of dormancy, I am starting back up the easy way, with some short notes on topics of interest:

1. Windows Home Server: 2 Months In

The best compliment I can give is that I just don’t think about my Home Server anymore. I let it do its thing and don’t worry about it. I am anxiously awaiting the Power Pack 1 additions, along with the data corruption bug fix (which has not affected me at all yet). In hindsight, I should have just gotten a prebuilt solution, avoided the install pain and had a machine with a smaller footprint, but my custom build does get the job done.

2. Microsoft OneNote: Better than TXT?

I’m still using OneNote at home to take notes on Core Python Programming (my current Python resource of choice), but at work I’m back to using mostly NotePad2 and other plain text editors. Even though OneNote does not fight you as much as Word, it still has the annoying AutoCorrect behaviors that sometimes make plain old text much less of a hassle. The feature to publish your notes to a web page format is nice, although Microsoft’s decision to use the obscure .MHT format is confusing. So after a few months of use, my opinion of OneNote is far more mixed.

3. Netflix Impresses

One of my favorite new technology toys is Netflix “Watch Instantly” feature. The quality is very good, and although the available content is mostly TV shows, seeing what you want, streaming, when you want it, is really an amazingly satisfying experience. If Netflix can integrate this into a set top box, watch out.

4. Indexed

I was going to draw a clever graph showing the inverse relationship between work load and my blogging frequency, but it seemed too obvious, so I’ll just plug the real Indexed site, which is far cleverer. I have been exploring the Python web framework Django, so work load non-withstanding, I hope to share the rare sight of some code on this blog in the near future.

Office's Undervalued Prospect: Microsoft OneNote

2008-02-25T00:12:14Z

If you use Outlook, Word and Excel frequently, there is a good chance you actually have the entire Microsoft Office suite, which includes some well known applications, like PowerPoint and some not so well known ones, such as Microsoft's OneNote. OneNote has been around since 2003 (the year and MS Office version), but I only started using it when I installed Office 2007.

OneNote is, as the name suggests, primarily a note taking application. Free form pages are organized by the note taker into notebooks. Notes are not just text, they can be images, links, sounds and videos. Notes and notebooks can be published as PDF documents and shared with other OneNote users. This may not sound like the most exciting application, but I've found it to be a surprisingly effective way to keep information organized.

I use OneNote (in addition to a paper journal) to organize ideas and research for blog posts. I use it at home with my dual monitor setup to take notes while watching screencasts and working my way through programming books. At work, I'm experimenting with using it to organize the many small notes to myself I make about projects that usually end up on notepads and then in the recycling bin.

OneNote's interface features tabs across the top for notebook sections, with tabs down the side for pages within the notebook, making it natural and quick to create a nested structure for the information. OneNote also includes its own screenshot tool to facilitate capturing snaps into notes.

Finally, when I open up OneNote it doesn't come with the heavy mental baggage that Word does. I've used Word for years and years, so when I open it up, I think of essays, reports, specs; pretty much all more formal documents. OneNote encourages me to be informal, which is what these notes should be. OneNote also fights against me far less often than Word does; so far I've yet to encounter the frustrating Word behavior where Word just won't let you type in anything until you adjust your formatting.

Long time readers of this blog will remember that I once discussed my somewhat unique diet of more, but smaller meals during the day. The book that explained this relies heavily on sports analogies, one of which is the "undervalued prospect", used in the book to highlight certain foods which lacked the reputation for being healthy, but were actually great things to have in your diet. I think the undervalued prospect is a good analogy for OneNote's place within Office. I never really see it get hyped, but if you take the time to get to know it, OneNote has a lot to offer.

The Fastest Web Form Out There

2008-02-17T22:28:15Z

Recently Google Docs added a Forms feature to their spreadsheets. This feature allows you to generate a form that can be filled out via email or on a page hosted by Google. The submission of this form then populates the data back into your Google spreadsheet where the responses can be collated online or exported as an Excel spreadsheet.

I think I just made that sound more confusing and difficult than it is; it takes under a minute to set up, and you can see my "Ice Cream" sample form by following this link: http://spreadsheets.google.com/viewform?key=p4Wm_X_NqY4oWmEuN8g25YA.

This functionality has a lot of potential and is a great tool for creating small and simple surveys. It exists in a similar space to Microsoft's InfoPath, but is much simpler and being Google, free. Speaking for lazy developers (is there any other type?), it is certainly not a bad thing to let non-technical users create and manage these kind of forms and save the technical users' efforts for more complex tasks. Or trying to speak from the non-technical users perspective, this is great since you can accomplish these seemingly simple data collection tasks without dealing with any irritable tech folks. [1]

[1] Note: As a software consultant I am obligated to point out that we never fit into this "irritable tech folks" category :)

Windows Home Server: Install Impressions

2008-02-17T21:30:27Z

Following the recommendation of my co-worker Mike Frank, I decided to give Windows Home Server (WHS) a try. WHS has the capability to do a lot of different things, but straight out of the box I was looking to take advantage of its automated backup capabilities, data duplication, file sharing and remote access capabilities.

I have only had WHS installed for a few days, but my first impressions are very positive. I don't know what I expected, but it just works. After I got it installed, setting up the connector software on my machine was easy and I was immediately setup for nightly backups. File sharing was also quick to setup and after a few well explained configuration changes to my router, I had a web site up so I could access my shared files and computers from anywhere with an internet connection. There is a lot more I want to do, including exploring Xbox 360 integration, putting a source control server on it and installing some of the many user created plug-ins, in particular for media management.

On the install side, I did run into some trouble. I built my own system, which is not the route that most consumers would probably take as there are a lot of solid looking pre-built options out there. I did some research beforehand to find parts that were compatible, but still had a problem getting my SATA hard drives recognized.

The first part of the install went smoothly, but then there is a part of the install that looks like it uses the Windows XP text based installer which does not have built in drivers for SATA drives. So, I was confronted with having to do the F6 "provide additional drivers" step, which means I had to have drivers on a floppy disk.

Since I have built a fair number of systems, I have a floppy drive (which sadly is still an essential tool for a system builder). However, I didn't have any other computers around that I could install the drive on to load a floppy disk with the SATA controller drivers. The motherboard came with a CD of the drivers, so I could take the drivers off there to make a floppy, but I needed a machine I could boot into to do the copying. So kind of a chicken and egg problem since the server machine was not in a bootable state.

My solution was to reboot the server off of the install media for WHS and then pretend to load additional drivers in the first GUI based part of the installer. This part of the installer uses standard Windows file dialogs to let you search for additional drivers. Standard Windows file dialogs let you right click, copy and paste files, so I used this dialog to navigate to the drivers (on a USB key) and then copied and pasted them to the floppy disk which I had hooked up to the server. I then canceled out of that dialog, got back to the text installer and was ready for the F6 to floppy disk, which worked perfectly. So, that was annoying and more time consuming than I care to admit, but beyond that the install was straight forward. I will wait until I've had the server up and running for a while before I give WHS a full review though.

Graham's Law of Power

2008-02-09T21:03:45Z

Because making programs short is what high level languages are for. It may not be 100% accurate to say the power of a programming language is in inverse proportion to the length of programs written in it, but it's damned close.
Paul Graham, "Take the Arc Challenge"

By definition, there is some optimal path from axioms up to a complete language. The Platonic form of Lisp is somewhere inside the block of marble. All we have to do is chip away till we get at it.
Paul Graham, "First Priority: Core Language"

Paul Graham, Lisp guru, founder of the startup company incubator Y Combinator and essayist, has always been somewhat of a lightening rod for controversy. He is highly opinionated and has a wonderfully concise and direct style of writing that does not back away from confrontation. So it was not too surprising to see the relish with which his detractors have criticized Arc, the language he invented, since an early alpha version was released recently.

Arc is a dialect of Lisp, with a focus on "exploratory programming" or rapid prototyping. A distinguishing feature of Arc is that it was/is built from the ground up with a focus on terse syntax that leads to short programs. As the first quote in this post illustrates, Graham believes that power is inversely proportional to program length. Program length is not equal to character counts necessarily, rather it is based on the size of the tree you'd need to represent the program. So, the first example from Arc is a program that displays a web form with an input field that parrots back what you type in when you submit it:

(defop said req
  (aform [w/link (pr "you said: " (arg _ "foo"))
           (pr "click here")]
    (input "foo") 
    (submit)))

This example has 23 nodes according to Graham's counting technique and he challenges anybody to produce a shorter program using a different language.

I'm sure there will be no lack of responses to this challenge, but I find the whole premise to be flawed. The idea that a language inherently has a measurable amount of "power" in the first place is suspect. Having power means to possess controlling influence. In the case of a programming language what would it have influence over? The answer could be loosely summed up as the domain of the problem.

To be more specific, if you are writing a telecommunications system, Erlang would be a very powerful language as it grew out of that domain; however, if you were creating a rich client side UI, you might use C++ or C# with WPF. Because programming languages are used to solve such a wide swatch of problems, it is impossible to have one controlling definition of power. Power for a programming language is completely dependent on context.

The contra-argument to this is that I'm describing the libraries of languages, not their core syntax, structures and features. I would argue that you cannot evaluate a language outside of its libraries; even the definition of libraries is difficult, as in the case of Arc where a "core" language feature is a construct for creating web forms. This might be called a library in another language, but if that language decided to incorporate that library into the core, then would that language suddenly become more powerful?

To be fair let's say we ignore the problems with using the term power and accept the premise that certain languages will in general make a programmer more able to solve the problems they encounter. Graham's premise is that brevity is the key to achieving this competitive advantage.

Why would brevity be such an advantage? Less boilerplate/setup code, fewer things for the programmer to keep in their mind; these are what I could think of. I don't find either of those reasons all that compelling. Certainly attractive and nice to have, but not enough of a reason to ditch one language for another. Unfortunately, Graham seems to find the idea pretty much self evident and does not offer much in the way of supporting evidence.

Since Graham is obviously a smart guy, I got to thinking about why he would be so sure that brevity is the key component in designing a powerful language. I looked back at the example of Arc and was struck by the similarities to Shoes, a domain specific language in Ruby for creating GUI's. I spent some time creating an extremely primitive flash card program in Shoes and really enjoyed it. Here's an example of the syntax to create a simple application with a button and an attached click event:

Shoes.app {
  button("Press Me") { alert("You pressed me") }
}

So Arc and Shoes possess similar compact syntax and Arc's aform construct looks like a DSL for creating web apps. Pushing the comparison further, Shoes is written in Ruby, a language strongly inclined to produce DSLs, which was strongly inspired by Lisp. And here we come full circle to what I think is the core of Graham's argument.

Graham's affection for Lisp is well known; Arc is a dialect of Lisp and he explicitly states (as shown in the quote above) that he set out to design Arc with the goal of creating the true and pure (or Platonic) Lisp.

So when Graham is saying that succinctness equals power, he is not saying anything he hasn't said before. This is really another front in the long running argument between Lispers and advocates of other languages, where Lisp, for a laundry list of reasons (including the power of macros to make short programs), is held up by its advocates as the one true language and by its detractors as a jumbled mess of parentheses and arcane incantations.

So Graham's Law of Power could perhaps more clearly be stated as the power of a programming language is directly proportional to its "Lispness." Or the more vulgar form that every other successful programming language is really just a poor implementation of Lisp.[1]

[1] I disagree with that argument, but if you are interested in hearing more, allocate a few hours and a couple of Google searches and you will find more than enough material from both sides to keep you busy.

What a marvelous predicament!

2008-02-01T14:17:00Z

Fred Brooks' seminal work, The Mythical Man Month, is rightly considered a classic on the topic of software development. Although it was originally written over 30 years ago, it is still a remarkably insightful and interesting read today.

The apparent timelessness of Brooks' work is surely due to its focus on managing and improving the software development life cycle, not on specific language techniques. Brooks' anticipates some of the tenants of extreme programming by suggesting rapid prototyping, extensive use of test cases and continuous customer interaction. He is optimistic about the benefits of object oriented development despite some initial reservations to the idea of keeping the code within components hidden from plain view.

He discusses the difficulties of estimation, the utility of keeping a project workbook (kind of a proto-wiki) and suggests a "surgical" team structure to maximize productivity. And of course, he presents and later defends the so-called "Brooks' Law", that adding people to a project that is late will only make it more late.

In the epilogue to the 20th anniversary edition of The Mythical Man Month, Brooks discusses how it has become impossible to keep up with all the new developments within the software industry:

The computer related intellectual discipline has exploded as has the technology... Today my intellectual life has seen me regretfully kissing subdiscipline interests goodbye one by one, as my portfolio has continuously overflowed beyond mastery. Too many interests, too many exciting opportunities for learning, research, and thought. What a marvelous predicament!

While marvelous, this is a predicament and as such, begs a solution. Brooks' hints at one when he discusses the difference between the essence and the accidental in software development in the chapter, "No Silver Bullet". Briefly stated, the essence of software development is the modeling or description of a problem into the software realm, while the accidental is the actual implementation of the essence (or specification) into working code. Brooks' estimates that 90% of the effort in a project is the essence. And yet we spend a lot of time trying to maximize the 10% of the accidental.

Based on this ratio, if object oriented programming can streamline the 90% of the essence by providing a good conceptual framework to plan a product around, then it is a worthwhile addition; however, writing a program using an object oriented language without following object oriented concepts is not in and of itself an advantage.

Let's return now to our marvelous predicament. It is almost tautological at this point that one developer cannot possibly master all their is to know within the field. Yet, most developers are by nature are very curious (a point Brooks makes early on) and so they will want to keep learning new things. So some selection criteria must be used to winnow the available choices. Looking to Brooks, a good criteria might be to concentrate on learning new things that let you more effectively attack the essence of software development, not the accidental.

Earth Toned Threads - .NET Parallel Extensions

2008-01-29T07:11:00Z

Back in the pre-Facebook era, there was a certain political figure who has accused of suddenly dressing in earth tones on the request of an advisor who thought that these muted colors were reassuring to audiences. This plan, especially after it was revealed, did not have the desired effect.

However, code is by nature easier to dress up than a person. And with the .NET Parallel Extensions, released as an add-on to the 3.5 .NET Framework as a CTP in December, Microsoft is attempting to take the edge off of threads by clothing them in an extra layer of reassuringly subdued abstraction.

This weekend, I attended a session of the SoCal Rock & Roll Code Camp at Cal State Fullerton about these newly released extensions. The presenter, Rinat Shagisultanov, gave a quick overview of what was in the extensions, as well as some working examples. There is a lot in there, so check it out yourself for all the details, but what really stuck out to me was a point Rinat made early in the session. To paraphrase, his point was that Microsoft already gave us the Framework with its threading capabilities and this extension did not change this, rather it gave us some nice building blocks on top of the existing Framework to make writing the code less intimidating.

The curious might ask, like what? The most basic example would be the addition of the Parallel.ForEach construct , which can be used to quickly transform a strictly sequential execution path into a parallel one. This construct gets a little more complicated when you are actively working with shared state, but the basic idea is that the Parallel Extensions spins out a thread for each core and distributes the work between the cores without you having to write the plumbing code. There is also PLINQ (Parallel LINQ) for querying objects in parallel, as well as the Task, Future and TaskManager objects which allow for finer grained control of the distribution of items of work in parallel. (1)

I won't get any further into the details of the API's at this point (although perhaps I will in a later post) because I want to return to the idea these extensions are really less intimated or earth toned threads. Threads could use the help, as they are pretty universally regarded as one of those "hard" topics in programming.

Lots of programmers cringe when they see code in an application that starts spinning out threads. They begin to imagine the nightmares of debugging seemingly random errors and designing the appropriate access schemes for shared memory. It is somewhat a badge of honor to brave the depths of multi-threading and come out with an application that is reasonably stable.

If threads are to be one of the prime mechanisms for parallelism and the importance of parallelism is increasing with multi-core machines, isn't it bad that they are perceived to be so difficult and obtuse? One answer to that question is that they are not really all that hard and most programmers are just stupid. I call this the unintentionally dismissive guru stance; once you've figured something out, you cannot really remember how hard it was for you to do it.

Another answer is that they are hard, but that's OK, because threads are too powerful and dangerous for the average programmer. This is the intentionally dismissive grizzled vet stance; you've earned your stripes working on multi-threaded applications and you don't need some newbie bumbler coming along and fouling things up.

The answer that Microsoft has given with the Parallel Extensions is yes, threads hard and yes that's a problem, so here's some tools that give you a chance to get the parallelism you want (using threads under the covers), but only dive as deep into the intricacies of the threads as you absolutely have to. I call this the populist approach; give something to the masses (even if it's not perfect), at the possible expense of being looked down upon by the elite.

I have not gotten a chance to dig deep into the Parallel Extensions yet, but apparently it is due to be incorporated into the next base distribution of the Framework and what I have seen looks really promising. These extensions, especially when you deal with shared state, will still open up some dangerous traps for the unwary, but they lower the bar for entry and provide some nice base abstractions to work with. In fact, coupled with WCF services, you can see the outline of a .NET grid computing architecture with multi-threaded services distributed across multiple machines pretty easily out of the box. (2) So do yourself a favor and add this CTP to your long list of new parts of the .NET Framework to learn.

(1) Rinat gave a more comprehensive breakdown of what these different API's are and what they could be used for, so any confusion you may experience in reading my description is of my own doing. The slides and code he presented in the session are on his blog at: Code Camp, Fullerton, CA Jan 2008: Concurrent Programming in .NET (Parallel FX)

(2) Some would actually argue that you should not really use multi-threading paradigms anywhere for concurrency; everything should be based on message passing between processes and thereby avoid the problems of accessing shared memory. That is an argument I am not prepared to take on, but for a strong version of the pro-argument see the language Erlang, which is built around the principles of no-shared memory, processes and message passing.