A few days ago my attention was drawn to a tweet spat between Karl Seguin and Scott Hanselman around the relaunch of ASP.NET and the title element in HTML. Tempest in a teapot of course, but worthwhile as I did some googling on Karl and found his blog at codebetter.com.
From there it was a short jump to his free e-book, The Foundations of Programming. This short book is distinguished by its orientation, opinionated, its tone, mentoring and its honesty, which is refreshing. In Foundations, Karl covers what he considers the basics of programming and good design, including test driven development, dependency injection and domain driven design.
Karl is opinionated, as the topics suggest, and doesn’t bother to pretend that he doesn’t think what he’s suggesting is the better way, not just another way. He is aligned with ALT.NET, and gives an excellent overview of what that means; an overview more enlightening than the ALT.NET site. ALT.NET has its critics, but presenting a strong opinion grabbed my attention as a reader.
It is a short walk from opinionated to hectoring, but Karl held my attention without insulting me. He takes the time to explain, with examples, from the ground up, the problems that test driven development and dependency injection solve. So for dependency injection he builds it up from no DI, to a hand crafted approach, to a full fledged DI framework. This approach is more persuasive than just proscriptive and engaged me as the reader to follow along with his train of thought.
Foundations is not as pedantic as I am making it sound. The final ingredient in Karl’s mix is honesty. He acknowledges that sometimes unit testing does cost more up front and take more time. He admits that sometimes he designs something a certain way just to be testable. He also warns that focusing too much on DI and loose coupling can lead to the poor design you are trying to avoid. These points add depth to his argument as I could tell he’s speaking from experience, with some hard won lessons.
I enjoyed The Foundations of Programming. When I was done with it, I was amazed how much I got a lot out of its 80 some pages. It is a rarity to come across something worthwhile that is longer then a tweet, but shorter than a tome these days. Well done Karl.
-- Relevant Links --
The now titled and newly relaunched page in question: http://www.asp.net/
The pleasantly confusing ALT.NET homepage: http://altdotnet.org/
A longer review, with details, chapter listings and all that important stuff: http://accidentaltechnologist.com/book-reviews/book-review-foundations-of-programming-by-karl-seguin/
In my last post I discussed how some creative thinking on my part led me to redo large chunks of a task scheduling library. As promised, here are some of the major changes and what I learned from them.
Lush Verbiage
In the refactor, I expanded upon the concept of a timekeeper and extended the metaphor to the rest of they component. So a timekeeper, keeps time, or in this case, a timetable. A timetable is composed of entries, a combination of time slot and actions scheduled for that time. A timekeeper has a worker which “works” the timetable using a simple procedure. Find the next entry in the timetable, keep track of the passage of time until it is past that entry’s time, execute the actions for that entry, find the next entry, continue on.
The metaphor is strained at times, but by carefully choosing descriptive names for classes from the physical world, there was less mental friction when trying to implement or explain the component.
Design Simplicity
Dealing with time based code in .NET is harder than it should be. Or more accurately, harder then I’d want it to be. Even the conceptually simple idea of scheduling a task to happen every hour is fraught with peril. I started with a Timer that went off at the next hour and repeated with a period of 60 minutes. This mostly works, but that timer will sometimes fire before the hour is up. Only a fraction of a second before the hour, but that is time enough to wreck havoc on logic that looks for the next event chronologically or on other components that depend on the hourly events occurring on or after the hour. If not havoc, at least make it messier.
Ultimately, I could have gotten past that challenge, but I decided to simplify the code by making the main “tick” loop of the timekeeper (what looks for timetable entries that need to be executed) a brute force style thread.sleep, check, thread.sleep, check, loop. The idea of this approach grates on me as it is making the program spin more then it needs to, but a sleep of 100 ms was enough to not noticeably affect CPU utilization. In addition, this made it very easy to add support for a clean stop of the timekeeper worker:
internal class TimekeeperWorker : BackgroundWorker
{...
protected override void OnDoWork(DoWorkEventArgs e)
{...
while (!CancellationPending)
{
Thread.Sleep(100);
… “tick” loop logic
}
//cancellation pending, so exit out
e.Cancel = true;
Even after this latest refactor, there are more places I could simplify this code. However, just like optimization, there is an effort associated with simplification that isn’t always needed. At least not yet.
Daylight Savings Time
This is kind of a subset of Design Simplicity, but one of the concerns I had with this component was how it would handle Daylight Savings Time. When the clock jumps from 1:59AM to 1AM or 3AM for example. After some discussion with the team, the behavior we decided on was that in the case of jumping back, the timekeeper would fire the 1AM action twice, jumping forward, the timekeeper would fire the 3AM action, skipping any 2AM actions.
With that expectation set, how to implement that? I debated using the TimeZone and TimeZoneInfo classes to get to the DaylightTime objects and figure that all out. After scratching my head over that for a while without making headway, I decided to again take a bit of a brute force approach in that timekeeper worker’s “tick” loop:
Thread.Sleep(100);
timeNow = TimekeeperClock.Now();
if ( Math.Abs( (timeNow - lastTime).Ticks ) > TimeSpan.FromMinutes(1).Ticks)
{
//if the time changes more then one minute (plus or minus)
//after a 1/10 of a second sleep, then
//we've had some kind of major clock adjustment
//(potentially daylight savings time)
//so recalculate what the next entry would be
//have to back up the time a bit, in case it went from
//1:59:59am to 3:00:01am on dst change
_logger.InfoFormat("Clock drift detected from {0} to {1} in one cycle.",
lastTime, timeNow);
DateTime lookBackTime = timeNow.Add(TimeSpan.FromMinutes(-1));
_logger.InfoFormat("Recalculating next entry based on a time of {0}",
lookBackTime);
_next = _timetable.GetNextEntry(lookBackTime);
}
The worker knows the last time it got from the clock, so if the next time it gets is significantly different, something has happened. The worker assumes it is a daylight savings time adjustment and recalculates the next timetable entry from the new time. In order to actually catch the hour changing, that next entry is actually calculated from the new time minus a minute. A bit goofy, but workable, and easier then dealing with the .NET time zone classes.
Testing Time
Testing time based code is tough. I found it useful to just let the component run for days in our test environment and watch the log, but that’s not particularly efficient. So I wrote some automated tests for the timekeeper, for example, to test that the daylight savings time behavior was what I wanted:
[Test]
public void Should_Execute_One_AM_Task_Twice_On_Jump_Back_DST()
{
var tt = new Timekeeper();
int numberTimesExecuted = 0;
var incrementTimesExecuted = new Task
{
Execute = dt =>
{
numberTimesExecuted += 1;
return new TaskResult { Message = "Incremented" };
},
FriendlyName = "Increment Times Executed"
};
tt.ScheduleFor(new TimeSpan(1,0,0), incrementTimesExecuted);
TimekeeperClock.Now = () => new DateTime(2010, 3, 14, 0, 59, 59);
tt.Start();
System.Threading.Thread.Sleep(200);//let timekeeper tick
TimekeeperClock.Now = () => new DateTime(2010, 3, 14, 1, 0, 01);
System.Threading.Thread.Sleep(200);//let timekeeper tick
TimekeeperClock.Now = () => new DateTime(2010, 3, 14, 1, 59, 59);
System.Threading.Thread.Sleep(200);//let timekeeper tick
TimekeeperClock.Now = () => new DateTime(2010, 3, 14, 1, 0,01);
System.Threading.Thread.Sleep(200);//let timekeeper tick
Assert.AreEqual(2, numberTimesExecuted);
tt.Stop();
}
[Test]
public void Should_Execute_One_AM_And_3_AM_And_Not_2_AM_Task_On_Jump_Forward_DST()
{
var tt = new Timekeeper();
List<int> hoursExecuted = new List<int>();
var addHourToExecuted = new Task
{
Execute = dt =>
{
hoursExecuted.Add(dt.Hour);
return new TaskResult();
}
};
tt.ScheduleForEveryHour(addHourToExecuted);
TimekeeperClock.Now = () => new DateTime(2010, 3, 14, 0, 59, 59);
tt.Start();
System.Threading.Thread.Sleep(200);//let timekeeper tick
TimekeeperClock.Now = () => new DateTime(2010, 3, 14, 1, 0, 01);
System.Threading.Thread.Sleep(200);//let timekeeper tick
TimekeeperClock.Now = () => new DateTime(2010, 3, 14, 1, 59, 59);
System.Threading.Thread.Sleep(200);//let timekeeper tick
TimekeeperClock.Now = () => new DateTime(2010, 3, 14, 3, 0, 01);
System.Threading.Thread.Sleep(200);//let timekeeper tick
Assert.Contains(1, hoursExecuted);
Assert.Contains(3, hoursExecuted);
Assert.False(hoursExecuted.Contains(2));
tt.Stop();
}
Pretty, these are certainly not. The lowlights include the numerous thread.sleep calls, which allow the timekeeper’s background worker to catch up and notice that the time has changed. The inline definition of tasks also ends up taking up a lot of screen real estate. Finally, the setting of method local variables from the action delegates seems clunky.
With all that said, these tests do make me feel more confident that the timekeeper will behave as expected, and furthermore express the expected behavior of the component to anybody else who reads them.
The reliance on using thread sleeps really bothers me; so far the only way around it that I could think of was to extract the main loop logic from worker into a pluggable algorithm, that is passed into the timekeeper as a dependency. Then I could test just that dependency in isolation. To put flesh on that, I’d have a property or constructor argument on the timekeeper that was an interface with a method that takes in the last known time, and a reference to the timetable, and then calculates what if anything to execute based on the current time.
Wrapping Up
This has been fun for me, and fixing up this component will be valuable to my current project in helping us reduce our day to day support load. I also got more experience with unit testing, and some experience using git to put the code up on github. Hopefully you learned something along the way as well. If you think this code would be useful to you (or part of it), check it out at http://github.com/phmiller/Timekeeper. And thanks for reading this far <g>
--- Relevant Links ---
The post wasn’t lengthy enough for you? Maybe try a dead-tree book, like Kent Beck’s XP Explained (2nd edition). Well worth your time:
http://www.amazon.com/Extreme-Programming-Explained-Embrace-Change/dp/0321278658/ref=dp_ob_title_bk
In the collection of interviews, Coders At Work, a common theme was the idea that when confronted with a difficult debugging problem, it can be better to just redo the troubled code rather then fix it.
This may seem like an invitation to put on your cape and do some hero-programming. This kind of creative destruction is only warranted when you are confident you can reproduce the functionality more coherently and with fewer bugs in less time than it would take to fully understand and tweak the existing code.
As I mentioned in a previous post, I’ve been working on a component to keep time and schedule tasks. The basic requirements of which are:
- Keep time in any time zone
- Provide a way to schedule tasks for every hour change, day change or arbitrary time
- Provide a mechanism to monitor the results of these tasks and/or any exceptions
The implementation I described in that post mostly worked, but would occasionally just fall over in our test environment. I couldn’t find the bug and looking through the code, I saw a lot of room for improvement. So after some creative destruction, I had a new and cleaner implementation:
public interface ITimekeeper
{
Action<Task, Exception> ExceptionHandler { get; set; }
Action<Task, TaskResult> CompletedHandler { get; set; }
void ScheduleFor(TimeSpan timeOfDay, Task task);
void ScheduleForEveryDay(Task task);
void ScheduleForEveryHour(Task task);
void Start();
void Stop();
}
public class Task
{
public Func<DateTime, TaskResult> Execute { get; set; }
public string FriendlyName { get; set; }
}
public class TaskResult
{
public bool Error { get; set; }
public string Message { get; set; }
}
Even just from a usage perspective this code reads much nicer to me:
var tk = new Timekeeper("Central Standard Time");
tk.ScheduleForEveryHour(task1);
tk.ScheduleForEveryHour(task2);
tk.ExceptionHandler +=
(task, ex) => System.Console.WriteLine(
"[Exception Handler] Exception for {0}: {1}",
task.FriendlyName, ex);
tk.CompletedHandler +=
(task, result) => System.Console.WriteLine(
"[Completed Handler] {0} completed with a message of {1}",
task.FriendlyName,result.Message);
tk.Start();
Under the hood, the changes were pervasive, and we’ll look at those changes in an upcoming post.
--- Relevant Links ---
Coders At Work was a fascinating read for the insights into how some of the giants of our field work. The homepage is also notable for an excellent salmon background:
Coders At Work Homepage
Can’t wait for the next post? Want to take a look at the code for yourself? View at your own risk:
Timekeeper on Github
For the time keeping component that I’ve mentioned before, one of the requirements was that it be able to run multiple actions or tasks at a given point in time. In case some of these tasks were particularly long running the idea was to run them in the background, rather than potentially block the main thread and miss a subsequent scheduled time. We also wanted to keep track of a task failing to execute properly and report back on that.
C# 3.0 provides some elegant and concise ways to do this. My exact implementation was pretty particular to our problem space, but for blogging’s sake, I abstracted out a basic and version of it for your convenience.
We’ll start from the top with usage:
class Program
{
static void Main(string[] args)
{
int i = 0;
List<Action> actionsToExecuteAsync = new List<Action>()
{
() => System.Console.WriteLine("executed action 1"),
() => System.Console.WriteLine("executed action 2"),
() => System.Console.WriteLine("exception time: {0}", 1/i),
() => System.Console.WriteLine("executed action 4")
};
AsyncUtil.ExecuteAsync(actionsToExecuteAsync, HandleException);
System.Console.ReadLine();
}
static void HandleException(Exception e)
{
System.Console.WriteLine("exception: {0}", e);
}
}
There is a heavy helping of syntactic sugar here as I create some anonymous delegates (Action’s) to execute and put them in a list using a collection initializer. Next up I call the ExecuteAsync method and wait to see the results. Which look something like this:
executed action 1
executed action 2
executed action 4
exception: System.DivideByZeroException: Attempted to divide by zero.
at AsyncWorker.Program.<>c__DisplayClass8.<Main>b__3() in C:\Users\Peter\src\
AsyncWorker\AsyncWorker\Program.cs:line 17
at AsyncWorker.AsyncUtil.<>c__DisplayClass3.<ExecuteAsync>b__0(Object state)
in C:\Users\Peter\src\AsyncWorker\AsyncWorker\AsyncUtil.cs:line 20
So we’ve got our tasks executing and an exception in one of them being handled by the HandleException method.
Looking behind the curtain:
public static class AsyncUtil
{
public static void ExecuteAsync(List<Action> actions, Action<Exception> exceptionHandler)
{
foreach (var action in actions)
{
var actionToExecute = action;
ThreadPool.QueueUserWorkItem(state =>
{
try
{
actionToExecute();
}
catch (Exception e)
{
exceptionHandler(e);
}
}, null);
}
}
}
Almost more whitespace and curly braces in there then code, but that won’t stop us. Again, with some sugary syntax, I’m throwing the action to a background thread through the call to ThreadPool.QueueUserWorkItem.
A gotcha was that I have to create the actionToExecute variable to ensure that all threads don’t just execute the last action in the list. Why? Because each thread would get a reference to “action” (from the foreach loop) and by the time they executed, “action” would be the final item in actions. Which is probably more confusing to you then it would be if you just took this code and tried it with and without the extra variable.
So take it for what it’s worth, but I found this snippet of code to be useful and a pleasant reminder of how expressive you can be in C#.
---Relevant Links---
If you care to learn anything about threading in .NET, read this whole threading guide by Joseph Albahari:
http://www.albahari.com/threading/
If you want to see what’s coming in .NET 4.0 that makes this even easier and more powerful:
http://www.codethinked.com/post/2010/01/25/NET-40-and-SystemThreadingTasks.aspx
Truly paying attention can be one of the hardest things to do. Even when you are doing something you want to do, say solving an interesting problem, your mind flits about. In a five minute interval you might answer an IM, check your email, check twitter and oh yeah, think about that problem you are working on.
There is a lot of literature out there around “flow” and the efficiency of focusing your attention on just one task. If you are working as an engineer or implementer on a project, common ways to help reduce these distractions are to put up a red flag (physically or with a do not disturb online status) or put on your headphones. You can also try to increase your focus by self imposing time boxes on your schedule, a la the Pomodoro Technique of 25 minute bursts of concentrated activity.
That’s at an individual level. To keep a software team focused, I’ve learned by counter example that the key is for the team to be highly focused on the smallest number of tasks you can get away with. Again, this is not easy to accomplish, but Agile methodologies have a lot to say on this.
From agile in general, comes the phrase “done done.” Meaning work on a task until the code works, is documented, tested, builds and deploys smoothly. This requires extra discipline on the part of the developer as making the code work is the fun part, whereas the rest just seems like extra effort. This also requires discipline on the part of the team lead to allow this “extra” work to be completed when the developer could go on to make something else work. The benefit is you don’t have to backtrack as much or have a stressful big bang at the end of a cycle to do all your integration, building, testing and documenting.
“Done done” talks about how to work on a task, which doesn’t help if you are overloaded with too many. Scrum addresses this problem through sprint planning, where tasks are given estimates and only X amount of work is allowed into a sprint. This should result in fewer tasks per developer, but not always. Again, discipline is the key as the scrum master and product owner have to engage in a give and take to keep scope manageable. If the product owner says you have to do everything this sprint, no matter what, you aren’t doing agile anymore, you’re just fire fighting.
Kanban or lean methodologies, deriving from production environments, focus on limiting the number of tasks going on at once. Every task goes through distinct stages and a new task cannot be added until another task exits the last stage. Progress is tracked visually on a kanban board. So there are not really sprints, so much as a continuous low flow of focused activity. I haven’t done kanban myself, but like scrum, it implies a give and take between the team and the other stakeholders to understand that doing less at once, does mean doing less for the project lifecycle.
So while Agile is becoming mainstream, and the concept of flow is almost passé at this point, I don’t get the sense that it is mainstream to follow through on their convergence of focusing more on fewer items at a time. This seems to be a general trend, where we all like the idea of being Agile, but think a light process = a free and easy process. After a rather unsuccessful attempt at Scrum with that concept in my mind, I can assure you it is not the case. Good process is difficult, but when did difficulty ever stop us before?
--Relevant Links--
The Pomodoro Technique and tomato timers:
http://www.pomodorotechnique.com/
If you like “Done done”, check out:
http://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffalo_buffalo_buffalo_Buffalo_buffalo
More on “done done”:
http://blog.phpdeveloper.org/?p=148
InfoQ’s stuff is pretty good, haven’t read this yet myself, but looks promising:
http://www.infoq.com/minibooks/kanban-scrum-minibook
Agile is now mainstream:
http://www.computerworld.in/articles/agile-software-development-now-mainstream
In a previous post, I mentioned working on a time keeping component. We used this component as the basis of a task scheduling application. Our tasks were represented by classes that implemented a common task interface:
public interface ICustomTask
{
CustomTaskResult Execute();
DateTime? ExecutionTime { get; }
TimeZoneInfo ExecutionTimeZone { get; }
string Description { get; }
}
public class CustomTaskResult
{
public bool WasSuccess { get; set; }
public string Message { get; set; }
}
So far, so good. Next challenge was how do we specify which of these tasks to run, and when? My initial thought was to put these details in an app.config or custom XML file. Task elements with the class name and time as attributes for example. Then additional elements for custom parameters that a task might need to initialize.
Fortunately enough, I work with some smart people, one of whom gently suggested that rather then create and manage my own custom config file, I instead leverage Castle Windsor (our dependency injection framework) and its configuration.
I’ll let the code do the talking; here’s a windsor section of an app.config for our task scheduler which will run two tasks at 9pm and the end of every day (the default if no time is specified) respectively:
<castle>
<components>
<component id="NightlyProcessing" lifestyle="transient"
service="Clarity.ICustomTask, Clarity.Task"
type="Clarity.Processing.NightlyTask, Clarity.Processing">
<parameters>
<timeZone>Eastern Standard Time</timeZone>
<processingTime>9:00 PM</processingTime>
</parameters>
<component id="RunSSISPackage" lifestyle="transient"
service="Clarity.ICustomTask, Clarity.Task"
type="Clarity.SSIS.PackageTask, Clarity.SSIS">
<parameters>
<description>Nightly Data Archiving</description>
<pathToPackage>c:\Data_Archive.dtsx</PathToPackage>
<pathToConfigFile>C:\DEV.dtsConfig</PathToConfigFile>
</parameters>
</component>
</components>
</castle>
The namespaces have been sanitized, but the point is that you can specify parameter elements that map to constructor arguments. So for example, the NightlyTask class has a constructor with a timeZone and processingTime argument that get mapped to the ExecutionTime and ExecutionTimeZone properties on the custom task.
--- Relevant Links ---
To take this to the next level (or if you just don’t like the angle bracket tax):
http://sheitm.blogspot.com/2009/05/fluent-castle-windsor-and-configured.html
As I continue to explore agile techniques at work, I had the opportunity to do some Test Driven Development, specifically Test First Development. The problem at hand was that our system is composed of several applications, all of which have various actions they have to take every hour or every day at a specific time. As the applications were developed, each application grew out its own time keeping infrastructure.
If you haven’t had the pleasure yet, working with time, dates and durations in .NET is not as easy as you’d like. It has gotten a lot better since framework 1.0, but it is still a great place for bugs to hatch. And not surprisingly, while each application was able to implement a system that worked most of the time, the code base was sprawling and behavior was unpredictable.
With the hindsight that comes after pushing a system to production, my team became convinced that we needed more then working most of the time, we wanted to prove that it worked all the time. Also, having abused the coding principle of (D)on’t (R)epeat (Y)ourself with multiple competing implementations we wanted to settle on one common implementation.
The approach we took was to throw out the old code, and first define an interface for our time keeper. Now it looks something like this:
public interface ITimekeeper
{
event EventHandler HourChanged;
event EventHandler DayChanged;
void Initialize(TimeZoneInfo timeZone, DateTime dayChangeTime);
void RegisterCustomEvent(DateTime time, TimeZoneInfo timeZone,
CustomEventCallback callback, object state);
void Start();
void Stop();
...
IClock Clock { get; set; }
}
Now is an important qualifier. We didn’t start out with the right interface, we took an educated guess and then started writing some tests. Importantly, and somewhat surprisingly to me, we got a lot of value out of just trying to write the tests themselves. It was a feedback loop as we’d try to write a test, find the interface clunky, refine the interface , write the test again and then move on.
We deliberately ignored the implementation at first and defined a reasonably wide set of tests that covered the basic functionality of our component. Then we subjected the interface to its sharpest critique by handing these tests off to another developer on the team and having him implement the code to make them pass.
In short, TDD drove our component level design to be more elegant, provided a natural way to break out tasks amongst the team, and of course provided us some confidence that the implementation would work.
I think I am taking a break from the bold prediction business, but this experience definitely has lead me to reconsider some of the points made over a year ago in this post of questionable origin:
http://blogs.claritycon.com/blogs/peter_miller/archive/2008/10/06/why-test-driven-development-is-a-hard-sell.aspx
<--- A questionable dude…
--- Related Links ---
Hat tip to Philipp Sumi, who implemented a nice scheduling library of his own (which I found after I finished my efforts) and provided me some good feedback and insight on TDD.
http://www.hardcodet.net/2010/01/lightweight-task-slash-job-scheduling-with-silverlight-support
Also, for the eagle eyed amongst you, yes we do have an interface for the system clock, IClock. See Ayende Rahien’s post on why this seeming non sequitur can be useful:
http://ayende.com/Blog/archive/2008/07/07/Dealing-with-time-in-tests.aspx
Dependency injection (DI) is a powerful technique to make your code more testable and your application more adaptable to future changes. Working with an application that uses DI in Visual Studio can be painful. You want to be able to make changes to your dependencies, and have those changes automatically picked up and put in the right place, so that when your application tries to find them, it uses the latest version. And you want that all to happen just when you press F5 to debug.
I posted a question to stackoverflow.com about this, laying out some options, including copying files by hand or creating a post build event per dependency per application. In keeping with stackoverflow rules about open ended questions, I quickly marked an answer as accepted.
The answer I accepted was to change the output directories of the dependency projects to some common folder, which the application would then query for dependencies at runtime:
This sounded good, but after some thought, I was worried about the unintended consequences of having these projects not build to the normal bin/debug, bin/release folders.
After some experimentation, I ended up with a solution that is similar in spirit, but leaves the normal build output directory alone:
- Add post build events to each dependency project to copy the dll and pdb files to a common directory
- xcopy $(TargetDir)*.dll $(SolutionDir)diBin /iy
xcopy $(TargetDir)*.pdb $(SolutionDir)diBin /iy
- Add a post build event to the application project to copy the DI dll and pdb files to a subdirectory of its bin
- xcopy $(SolutionDir)\diBin\*.dll $(TargetDir)diBin\ /iy
xcopy $(SolutionDir)\diBin\*.pdb $(TargetDir)diBin\ /iy
- We use Castle Windsor for DI and to see the dependency files in diBin\ I had to add this to the app.config:
- <runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="diBin"/>
</assemblyBinding>
</runtime>
All of the projects in our solution live one level below the solution, and all are solutions are in the same folder, so $(SolutionDir) is common.
So far this has worked well. This does NOT solve anything having to do with deployment, but it does make the F5 debug experience much nicer.
Relevant Links:
Stack Overflow: How do I get a smooth debug experience in visual studio 2008 when using dependency injection?
MSDN documentation on the probing element
As a consultant, I am pulled in two seemingly opposing directions in regards to new technology. To be a good consultant, particularly at my company Clarity, you have to be engaged with technology. Reading, talking about, working with new technology is what keeps us fresh and relevant. Here I'm using technology to mean more then just languages or libraries, I also mean the processes and techniques of delivering quality software solutions. The "opposing" force I mentioned earlier is that we have to focus relentlessly on delivering the particular project we're on, with its particular timelines and constraints.
So while I constantly am tinkering with technology in my spare time, I have a much higher bar for what I'd include in my project work. For every upside something new promises, I have to decide if it outweighs the downside or risk to the project. With that said, for the past six months plus, I've been working in product development, so outside the context of my regular on site, client engagement. Our team still cares about the quality of our deliverable, but there is more freedom to experiment with new technology. Using my expanded definition of technology, our team has been experimenting with the new (to us) technology of agile processes.
One of these processes is pair programming. Unfortunately, to the skeptic, a process where two people work together on the same machine on the same task, probably brings the following images to mind:
The Guy Driving
The Other Dude
This is a mistaken view. First off, having worked previously in corporate IT, I can assure you that people are quite capable of doing nothing all day all by their lonesome. Second, we all value pair programming. Don't believe me? Think about what you do when you are working on a particularly nasty bug or making a big architectural decision. You don't usually do that alone, you bring in at least one other trusted person to help out, to bounce ideas off of. I've over simplified here, this is not what agile adherents really mean by pair programming. As an agile practice, pair programming is the predominant mode of work for developers. What I just described is just plain old collaboration. Nevertheless, it is important to point out that the argument is not about whether or not two people working together produce better results, it is about how often they should work together.
Within our product team, we have not moved to pair programming for every task. But we have been using it more extensively then I have before. We've found it surprisingly effective over the past few weeks. In pair programming sessions each lasting a few hours, I and another team member were able to quickly get two components going from design to implementation. I was thrilled with the give and take as we reined in, revised and rejected each others' ideas as necessary. By the end of it, I was mentally worn out, but in a good, kind of post work out muscle burn type of way.
With some hindsight, I think the key to this success was that we were truly peer programming, not just pair programming. There was no project role hierarchy to make one of us hesitant to correct the other, we have similar levels of tech skills and experience so we could keep up with each other and perhaps most importantly we respected each other. Respect means we take each other seriously and actually think about what the other one is saying or coding. This level of engagement helps both sides of the pair get more of out of the experience. Lack of respect plus malice is contempt, which obviously will not work well, but even lack of respect without dislike is often expressed as indifference or pity. Neither one is going to help the pair be more productive then its parts.
Moving forward, I plan to use pair programming more often within our team. I am not ready to take the leap and say we should use it all the time. Part of the software development life cycle always involves some drudgery, linking up simple UI's, writing DB scripts, fixing a simple bug and the like. These tasks might be more pleasant with a partner, but the possible gains in efficiency seem minimal.
Relevant Links
http://xprogramming.com/xpmag/whatisxp#pair
http://menloinnovations.com/blog/?p=399
http://stackoverflow.com/questions/291630/does-pair-programming-work
While working with ASP.NET MVC, one of the mind shifts I made was to embrace the concept of the view or presentation model. Elegantly presented in ASP.NET MVC In Action, the idea is that you have your domain model as one set of classes and then another set of classes optimized for your user interface. Optimizing for user interface means understanding the limitations of the UI, in this case web pages and markup.
HTML is less friendly to debug and test, and the constructs within MVC markup are not as rich as within C#. So presentation models should be leaner, meaner, flattened out versions of your domain model. If your screen is not going to display it, it should not in your presentation model. If your screen is going to display it, it should be easy to get at. For example, if your Person domain object has a .Address.State property and you are displaying a mail order receipt for that user, you might have a MailOrderReceipt presentation model class with an AddressState string property.
The logical end state of embracing the concept of a presentation model is ending up with one model class per view or screen. At first blush this seems to fly in the face of good OO practices of object reuse and DRY. However, if you focus exclusively on these practices, you are ignoring the value of cohesion; meaning for a system to work well the parts should fit together. When views and models are tightly coupled then the views are simpler to write, debug and test.
There is one more pain though, you know have map from the domain to the presentation model. This code is easy to write, but also easy to mess up if you have to write a lot of it. The errors you might make here, such as forgetting to map a property should be quite obvious in testing, but an alternative is to use a library like AutoMapper to do the work for you.
As promised in part two, I’ll close the loop on OpenId by talking briefly about authorization. We’ve already seen how to authenticate, so once you’ve got an authenticated OpenId identifier, the trick is what to do with it. This does not end up being all that exciting; in my case linked back to an identifier column in my user table and to get information about the newly logged in user. Based on characteristics of that user, I could then determine their authorization level or roles. Stick the roles into the FormsAuth cookie in pipe delimited list and you can then write code like this in your controller action:
[Authorize]
public ActionResult Edit(long id)
{
if (!User.IsInRole(UserRoles.CRUD)) return View("AccessDenied");
So going from authentication to authorization is not that big of a jump. Where it gets a little trickier is what I glossed over before, going from the OpenId identifier to the user. In an open, public site where you want as many to create accounts as possible this association can be created as part of new account set up.
When the site is meant to mostly private or internal, you don’t just want anyone creating accounts with privileges to modify or access your data. Our solution was to set up users in the database, but leave the OpenId identifier blank. We then had a special url where we could point the actual user to go and register their OpenId identifier with the unclaimed user account. This still doesn’t feel quite right to me, but it is workable for small, internally addressed sites. It was nice to work with OpenId and the excellent DotNetOpenAuth library, but overall it seems that OpenId fits more naturally for a public-facing, community oriented site.
Unfortunately, upon deploying the OpenId enabled MVC application I described in the previous post, and doing some regression testing, I discovered that my OpenId login page was functioning erratically. I found that from certain machines I could not login at all, the redirect to the OpenId provider was being ignored by IE.
After banging my head against a wall for a few hours, I posted a question on stackoverflow.com. With help from the wonderfully responsive Andrew Arnott, main contributor of the DotNetOpenAuth library I was using, I was able to track down the issue to a bug in IE 8 on Win 7 RTM. IE was stopping redirects from a trusted site to a regular Internet zone site if the Protected Mode settings were different in the zones. That’s a mouthful, but more details are at the question link above.
So the lesson for today was that a) stackoverflow was great, working exactly as advertised by getting my question in front of a knowledgeable audience (thanks again Andrew), b) get yourself a web traffic analyzer like Fiddler and learn to use it; it was invaluable, and c) trust, but verify; very rarely will your web browser really be the problem, but it is possible.
Authenticating someone’s identity has been a persistent challenge throughout history. How can you trust that someone is who they say they are? Hence the FBI agent flashing her badge, the ancient king brandishing his magical sword, and the long lost relative regaling you with stories of the old times.
Authentication is a two way street. It is in your interest to know who you’re dealing with to not get taken advantage of. It is in the other party’s interest to make sure no one else is using their identity, impersonating them for their own gain.
When we move into the digital world, authentication is just as important. And the direction has been towards fragmentation and isolation. Your company network knows who you are, your email provider knows who you are, World of Warcraft knows who your are, but Amazon.com could care less. Maintaining all these identities is as much fun as discovering your wallet now doesn’t fit in your pocket due to all the preferred customer cards you have to carry around with you.
One solution to this problem is OpenId. With OpenId, you register your identity with an OpenId provider and then can use that identity to authenticate with any other site that trusts (relies on) that OpenId provider. It’s much like if you’re throwing a party and see someone you don’t know. You ask some friends and they say that this person is a friend of theirs, so they’re alright. You still don’t really know the guy, but you rely on your friends’ to vouch for them.
Using OpenId on your site can free you up of the hassle of managing passwords and provide visitors to your site a friendly experience that does not involve creating a remembering yet another password. Within .NET, implementing OpenId is pretty straightforward.
I recently implemented OpenId authentication for an ASP.NET MVC site I was working on. I used the classes and samples provided by the excellent open source library DotNetOpenAuth. I made some tweaks along the way, but before I get to that, let’s walk through the sample code that is in the DotNetOpenAuth download.
Although it is in some ways a competitor to ASP.NET WebForms, ASP.NET MVC (MVC) is not named by accident. It is derived from the ASP.NET architecture that WebForms runs on, including the security infrastructure. So for security, you still have FormsAuthentication to work with. For more details on FormsAuthentication, there is a well written synopsis on O’Reilly’s site.
Not surprisingly then, we can start in the web.config to see how we’re going to start hooking up OpenId for authentication:
<authenticationmode="Forms">
<formsdefaultUrl="/Home" loginUrl="/User/Login" name="OpenIdRelyingPartyMvcSession"/>
<!--named cookie prevents conflicts with other samples-->
</authentication>
This is standard FormsAuthentication, letting the ASP.NET framework know that when authentication is needed, it can look for a cookie called “OpenIdRelyingPartyMvcSession” and if that’s not found, redirect to the /User/Login page. How do we tell if authentication is needed? In MVC, any controller action can be decorated with the [Authorize] attribute.
Getting back to the aforementioned Login page (notice the clean url). It is not much to look at. It has a form with a textbox called “openid_identifier” and a form action of Authenticate. I’ll spare you the rest of the markup, which is readily available from the sample download.
So, we now can look at Authorize action that this form calls:
[ValidateInput(false)]
publicActionResult Authenticate(stringreturnUrl) {
var openid = newOpenIdRelyingParty();
var response = openid.GetResponse();
if(response == null) {
// Stage 2: user submitting Identifier
Identifierid;
if(Identifier.TryParse(Request.Form["openid_identifier"], outid)) {
try{
returnopenid.CreateRequest(Request.Form["openid_identifier"]).RedirectingResponse.AsActionResult();
} catch(ProtocolException ex) {
ViewData["Message"] = ex.Message;
returnView("Login");
}
} else{
ViewData["Message"] = "Invalid identifier";
returnView("Login");
}
} else{
// Stage 3: OpenID Provider sending assertion response
switch(response.Status) {
caseAuthenticationStatus.Authenticated:
Session["FriendlyIdentifier"] = response.FriendlyIdentifierForDisplay;
FormsAuthentication.SetAuthCookie(response.ClaimedIdentifier, false);
if(!string.IsNullOrEmpty(returnUrl)) {
returnRedirect(returnUrl);
} else{
returnRedirectToAction("Index", "Home");
}
caseAuthenticationStatus.Canceled:
ViewData["Message"] = "Canceled at provider";
returnView("Login");
caseAuthenticationStatus.Failed:
ViewData["Message"] = response.Exception.Message;
returnView("Login");
}
}
return new EmptyResult();
}
While there is a distinctly pasta like consistency to this code, it is demo code and gets the job done. The first thing to understand about this method is that it is meant to be called twice. OpenId authentication involves hopping outside of your site’s domain, letting the user log in to the provider site, and then reading back the response from that provider.
Hence the strange looking conditional around the response variable. When a user types in an OpenId identifier, such as “http://myname.myopenid.com” and clicks log in, that request goes to our site first. At this point, there is no OpenId response, since we haven’t asked for anything yet. So, with a null response, we send out a request to the OpenId provider, who’ll get back to us when the user logs in.
The second time this method is hit, it is being hit by the OpenId provider, who passes along a bunch of data that the DotNetOpenAuth library kindly parses for us into a response object. If the request was authenticated, we can complete the final step of FormsAuthentication, which is to create a cookie with authenticated user’s info so they don’t have to log in for every request. With the user’s OpenId identifier verified, we send them on their way with a redirect back to the page they first requested.
I hope you enjoyed this brief look at OpenId and the DotNetOpenAuth library. Up next, linking an OpenId identifier back to objects in our domain for the purpose of access control.
A product I’m working on is nearing release and I am in the midst of finishing off some basic administrative pages for it. I started out in ASP.NET WebForms, but quickly ran into a road block. Throughout the rest of our system, we maintain at least a token separation between the data and behavior of our domain and the persistence mechanism for that data. In other words, we frown on a UI running directly against the database.
In ASP.NET WebForms the playing field is titled towards this type of direct binding. If you want a grid up and running quickly, you bind it to the data source directly, like a LINQ to SQL context or to a DataSet. If you do that, you get sorting, paging, alternate row striping and the like for free. It is satisfying and resonates on the, I am programmer, I am skillful, yet lazy when appropriate level.
Going with the satisfaction, you accept the structures that work best with WebForms. However, your troubles are not all solved, as I find you always end up with ragged edges in your application. These ragged edges are spots where you have to do something really nasty in JavaScript or get tangled up in the page lifecycle to get the page just how you want it. In particular, you end up with controls that render HTML that is almost right, and you can’t tweak the output, so you have to let it render and then mess with it after the fact.
Circling back to where I started, I was working on these admin screens and decided that I was sick of shoe horning the rest of my code base to fit into WebForms. What immediately came to mind as an alternative was ASP.NET MVC. I knew it could build up scaffolding for basic CRUD operations quickly, you worked directly with the HTML and it was geared for working with a more layered architecture.
I had been reading Jeffrey Palermo’s ASP.NET MVC In Action since it was in MEAP (early access), as well as the NerdDinner examples from the Microsoft crew. Of course it is different to work with production code for a client then just tinkering, but I have been thrilled with the experience so far.
Although I get fewer rich UI controls out of the box, it is not too bad to build them myself. While there is some helpful magic built in around data binding from forms and routing, the absence of magic in the markup makes the application seem a lot cleaner.
As I work with MVC more, I find myself wondering when I’d recommend to use ASP.NET WebForms other than for legacy reasons. If you want a deep and rich graphical user experience delivered over the web, then I’d gravitate to Silverlight. For most other apps, I’d want the flexibility of MVC. I'll give credit to Jeffrey Palermo for this idea, but it has resonated with me the more I work with MVC.
In coming posts, I will share some code that I’ve written for MVC around authentication and OpenId, as well as discuss my joy in creating view models or as I like to call them crufty models.
Continuous Integration
Continuous Integration (CI) is the practice of continually incorporating code changes into an automated build to ensure that breaking changes are detected as quickly as possible. CI is prevalent among open source projects in general and in the non-.NET ecosystems of Java, Python and Ruby. In my .NET work, the majority of projects have not used CI. Build was always a necessary evil, an annoyance to be avoided and sometimes even the entirety of a persons’ work for a given iteration.
After a lot of reading, but little practical experience with it, I came to view CI as a necessity on my current project. For this project, there are a number of separate components that intertwine to form the full system. This system needs to be demoable in short order, so it was too risky to have build and deploy be an ad-hoc affair.
An apparent disadvantage to CI is that there is a big investment upfront. I had code that compiled just fine inside of Visual Studio and a drag and drop style of deployment that basically worked. I still have a lot of effort to go from there to automated build and deploy.
However, there are tangible benefits to the process itself. Automating build and deploy forced me to really understand how the system I am working on fits together. It also revealed the extent of interdependencies in the code base.
Configurable Service Installers
Fighting with TFS and Team Build also lead me to this tip on how to make the installers for Windows service projects configurable from the command line. Windows services can be installed using installutil, which can take command line arguments.
These command line arguments are passed into the Installer class as the Context.Parameters. I overrode the OnBeforeInstall, OnBeforeUninstall and OnBeforeRollback methods to look at these context parameters and set the service name, username and password based on the command line arguments.
This allows me to easily have multiple instances of the same service running on the same machine without recompiling the service project for each new name.
So I end up being able to do something like this from the command line:
> installutil.exe /serviceName=”MyService” /username=serviceUser /password=servicePw serviceExe
And then I have code in my service installer class like this:
protected override void OnBeforeInstall(IDictionary savedState)
{
base.OnBeforeInstall(savedState);
ReadContextParameters();
}
protected override void OnBeforeUninstall(IDictionary savedState)
{
base.OnBeforeUninstall(savedState);
ReadContextParameters();
}
protected override void OnBeforeRollback(IDictionary savedState)
{
base.OnBeforeRollback(savedState);
ReadContextParameters();
}
private void ReadContextParameters()
{
if (Context.Parameters.ContainsKey("serviceName"))
{
string serviceName = Context.Parameters["serviceName"];
serviceInstaller1.DisplayName = serviceName;
serviceInstaller1.ServiceName = serviceName;
}
if (Context.Parameters.ContainsKey("username"))
{
serviceProcessInstaller1.Username = Context.Parameters["username"];
}
if (Context.Parameters.ContainsKey("password"))
{
serviceProcessInstaller1.Password = Context.Parameters["password"];
}
}
More Posts
Next page »