Something Sure Is Rank

Recently I ran across a few articles discussing the not-uncommon corporate HR practice of stack ranking.  Frankly, my jaw dropped when I read that that practice has a lot of traction in the corporate world.  The premise is so flawed that I cannot believe so many people swear by it.

For those not familiar, stack ranking is the practice of ranking a set of employees in a linear fashion, from worst to best, and then marking the top 20% as excellent, the middle 70% as adequate, and te bottom 10% as failures, and suitable for dismissal.  It assumes that the population follows the familiar bell curve.

So let's take a look at the problems with this.  First the technical ones

1) Sample Size
The bell curve ( or normal curve) is the expected distribution is valid only for sufficiently large populations.  Most workgroups are not nearly large enough that they would match the normal curve in the distribution of ability.  Use of stack ranking on small groups can incorrectly mark people of lesser ability as part of the bottom 10%.

2) Random Selection
The normal curve also assumes that there is no skewing influence on the population.  Most companies proclaim that they are selective in their hiring, so they should not have any poor performers.  Moreover, if the company has run any rounds of performance-based layoffs, the population of employees is no longer normally distributed.  Once you remove the lower 10% of the population, there is no longer a "lowest 10%" to choose from.

And now the social reasons against stack rank:
3) Arbitrariness
Since most people are aware of the basic fact that statistics requires large groups of people, they know that in a small group there can't be a bottom 10%.  So when one of them has to be sacrificed as the goat, they begin to see all the rankings are either arbitrary or bogus, so they become cynical as to the merit of the evaluations

4) Gaming the system
Once employees know that they are ranked against each other and avoiding being in the "bottom 10%" is vital, they will start to alter their behavior to try and avoid being the low man.  This leads to anti-social activities such as information hoarding, sabotage, and grandstanding.  They stop trying to further the group goals in favor of their own.

Given all this, I cannot understand why this is favored by corporate management except as a means to instill fear in the employees and enable the rewarding of those liked by the managers.

Getting Input, But Not The Wrong Kind

One of the things that I noticed a while back is that quite often it seems impossible to get others to provide their input to a design or document.  Nobody wants to respond to your questions, or doesn't have the time, etc.

So I started presenting every document or design as a strawman - I made sure to put in things that I thought good but controversial into every section.  Then I sent it out for review as version 0.1.

The controversial ideas would galvanize the reviewers to respond with their better ideas.  It seems that while few will stick their necks out to propose solutions, almost everyone is comfortable with criticizing someone else's solution.  It seems to allow everyone to assume their preferred role in the organization - wise sage, gadfly, elder statesman, or just plain old curmudgeon - without risk.  It's not their idea that's being looked at, and if their input is ignored, they are free from blame, but if the idea is a success, they contributed to the discussion, and therefore helped bring it to market!

Curiously, there is a counterpoint to this situation - sometimes you have a manager/PM/customer who cannot allow a document to pass by without making some changes.  I'm not the only one to notice this, not by far.  And the solution is exactly as described - make some very specific things either wrong or superfluous, and let the manager/PM/customer tell you to change them. 

A corollary to this happened to me at my first job.  We had a group of senior customer executives visiting to see our initial demo of the product.  We showed them what we had - a mockup running on a terminal, and then they were shown the various other bit s of technology that were being assembled into the final product.  Now, this product had some very specific color-coding of icons on the UI, and we were already planning on making the color specifications loadable from a configuration, to allow for the differences in the color monitors of that time, yet we had the whole group of executives staring at a screensaver trying to point out some specific colors that specific icons needed to be.  It was something that was best dealt with in a document, but they felt the need to impress us (or each other) with their acumen at determining between close shades of magenta.

False Urgencies

Recently, The Job has offered me a number of occasions where the usual software development process of asking for an estimate has been skipped entirely,  Several times, the Manager is sending out an email announcing the next code freeze in 2 days, with a list of unfinished bug reports.  So there is a sudden scramble to finish things that night well be too much to finish in the remaining time.

This rant about bogus deadlines addresses this issue.  One of the tenets of good software is that is takes some time, and it is at least somewhat a creative endeavor.  So rushing your developers just to meet an arbitrary deadline is counterproductive.

Give them room.  Provide the list of desired changes, get estimates, and then juggle the schedule or the list to fit.

Oh, and never think that twice-daily, 2+hour progress meetings with VPs, EVPs, and the rest of the C-level wannabees are going to help.  Nothing ruins concentration more than knowing that in 3 hours you are going to have to explain why you haven't made "satisfactory" progress solving a problem, where "satisfactory" means "saving my division customer satisfaction rating so I get my 35% bonus" for the EVP

The Tyranny of the Best

I ran across this post about different skill types among programmers and it struck a chord.  I've seen a great many places described as looking for "The Best Programmers", implying that the rest of the programmers need not apply.

As that post notes, those places may be missing out on some other Best Programmers due to their different approaches to things.  However, I think that it is also a bad thing to expect all your developers to be the best at their craft.

If every company is gunning for the best, the salary budget will be stressed - if they are the best, won't they expect the best pay, or at least more than the average/market rates?  So there will be smaller teams, or fewer teams of a given size, which means your company will not be able to work on as many things.  This might be ok in a startup, where the goal is often a single product with relatively few sections, but in a large established company, there can be hundreds of projects.

And then what happens when your Bests decide to jump ship, because your projects no longer excite them, or the next company offers a better deal?  You will have to hire some lesser lights, and then either you will hit a slowdown in your development rate, or you will see problems.

Of course, if you let slip that the current team is not regarded as highly as their predecessors, you'll have a morale issue.  Nobody likes to be looked down on, especially if they are working hard.

So think long and carefully about the phrases you put in your job ad, or you might end up driving off some people that would do a great job.

A Little Python Partisanship

Although most of my time at The Job is spent in compiled languages, I have been trying to develop some skill in Python to enliven my technical skills.

This week, I got a chance to try something that I know would have been a real pain in a comp[iled language like C++.

The gist of the issue is that I want to automate some integration tests I'm writing for the system, so that I can just run them all with one command.  I am trying to make them as modular as possible so that I can add new ones easily, since I know from past experience that if it takes any significant time to add tests, I will not have the time to add them.  So I come up with a skeleton program that will run the tests, dividing them into preconditions that must pass, the test itself, and postconditions that must pass.  Now I run into the issue that at the moment I do not want to package my tests, so I need to import the modules directly and individually.  However, that makes it hard to add new tests, since the import list will grow with each test.

A quick search of www.python.org/documentation and I find the import statement is a shortcut for __import__(), and __import__() can take runtime strings as arguments.  This means I can pass the test modules' names into the test skeleton.  A little help from exec/eval and I can call an arbitrary function from my test module based on command line input.

Here's the code I have so far:

d = {}
with open(sys.argv[1]) as f:
for line in f:
(key, val) = line.split()
d[key] = val

execStmt1 = d['modulename'] + " = __import__('" + d['modulename'] + "', globals(), locals(), [], -1)"
execStmt2 = d['modulename'] + '.' + d['functionname'] + '(d)'

print execStmt1
print execStmt2
exec execStmt1
exec execStmt2
targetFunction = eval(d['modulename'] + '.' + d['functionname'])

And the data file sent in looks like this:

modulename dummy
functionname dummy2
arg1 1
arg2 2


And the dummy module dummy.py looks like this:

def dummy(arg1, arg2):
print "dummy(", arg1, ",",arg2,")"

def dummy2(d):
print "dummy2(",d['arg1'], ",", d['arg2'], ")"

if __name__ == '__main__':
dummy(0,0)

So this gets me most of the way there.  I still need to rework how the preconditions and postconditions are run and their return values checked, and a few other things, but this was just so amazingly EASY to do, I thought it was worth a post

Taking Out A Contract

Earlier this week, I was involved in a discussion about the relative merits of Design By Contract vs Defensive Programming.  While both are good, I was struck by the thought that in my experience, it's hard to get other developers to keep up their end of the bargain for DBC.

For example, one protocol I programmed to had a field in one message that specified the maximum time it would take for a file to be transferred across a link,  When we wrote the code, the client received that message, took that time value, added a small fudge-factor to account for some specific conditions on their end, and all was well.  However, at some later point, a new group of programmers for that project decided that "No file will ever take longer than 30 seconds to be transferred", so they would wait a maximum of 30 seconds before timing out, even though we routinely had files that took 2 minutes to transfer.

Basically, any program that is built in a work environment where the original programmers will be replaced at some point is almost guaranteed to have one or more of the new programmers ignore the constraints at some point, rendering the contract useless.  And unless you're programming in Eiffel, you get bad results.

Defensive programming, however, differs in that the receiving side will check the inputs and can complain when its expectations are violated.  The big problem with Defensive programming is that it tends to clutter up the code, and foster paranoia.  The compromise I have come up with is to but string defensive checks on things that cross object boundaries - the public methods, API calls, etc.  It's generally easy to at least document the expectations within a class so that later developers will see them and follow them, but once you have to call something defined in another file/directory/subsystem, you risk not being able to find the details.

Hey, some programming content, at last! Finite State Machines!

So I decided that I've been using this blog as a ranting place instead of an informing place.  That;s not really what I was intending when I got started, so I'm going to try and focus on informational content for a while.  Not quite a New Year's Resolution, but it's worth a try

Today's Topic:  Finite State Machines
Finite State Machines (FSMs) are a powerful tool for handling program behavior in a flexible manner without losing track of the details.  If you are not familiar with FSMs, here's the Wikipedia article on them, which is a bit obtuse, but gives some of the details.

Typically, you will use an FSM in a situation where you have a set of behaviors you need to model, and the actions that get taken on certain inputs depend on the state of the object.  Most examples use mechanical devices, like turnstiles, automatic doors, traffic lights, and vending machines.

The real power of FSMs comes from when you combine a generator utility (easily found online) to generate the skeleton code for the state transitions, and the Command pattern to encapsulate the events/inputs.  This leads to compact code for the transistions, and lets you concentrate on the meat of your functionality.

The GoF State pattern might be appear to be useful here, but that is more helpful when you have something that does multiple things depending on its internal state, as opposed to the FSM itself.  The example in the link shows that this is less about the transitions and more about the actions, with the transitions not doing useful work, or being almost dataless signals.  The typical FSM even is more diverse, being a value or a signal whose meaning will differ.

One of the drawbacks to FSMs is that the logic of the transitions can be absent from the code, making it difficult to follow.  This is of course solvable with good documentation, but you need to keep it clear.  Most of the FSM generating programs have a data file that drives the process, so keep the descriptions there, and link to the in the generated/skeleton code, or if you generate skeletons and then modify them manually, in that code.

One other common use of FSMs is in regular expression recognition, but that's fairly tightly tied into the fields of compilers and interpreters, which is a really complex subject to cover.