I ran across this post about different skill types among programmers and it struck a chord. I've seen a great many places described as looking for "The Best Programmers", implying that the rest of the programmers need not apply.
As that post notes, those places may be missing out on some other Best Programmers due to their different approaches to things. However, I think that it is also a bad thing to expect all your developers to be the best at their craft.
If every company is gunning for the best, the salary budget will be stressed - if they are the best, won't they expect the best pay, or at least more than the average/market rates? So there will be smaller teams, or fewer teams of a given size, which means your company will not be able to work on as many things. This might be ok in a startup, where the goal is often a single product with relatively few sections, but in a large established company, there can be hundreds of projects.
And then what happens when your Bests decide to jump ship, because your projects no longer excite them, or the next company offers a better deal? You will have to hire some lesser lights, and then either you will hit a slowdown in your development rate, or you will see problems.
Of course, if you let slip that the current team is not regarded as highly as their predecessors, you'll have a morale issue. Nobody likes to be looked down on, especially if they are working hard.
So think long and carefully about the phrases you put in your job ad, or you might end up driving off some people that would do a great job.
Saturday, February 16, 2013
Sunday, February 03, 2013
A Little Python Partisanship
Although most of my time at The Job is spent in compiled languages, I have been trying to develop some skill in Python to enliven my technical skills.
This week, I got a chance to try something that I know would have been a real pain in a comp[iled language like C++.
The gist of the issue is that I want to automate some integration tests I'm writing for the system, so that I can just run them all with one command. I am trying to make them as modular as possible so that I can add new ones easily, since I know from past experience that if it takes any significant time to add tests, I will not have the time to add them. So I come up with a skeleton program that will run the tests, dividing them into preconditions that must pass, the test itself, and postconditions that must pass. Now I run into the issue that at the moment I do not want to package my tests, so I need to import the modules directly and individually. However, that makes it hard to add new tests, since the import list will grow with each test.
A quick search of www.python.org/documentation and I find the import statement is a shortcut for __import__(), and __import__() can take runtime strings as arguments. This means I can pass the test modules' names into the test skeleton. A little help from exec/eval and I can call an arbitrary function from my test module based on command line input.
Here's the code I have so far:
d = {}
with open(sys.argv[1]) as f:
for line in f:
(key, val) = line.split()
d[key] = val
execStmt1 = d['modulename'] + " = __import__('" + d['modulename'] + "', globals(), locals(), [], -1)"
execStmt2 = d['modulename'] + '.' + d['functionname'] + '(d)'
print execStmt1
print execStmt2
exec execStmt1
exec execStmt2
targetFunction = eval(d['modulename'] + '.' + d['functionname'])
And the data file sent in looks like this:
modulename dummy
functionname dummy2
arg1 1
arg2 2
And the dummy module dummy.py looks like this:
def dummy(arg1, arg2):
print "dummy(", arg1, ",",arg2,")"
def dummy2(d):
print "dummy2(",d['arg1'], ",", d['arg2'], ")"
if __name__ == '__main__':
dummy(0,0)
So this gets me most of the way there. I still need to rework how the preconditions and postconditions are run and their return values checked, and a few other things, but this was just so amazingly EASY to do, I thought it was worth a post
This week, I got a chance to try something that I know would have been a real pain in a comp[iled language like C++.
The gist of the issue is that I want to automate some integration tests I'm writing for the system, so that I can just run them all with one command. I am trying to make them as modular as possible so that I can add new ones easily, since I know from past experience that if it takes any significant time to add tests, I will not have the time to add them. So I come up with a skeleton program that will run the tests, dividing them into preconditions that must pass, the test itself, and postconditions that must pass. Now I run into the issue that at the moment I do not want to package my tests, so I need to import the modules directly and individually. However, that makes it hard to add new tests, since the import list will grow with each test.
A quick search of www.python.org/documentation and I find the import statement is a shortcut for __import__(), and __import__() can take runtime strings as arguments. This means I can pass the test modules' names into the test skeleton. A little help from exec/eval and I can call an arbitrary function from my test module based on command line input.
Here's the code I have so far:
d = {}
with open(sys.argv[1]) as f:
for line in f:
(key, val) = line.split()
d[key] = val
execStmt1 = d['modulename'] + " = __import__('" + d['modulename'] + "', globals(), locals(), [], -1)"
execStmt2 = d['modulename'] + '.' + d['functionname'] + '(d)'
print execStmt1
print execStmt2
exec execStmt1
exec execStmt2
targetFunction = eval(d['modulename'] + '.' + d['functionname'])
And the data file sent in looks like this:
modulename dummy
functionname dummy2
arg1 1
arg2 2
And the dummy module dummy.py looks like this:
def dummy(arg1, arg2):
print "dummy(", arg1, ",",arg2,")"
def dummy2(d):
print "dummy2(",d['arg1'], ",", d['arg2'], ")"
if __name__ == '__main__':
dummy(0,0)
So this gets me most of the way there. I still need to rework how the preconditions and postconditions are run and their return values checked, and a few other things, but this was just so amazingly EASY to do, I thought it was worth a post
Saturday, January 26, 2013
Taking Out A Contract
Earlier this week, I was involved in a discussion about the relative merits of Design By Contract vs Defensive Programming. While both are good, I was struck by the thought that in my experience, it's hard to get other developers to keep up their end of the bargain for DBC.
For example, one protocol I programmed to had a field in one message that specified the maximum time it would take for a file to be transferred across a link, When we wrote the code, the client received that message, took that time value, added a small fudge-factor to account for some specific conditions on their end, and all was well. However, at some later point, a new group of programmers for that project decided that "No file will ever take longer than 30 seconds to be transferred", so they would wait a maximum of 30 seconds before timing out, even though we routinely had files that took 2 minutes to transfer.
Basically, any program that is built in a work environment where the original programmers will be replaced at some point is almost guaranteed to have one or more of the new programmers ignore the constraints at some point, rendering the contract useless. And unless you're programming in Eiffel, you get bad results.
Defensive programming, however, differs in that the receiving side will check the inputs and can complain when its expectations are violated. The big problem with Defensive programming is that it tends to clutter up the code, and foster paranoia. The compromise I have come up with is to but string defensive checks on things that cross object boundaries - the public methods, API calls, etc. It's generally easy to at least document the expectations within a class so that later developers will see them and follow them, but once you have to call something defined in another file/directory/subsystem, you risk not being able to find the details.
For example, one protocol I programmed to had a field in one message that specified the maximum time it would take for a file to be transferred across a link, When we wrote the code, the client received that message, took that time value, added a small fudge-factor to account for some specific conditions on their end, and all was well. However, at some later point, a new group of programmers for that project decided that "No file will ever take longer than 30 seconds to be transferred", so they would wait a maximum of 30 seconds before timing out, even though we routinely had files that took 2 minutes to transfer.
Basically, any program that is built in a work environment where the original programmers will be replaced at some point is almost guaranteed to have one or more of the new programmers ignore the constraints at some point, rendering the contract useless. And unless you're programming in Eiffel, you get bad results.
Defensive programming, however, differs in that the receiving side will check the inputs and can complain when its expectations are violated. The big problem with Defensive programming is that it tends to clutter up the code, and foster paranoia. The compromise I have come up with is to but string defensive checks on things that cross object boundaries - the public methods, API calls, etc. It's generally easy to at least document the expectations within a class so that later developers will see them and follow them, but once you have to call something defined in another file/directory/subsystem, you risk not being able to find the details.
Sunday, January 20, 2013
Hey, some programming content, at last! Finite State Machines!
So I decided that I've been using this blog as a ranting place instead of an informing place. That;s not really what I was intending when I got started, so I'm going to try and focus on informational content for a while. Not quite a New Year's Resolution, but it's worth a try
Today's Topic: Finite State Machines
Finite State Machines (FSMs) are a powerful tool for handling program behavior in a flexible manner without losing track of the details. If you are not familiar with FSMs, here's the Wikipedia article on them, which is a bit obtuse, but gives some of the details.
Typically, you will use an FSM in a situation where you have a set of behaviors you need to model, and the actions that get taken on certain inputs depend on the state of the object. Most examples use mechanical devices, like turnstiles, automatic doors, traffic lights, and vending machines.
The real power of FSMs comes from when you combine a generator utility (easily found online) to generate the skeleton code for the state transitions, and the Command pattern to encapsulate the events/inputs. This leads to compact code for the transistions, and lets you concentrate on the meat of your functionality.
The GoF State pattern might be appear to be useful here, but that is more helpful when you have something that does multiple things depending on its internal state, as opposed to the FSM itself. The example in the link shows that this is less about the transitions and more about the actions, with the transitions not doing useful work, or being almost dataless signals. The typical FSM even is more diverse, being a value or a signal whose meaning will differ.
One of the drawbacks to FSMs is that the logic of the transitions can be absent from the code, making it difficult to follow. This is of course solvable with good documentation, but you need to keep it clear. Most of the FSM generating programs have a data file that drives the process, so keep the descriptions there, and link to the in the generated/skeleton code, or if you generate skeletons and then modify them manually, in that code.
One other common use of FSMs is in regular expression recognition, but that's fairly tightly tied into the fields of compilers and interpreters, which is a really complex subject to cover.
Today's Topic: Finite State Machines
Finite State Machines (FSMs) are a powerful tool for handling program behavior in a flexible manner without losing track of the details. If you are not familiar with FSMs, here's the Wikipedia article on them, which is a bit obtuse, but gives some of the details.
Typically, you will use an FSM in a situation where you have a set of behaviors you need to model, and the actions that get taken on certain inputs depend on the state of the object. Most examples use mechanical devices, like turnstiles, automatic doors, traffic lights, and vending machines.
The real power of FSMs comes from when you combine a generator utility (easily found online) to generate the skeleton code for the state transitions, and the Command pattern to encapsulate the events/inputs. This leads to compact code for the transistions, and lets you concentrate on the meat of your functionality.
The GoF State pattern might be appear to be useful here, but that is more helpful when you have something that does multiple things depending on its internal state, as opposed to the FSM itself. The example in the link shows that this is less about the transitions and more about the actions, with the transitions not doing useful work, or being almost dataless signals. The typical FSM even is more diverse, being a value or a signal whose meaning will differ.
One of the drawbacks to FSMs is that the logic of the transitions can be absent from the code, making it difficult to follow. This is of course solvable with good documentation, but you need to keep it clear. Most of the FSM generating programs have a data file that drives the process, so keep the descriptions there, and link to the in the generated/skeleton code, or if you generate skeletons and then modify them manually, in that code.
One other common use of FSMs is in regular expression recognition, but that's fairly tightly tied into the fields of compilers and interpreters, which is a really complex subject to cover.
Friday, January 11, 2013
Get Outta My Way!
You know what I really hate about software development process?
When the blasted parts of the process fail to work and make more work for developers!
Case in point - The Job has a number of scripts that are run when the product is near a final build for a release, that check for all the expected fixes being complete, all the process steps are done, etc.
So today I get the nasty email from the PM that I have some unfinished fixes. I look, and I've done everything that I am supposed to do, all parts are present, but the scripts are not finding them. And this was noted a week ago and nothing was done when I noted that part of it was clearly wrong (as in I showed the supposed missing parts by ID)
But the dysfunctional process favors PMs bothering developers instead of following through on responses.
I repeat the gathering of part IDs, and then have to re-save some things to trigger the scripts again. Why they do not run automatically on the production build, I'll never know.
When the blasted parts of the process fail to work and make more work for developers!
Case in point - The Job has a number of scripts that are run when the product is near a final build for a release, that check for all the expected fixes being complete, all the process steps are done, etc.
So today I get the nasty email from the PM that I have some unfinished fixes. I look, and I've done everything that I am supposed to do, all parts are present, but the scripts are not finding them. And this was noted a week ago and nothing was done when I noted that part of it was clearly wrong (as in I showed the supposed missing parts by ID)
But the dysfunctional process favors PMs bothering developers instead of following through on responses.
I repeat the gathering of part IDs, and then have to re-save some things to trigger the scripts again. Why they do not run automatically on the production build, I'll never know.
Thursday, January 03, 2013
The Horizon Effect
Today I got started on one of the more fun things in software development - removal of old code. Not a refactoring of functionality, but outright removal. It's been a while since I had the chance to do this.
It's cool, getting to excise code and make the codebase smaller and more efficient. The feature unfortunately was not wrapped in a simple method or methods, but the functions and data are all well-named, so it's a simple search anddestroy replace.
But I was musing that this was something that gets ignored by many shops - getting rid of old features that are unused. Old hardware, obsoleted features, invalidated combinations, all of these lurk in many (most?) applications, waiting to spring on the unwary. But I rarely see mention of this pruning in the literature - refactoring gets mentioned as a response to changing requirements, but the requirements-gathering always gets described from the viewpoint of a new project, not a mature product.
One of the hard parts is that it's often impossible to determine which features are no longer valid, because the reasons for them in the first place are lost in the mists of time (or Bob, the original architect, retired 2 years ago, and is sipping Corona on the beach at Cozumel). So I was thinking that it would be a cool feature of an advanced source code management system to tag a set of changes with an expiration/re-examination date, so that the developers would be reminded of the need to do this, and contain the explanation of the feature so that this later validity could be checked.
It's cool, getting to excise code and make the codebase smaller and more efficient. The feature unfortunately was not wrapped in a simple method or methods, but the functions and data are all well-named, so it's a simple search and
But I was musing that this was something that gets ignored by many shops - getting rid of old features that are unused. Old hardware, obsoleted features, invalidated combinations, all of these lurk in many (most?) applications, waiting to spring on the unwary. But I rarely see mention of this pruning in the literature - refactoring gets mentioned as a response to changing requirements, but the requirements-gathering always gets described from the viewpoint of a new project, not a mature product.
One of the hard parts is that it's often impossible to determine which features are no longer valid, because the reasons for them in the first place are lost in the mists of time (or Bob, the original architect, retired 2 years ago, and is sipping Corona on the beach at Cozumel). So I was thinking that it would be a cool feature of an advanced source code management system to tag a set of changes with an expiration/re-examination date, so that the developers would be reminded of the need to do this, and contain the explanation of the feature so that this later validity could be checked.
Tuesday, January 01, 2013
New Year Hopes, Plans, and Ideas
Well, it's now January 1st, and it's time for the obligatory New Year's post about resolutions, plans, and so on.
What, you thought I was going to do something different?
So here are my plans:
Professionally - improve my coding skills; work more on getting testing infrastructures built around my work code; start learning more about the other areas of The Job's new products; build something useful to the public in Python; learn more about VMs and use of the cloud.
Personally - sustain an exercise program for more than 2 months; work on my photography project; declutter the house (at least one item per day disposed of by trash, sale, recycling, or reuse); blog at least once a week; write some fiction every month.
Socially - well, let's just say I need to see how much time I have left after doing all of the other things.
I'll post later with the list of specific things of interest to technical types.
What, you thought I was going to do something different?
So here are my plans:
Professionally - improve my coding skills; work more on getting testing infrastructures built around my work code; start learning more about the other areas of The Job's new products; build something useful to the public in Python; learn more about VMs and use of the cloud.
Personally - sustain an exercise program for more than 2 months; work on my photography project; declutter the house (at least one item per day disposed of by trash, sale, recycling, or reuse); blog at least once a week; write some fiction every month.
Socially - well, let's just say I need to see how much time I have left after doing all of the other things.
I'll post later with the list of specific things of interest to technical types.
Subscribe to:
Posts (Atom)