Tuesday, December 19, 2006

Soft Issues

Last week I was forcibly reminded that people issues are sometimes more important that technical ones.
A Cow-orker dressed me down for an alleged shortcoming in my code - a naming issue in an interface.
To be fair, I had not commented the suspect code to note it was re-using an existing call, with different meaning for the parameter, and I had left some unused code in the file. However, the big issue is that I was accused of not just bad coding, but of something akin to thoughtcrime - I was willing to leave API code unchanged (to help the external developers) rather than change it (to help internal developers). The kicker was that the code under consideration was actually correct for the one use - I had re-purposed that call for an internal issue, making things much better for the end user, at the cost of two differing uses for a parameter in the call.

Frankly, the incident pissed me off for almost the whole week, and more so because it was totally due to the abrasive manner of this cow-orker. If I had been approached with a simple questioning attitude, the issue would have been explored and settled with no rancor. But because my technical skills were called into question, I took (understanable) offense, and the incident escalated.

The moral of the story is - never assume that you have the whole story for something in a system when that story involves something you consider stupid and a developer that has significant real experience. Always tread with caution, and a humble attitude will get you farther than arrogance, and will save you from eating crow.

I am interested in seeing if I get an apology.

Technorati Tags --
, , ,

Saturday, December 09, 2006

A Digression into AI

I was thinking the other day about the advent to true AI - an artificial mind, and that fraternal twin of a concept, uploading a human mind into a digital device.

When we have these, what are the criteria that determine if one copy of a digital mind is an individual? Making a new flesh person is a long process, and once made, their mind is their own thing. But a digital mind can be copied exactly to another unit (physical memory or whatever). Suppose digital minds are granted voting rights - what if they decide to replicate and register to vote? How could flesh minds ever win a democratic vote? Or should identical copies be considered just one mind, and any number of them count as just one "person"? But after a period of time, those copies would have different experiences and be different "people", would they not?

Just something to consider....

Technorati Tags --
, , ,

Tuesday, December 05, 2006

Look at what you have

Today I found myself somewhat nonplussed by the behavior of 2 cow-orkers who are not dumb, but did not appear to see the forest for the trees.

They were working on an attempt to get a 3rd party to provide a data set to us, for internal use, and were writing impassioned emails to said 3rd party to justify our asking for this data. When this was explained in my presence, I suggested that since a 2nd party was providing us with data that might reasonably include that info, we just get said 2nd party to give us the detailed format of their files (which we might already have), and we extract the desired info from those files.

They seemed rather stunned at this suggestion.

So the Ninja Developer learns from this: if you need some data, look at the data you already have, and see if what you want is what you already have.

Technorati Tags --
, , ,

Friday, December 01, 2006

Too Much Reality Lately

Alas, Dear Reader (I suspect that there is indeed only one person ever reading this...), the vicissitudes of wage slavery have kept me from posting...

[perhaps a confessional tone "Forgive me, Blogger, for I have sinned..."]

Nearly 6 months since the last post. I have been busily diving into new sections of the product, ostensibly to "broaden" my experience.

The Ninja aspect of this has been the lack of input from me on the schedules - dates were set by management without my being asked how long it would take to develop the code. Most deadlines were on the 8-week plan, with the notable exception of the first, which was 10 weeks, 6 of which were consumed by the requirements team pissing away on the requirements document, and the most recent, which went 7 weeks late, and one week pulled in by the VP.

So what is a Ninja to do? Major changes to the product and not enough time to develop it properly? I punted - I developed the new server in Python, driven by text configuration files, and wrote 2 small C++ interface utilities to reach into the product where Python could not. I avoided working significant overtime, and came in almost on schedule - and I plead my case there on the many meetings that wasted my time. ["We will hold daily 2-hour meetings to determine why productivity has plummeted. All the VPs will be there to hear explanations"]. And the decision not to work excess hours lent a Zen-like calm that helped. I did not panic over delays or setbacks.

The lesson to be learned here is that you sometimes need to work outside the system - use a different language, ignore the micro-management, and solve the problems without worrying about how to fit them into the existing framework. I got the chance to use Python in a production project, and I didn't kill myself.

As an aside, I really have to compliment the Python developers - the suite of modules available in the standard package is outstanding - FTP library, tools for handling gzipped tarballs, easy access to the command line, all in the "stock" package. Kudos, folks!

Technorati Tags --
, , ,

Thursday, June 29, 2006

Reminders of reality

My job brought me a dose of reality this week.

No matter how much Management claims to want to produce good software, they don't want to give up the idea that they know better about how to produce it.

Specifications that are never frozen - schedules that do not include the time for documents required by company process, etc.

The Ninja Solution(tm)?
You need to understand your software well enought to have the basics of a feature hashed out in a hallway conversation with the other developers. If you have clean interfaces, and well-factored classes, you can have the design in mind before they ever finish the spec, which will change anyway.

Technorati Tags --
, , ,

Sunday, April 23, 2006

The First Rule about Optimization Club....

...Is you don't talk about Optimization.....

[yeah, cheesy, but this is the web.]

I ran across this, an article about code optimization, and I gotta say, it's dead on. I've been burned by bad guesses as to where a program was inefficient, and using a profiler to start with is vital.

I do think that the final steps of optimization need to be weighed against the maintainence load of tighter code, and only done if the program needs the speed.

Technorati Tags --
, , ,

The First Rule about Optimization Club....

...Is you don't talk about Optimization.....

[yeah, cheesy, but this is the web.]

I ran across this, an article about code optimization, and I gotta say, it's dead on. I've been burned by bad guesses as to where a program was inefficient, and using a profiler to start with is vital.

I do think that the final steps of optimization need to be weighed against the maintainence load of tighter code, and only done if the program needs the speed.

Technorati Tags --
, , ,

Wednesday, April 19, 2006

When Managers Attack, Pt 1

Schedules are such fun.

So we have a tentative schedule for the next product release, and so we set a date for code complete. After a few managerial meetings, the Powers-That-Be decide to add a couple of new features. Fortunately, they also added 2 weeks to the schedule. So my boss publicizes that to the group, and we're happy.

So early this week, the Higher-Powers-That-Be decide that we need to pull the schedule in. A quick analysis determines that the new features are not actually affecting the code in my system, so we can safely cut the additional 2 weeks off again.

And now, they realize that they had not specified a feature, and so they add it into the schedule, but DO NOT extend the deadline. No idea about impact, no idea as to effort. So I do what every good Ninja Developer does - consult the experts and determine that there is a clever, clean design change that will be easy to do.

Technorati Tags --
, , ,

Sunday, April 09, 2006

I Told Ya So!

A few posts back, I discussed offshoring, and I mentioned that one of the big things standing in the way of American companies when they try to hire in India is the relative volume of applicants. Browsing Google's gmail feeds I found this article, which details the problem confronting Inidan hiring - the sheer volume of people they have to process in the short times. It also notes that there is an impending skills shortage there. All this is good news for teh American IT/software market - as offshore talent gets scarce, their salaries rise, and it becomes less effective to move the jobs away.

Technorati Tags --
, , ,

Wednesday, March 29, 2006

Are Geeks Countercultural?

It's hard to be a real geek these days.

Online gaming is almost mainstream; everyone is hooking up through MySpace or some other similar website; collaborative sites like Flickr abound; everyone has email.

Even geek fashion is everywhere - between the 70's revival and Napolean Dynamite, plaid is scarce.

During the dot-com boom, computer people were trendy and popular, and everyone wanted to be a webmaster. We're still in the halo from that time.

So geek culture is slowly inching its way into the mainstream, but from a flanking angle.

We're not yet totally accepted, but we're no longer pariahs. The schoolkids are still vulnerable to Lord Of The Flies scenarios, but less so after Columbine.

The downside is that with this explosion of mainline geekiness there is a greater capability for the less-geeky to better harass those they dislike. Online bullying is almost trivially easy, and the posting of embarassing photos to websites can draw volumes of ridicule from across the nation.

Technorati Tags --
, , ,

Wednesday, March 22, 2006

I'm Baaa-ack!

Apologies, Dear Readers (all 3 of you), for the long delay. My employer was busy being bought out by a larger company, so I was distracted by much reality.

The purchase process was interesting in a cynical way. There was a great deal of enthusiasm from the buyers about how great it was that the two companies were combining - too much enthusiasm. My cow-orkers and I began to make Jonestown and Heaven's Gate references. There was much reference to "synergy" and "Actualization".

All this has fed several ideas for later columns.

Technorati Tags --
, , ,

Sunday, March 12, 2006

Automation, a developer's best friend

This is not going to be a big post, but I definitely wanted to push automation as a Good Thing.

You don't need to go whole hog, with an expensive framework supporting email and IM's the build admin when things go wrong. Shell scripts, SQL, and text files will do.

Automate your builds, so you can build any part of your system with a single command.

Automate you build notifications, so the build admin can be aware of errors ASAP, and notify your developers.

Automate your regression tests, so you can make sure you don't lose anything when you add new features.

If you are fortunate enough to have management that supports agile development, automate your new feature tests so you can confirm correct function of the new stuff.

Where possible, make your GUIs build automatically from text-based description files, so you can enable/disable features without requiring code changes.

Technorati Tags --
, , ,

Thursday, March 02, 2006

There's never just one release

One of the biggest issues that I've had career-wise has been the even distribution of new development work with support work. One of my past jobs, I ended up spending all my time supporting the released products, and no time developing the new products. This caused problems because the support effort was not well-regarded - it wasn't the sexy new stuff that fired everyone's imaginations. I got quite fed up about it, so it's now one of my discussion points at interviews and reviews.

So when you planning software products, be aware that someone will have to be the support team, and they either have to like the job, or you'll have to have a rotation between the people to prevent burnout.

Support duty is hard for many developers, both from the technical side and the glamor side. Some of the difficulties are:
  • It's difficult to keep answering the same questions time and again from the different customers that run across an issue.
  • It's difficult to see where a customer is having a problem when you know the code well - you expect things to go as you designed them, and not as the user does them.
  • you have to make the quick fix, because the customer can't wait long for a solution.
  • you don't get to exercise your design muscles because the product is released, and nobody will want to rework it.

Technorati Tags --
, , ,

Friday, February 24, 2006

The Obligatory Hacker Culture Post

Ok, so I'm predictible.

As someone who discovered computers just about the time the idea of the PC was reaching the market, I'm one of those who has been fully immersed in the hacking culture. I never was as hardcore as the guys who wardialed the Pentagon, or build my own computer from NAND gates, but I got my PC as soon as I got out of college, and have been programming in some form for almost 30 years.

For me, the defining hacker trait is curiousity. You want to know how something works, or want to build one for yourself. It manifests most strongly in computers, I think, because software is so malleable that once you've bought the computer and get a compiler/interpreter, you can make nearly anything. In contrast, electrical engineering hacking requires you to buy parts, which limits your work. It's also much easier to prototype in software than in hardware, and you almost never blow up parts in the testing.

At the present time, it's a sad truth that the US is drifting away from the hacking spirit. The advent of offshoring has made software a precarious career, and colleges are starting to move their curricula away from the fundamentals of software. So many of our products are packaged so that there are no repairable parts in them. But a few rays of hope are shining - Make is publicizing hacks, and the very flood of disposable technology is spurring hackers to try things, since they are so cheap to buy.

I hope the trend picks up speed. I'm looking forward to some of the cool things lurking around in people's skulls.

Technorati Tags --
, , ,

Tuesday, February 21, 2006

Software Tools

These days there are a great many tools that developers can use to create software. There is UML and the various products that use it. The Java world has Struts and its successors for application development. Ruby on Rails is currently enjoying popularity as a web application development platform.

What all of these have in common is a development style that they will tend to enforce on your team. This is not a bad thing, but it is something to be aware of, especially if you are using two or more with styles that conflict, or any that conflict with the company's process style.

An example of this is most UML products. They tend towards the process-heavy side of the spectrum, and recommend the full Analysis/Requirements/Design/Development/Testing cycle. If you are working in a Agile style, these tools will be a significant deviation from your accustomed process, and you will need to take this into account when shopping for the UML tool.

Another example is Rational Rose. One of the earlier versions was integrated with a relational DB mapping tool called Persistence. You could draw your UML class diagrams in Rose, and drop them into a generator that would generate the files that Persistence would use to generate the database access library code. The wrinkle came in that there was no way to split the UML into sections and have it generate database code that referenced tables not in the specific UML diagram. So you ended up with one huge monolithic database library, that took a long time to generate and build, making it painful to make database changes; the alternative was to split the UML into logical sections, but you would lose the database integrity checks, because the tools could not link across diagrams.

Technorati Tags --
, , ,

Saturday, February 18, 2006

Thinking Outside the Box

Sometimes you gotta stretch the definition of things...

If you have a new application that needs interprocess communications, but you don't want to build the whole infrastructure, consider using something like email or instant messaging for IPC. Many modern languages have libraries that support email, so it's not a difficult thing to add, and allows you to take advantage of the entire internet connectivity. You also can use it for communications back to developers, when significant errors occur.

Instant messaging can be used as well, but there are fewer libraries built-in to langauges. It's faster and more connection-oriented, but less well-supported across the networks. You will have a variety of IM protocols to choose from, and a plethora of servers to run, should you choose to run a private server.

Another useful hack for developers is to use Gmail as a poor-man's FTP utility. Since you have over 2.5GB of space on an account, it will be large enough for most any transfer. You probably will want to encrypt your files, however, to preserve security. This will pose a slight problem, as Google blocks encrypted archives. Never fear, because you can use something like uuencode to convert an encrypted file into pure ASCII. After downloading the file, uudecode will convert it back into its original binary form.

Andrew Tannenbaum has been quoted as saying "Never underestimate the bandwidth of a station wagon full of tapes shurtling down the highway". As network speeds have increased, so has portable storage density, so the saying is probably as true now as it was then, with only the vehicle and media changing (I'd probably reference a minivan full of USB thumb drives, personally). Sometimes the brute force approach is the best combination of simplicity and utility. If you need to be able to do a rapid upgrade with the ability to just as rapidly return to the initial setup, look into the cost of simply making a complete copy of the machine in the new configuration, and perform the switch by swapping the network connections.

Technorati Tags --
, , ,

Friday, February 17, 2006

How Not To Be Seen

It's relatively rare these days to be part of a team that is just forming. Most of us start a new job with a bunch of existing developers already working on the product. So there is a period of adjustment as you learn the new team's process and culture.

One of the worst things to do during this time is to start commenting on the team's practices in a critcal manner, even if they are suboptimal. This will not endear you to your new colleagues.
  1. Don't assume that they are not reading current journals. If they are not taking advantage of the latest technology, it may not be because they haven't heard of it.
  2. Don't act surprised if they don't have all of the Best Practices implemented. Company politics, inertia, and scheduling may prevent them from implementing them
  3. Don't try to impose your favorite practice. You will appear arrogant. No matter how good an idea it is, you will piss people off.
  4. If you really wish to change, build support from within. No group welcomes an outsider trying to change things by fiat.
  5. Don't go running to the manager to get things done without first. It looks to coworkers like you can't support your idea, and want to short-circuit the team's evaluation of the idea.
  6. Never talk about how great the process was at your last job. The team has had to work under this company's system, and they are probably making the best of it here.
The executive summary: Don't assume that the team is full of idiots, and don't act superior.

Technorati Tags --
, , ,

Monday, February 13, 2006

Hints for new programmers - someone else's view

I was poking about today and happend across this article about what advice to give a new programmer. Overall, I agree with it, except for the parts about a coding standard, Singletons, and comments.

Singletons are more than just global variables. Even if you code them to just be wrappers around global variables, you've gained some encapsulation, and that will help you when you need to track changes to something global - you put a trace statement into the set function for the value, and you'll see where it's getting changed.

A single coding standard is important - the article shows a small function with 2 different styles and states that it's easy to work out the code flow and intent. Well, sure it is, when you've got only 5 lines of code. Where a coding standard shines is in functions that are longer, with more complex interactions. I had the misfortune to be looking at some code recently that as far as I can tell had been developed by at least 3 different people, each with a different indentation setting and brace style. I found differences in one function (about 30 lines) in intertwined sections of the code. Following the logic was difficult because I had to keep going back to the start of each clause to see if I was exiting a layer. If you can't get your development team to agree on a standard, enforce one with an automated tool - you will be thankful later.

And finally, comments. The article does recommend using comments for the WHY and not the WHAT, and rightly ridicules commenting each change in the top of the file, but if you don't comment on the why of each non-obvoius path in the code, the developers that come after will miss it.

Technorati Tags --
, , ,

Sunday, February 12, 2006

Developer Tip: Yet Another API point

Just a quickie -
When you write the functions for an interprocess API, don't put the work part of the API in the catching functions. If you put the guts of the work into the catching function, changes to the API directly affect the work part. Instead, if you have a small dispatching function as the catcher, and delegate the work to another function, you can reuse that function for later version fo the API functions, without needing to completely rewrite the work function.

You will need to make the work function fit the newer version of the API, and the older version catcher function will need to be modified to send the new parameters to the work function, but that is relatively minor work compared to a total clone, or complete rewrite.

Technorati Tags --
, , ,

Wednesday, February 08, 2006

Developer Tip - One Code Stream To Rule Them All

Here's something that I recommend that might fly in the face of conventional wisdom regarding the use of your CVS to manage multiple branches of code in the corporate world - if at all possible, DON'T!

It's not because the tools can't handle them - most modern CVS tools are adequate for managing separate code streams. It's the overhead in managing the streams that will kill you. Every bug found will need to be tracked once for each stream, fixed once for each stream, and tested once for each stream. If you have different code in modules along the branches, the fixes are more complex, since you can't just plug in the same lines of code.

If you have a fluid feature set for a given release, as is unfortunately common, you'll find that features jump between releases depending on:
  • Progress - if the work can't be done in time, the feature moves to the next release.
  • Contractual obligations - someone signed papers promising it earlier than previously scheduled.
  • Hail Mary business deals - the future of the company depends on closing this deal, and this feature is the only way to do that.
  • Customer release cycles - a common problem. The customer wants the feature by time N, and the schedules are drawn up, and suddenly the customer decides they need 6 weeks of certification testing to accept the new code you give them, meaning that you have to pull the important features back into the current release to get it to the customer before the certification deadline.
  • Customer A doesn't want Feature X, but Customer B does. Several releases later, Customer A wants Feature X, after a lot of divergence in the code streams.
So how to solve this?

For every feature in the product, have an activation flag. You may need more than one flavor of this, to account for the differences between optional features that everyone gets to choose to activate or not, and bespoke features that only some customers will be allowed to activate, or even know about. In any case, build a library that will have a set of simple functions that will determine if a given feature is in use at runtime. Don't forget multi-valued options, like a staged conversion between file formats - you'll have a value for pre-conversion, mid-conversion [when both old and new formats are valid], and post-conversion.

Once a feature has been defined, write all the code to handle that feature in all of its incarnations, so that there is no need to branch your code stream. You may need to branch for customer-specific configurations to present the options in a desired manner, but that code should be very small, and managing it on a per-customer basis will not be taxing.

Another important facet of this is to concentrate the decision-making for a feature in as few places as possible, and have the rest of the code handle all possibilities without checking for feature activation. An example of this would be a system that talks to some specific type of equipment only if licensed. The communication code should be written to handle messages from all types of equipment, and the next layer up would be the code that decides to send messages to that type only if licensed, and to respond to any message from that type only if licensed. The fact that messages from an unlicensed type of equipment are passed up to the higher layer instead of being rejected at the lower layer is not a problem - the logic of licensing has no business in the communications layer of a system.

This will develop your system as both flexible and simple, and spare you the headaches of trying to port features into radically different code streams. Your CVS tool will present simple trees instead of briar patches of inheritance. If you use a database, the schema will easier to upgrade, since there will be no releases that have different parts of a later schema - they will all be the same, or newer ones will be supersets of older ones (barring removal of tables, in which case they will be subsets, but functionally complete subsets).

Technorati Tags --
, , ,

Tuesday, February 07, 2006

In on the ground floor

Let's talk for a minute about the ideal software developer situation - when you are there before the product exists. Getting in on the ground floor is exhilarating; there is no cruft in the code, there is no backwards compatibility to maintain, there is no build-up of sludge in the process. But it's also a challenge - there is no structure to work from, no library of existing code that you know fit your problem space. So what do you do?

If you've got a team already, then you probably have a work style that the team is comfortable with. Take a few days to meet and analyze this, to see if there are issues that need to be handled, and if there is anything the team really wants to change. Get consensus, but you don't need unanimity - as long as the dissenters are recognized and taken seriously, things can proceed.

From that point, you have a number of things to decide:
  • pick your code management system. Make sure it fits your development process, and your company's release style. If you don't plan to have more than one code stream, things like RCS and SCCS may be perfectly suitable. Otherwise, something like CVS, or Subversion may be more suitable. For large companies, something like Telelogic's Synergy or IBM/Rational's ClearCase might be mandatory. In any case, settle the question and set up the project in the system.
  • System startup and shutdown. Decide how the system will start and stop. If the product is a single program, this may be a no-brainer - a command line or desktop shortcut may be all that is needed. If the product is a full suite of programs that need to be running all the time, you may need to interact with the operating system in some way - inittab for UNIX-style OS's, for example. Don't forget to examine the need to stop the system for upgrades!
  • Configuration. Determine how your applications will get their configuration. The two obvious choices are a database or a configuration file. If the application does not have a DB, then file(s) are your only choice. If you have a DB, then you can choose. The factors influencing you will be the type of data you need for configuration, and the amount of data needed. Be sure to make your programs capable of re-reading configuration on demand, to make runtime changes possible and easy.
  • Logging. Your programs will need to log abnormal conditions, errors, and other information. Decide how you will need to report this data. On UNIX-like systems, syslog is a good choice. if you want to piggyback on the OS facilities. One thing to consider with your logging system is rollover - you do not want to fill up the disk with one large file. Design the logging system to allow the log file to be moved out and replaced, and manage the saved files so you don't fill the disk. Another consideration is whether or not you have multiple copies of a single program running - your file naming scheme should make allowances for this, as well as the rest of your logging scheme - if you have a single logging process that connects to the work processes by name, or some such.
  • Interprocess communication. You've got a lot of options - RPC, CORBA, plain sockets, shared memory. Don't overlook some newer options, given the versatile libraries of modern languages, of email and instant messaging; also don't discount such old-school options like using the database, or files on the disk as communications channels. Build your libraries using the minimum number of options, but keep the interfaces clear (see my previous entries on this )
  • Coding Standards. I'll address this more some other time, but work out a coding standard for your project with the developers, and if possible, make that format automatic - use hooks in the CMS to convert files into that standard upon check-in. This will prevent disagreements from stubborn developers - they can code their own way if they insist, and the code gets dropped into the common area in the standard form.

That's at least a few of the things that I feel important about a starting project.

Technorati Tags --
, , ,

Thursday, February 02, 2006

Developer Tip: Defining an API

Ok, enough blather about theory, company politics, and other crap. How about something useful to an everyday developer?

When defining an API, make it as simple and as obvious as possible what data is actually being passed across the API.
  1. This will make the API much easier to document - the names of the fields in the structures/objects/messages will be mnemonic, and you can document them as comments in the IDL file.
  2. You can use an IDL translator to convert from one IDL to another, like RPC to CORBA, or from one language to another, like C to Python
  3. You can automate the argument marshalling code if you want to write one or both sides of the API in another language.
An example I have seen that violates this was an API that wrapped all the arguments inside functor objects that were templates typed by the number of arguments they took. It's a great utility mechanism, because all your calls across the API are compact, taking one argument, and no return value because the functor inside will handle all that when the callback is made.

HOWEVER, this API was aboslutely miserable to develop with, because you had to go deep into the code on both client and server sides to discover what the arguments being passed really were. We wanted to build a test framework to exercise a server, but we had to manually build the structures populate the arguments, instead of being able to build them automatically from the IDL files.

A counterexample is a typical RPC interface to C programs. It will have a number of structures defined in the .x file, and it's a small matter of programming to build, buy, or Google a program to parse the .x file and produce the other files to build a test scaffold from in the language of your choice.

Technorati Tags --
, , ,

Monday, January 30, 2006

A bit of explanation

Why is this blog called "Ninja Software Development", you may ask?
(Well, you might ask. Someone might. Ok, nobody would, but I'm going to explain it anyway)

It's not from a fannish infatuation with the Ninja as super-stealthy assassins, or an anime fixation. The key elements that I wished to convey were the secrecy of the ninja, and their resistance to a stronger force. The forces of business are arrayed against the development of good software, and the developers need to work behind the scenes to counter the greater army.

A colleague of mine once commented that it was a sad state of affairs when developers had to resort to skunkworks methods to improve software quality. He was right, in that the business risk management practices often require the reduction of change, which mean that quality changes will be delayed, and the business case of quality improvements is hard to make. And that, given the current business climate of making the numbers for the current quarter, means that nobody wants to spend time developing things with no new features. So the only way to improve software quality is to do so stealthily, since process improvements are equally difficult to make a business case for these days. Refactoring is done as part of a new feature; algorithmic changes are slipped in as a bug fix. And thus are we able to keep entropy at bay.

Technorati Tags --
, , ,

Sunday, January 29, 2006

Lots of programmers == lots of trouble!

Yesterday I backranted about offshoring, particularly to India. I discussed a number of things that I feel are difficulties for corporations offshoring to India. I forgot a more fundamental difficult for the company wanting to hire India programmers.

If they are intending to hire directly, they will be running into a problem of scale.

For a typical American job opening, a company may get (for example) 100 applications. About 50 will be dropped immediately for various reasons. Around 30 will be dropped at first resume review for insufficient capabilities. From the remaining 20, 10 will be picked as the best qualified, and get phone interviews. The field will be narrowed to 4, and the interviews will pick the best 2 of them to make offers to. When all is done, the company can feel reasonably certain that they have made a good choice in hiring the best qualified candidate.

Now consider this in India - where the number of incoming resumes is an order of magnitude larger. Instead of 100 applications, there are 1000. Assuming a typical bell curve (India may have a lot of good programmers, but statistics says that they will have an equivalent number of bad ones!), 500 of these will be dropped immediately, for the same sort of reasons as the previous example. Another 300 will fail to pass a resume review. This leaves 200, which will be sorted to 100 "best fit" candidates. Now, phone interviews for 100 people will be much less effective at narrowing down the field, compared to interviews for 10 people (assuming that phone interviews are even done in India, in which case this point is even more valid), so it will be a much more random selection out of the final 100 to get the 10 to interview. This boils down to a less-good fit for the selected candidate. And this does not even consider the amount of time to interview the larger number of "final" candidates!

Now, some would suggest that this can be countered by using local firms to do the hiring. I don't think that this will improve the situation much, because if the people hiring are not those who know what the position requires, they will not find people who match the need.

Technorati Tags --
, , ,

Saturday, January 28, 2006

Software Infrastructure and You

In a recent rant, Rockford Lhotka tears into developers as spending too much time on the software infrastructure and not enough on delivering business value. He uses a Visual Basic application written some time ago as an example of something that worked just fine then, and is even better now that PCs have increased speed by at least one order of magnitude.

As a developer, I quite naturally take offense to this. We are not, by and large, avoiding improving the product by playing with the background stuff. Here are a few counterpoints:

  1. Business app are rather pedestrian to write, but the bigger question is why are we still writing them? Because someone wants something a little bit different from anything else out there, and instead of living with the existing applications, they want someone to build an application that fits their precise needs. IBM had a small business suite back in 1985 that covered everything, and I mean everything, so all businesses should be using that app now, right? But instead, a business-type wants to be creative, and requests new software.
  2. He says that the customer doesn't care about infrastructure. That doesn't mean infrastructure is not important. The customer may not care about the condition of the roads when he asks for his supplies to be delivered to him and his products to be delivered to the buyers, but bad roads will damage trucks, old narrow roads will delay trucks, and low overpasses will stop trucks, and the customer will lose money. The infrastructure of the dialup days will not handle today's DSL/cable modem traffic levels; today's high-perfomance network no longer needs the heavy error-correction protocols required by the slow and noisy networks of old. A prime example of this is email - remember when an email address was a series of machine names separated by "!"s, with other odd characters in the last part? It was no worse an email system then that it is now (as far as getting a message from one computer to another), but the infrastructure changes (DNS, STMP/IMAP/POP everywhere, and generally universal TCP/IP connectivity) have made email much better for the end-user. [And due to some original decisions about this infrastructure, worse, from SPAM, because the infrastructure was not developed with expectations of anti-social use. To fix this, there is ongoing infrastructure work in a number of projects]
  3. He bemoans the trends to web services and similar things away from the older methods. In my experience, one of the biggest forces in driving changes in infrastructure is the business press. I recall one boss asking I investigate moving our application to a CORBA interface, because he had read about CORBA in the business press and thought to jump on the bandwagon. There was no reason to move to CORBA at that point - our application was only talking to other applications we wrote, and the socket-based protocol was adequate for our expected needs. Other times, the move to new technology is mandated by the government, like the demise of analog TV signals.
  4. Another case is that sometimes the customer drives these changes. The big draw of web-based services is that the customer does not need to install something on every PC in the organization in order to use the application - each user has a PC with a web browser already on it. Or the customer favors something else. I know of one vendor whose application was chosen over another vendor's because it had drop shadows on the buttons, while the application not chosen used Motif widgets that "only" had bevels on its buttons.
Sure, given the choice, I'd rather develop something cool, instead of Yet Another Accounts Payable system. But plenty of customer-value things are cool - Firefox/Thunderbird, flickr, deli.cio.us, blogging, P2P software, VOIP, and so on - we developers hardly need to play with infrastructure to be doing cool stuff.

Technorati Tags --

Offshoring Software - a Developer's Opinion

A brief scan of Google's Blog Search looking for other blogs on software development and I happened upon a post by a Rockford Lhotka about how software is too hard, and that the developers have brought offshoring down on themselves because they are always re-inventing things that already work well enough for the business purposes.

Well, I have an opinion about that (qu'elle suprise)

Offshoring is driven almost totally by costs. Companies look at what it costs to develop software, and it's significantly cheaper to do it offshore in India (or sometimes China), because of the much lower cost of living over there. The additional expenses of getting a reliable development center set up, sending managers over there to hire and spin up the offshore team, and communicate are smaller than the saving on salaries and benefits. So the company have a lower total cost to develop, and that looks good to them.

There are 2 flies in this profitable ointment, however: the offshoring boom is driving Indian developer salaries up fast; and dealing with a remote team from a different culture, with significant communication barriers, makes it difficult to develop good software.

Let's take a quick look at the first point - salaries. Indian cities like Bangalore, Hyderabad, and Chennai (formerly Madras) all compete for the local developers. The boom has brought many US companies to the country, where they have lots of jobs to fill. So they offer better salaries to attract people. Then the next big company arrives, and has to offer bigger salaries to draw people their way. Sure, there's a long way to go before they cost as much as Americans, but they are cutting into the "big savings" with every raise. A corollary to this is turnover - as salaries rise in Bangalore, the Hyderabad developers see a chance to better their lot by finding a job over there; 2 months later, Chennai salaries create envy in Bangalore developers.

Now for the second point - developing with a remote team. I often find it ironic that American managers who won't let their staff telecommute "because if I can't see them, how do I know they're working?" are willing to create an entire department half a world away where they will meet face-to-face no more then once a month at best. Indians in software all speak English fairly well, in theory. But quite often the combination of accent and speech patterns conspire to make them difficult for Americans to understand, particularly over the phone. This make collaboration less efficient, and truly harder to resolve misunderstandings.

The time difference is another factor. If the finishing touches to a product must be made in the US, the Indian team will need to have finished them the previous workday, meaning 36 hours ago, to allow for their changes to percolate through most CMS systems. Even if your CMS system can take changes rapidly, the 10.5 hour time difference means that the morning is spent pulling in the India work and rebuild the application, which for local development could have been done in the overnight hours. And if there are bugs, the Indian team has gone home for the night, making it harder to get their input to fix things. So the Indian team has a deadline that may be 2 days earlier than the local team.

Another issue is the development culture. American developers, especially after the dot-com boom, are wont to question everything about a design. While this can be very frustrating for the designers, it does tend to flush out any problems with the design, the design documents, and sometimes even with the project itself. I have found a fair number of errors in the documentation of a design by questionsing the intent. From anecdotal evidence, significant numbers of Indian developers do not question such things, for whatever reason, leading to incorrect software. It may be that they see the inconsistency, but do not feel comfortable bringing it to the attention of the designer, or that they assume that the designer intends it for a reason, but does not explicitly state the reason. Whatever the cause, it does not further the project. Add to this the reluctance the remote team may have for reporting difficulties they are having back to the home office - few people like to report bad news, and the cultural divide makes this reluctance greater.

A large proportion of Indian developers have come out of longer academic careers than typical for American developers. This is problem in a production environment, regardless of the location of the developer, because the rules of software development in academia are quite different than in business. Many former academic programmers have difficulty transitioning to an environment where the projects are usually quite detailed on how it looks, and less detailed on how it works, yet requiring high-perfomance output. I've seen fresh-out developers struggle over what to name database columns, because their boss did not give them a detailed naming plan. Other common academically-induced quirks are oddball variable naming schemes (sports teams, etc), and a really rigid view of commenting style.

[I hasten to add here, that I'm not in the least saying that Indian developers are not smart, or capable of developing killer software, but rather that in the context of developing software as part of an American company, they have some hurdles to reaching peak performance, most of which are purely part of the distance and differences in language and culture. Such issues arise between American and European development teams, as well, but the greater commonality of history makes the cultural issues a little less complex.]

And another thing - one of the key issues with a remote team is being able to provide them with good documentation - requirements, architecture, design, and so forth. Many (most?) American development shops, frankly, suck at development documentation. So, no matter how great the Indian team may be, they may not have any chance of developing the desired product because they don't know features to develop!

Part 2 of this will address the other side of Rockford's rant - that developers spend too much time fiddling with the internals of software and not enough time working on business value.

Technorati Tags --

Thursday, January 26, 2006

Meeting your Schedule - A Few Tricks

So your boss has just walked in and informed you that the project has 4 new requirements from the customer, one of which forces a redesign of some basic part of the project. The deadline doesn't change, of course, because that was set in a contract a few months ago, and the customer would be shocked at the suggestion that these newly-discovered requirements should make the project take longer!

I can't tell you how to redesign your project to meet the new requirements, but I can give you some tricks that, taken at the start of the project, can make your life easier when this situation occurs - and believe me, it will occur.

  1. Buy and read "The Pragmatic Programmer". I cannot stress enough how utterly cool this book is for developers. This book codifies a great deal of wonderfully useful information.
  2. Put your program configuration into a file or a database. Resist the urge to use environment variables. If you make your programs read from a database or file from the start, and you have a trigger to re-read the configuration when desired, you will be able to adjust configurations on the fly.
  3. Put your configuration in a Singleton object, and have methods that return the various values you need from this object. The re-read function can be done either as an explicit method, or part of the constructor, where the signal to update the configuration will delete the configuration and the next use will resurrect the object from the modified file/database.
  4. (added later) If you have a configuration option that is intended for a old feature/new feature type of thing, write the code to default to the new feature version in the absence of any configuration, and only do the old feature stuff if the configuration is set to 'use old'. This will allow you to phase out the configuration flags as the old features drop of the supported list, and your code will be cleaner as you delete code the references the unused configuration flags.
The point of all this is to make it easy to introduce flags and values for new features into your system, so that when the boss tells you that there are new requirements, you at least can have your programs detect these variations easily and in a consistent manner. You need to build this in from the start, so that the code will have only one method of getting configuration information and you will not have to spend much time tracking down oddball data accesses.

Technorati Tags --

Wednesday, January 25, 2006

Development in the Corporate World, Part 2

Schedules are the trickest part of software development, because it is difficult to predict the future, and predict software complexity from a distance. Predicting just one would be a feat to rival the Amazing Kreskin, but the two together....

From a business standpoint, schedules are vital; businesses must coordinate their activities, and the schedule is the major junction of alignment. So the use of a schedule is seen as a binding agreement, the Holy Grail of the endeavor. So it is set in stone, early in the process, and rarely negotiated.

From a development standpoint, schedules are vital, too. But they are fluid - things change as design uncovers hidden assumptions, and new requirements are uncovered. As new work is added, the sane assumption is that the schedule will expand. But because of the business importance of the schedule, it cannot expand. So in the grand tradition of "Good, fast, cheap; pick any 2", the time dimension is fixed, and the only variables are cost and quality.

Now, compunding this is the problem that attempting to maintain quality by spending more money doesn't work. If the bosses intend to give bonuses to the existing team to reward them for overtime (a rare and noble group of bosses, indeed), it will work to a certain extent, until the team hits the wall from overwork. If the plan is to increase the team, they run into Brook's Law - adding manpower to a late project makes it later. The effort of bringing the new people up to speed, and keeping the larger team in communication will swamp any benefits of added manpower.

So it appears that the only likely outcome is for quality to go down.

Now that I've painted a doom and gloom scenario, I'll give some suggestions on how to better survive when you're working in a shop where schedules are set in stone.

  1. Learn how to estimate your work accurately. If you don't have a good feel for how long it will take you to develop a piece of software, you can make accurate estimates
  2. Learn your company's overhead load. Some companies have a process that's very streamlined - developers are allowed to spend most of their time developing. Others have lots of meetings, documentation, etc., that consume time that would otherwise be spent developing. Knowing what percentage of your work day will really be free to use for develop the software is critical to your estimation process. Your original estimate should be adjusted by this factor to get a 'real day' estimate.
  3. Get a feel for your boss's estimate slop - how much does s/he adjust your estimate when adding it to the overall schedule? Some managers take your words as gospel. Others, more cautious, add a few days/weeks/months to your numbers to get a 'safe' schedule. Knowing this will allow you to pad your estimate a bit to account for a brave manager's lack of adjustment.
  4. Learn how often developers are called on to support code in the field. Field support always trumps development work, because paying customers trump potential customers. This, of course, introduces a whole new set of issues with schedules, because at any moment your development time may be pre-empted by a support call that may take the rest of the day and into the night, as well as blossom into a string of meetings, conference calls, and possibly site visits. So the percentage of time you might spend dealing with the field will be a factor in your estimations.
Next: Tricks of the wily developer, or how to prevent mental breakdowns when they move your dates.

Technorati Tags --

Monday, January 23, 2006

Development in the Corporate World, Part 1

"The difference between theory and practice is smaller in theory than in practice" - Unknown

When I was in school, learning software development, there was only one accepted methodology in the industry - what is now called the Waterfall model, ironically proposed as a flawed approach. Despite that, even the iterative variation of the Waterfall model was influential in supporting the rigid separation of the phases of development - Requirements was to be finished before Design started; all Design was done before Development was started, etc. As the referenced Wikipedia article notes, many forces work against such a system being successful.

The first and foremost of these is that, except in government contracts, requirements gathering is never finished until the product goes obsolete. Few customers have a complete idea of what they want at the start of a project, especially one that is a new product for them. The initial idea they will have will typically be a computerized version of their existing workflow. As the storyboards are presented, they will usually OK them without much thought, spending most of the time debating niceties of field placement, drop shadows, and background colors. The Requirements Document is blessed by both sides, and Design begins. Once the Design Document is produced, reviews will start.

Document reviews can be a Hell all their own, but in most cases the customer does not have the expertise to critique a design in detail. Once they see the storyboard for workflow, however, they start to feel comfortable to assert themselves. They have some new ideas, or perhaps their workflow has changed in the meantime; sometimes government regulations have changed, or a new technology has emerged to catch the fancy of the VP. [I once lost a job when the Director in charge of my project was distracted by the new Apple Newton, and withdrew his attention from the politics of my project. But more about that another time.]

So new requirements roll in from the customer, after the Design is supposed to be done. But by now, the schedule clock is ticking, and so some development needs to be started, or the project will be late. So the team starts development, and the analysts respin the Requirements Document, and the designers respin the Design Document, and another review ensues. With luck, the new design does not seriously impact the work already done, but sometimes it will invalidate much of it, requiring a fresh start by the developers, and wasting more time.

And this brings us to the second big problem with Waterfall, as commonly seen in the wild: the Schedule never gets longer, even if one of the phases gets redone. I'll cover this in the next entry....

Technorati Tags --

Sunday, January 22, 2006

Watts Humphrey's TSP/PSP

I can't say much about the TSP/PSP because I have not been through their training, and their official web site has minimal details about the methodology - odd, considering that they charge money for the training (heh).

What little I do know makes it appear to be a very heavy methodology, requiring significant investment by a whole organization. This makes it a difficult methodology to introduce into a company for just a small group, and almost impossible bring in from the bottom up, since the training requires managerial approval, and costs $1000 per head for the initial course (as of January 2006).

The one anecdote I have about this is that one developer of my acquaintance was on a team scheduled to take the PSP training. Management was told that they were expected to attend, offsite, and turn off their cellphones during the training, to show that they were committed to the new process, and to make sure they understood the training.

The management agreed to the offsite, but not to turn off their cellphones. Less than halfway through the first day, the upper mamangers were out of the classroom, taking calls. So much for commitment.

Technorati Tags --

Friday, January 20, 2006

Scrum Methodology

The next stop on the Magical Methodology Tour is Scrum. In contrast to XP,
Scrum is a top-down methodology, detailing more about how the project is managed. The development effort is broken up into 4-week units called Sprints, with a Sprint Goal - a short summary of the desired end result of the Sprint, as defined by the Product Owner, who is the person in charge of definied the product and priorities of features. Each Sprint is managed by a ScrumMaster, whose job is to keep an eye on the development, and remove any roadblocks the team encounters, and to hold the short Daily Scrum meeting, in which the team recounts the previous day's work, the planned day's work, and any roadblocks they face. The list of tasks to be done for the current Scrum is called the Scrum Backlog, and is set during the Sprint Planning Meeting, held at the start of the Sprint, where the highest priority remaining tasks in the Product Backlog are moved to the new Sprint Backlog. The ideal goal of every Sprint is a working version of the product, with each Sprint adding more functionality until a complete product results.

Now, as to the details:
Scrum is a lot less specific than XP, with fewer required practices, and a lot more freedom in the development techniques. It also is much more palatable to management in that it requires few changes to the actual development process - programmers still write code the same way, and the Product Owner role maps well to the common Product/Project Manager position in many companies. Priorities are still set by management. The most likely difficulties arise in the ScrumMaster's role - removing obstacles. This is not an easy job, nor one for a first-line manager - it requires authority beyond the usual bounds. The ScrumMaster must be allowed to prevent developers from being pulled into other projects, or redirected to support. And this is the big stumbling block - the development teams are usually low enough on the food chain that the ScrumMaster is not someone with such clout, and will be unable to deter these threats to the integrity of the team.

And that is a detail that I will return to in a later post.

Technorati Tags --

Thursday, January 19, 2006

Extreme Programming

Extreme Programming (XP) is arguably the hottest methodology in the Agile bunch. Entire libraries of books have been written about it in the last 10 years (if that long). Lately it appears to be dropping off the trend radar, but the boost it gave to Agile Methodologies ensures it will not fade away anytime soon.

XP consists of 12 Core Practices:
  • The Planning Game
  • Small Releases
  • System Metaphor
  • Simple Design
  • Test-Driven Development
  • Refactoring
  • Pair Programming
  • Collective Code Ownership
  • Continuous Integration
  • Sustainable Pace
  • Onsite Customer
  • Coding Standard
Now, some of these are things that nearly every development group will take as a given - Coding Standards and Collective Code Ownership, for example, are common. Others are likely to run into heavy resistence, either from the developers themselves, or from the management above the developers. Pair Programming often runs afoul of both groups; programmers are accustomed to working solo in many cases, while managers dislike the appearance of half their staff not working.

Refactoring is one of the hardest to explain to a non-programmer. As every developer knows, the design as originally planned never quite matches the code that you end up with. Reality forces numerous small deviations from the blueprint, and so the end product is full of hacks, compromises, and cruft. So developers need to spend some time periodically reorganizing the code structure and clean out the digital dust bunnies.

The problem is that managers often don't want working code to be touched, and since refactoring is intended to change the code structure without altering the function, it can be difficult to get buy-in for it. The best suggestion I have heard is to use Rube Goldberg as the metaphor - his contraptions worked, but they were hardly streamlined. A little time to replace the cats, candles, and buckets with motors, switches, and tubing, and voila!, it works, better.

Sustainable Pace is another one likely to stick in the craw of management. In XP vernacular, this means a 40-hour week, all the time. No Death Marches, no 80-hour-weeks, nada. XP holds that with all the Core Practices in place, the team should not need to work overtime. As far as I can tell, current American management is quite unlikely to accept "No, we're all going home at 5", if there is work left to do, especially if there is a deadline looming. Having been part of at least 2 Death Marches, and one pre-planned Zombie Stroll, I can attest to the diminishing returns of 60-80 hours a week for more than 2 weeks.

Onsite Customer is the one practice that I feel is hardest to introduce. Too many development shops are kept away from the customers because the managers feel uneasy with the developers in close proximity to the customers. Others have the problem that they have more than one customer with differing needs, so that it is difficult to reconcile all the customers' needs. Also, few customers want to dedicate a knowledgable staffer to the project for that much time - they'd rather throw a few hours into a RFP-style document and have the development shop do all the work from there, until the time for testing arrives.

Technorati Tags --

Wednesday, January 18, 2006

Hello, World!

Hello, World!

This will be my attempt to expound some of the wisdom I've gleaned from 20 years of software development.

All the methodologies and books describe things in the perfect world of full management buy-in, clean handoff between phases, complete documentation, and such. Real-world developers find that these plans are scrambled by management using archaic practices, bar-napkin documentation, schedule changes, and features moving between releases.

So I'm hoping to describe some of the things that I've learned in 20 years of developing software in a variety of environments; things that are not part of methodologies, processes, and plans; things that might help newer developers have an easier job.

I'm keeping my name and employer(s) confidential so I can speak freely about some of the less-desireable things that I've experienced without facing legal and social consequences - I'm not looking for trouble, but I also want to honest about things.