Practical advice re CSV files

Morning, everyone!

I'm sure everyone has had a situation where they were generating data for another system, and needed a text format, and decided that comma-separated values (CSV) would work nicely.  Well, here's a little advice:  if you have any reason to expect a) human-driven input for fields, or b) reformatting of fields by some programatic means, then chose to use a pipe '|' as the separator instead.

The reason is that there is little use for that character in normal human text usage, so it will not get used within a field, the way commas often are in things like addresses, or names.  And as I found out recently, some encryption software is perfectly happy producing cyphertext with commas embedded

Losr in Corporate Hell

I just got out of a meeting where I was gobsmacked at the insistence by some architects that no development could be done unless the specific IP addresses of the machines in the system were set in a design document.

Think about that.

No design or development could be done unless the specific machines were known, despite the interchangability of any machine.

IP addresses are standardized, so the only thing needed to be known is the number of machines.

As I said, gobsmacked.

Muzzling diversity

I've got a new corporate master now, and one of the big "perks" is working in an open office plan.

There are plenty of blog posts, articles, studies, etc that show such offices are crap for software development.

However, I realized something this week.  They also serve to minimize dissent in the company.

If you are surrounded by people who can watch your every move, you are likely to self-censor your speech, because you have to get along with the cow-orkers every day, so setting yourself up as someone who disagrees with your colleagues is career-limiting behavior.

If you had an office or a halfway decent cubicle (6-ft partitions, minimum) you could have soft conversations about topics that your neighbors did not agree with.

Open offices lead to groupthink.  Just say no

Yet Another Bad Catchphrase

"...It's in our DNA"

Gaaahh.  I *HATE* when corporations use that phrase.

It's not in your DNA, even metaphorically.  If it were, you would not have to keep apologizing for security breaches, or for crappy customer service, etc., etc., etc.

It's just some uneducated marketing hack who wants it to sound like they can't fail.

Nearly as bad is "laser-focused".  'Cause the whole point of a laser is that it is not focused, it's collimated.  If you focused it, it would no longer be a laser, and no longer any use.

Juggling a few too many things

At The New Job, the product I'm supporting is a good example of how a small skilled team can do amazing things.  It's going strong after 3 years when it was supposed to be a stop-gap lasting only 6 months.

But in the course of my learning the ropes, I've run into a situation that allowed me to finally put into words something that I've felt for a long time about systems design - simplicity of all things is a virtue.

The setup - the product is a collection of scripts and binaries that process incoming files.  There are bash scripts, Python scripts, Ruby scripts, C binaries, cron jobs, and even a Java binary.  The upshot of this is that it requires a fairly experienced developer to be able to work on the system as a whole.  Now, this can be an issue in finding suitable developers, but it's not a high hurdle.  But where it does cause a problem is that the corporate sysadmin environment is very controlled, and it is not a simple thing to get tools installed, and the more tools you use, the worse it gets.

So, I'm having to navigate the labyrinth of corporate IT to get the desired versions of Python and Ruby approved and installed on our production systems, as well as the boilerplate necessities of user accounts.

Moral of the story - consider the overall ease of use when choosing languages to build a system in

Movin' On Up

Well, Dear Readers, you can rest easy - I am gainfully employed (as of the middle of May, actually)

I moved to a small group in a large corporation, with all the chaos that implies.  I"m also working a project that an old friend started, so I have a bit of a narrow ledge to walk, not wanting to disparage work that is provably profitable.

I will not, however, spare the company.  They can take it.

The big thing in my immediate headlights is the weird corporate labyrinth of permissions, requests, and partitioning of tasks.  It seems to take 6 different groups to get anything done.

Gambling with your future

I ran across this article about where new CS grads are going, and was somewhat disturbed by the 'advice' being given by Paul Graham, and someone named Harj Taggar.  They push the idea that new grads should not be looking to work for the established companies, but instead should be looking to work for companies that will be "the next big thing".  They hold out a number of (probably made-up) statistics comparing overall earnings (Going with a known company gets you a 90% chance for $1.1 million, while the new thing gets you 10% chance for $10 million).

Now, Paul Graham has a history of success, but I think he's being disengenuous here in claiming that a new grad has a 10% chance of selecting the next big thing (NBT) at a point where they will get the big payoff.  VCs are much more predatory these days, locking up preferential stock so that they get their money back before any of the engineers get a payout, so even if you were able to find the NBT and get hired, you might not get anything more than a few years' worth of income from it.  He also seems to be forgetting that there were so many new things back in the late 90's that it was very hard to pick the winners.  Graham picked Google, but it was in no way a sure thing at that time.  I'm sure the engineers at WebVan thought well of their chances of becoming millionaires.

Finally, there is the issue of basic compensation.  Startups generally cannot pay as well as established companies, and often have much lower benefits, which makes working for a startup in Silicon Valley much less viable, given the cost of living out there.  Seattle is similar; likewise NYC.

(PS.  A conspiracy-theory view is that VCs want to keep the supply of engineers working at startups high, to keep salaries lower (you can pay less, offer more stock options that may not vest fully, etc) so that there are enough startups for the VCs to hit their percentages for success)