Get yer goddamn ducks in a row first

Ok, so today I had a remote cow-orker piss me off greatly.

I'm fresh out of another meaningless "town hall" meeting with upper brass - the usual type, full of platitudes and Orwellian euphemisms for corporate anti-social goals, and I get a message form this cow-orker - "Did you stop sending messages from X" (X being the system I am lead on)
I check my output logs - output volume is down, but nowhere near 0.
"No, we're still sending"
"Well as of N minutes ago, I stopped seeing any data.  You can look at my logs and see"
"I believe that you are not receiving data.  But we are transmitting"
"Can you check again?"
"I have, and we see no errors on send, and our receiver is getting data in a timely manner"
"Well there must be something wrong with your system"
I send the receiver log snippet showing we have received messages across this interval.

Radio silence for a while.  Then "I think I know what happened - another receiver grabbed the same ID"
In other words, your receiver was borked, and you failed to believe me when I showed you that it was not our transmitter.

Don't do that.  Even if you are certain it's their fault, do not cast balme until you have proof.

Another Reason to Hate Open Offices

I realized something the other day.

I've been at a new job for a couple of years, and I've noticed that I've been much less interested in doing activities after work.  I get home, and all I want to do is veg out.  While some of this I attribute to the realization that I've got over 10 years left until I can retire, there is more to it.

And then I realized that I've been working for over 2 years in an open office environment, as opposed to my previous workplace, where I had a tall cubicle (tall enough for an actual door).  I spend all day in the middle of an open room, with several large TVs(mercifully muted), 40-some-odd cow-orkers, and all the attendant noise.  I literally have no privacy except when I go to the toilet, or take  phone call in one of the little booths we have for that purpose.

I'm an introvert (shock, that a programmer is an introvert!), so all that exposure is exhausting me, so much so that I just want to cocoon myself when I get home.

Amazing differences

We've got a new contractor in my group, and I"m pleasantly surprised - they're proactive, smart, and easy to get along with.

I'm very happy to have someone who I can describe a task to, and find it done in only slightly more time that I would take (due to my being already up to speed on the issue)

The only mild flaw is needing rather longer explanations than I prefer to give, and being a no-UNIX type.

Humility as a Programmer virtue

Humility is good in programmers.

Not just 'humility as lack of arrogance', but 'humility as not assuming you know everything'

A few days ago, a cow-orker had an issue pushing some code into our repository, and had to recreate the work.  He pushed the fix, or so he thought.  I went to our build machine (yes we don't had automated builds, mea culpa), cloned the repository, and tried to build the system.  When it got to his code, the build failed due to a missing file.

When he arrived, I mentioned that I was unable to build the repository version of his code, and his reply was "And yet I can build it in my workspace"

It pissed me off because he has had previous cases of him forgetting to add files to the push, so he should know that it's one of the possible errors he could make that would show up on another machine.

So be humble.  If someone mentions something you worked on is not right, ask them to show you what is going on before you attempt to evade blame.

Exceeding Authority

One of my cow-orkers keeps acting like he can dictate what I do.  We report to the same supervisor, and work on separate projects, although my projet is a data source for his.  But today we had a meeting about data formats and such, and he starts handing out tasks to me.  And it's a task that makes his life easier at a cost to my usability.

THIS IS NOT ACCEPTABLE.

I do not work for him, and he knows this.  So I fire off an email to our common boss to outline that of the 3 items in the email, one is already in progress, one is clearly a bug that we will be addressing, and the third is a new *request*, and one that has ramifications to prospective other clients of our system.

I am pissed off that I had to do that.  This is not a case where there is a real vacuum in the structure.  And it comes from a guy who was all over our not sending as much data as we claimed we were, until he realized he had forgotten the multiple messages in a payload that we all had to deal with.

Ctrl-Alt-Delete

Have you ever had a colleague who does not seem capable of quickly jumping to a new topic?  One whom you can see rebooting the monitor process when you ask a question that is not about exactly what they are doing at that moment?

Not the context-switch of "I'm deep in thought about X, when you interrupt" but the part after that, when you have their attention.

Then add in the annoyed "I have no idea what you are talking about"

my life, several times each week

Monolingual Programmers

Perhaps I'm getting cranky in my old age, but I find it extremely irritating that there are programmers out there who do not attempt to use more than one programming language in their jobs.

It's a fucking 21st century, people, it's not that hard to learn a programming language enough to work in it.

Followup on "NOW we need to test"

As expected, the test plan I wrote has been soundly ignored.  I provided it, and the priorities are nowhere near that product.  My originakl individual assessment has been deemed satisfactory.

I'm so glad that I spent the time writing up a test plan.

Saw it on a button

When I was in college, I had the hobby of collecting pin-on buttons with geek-related sayings.  Stuff like "God is real unless declared integer" and the like.
During one exam, I wore one that read "I'm surrounded by idiots!"

I feel that today.

At The Job, we are realizing that a departed contractor left a lot of buggy code in his wake.  As we find the problems, we are fixing them.  One of them involved code that filtered out excess notifications based on time and some other factors.  The other dev was done with his tasks and so he started looking at the code.  We realized that it was not filtering correctly, so he started making fixes.  When he was done, he asked how we were going to test it.

This is not a n00b, this is a dev with 25+ years experience.  Sure, the project is fed gigabytes of data per day, but we already know that we can test a binary in isolation.  So the obvious way to test it is to create a data file with the desired qualities for the filter (pass and block values) and run that file through the code.
So it seems to be to be blindingly obvious that this was the way.  But it did not to him.  So I ended up writing the generator and running the tests, showing hiom the results, waiting while he fixed things and put the new binary on the test machine, etc, etc, etc.

Sheesh. 

I was trying to let him run with a section of the project as his domain, where he would be the SME, and I would stay out of it, but he just does not seem to cotton to the idea that he needs to be proactive and look ahead of the one thing he's currently doing.

All I gotta say is that my boss better put me in for another level bump ASAP

Sure, NOW they decide we need to test

So today the PM gets insistent in the daily that I need to write up a test plan for the rework of our DB population module.  It was written incorrectly by a contractor who left recently, and there seems to be lingering blamestorming on how we did not know he was not doing the right thing (hint: when you have 5 priorities and only 3 developers, you end up having to assume everyone knows enough to ask for help).

Now, I get the desire for testing, but until this very week, nothing, I mean NOTHING has had an official test plan on this faux Agile fustercluck of a project.  So why does this suddenly need a test plan?  And who is going to test it?  The DB relationship are part of the complexity of this part, and I'm the only one who has any idea of what those relationships are.

I swear, it's going to turn into me teaching someone the whole mess including how to test this, all the while being given other work, because someone else is 'doing the testing'

Thinking outside the paradigmatic box

I've probably blogged about this previously, but I'm too lazy right now to search.

I've determined one crucial thing about good older programmers - they are not at all afraid to exit the programming language and use the system around it.

I have been helping a junior programmer with some tasks - he inherited a program partially built, and I was one of the builders.  It's a multi-program system, with data being processed by a number of Python and C programs in sequence.  He's been tasked with converting everything into a single program, and is having issues with getting the data from some steps into the next ones.

Where the box is, it seems, is in trying to stay totally inside Python.  He asked for my help, and my immediate answer when he described the issue, was "Just redirect the output to a file, and read that file in as the next step.  It avoid having to worry about the string/list formatting"

He seemed reluctant, and eventually found the step needed to convert the data formats (which I would have done in a similar fashion, but I would not have had the initial problem for more than 3 minutes before converting the data)

So one of the skills to develop is that of seeing where the box edges are, and that they are often just lines you can cross

Mega Low Maniacal (venting)

Ok, folks, I'm starting to get concerned that I'm either losing all estimation of my skills, or going full-bore diva.

I'm currently part of a small team (2 devs, one PM), and I'm finding it increasingly frustrating that I seem to be the only one on the team who has any idea of how to develop a software system.  If anything needs doing, I seem to be the only one who has any idea how.  If there is a bug in code, my suggestions as to cause or how to proceed are ignored, then turn out to be correct.  Members of another associated team make suggestions that for the most part are either already in place (sometimes obviously), or are totally irrelevant.

The other dev is very good at doing one thing at a time at best, and while they do create test harnesses for the code, does not seem to grasp the need to live testing *and* actually take the step to do it.  I have to push for the live tests.  If a stumbling block appears, all progress halts and it's a "what do I do now?"

I appear to be the only one looking at the system as a whole, and the only one taking action to do the things that need to be done.  Several times we have had the production build fail due to some issue with the other dev's code or build process, and I get to diagnose and fix that.

And I'm not even bitching about the housekeeping issues that I'm handling, which, due to the draconian security policies, are almost entirely manual, because we are not trusted with admin access to the systems.

Use what's already there, dammit

Venting post today.

Cow-orker and I are trying to debug performance in a multi-threaded application.  I already added threads initially, so there was an example in the code to base anything new on.  Of key significance is the casting of some references to arguments.

Yet cow-orker adds a new set of threads, and not only does not use that for the new code, but removes it from the part that it was already in.

So the code compiles under the old compiler mode, but fails when we get to the official build machine.

And the correct example was in the code to begin with.

More Practical advice - Producer-Consumer threads

Two in one day!  What's the world coming to?

Ok, a common paradigm is a producer-consumer threaded application, using a queue to pass the message to be processed.  I recently coded one, and we were discussing how to manage the number of threads in the future, when we would want the application to dynamically allocate threads to match load.

Now, adding threads to increase capacity is simple - you just need to create the thread with the appropriate parameters (input queue, output sink, etc), save off the info so you can reap the thread at termination, and Bob's your uncle.

But removing threads is more complicated, because you don't want to kill a thread in the middle of processing a message.  So you need a means to tell a thread to die when it's done with the current message.

What I had come up with for shutting down all the threads at program end was pretty simple - the Producer emits a message with a specific format that when read by a thread, tells the thread to exit its  main loop and return.  What was moderately cool was that I had the thread re-emit the message into the queue, so that the next thread to read it would also stop, and would re-emit the message again, until all threads stopped.

The today I realized that if I added a simple counter onto the message format, I could use that as a way to reap a specific number of threads without any new mechanism.  The Consumer thread sees that flag message, exits its main loop, and then checks the counter value for 0.  If it's not 0, it decrements it by 1, and re-emits the message with the new counter value.  So start the message with a counter of 3, and 3 threads see it, stop, and re-emit it once each.  The last one sees the counter is 0, and does not re-emit it.

This can be used to kill all the threads but setting the counter to -1, which will never be 0, so all the threads will see, process, and re-emit the flag message (unless you have so many thread that the value wraps, but that's an entirely different problem)

Practical advice re CSV files

Morning, everyone!

I'm sure everyone has had a situation where they were generating data for another system, and needed a text format, and decided that comma-separated values (CSV) would work nicely.  Well, here's a little advice:  if you have any reason to expect a) human-driven input for fields, or b) reformatting of fields by some programatic means, then chose to use a pipe '|' as the separator instead.

The reason is that there is little use for that character in normal human text usage, so it will not get used within a field, the way commas often are in things like addresses, or names.  And as I found out recently, some encryption software is perfectly happy producing cyphertext with commas embedded