Sure, NOW they decide we need to test

So today the PM gets insistent in the daily that I need to write up a test plan for the rework of our DB population module.  It was written incorrectly by a contractor who left recently, and there seems to be lingering blamestorming on how we did not know he was not doing the right thing (hint: when you have 5 priorities and only 3 developers, you end up having to assume everyone knows enough to ask for help).

Now, I get the desire for testing, but until this very week, nothing, I mean NOTHING has had an official test plan on this faux Agile fustercluck of a project.  So why does this suddenly need a test plan?  And who is going to test it?  The DB relationship are part of the complexity of this part, and I'm the only one who has any idea of what those relationships are.

I swear, it's going to turn into me teaching someone the whole mess including how to test this, all the while being given other work, because someone else is 'doing the testing'

Thinking outside the paradigmatic box

I've probably blogged about this previously, but I'm too lazy right now to search.

I've determined one crucial thing about good older programmers - they are not at all afraid to exit the programming language and use the system around it.

I have been helping a junior programmer with some tasks - he inherited a program partially built, and I was one of the builders.  It's a multi-program system, with data being processed by a number of Python and C programs in sequence.  He's been tasked with converting everything into a single program, and is having issues with getting the data from some steps into the next ones.

Where the box is, it seems, is in trying to stay totally inside Python.  He asked for my help, and my immediate answer when he described the issue, was "Just redirect the output to a file, and read that file in as the next step.  It avoid having to worry about the string/list formatting"

He seemed reluctant, and eventually found the step needed to convert the data formats (which I would have done in a similar fashion, but I would not have had the initial problem for more than 3 minutes before converting the data)

So one of the skills to develop is that of seeing where the box edges are, and that they are often just lines you can cross

Mega Low Maniacal (venting)

Ok, folks, I'm starting to get concerned that I'm either losing all estimation of my skills, or going full-bore diva.

I'm currently part of a small team (2 devs, one PM), and I'm finding it increasingly frustrating that I seem to be the only one on the team who has any idea of how to develop a software system.  If anything needs doing, I seem to be the only one who has any idea how.  If there is a bug in code, my suggestions as to cause or how to proceed are ignored, then turn out to be correct.  Members of another associated team make suggestions that for the most part are either already in place (sometimes obviously), or are totally irrelevant.

The other dev is very good at doing one thing at a time at best, and while they do create test harnesses for the code, does not seem to grasp the need to live testing *and* actually take the step to do it.  I have to push for the live tests.  If a stumbling block appears, all progress halts and it's a "what do I do now?"

I appear to be the only one looking at the system as a whole, and the only one taking action to do the things that need to be done.  Several times we have had the production build fail due to some issue with the other dev's code or build process, and I get to diagnose and fix that.

And I'm not even bitching about the housekeeping issues that I'm handling, which, due to the draconian security policies, are almost entirely manual, because we are not trusted with admin access to the systems.

Use what's already there, dammit

Venting post today.

Cow-orker and I are trying to debug performance in a multi-threaded application.  I already added threads initially, so there was an example in the code to base anything new on.  Of key significance is the casting of some references to arguments.

Yet cow-orker adds a new set of threads, and not only does not use that for the new code, but removes it from the part that it was already in.

So the code compiles under the old compiler mode, but fails when we get to the official build machine.

And the correct example was in the code to begin with.

More Practical advice - Producer-Consumer threads

Two in one day!  What's the world coming to?

Ok, a common paradigm is a producer-consumer threaded application, using a queue to pass the message to be processed.  I recently coded one, and we were discussing how to manage the number of threads in the future, when we would want the application to dynamically allocate threads to match load.

Now, adding threads to increase capacity is simple - you just need to create the thread with the appropriate parameters (input queue, output sink, etc), save off the info so you can reap the thread at termination, and Bob's your uncle.

But removing threads is more complicated, because you don't want to kill a thread in the middle of processing a message.  So you need a means to tell a thread to die when it's done with the current message.

What I had come up with for shutting down all the threads at program end was pretty simple - the Producer emits a message with a specific format that when read by a thread, tells the thread to exit its  main loop and return.  What was moderately cool was that I had the thread re-emit the message into the queue, so that the next thread to read it would also stop, and would re-emit the message again, until all threads stopped.

The today I realized that if I added a simple counter onto the message format, I could use that as a way to reap a specific number of threads without any new mechanism.  The Consumer thread sees that flag message, exits its main loop, and then checks the counter value for 0.  If it's not 0, it decrements it by 1, and re-emits the message with the new counter value.  So start the message with a counter of 3, and 3 threads see it, stop, and re-emit it once each.  The last one sees the counter is 0, and does not re-emit it.

This can be used to kill all the threads but setting the counter to -1, which will never be 0, so all the threads will see, process, and re-emit the flag message (unless you have so many thread that the value wraps, but that's an entirely different problem)

Practical advice re CSV files

Morning, everyone!

I'm sure everyone has had a situation where they were generating data for another system, and needed a text format, and decided that comma-separated values (CSV) would work nicely.  Well, here's a little advice:  if you have any reason to expect a) human-driven input for fields, or b) reformatting of fields by some programatic means, then chose to use a pipe '|' as the separator instead.

The reason is that there is little use for that character in normal human text usage, so it will not get used within a field, the way commas often are in things like addresses, or names.  And as I found out recently, some encryption software is perfectly happy producing cyphertext with commas embedded

Losr in Corporate Hell

I just got out of a meeting where I was gobsmacked at the insistence by some architects that no development could be done unless the specific IP addresses of the machines in the system were set in a design document.

Think about that.

No design or development could be done unless the specific machines were known, despite the interchangability of any machine.

IP addresses are standardized, so the only thing needed to be known is the number of machines.

As I said, gobsmacked.