My client, a value-added-reseller and system integrator of bar
code and computer equipment, had some installations that were
experiencing some very annoying problems with their bar code
readers. Occasionally, the readers would stop working and display
random junk on their liquid crystal displays. These readers needed
to be reprogrammed before they would work again. The customers, who
depended on these readers for their manufacturing operations, were
irritated. The manufacturer of the bar code equipment denied there
were any problems with the equipment, and did not offer to solve the
problem. My client asked me to look into the situation.
This turned out to be much more complicated than we had hoped.
The bar code equipment was programmed with parameters stored in
EEPROM, a relatively new technology at the time. One problem that
designers took very seriously, especially in the early days of
EEPROM, was accidental reprogramming or erasing of the part. I was
given a schematic of the reader, and I found the control lines for
programming the part were I/O pins on the microprocessor. This meant
that all safeguards for accidental erasing or reprogramming the
EEPROM had to be in the firmware; not a good idea. I did not have
access to the firmware so I was not able to verify that adequate
safeguards were incorporated during the power-up and power-down
cycles.
In addition to the EEPROM, there was a static RAM chip that was
maintained with a backup battery that contained the firmware for the
reader. It was easy to infer that either the EEPROM or the RAM or
both were being corrupted by something.
I looked at the power supply profile as power was turned on and
off and found that there was a period of time during which the power
supply voltage remained at a constant voltage that just happened to
be very near the switch-point between battery backup and normal
power. This looked like a candidate for causing corruption of data
in the RAM chip.
Finally, I needed to find a way to reproduce the problem at will.
I cycled the power on and off, and tried to cause spikes in the
power lines but I was not able to reproduce the problem. Then I
tried to lower the power line voltages and I found that if the
voltage became very low, I could induce the failure by introducing
noise on the power line. When I spoke to one of maintenance people
in a plant that had severe problems with this reader, I learned the
plant experienced power outages regularly and had a severe brown-out
problem. It was not unusual for their power line voltages to run
around 85 VAC.
I reported back to my client that I had found severe
susceptibility to low power line voltages in the product and that
the vulnerability was caused by a combination of hardware and
firmware design. I was able to demonstrate the problem by causing it
at will using a low line voltage and a vacuum cleaner as a power
line noise source. Unfortunately, we really could not do anything to
fix the problem, although my client did eventually get the bar code
manufacturer to admit to the problems. Since that time, components
have become more robust, and I hope these problems have become
obsolete. But we can all take a good lesson in product design from
this case: Take great care to handle unusual power cycling and power
quality situations in a way that fails gracefully without damage or
inconvenience to the user.
