A Hydraulics Pro Club member from South Africa sent me this story about a hydraulic system overhaul that turned into a troubleshooting nightmare:
“My company was contracted to overhaul the hydraulic system in a beverage pasteurizing unit. The system was extremely simple but the pasteurizer, after 20 years of trouble free service, was going to be critical in the launch of a new alcoholic beverage in South Africa. Cider at the time was virtually unheard of here and the whole lead up to the launch involved millions being spent in TV and print advertising. Promotions and launch parties were arranged throughout the country for the launch date – it was going to be a big thing.
Anyway, the production line was restarted after the overhaul and after a few days everything was running smoothly. I can’t recall the output but it must have been in the region of 40,000 cans per hour. In the early hours of the morning, after about 12 hours of production, I got a phone call. About 100 litres of oil had sprayed out contaminating the water used to pasteurize, the outside cooling tower and 100,000 cans of cider now wrapped in plastic and packed in cases.
Control was through 8 of NG10 hydraulically piloted valves on a common sub plate. The leak was between the valves and the sub plate. The rest of the day was spent chemically cleaning the machine, piping and cooling tower and looking for the origin of the problem. Oh yes, and me on the carpet, facing the directors and plant engineers. The loss of product was of little concern to them, meeting the launch deadline was and it couldn’t be changed. I acknowledged that it was our fault and assured them I would do everything personally and they would be running by the following morning.
In one extremely hectic day and night I removed the valves and sub plate (all the cap screws were tight), had the sub plate laser checked for flatness and X-rayed for cracks, fitted all the valves back (and I used a torque wrench and new O- rings to be extra sure) and started up. It ran like a dream. I sat on top of that machine for the whole of the next day making sure there were no leaks until tiredness overtook me and I went home to sleep. I got about 6 hours sleep before my phone rang. Same problem again!
I had done my training as a submariner in the Navy where there was no margin for error. We were trained to check everything 3 times and I was at a loss as to where I could have gone wrong. The meeting with the client the following morning, for obvious reasons, didn’t go nearly as well as the previous one had. I stripped everything out again, while the whole system was chemically cleaned again.
Back at the shop, one of my fitters suggested to me that perhaps the absence of spring washers under the cap screws was the reason the O-rings were popping. I was stressed and I asked him where he got such a bloody stupid idea from? The next 5 minutes were very insightful. There had originally been spring washers under the cap screws but he hadn’t put them back after the original refurbishment because sub plate mounted valves “never have spring washers”.
It would appear that the original manufacturer installed spring washers because the cap screws he had were slightly too long. This is the first and only time I have ever seen this. The valves had pulled up against the O-rings enough to prevent them moving before the cap screws bottomed out in the threaded holes. Such a simple thing and about ½ million cans of cider were ditched for it.
The pasteurizer ran perfectly for 5 years after that, the supplier made the launch, just. And amazingly, I retained the client. I told the factory engineer the truth about the problem and got to fully refurbish and automate it again before they shipped it to another factory in Greece. The specific cider is still the top selling cider in South Africa.”
Lots of valuable lessons here. And in no way do I want to be critical of this member’s herculean effort to get his client back up and running. There’s no doubt his personal efforts were the key reason he kept the customer. But the whole point of conducting any debrief is to be wiser after the event.
Even though this job began as an overhaul, it morphed into a troubleshooting situation when the sub plate mounted valves leaked. A review of the troubleshooting checklist that appears on page 83 of The Hydraulic Troubleshooting Handbook would have prompted the application of Troubleshooting Principle #10: Elimination by Backing Out. This is VERY relevant in this situation because when a system is overhauled, lots of things are changed.
Application of this principle involves listing everything that was changed, no matter how small or seemingly insignificant–so they can be methodically un-changed, as necessary, in order to eliminate the problem. As soon as this problem appeared, a meeting with everyone involved in the overhaul – in a room with a white board and marker – would have shaken out the fact that the valves’ cap screws had spring washers that were not reinstalled. And given how easy this change is to reverse, it would almost certainly have been top of the list of things to ‘back-out’ of.
Persistence prevails. And so did this member in the end. But when troubleshooting you always want to get to the solution by the least torturous path. To this end, The Hydraulic Troubleshooting Handbook explains THE 12 principles that ensure the correct diagnosis in the shortest possible time, every time. So if you haven’t read it yet, get your copy here today.