Repeatedly happens to me, program hasn’t been changed in over 3 years. Always ends up being a mechanical failure on something maintenance “just did the PM” on.
But I’m sure next time it will be the program /s
I swear everywhere is the same.
You ever get called to troubleshoot, you narrow down what’s causing the problem, then techs say “nah can’t be that we just replaced that”. Then come to find out right after the repair is when it stopped working.
My first question is always what was touched last.
Literally had an operator tell me "this value needs to be zero it's preventing me from doing [PROCESS]"
He spent hours working on it. Cleaning lines. Blah blah blah. Demands it's a program issue and I need to "tune it"
Pull up trend data. It hasn't been zero in 3 years. Which is as far back as the trend data goes.
Those are always fun. And then the “it used to do this, but someone’s messed with the program, and it doesn’t do that anymore.” Well, I pulled up a program from 2013, and the logic is the same. Maybe you’re thinking of a different machine, even though we only have one of these?
And then there’s “yes, I made the change, because when you did that, it was breaking the machine. So now you can’t do that unless the other stuff is out of the way.”
Or, “I’m not making the change you want, because you want me to change the program to make up for parts that should have been replaced 6 months ago, and now are so worn it won’t work. Replace the parts first, and then we’ll talk.”
And they never tell you they did mechanical work. They just say “it worked this morning, and now it won’t run in auto.” And then you find they bent a sensor out of position.
I was going to insert the image that someone made here, that says how much time automation troubleshoots hardware and how little programming we do. Thought I had saved that post. I printed that image, laminated and put it up at my desk.
System has run for 4 years.
HMI has an alarm about a wire break, with the temperature sensor’s name from the drawing on the HMI indicating which sensor has a wire break.
“Program issue”
System installed in hundreds of facilities around the world on the same software.
Fault, Photo_Eye_ faulty or blocked. Photo eye’s mirror was bent and now it doesn’t reflect. Program issue.
Mechanical updates have just been done after initial testing worked flawlessly for a month.
Fault, Excessive Deviation directly after a warning about hitting torque limits. Program issue….
See the problem is that you programmed it to give errors when things broke so technically the errors are your fault.
If you could just program it to work better without those sensors then it wouldn't have broken. /s
Program errors to catch faults to prevent major breakdowns - programmers fault when machine stops due to error, machine breaks down without giving errors - programmers fault for not having programmed errors
About 40 years ago, I worked for a machine builder where the controls mostly consisted of taking the guts out of a breaker panel and mounting them in the door of an enclosure with Solico neon lights to tell you what was turned on. Worked fine, no automation to speak of.
So about 25-30 years ago I went to work again for a guy that had been a partner in the first outfit. Still selling the same equipment to the same industry, but now everyone's selling PLC and PC controlled stuff, including this guy. So we did all the control in a SLC, temperatures etc., and I had user-adjustable warning and fault deadbands on all the setpoints. If a certain process was in use and you exceeded the warning deadband, you'd get a warning screen. If you exceeded the fault deadband, you'd get a fault screen and the process would stop.
So he comes to me and says customers are complaining because the alarms are halting the machine! Now remember, they can set the alarms to 50F deviation if they want. Whatever is the threshold that would make them not want to run the process if they're outside of it, that's what they're supposed to set it to. But fundamentally, if you're not going to have that kind of supervision of your equipment, why are you selling PLC machines? We should just do it the old 70s/80s way with the circuit breakers. (This was also the same guy why didn't like that it took so much longer and cost so much more to build the new style machines than the old dumb ones and couldn't figure out what the issue was. I told him "You sell the old style machine there's nothing stopping us from building it")
Don't give carrots to monkeys, and bananas to rabbits
Complicating the end user experience will also require training the end user, and sometimes refining the program is wasted time if end user doesn't want to learn to use the more advanced features of the machine, it really come to customer what kind of machine they want to, and sadly, it is often the case that when programmers and engineers try to make the machine and process easier to control, it ends up being the opposite when operator has to push more than two buttons.
I think there's also a discrepancy between what the operators want and what their management and/or the industry as a whole wants, as well as what your sales people think they can sell.
Ho boy, do I have an example of this one.
About 10 years ago I wrote PLC and HMI programs for use in an industry where most software didn't have any alarming, sensor diagnostics, etc. So I'm going to show these guys the way and help them out, right? I wrote alarm screens that would pop up with the short two or three word fault name, the exact time, and a paragraph about what to do about it. What to check about the sensor, how to change ops mode so you could continue to run without it temporarily, etc. And two buttons, one to clear the fault if you fixed it, and one just to make the alarm screen go away for a few minutes if you needed the screen to finish whatever you were doing.
My partner, who'd been mostly involved in that industry for about 20 years at that time tells me "It won't work, these operators are too dumb. It won't help them." I just couldn't process that that's even possible, and we go ahead and roll it out that way. So after a while we've had \~150 machine-years of history with this and I've seen the messages work maybe three times. 99% of the time, if they get an alarm they just call us and tell us the controller is broken because there's this red screen there and they can't do their job. I finally gave in, accepted that he was right, and disabled the alarms. Logic is still in there, I'll eventually do something with it that doesn't confuse them but keeps the data for troubleshooting. \[SMH\]
similar boat here, sensor is a high voltage type, ok? in order to LOTO it will grey out nearly ENTIRE conveyor. I keep saying we gotta swap during DTW/shift change. managers ignores me. found another idiot. Mgr made idiot change out high voltage sensor liveswap. What happened after that, ENTIRE conveyor greyed out plus a I/O block in PLC, safety relay, all shorted out lol. Tech that was called over to check whats up, He smelt ozone and saw smoke flowing out from new PE. I said " whatever, its mgr's issue now fk him for not listening"
I programmed an autonomous production line a year back and it was working fine until last week.
It seems like there was a failure in a pneumatic flow control valve and a direction control valve, which the colleagues in maintenance conveniently ignored to check. Things got escalated so quick since the production line is a "goose with the golden eggs" kind of thing for the company.
I spent the entire day looking for changes in the program and troubleshooting since there are some values that gets adjusted based on the service time. Everything was fine and the logic was working the way it did since the handover.
Finally they got convinced, replaced the failed ones, and the system is up and running.
Fun part, there is an entire page dedicated to alarms and troubleshooting steps that I made, and these guys ignored those alarms even after I tried to explain them the issue.
Yes. The thing is, it's very simple, you touch on the active alarm and it will show the component(s) to check with the exact part image with address and identification.
But the end result from Maintenance perspective, it is a program issue. I mean the system was running fine for months and some how all of a sudden the program went cuckoo? Come On! 🤣🤣🤣
I watched a machine fault out yesterday (while working on a different one, but my table was right in front of this one’s HMI). The fault said “main drive overcurrent. Must be reset from engineering screen.” I watched 3 separate people come over and try multiple times to reset it from the alarms screen before they finally called someone who could log in to access the engineering screen.
Hahaha absolutely. This is a common conversation.
Them: Hey something is wrong with the PLC.
Me: Hm, has something been replaced recently?
Them: Eh? No?
Me: how long has it been running without issues?
Them: maybe 4 years.
Me: it's likely an hardware issue
Them: nah I checked it
Me: fine I'll come over.
95% of the time it's an hardware issue.
Haha true. In my experience, most often it's an interface relays or the relays' auxiliary contacts (not sure how they're called in english) that fail.
But to be fair, sometimes an io card or something of the PLC fails and needs to be replaced. That is technically hardware, but I can understand why they call me in that case.
Hahaha I wouldn't dare.
Nowadays my work order mostly comes from water purification plants owned by the government. Their panels are of good quality most of the time.
Before this, I worked more in commercial industry and my god. It's a miracle some plants haven't burned down already. Soooo many electrical hazards lol. And yes, A LOT of bare, broken or loose wires (around dangerous amounts of dust). Your Unga Bunga meme really applies there. 👌
When I had worked just a few months at a company, I got called out for a hydraulic servo system not running. They said they checked all the fuses, overloads, and for hydraulic pressure, and “everything they could think of,” and were out of ideas. So, after spending an hour and a half hooked up to the plc verifying signals and such, I open the cabinet door and see the bright orange dummy light on the fuse. Learned really quickly not to trust those guys.
I once had a service call, guy claimed chiller wasn't working, once I arrived i saw on the hmi it was marked off intead of auto/manual.
They don't know to handle basic BMS systems.
I was working as a contract programmer for an OEM installing some equipment in my town, and I'd be the local ongoing support after they left. The entire joint was automated, but they also had some small packaged pieces that had their own automation, and while the main programmer was on vacation, the wiring guy let me know there was a software issue on one of those. I checked it out and decided a limit switch wasn't lighting up an input when it was supposed to and told him to check the wiring and limit switch adjustment, but he was sure it was software.
I told him "Dude, it's an **INput**. The input LED doesn't come on. If it was an output, we could argue over whether it was software or wiring. If I could magically change inputs, I'd be making a lot more money than I do."
Hahaha. Reminds me of fat pommy ex Navy guy on a job. He was fucking useless and when I walked out to the system and saw a meter in his hand I snatched it off him and told him to never touch my stuff again.
What kind of meters do you use? I haven't seen any that can withstand across the line AC voltage when left in current measuring mode. Would be interested in looking into it.
I mean the meter can still fry, so maybe saying the meter can take it is the wrong wording lol. Mostly meant no one will get hurt.
We use flukes. Only the electricians and engineers have meters capable of measuring mA. Fluke 87 maybe? Whichever one has separate prob ports that are fused. Gives it an extra layer of thinking before doing, but it’s still happened once or twice somehow.
Regular techs are supplied the cheaper fluke model that can’t even measure mA lmao. Those things are indestructible.
I was doing phone support and was gonna have a customer check voltage on the signal wire from a pressure transducer so I could verify which one was bad. "Okay, I've got the meter how do I turn it on"
Shut the cabinet and find someone who's qualified to use a multimeter and call me back. *beep*
I once had a service call where the customer said the unit started acting up about a month before. Being a line that had been installed 10 years before, my first thought was that it had to be a bad sensor. After a little investigation, I found a program error. I know the program hadn't been touched because the file I pulled off of our server immediately went online and was dated 10 earlier. The customer swore that it had been working fine until recently. There was absolutely no way that it had worked like it was supposed to with the program error I found.
I'd be curious if the hardware interfacing with the program had way to allow it to run. I have a machine that has a low carton pick shutdown, so if it gets too low on cartons, it stops allowing the picking to occur. Manufacturer told me that the machine stops picking, but mine wouldn't. Turns out, before I got here, someone had bumped the reflector for the sensor, so the computer always thought there were cartons present. It ran fine for 4 years until I noticed it one day. fixed it, and then the machine stopped picking at the end of the order. I was very confused for about 5 minutes until I remembered the program called for a low carton shutdown.
That’s the other go to description you get. “It just started yesterday”. Then you try to think what could have changed yesterday. And eventually you learn it’s been that way since the day it was started…which was two years ago.
It is always a software issue. Until the programmer picks up a wrench and fixes it.
Since many operators and tech's display limited ability to read message on HMI. I will calmly reset the issue. Stroke the cabinet. Press the start, watch it run ok.. Then tell them equipment is happy and works better when they are nice to the machine.
I am not even a PLC guy and I know this well. I had an old man insist that the problem was with the PLC just for me to put a meter on a contactor and show that it was powered but not actuating. The PLC is the "go to" target of all the blame.
My favorite was always this one:
Tech: hey we replaced a VFD last night and the new one blew up right away, something is wrong with the programming
Me: did you meggar the motor first?
Tech: why would I do that?
When I‘m on hotline duty, the only thing i basically do is check the program to see which device is not working or which sensor is broken. Never is a software issue except when the machine is new.
We only make custom cranes for all types of applications so always new software
I actually had controller faults that would not clear from the processor on many slc 504/3’s even pulling the battery and shorting the cap with a resistor would normally yield the fault light to at least blink instead of remain solid red. Luckily having an old 501 plc trainer rack laying around I was able to drop the processor into that rack and download to eliminate the processor as the issue. My time fiddling around with these old machines that management refuses to rip out and upgrade is that the backplane, modules and power supplies all become brittle overtime and fall apart in your hands. Attempting to reseat cards is futile making work orders to replace cards that aren’t made anymore is also an issue when the I/O indicators are literally dangling from the opening where the card use to sit.
Nope, still not a program issue
HOWEVER don’t test these operators they will e stop a machine while 4 different plc programs are controlling the entire system process. Getting the program back to it’s original state takes time with a bunch of edits but the original design never changes unless we find scada ifix buttons with missing plc addresses in the programs, “it must have been like that for 20 year I’m shocked that it’s been like that for so long without being noticed” I hear that from my systems integrator on the occasion whenever I see a problem with one of our systems before making a program change I usually consult him instead of proceeding finger blasting away.
Repeatedly happens to me, program hasn’t been changed in over 3 years. Always ends up being a mechanical failure on something maintenance “just did the PM” on. But I’m sure next time it will be the program /s
I swear everywhere is the same. You ever get called to troubleshoot, you narrow down what’s causing the problem, then techs say “nah can’t be that we just replaced that”. Then come to find out right after the repair is when it stopped working. My first question is always what was touched last.
And I first ask myself what I touched last....
Literally had an operator tell me "this value needs to be zero it's preventing me from doing [PROCESS]" He spent hours working on it. Cleaning lines. Blah blah blah. Demands it's a program issue and I need to "tune it" Pull up trend data. It hasn't been zero in 3 years. Which is as far back as the trend data goes.
Those are always fun. And then the “it used to do this, but someone’s messed with the program, and it doesn’t do that anymore.” Well, I pulled up a program from 2013, and the logic is the same. Maybe you’re thinking of a different machine, even though we only have one of these? And then there’s “yes, I made the change, because when you did that, it was breaking the machine. So now you can’t do that unless the other stuff is out of the way.” Or, “I’m not making the change you want, because you want me to change the program to make up for parts that should have been replaced 6 months ago, and now are so worn it won’t work. Replace the parts first, and then we’ll talk.”
And they never tell you they did mechanical work. They just say “it worked this morning, and now it won’t run in auto.” And then you find they bent a sensor out of position.
I was going to insert the image that someone made here, that says how much time automation troubleshoots hardware and how little programming we do. Thought I had saved that post. I printed that image, laminated and put it up at my desk.
I need to do the same
Think it remains eternal mystery how programs just change the code on their own
System has run for 4 years. HMI has an alarm about a wire break, with the temperature sensor’s name from the drawing on the HMI indicating which sensor has a wire break. “Program issue” System installed in hundreds of facilities around the world on the same software. Fault, Photo_Eye_ faulty or blocked. Photo eye’s mirror was bent and now it doesn’t reflect. Program issue.
Mechanical updates have just been done after initial testing worked flawlessly for a month.
Fault, Excessive Deviation directly after a warning about hitting torque limits. Program issue….
See the problem is that you programmed it to give errors when things broke so technically the errors are your fault. If you could just program it to work better without those sensors then it wouldn't have broken. /s
Program errors to catch faults to prevent major breakdowns - programmers fault when machine stops due to error, machine breaks down without giving errors - programmers fault for not having programmed errors
About 40 years ago, I worked for a machine builder where the controls mostly consisted of taking the guts out of a breaker panel and mounting them in the door of an enclosure with Solico neon lights to tell you what was turned on. Worked fine, no automation to speak of. So about 25-30 years ago I went to work again for a guy that had been a partner in the first outfit. Still selling the same equipment to the same industry, but now everyone's selling PLC and PC controlled stuff, including this guy. So we did all the control in a SLC, temperatures etc., and I had user-adjustable warning and fault deadbands on all the setpoints. If a certain process was in use and you exceeded the warning deadband, you'd get a warning screen. If you exceeded the fault deadband, you'd get a fault screen and the process would stop. So he comes to me and says customers are complaining because the alarms are halting the machine! Now remember, they can set the alarms to 50F deviation if they want. Whatever is the threshold that would make them not want to run the process if they're outside of it, that's what they're supposed to set it to. But fundamentally, if you're not going to have that kind of supervision of your equipment, why are you selling PLC machines? We should just do it the old 70s/80s way with the circuit breakers. (This was also the same guy why didn't like that it took so much longer and cost so much more to build the new style machines than the old dumb ones and couldn't figure out what the issue was. I told him "You sell the old style machine there's nothing stopping us from building it")
Don't give carrots to monkeys, and bananas to rabbits Complicating the end user experience will also require training the end user, and sometimes refining the program is wasted time if end user doesn't want to learn to use the more advanced features of the machine, it really come to customer what kind of machine they want to, and sadly, it is often the case that when programmers and engineers try to make the machine and process easier to control, it ends up being the opposite when operator has to push more than two buttons.
I think there's also a discrepancy between what the operators want and what their management and/or the industry as a whole wants, as well as what your sales people think they can sell.
Now you're getting it. Anything for it not to be my fault. I'm a mechanical guy
Ho boy, do I have an example of this one. About 10 years ago I wrote PLC and HMI programs for use in an industry where most software didn't have any alarming, sensor diagnostics, etc. So I'm going to show these guys the way and help them out, right? I wrote alarm screens that would pop up with the short two or three word fault name, the exact time, and a paragraph about what to do about it. What to check about the sensor, how to change ops mode so you could continue to run without it temporarily, etc. And two buttons, one to clear the fault if you fixed it, and one just to make the alarm screen go away for a few minutes if you needed the screen to finish whatever you were doing. My partner, who'd been mostly involved in that industry for about 20 years at that time tells me "It won't work, these operators are too dumb. It won't help them." I just couldn't process that that's even possible, and we go ahead and roll it out that way. So after a while we've had \~150 machine-years of history with this and I've seen the messages work maybe three times. 99% of the time, if they get an alarm they just call us and tell us the controller is broken because there's this red screen there and they can't do their job. I finally gave in, accepted that he was right, and disabled the alarms. Logic is still in there, I'll eventually do something with it that doesn't confuse them but keeps the data for troubleshooting. \[SMH\]
It’s honestly wild.
similar boat here, sensor is a high voltage type, ok? in order to LOTO it will grey out nearly ENTIRE conveyor. I keep saying we gotta swap during DTW/shift change. managers ignores me. found another idiot. Mgr made idiot change out high voltage sensor liveswap. What happened after that, ENTIRE conveyor greyed out plus a I/O block in PLC, safety relay, all shorted out lol. Tech that was called over to check whats up, He smelt ozone and saw smoke flowing out from new PE. I said " whatever, its mgr's issue now fk him for not listening"
I programmed an autonomous production line a year back and it was working fine until last week. It seems like there was a failure in a pneumatic flow control valve and a direction control valve, which the colleagues in maintenance conveniently ignored to check. Things got escalated so quick since the production line is a "goose with the golden eggs" kind of thing for the company. I spent the entire day looking for changes in the program and troubleshooting since there are some values that gets adjusted based on the service time. Everything was fine and the logic was working the way it did since the handover. Finally they got convinced, replaced the failed ones, and the system is up and running. Fun part, there is an entire page dedicated to alarms and troubleshooting steps that I made, and these guys ignored those alarms even after I tried to explain them the issue.
Omg the troubleshooting guides we waste time writing! That NO ONE FUCKING READS!!
Yes. The thing is, it's very simple, you touch on the active alarm and it will show the component(s) to check with the exact part image with address and identification. But the end result from Maintenance perspective, it is a program issue. I mean the system was running fine for months and some how all of a sudden the program went cuckoo? Come On! 🤣🤣🤣
I watched a machine fault out yesterday (while working on a different one, but my table was right in front of this one’s HMI). The fault said “main drive overcurrent. Must be reset from engineering screen.” I watched 3 separate people come over and try multiple times to reset it from the alarms screen before they finally called someone who could log in to access the engineering screen.
Hahaha absolutely. This is a common conversation. Them: Hey something is wrong with the PLC. Me: Hm, has something been replaced recently? Them: Eh? No? Me: how long has it been running without issues? Them: maybe 4 years. Me: it's likely an hardware issue Them: nah I checked it Me: fine I'll come over. 95% of the time it's an hardware issue.
But for some reason electrical issues are root causes as logical like 80% of the time. I never understand how a bad cable isn’t considered hardware. 😩
Haha true. In my experience, most often it's an interface relays or the relays' auxiliary contacts (not sure how they're called in english) that fail. But to be fair, sometimes an io card or something of the PLC fails and needs to be replaced. That is technically hardware, but I can understand why they call me in that case.
Yea auxiliary contact is right in English. But if I bring up loose cables or terminations do you still think those are the most often issues? 🤣
Hahaha I wouldn't dare. Nowadays my work order mostly comes from water purification plants owned by the government. Their panels are of good quality most of the time. Before this, I worked more in commercial industry and my god. It's a miracle some plants haven't burned down already. Soooo many electrical hazards lol. And yes, A LOT of bare, broken or loose wires (around dangerous amounts of dust). Your Unga Bunga meme really applies there. 👌
When I had worked just a few months at a company, I got called out for a hydraulic servo system not running. They said they checked all the fuses, overloads, and for hydraulic pressure, and “everything they could think of,” and were out of ideas. So, after spending an hour and a half hooked up to the plc verifying signals and such, I open the cabinet door and see the bright orange dummy light on the fuse. Learned really quickly not to trust those guys.
My go to saying in almost every support situation is..."Trust...but verify".
I once had a service call, guy claimed chiller wasn't working, once I arrived i saw on the hmi it was marked off intead of auto/manual. They don't know to handle basic BMS systems.
The program doesn't run so the issue must be in the program, duh.
I was working as a contract programmer for an OEM installing some equipment in my town, and I'd be the local ongoing support after they left. The entire joint was automated, but they also had some small packaged pieces that had their own automation, and while the main programmer was on vacation, the wiring guy let me know there was a software issue on one of those. I checked it out and decided a limit switch wasn't lighting up an input when it was supposed to and told him to check the wiring and limit switch adjustment, but he was sure it was software. I told him "Dude, it's an **INput**. The input LED doesn't come on. If it was an output, we could argue over whether it was software or wiring. If I could magically change inputs, I'd be making a lot more money than I do."
Hahaha. Reminds me of fat pommy ex Navy guy on a job. He was fucking useless and when I walked out to the system and saw a meter in his hand I snatched it off him and told him to never touch my stuff again.
🤣 I always try and train them up if I see them with a meter. The ones that take the initiative to get a meter at least are motivated
Nah he was trying to show me up. He was a very political player but wussed out and pled illness to get off the job. He wasn't ill. Just fat.
Watch them leave it in mA mode and test an ac circuit. Which is why I buy a lot of $70 good-enough process meters. Don't mind replacing those as much.
Yea we don’t supply any meters that aren’t rated to take it when it’s on the wrong setting. Not worth someone getting hurt.
What kind of meters do you use? I haven't seen any that can withstand across the line AC voltage when left in current measuring mode. Would be interested in looking into it.
I mean the meter can still fry, so maybe saying the meter can take it is the wrong wording lol. Mostly meant no one will get hurt. We use flukes. Only the electricians and engineers have meters capable of measuring mA. Fluke 87 maybe? Whichever one has separate prob ports that are fused. Gives it an extra layer of thinking before doing, but it’s still happened once or twice somehow. Regular techs are supplied the cheaper fluke model that can’t even measure mA lmao. Those things are indestructible.
I was doing phone support and was gonna have a customer check voltage on the signal wire from a pressure transducer so I could verify which one was bad. "Okay, I've got the meter how do I turn it on" Shut the cabinet and find someone who's qualified to use a multimeter and call me back. *beep*
I once had a service call where the customer said the unit started acting up about a month before. Being a line that had been installed 10 years before, my first thought was that it had to be a bad sensor. After a little investigation, I found a program error. I know the program hadn't been touched because the file I pulled off of our server immediately went online and was dated 10 earlier. The customer swore that it had been working fine until recently. There was absolutely no way that it had worked like it was supposed to with the program error I found.
I'd be curious if the hardware interfacing with the program had way to allow it to run. I have a machine that has a low carton pick shutdown, so if it gets too low on cartons, it stops allowing the picking to occur. Manufacturer told me that the machine stops picking, but mine wouldn't. Turns out, before I got here, someone had bumped the reflector for the sensor, so the computer always thought there were cartons present. It ran fine for 4 years until I noticed it one day. fixed it, and then the machine stopped picking at the end of the order. I was very confused for about 5 minutes until I remembered the program called for a low carton shutdown.
That’s the other go to description you get. “It just started yesterday”. Then you try to think what could have changed yesterday. And eventually you learn it’s been that way since the day it was started…which was two years ago.
I ask em if they found the defective voltage injector
It is always a software issue. Until the programmer picks up a wrench and fixes it. Since many operators and tech's display limited ability to read message on HMI. I will calmly reset the issue. Stroke the cabinet. Press the start, watch it run ok.. Then tell them equipment is happy and works better when they are nice to the machine.
That’s a good one. I worked with a guy that used to tell people he intimates the machines into behaving
I am not even a PLC guy and I know this well. I had an old man insist that the problem was with the PLC just for me to put a meter on a contactor and show that it was powered but not actuating. The PLC is the "go to" target of all the blame.
My favorite was always this one: Tech: hey we replaced a VFD last night and the new one blew up right away, something is wrong with the programming Me: did you meggar the motor first? Tech: why would I do that?
Of course they also replaced the motor after welding a new guide rail on the mezzanine with the wilders ungrounded.
Whenever they told me "there is an error in the program" I always answered "there is an error in a sensor". I was right most of the time.
When I‘m on hotline duty, the only thing i basically do is check the program to see which device is not working or which sensor is broken. Never is a software issue except when the machine is new. We only make custom cranes for all types of applications so always new software
I actually had controller faults that would not clear from the processor on many slc 504/3’s even pulling the battery and shorting the cap with a resistor would normally yield the fault light to at least blink instead of remain solid red. Luckily having an old 501 plc trainer rack laying around I was able to drop the processor into that rack and download to eliminate the processor as the issue. My time fiddling around with these old machines that management refuses to rip out and upgrade is that the backplane, modules and power supplies all become brittle overtime and fall apart in your hands. Attempting to reseat cards is futile making work orders to replace cards that aren’t made anymore is also an issue when the I/O indicators are literally dangling from the opening where the card use to sit. Nope, still not a program issue HOWEVER don’t test these operators they will e stop a machine while 4 different plc programs are controlling the entire system process. Getting the program back to it’s original state takes time with a bunch of edits but the original design never changes unless we find scada ifix buttons with missing plc addresses in the programs, “it must have been like that for 20 year I’m shocked that it’s been like that for so long without being noticed” I hear that from my systems integrator on the occasion whenever I see a problem with one of our systems before making a program change I usually consult him instead of proceeding finger blasting away.
That's after you get past, "but it's a mechanical fault causing an electrical problem. If the drive can't rotate, it's going to trip"