Category Archives: coding

Heartbeats and not melting solenoids

One of the most worrying problems with pinball systems is that your software will crash, hang, or fall into some horrible infinite loop that leave a solenoid on long past it’s duty cycle, resulting in a nice melty pile of goo.

So what can we do about it? Well, first we need to identify the cases that are likely to result in a locked coil.

1) A variable hold solenoid (such as a diverter or magnet) is on and the C# controller crashes. The arduino doesn’t monitor these solenoids (as we can hold flippers forever!) so bad things can happen. Luckily only a few of these solenoids aren’t full duty cycle.

What we need to know here is if the C# controller is still alive. To do this the arduino watches for heartbeats in the solenoid line. If the PC misses 10 heartbeats (about one second of non-stop failures) the arduino will shut down all power to the pinball machine and wait for a “Power on” signal before continuing.

2) Something goes horribly wrong in the arduino code, while a solenoid is active. This can happen to any solenoid, momentary or otherwise; so this one is a bit scarier.

The solution is however nearly identical to the previous. The C# controller monitors the heartbeats from the arduino and if it doesn’t see any for a long enough period (about 1 second) it uses serial to force a reset. This is possible due to the mega’s 2 chip design, where the serial montior chip is able to watch for the reset command and pull the reset pin on the main ATMega. This case does not however handle a case where the serial monitor chip fails; though this is hopefully a rare case.

3) A transistor shorts.

It’s possible to solve this with hardware but I took the “Use really big transistors that can take the abuse” route instead. A later design may include the ability to detect the state of all transistors and cut power if one is bad, but that’s a lot of engineering for a fairly rare problem.

The heartbeat system itself is fairly simple. There is a timer on the arduino that sends the e_HEARTBEAT signal every 100 ms. When the C# controller gets this signal, it sends it back. This allows the C# side to know if the arduino is dead (No heartbeats being sent) and the arduino to know the same (How many heartbeats we’ve sent since the last time we got a response), which is hopefully a fairly durable system.

Animation editor

After quickly realizing how much I hate writing tons of giant arrays by hand, I took some time out to write an animation editor.

The goal was simple, write a quick tool to write out lamp state buffers into a hard file. These are then loaded at runtime and can be played at any time.

AnimationSized

Overall it was pretty straightforward. The perf of this tool is pretty damn awful and it’s very one-off, but it gets the job done. One thing I did notice is that transparent buttons load VERY slowly, which is kind of annoying since I use transparency to indicate the animation has no preference (Used for localization animations)

Importing and running the animations was dead simple with C#’s DataContractSerializer, it doesn’t even require the target class be the same code as long as the data structures match.

Indexes woo

For the last week I’ve been trying to sort out an issue with the serial bus. The basic problem turned out to be that the USB bus has a minimum packet size, which means sending lots of tiny 1 byte packets actually rapidly overflows the arduino’s hardware buffer.

The solution became fairly obvious, using locking I’d batch similar packet spans together, and instead of transmitting each byte individually, wrap it into a byte array. So if we need to update 5 lamps at once, we lock the serial buffer and write all 10 bytes out at once. While this does introduce a small amount of latency that could have been spared on some of the lamps, we want to synchronize these events anyway; so we’re actually just tightening up the timing a bit and ensuring the IO thread is waiting to send our data as soon as we’re finished populating the buffer.

This lead to a giant bug hunt caused by a bad for loop index. Lesson of the day: Arrays are 0 indexed, queue .count()s are not!

So as a more general progress update, I’m now able to spam the bloody hell out of the arduino and it seems to be quite content to keep up. There’s still some perf work to be done on the C++ side, but overall it’s looking quite good.