A common task in any engineering environment is realtime data logging. Of the many ways to do this, my current favorite is to casually stream sensor data at a fixed rate to some host computer. On a UART serial bus, this is pretty convenient because a host can “listen in” on the device anytime. Visually, this architecture may be represented as:
This post will focus on the host side of things since it is typically closer to what the end product (or prototype) will look like. From the host, the user can effectively monitor any kind of realtime process, save the results to a data file, or implement a GUI for controlling the hardware.
To keep things simple, we’ll use python, and let’s say you already have a device like the Arduino shown in the diagram above. We just want to throw together a quick and dirty terminal “GUI” for reading all device data. In fact, let’s just say someone has already written up some clues as to how the Arduino does its job
Arduino Specification:
- Collects all sensor data at a rate of 40Hz
- At the end of every cycle, the arduino will print all sensor data to the serial bus. It is up to the host machine to decipher the package of data and pull out the individual bits of sensor information from it.
Seem like a reasonable specification for an Arduino (or any other microcontroller)? Upon this, we draw up the following spec for the host machine:
Host Specification:
- Continuously listen for any data that comes from the Arduino.
- Speed/latency requirements behind data collects are going to be pretty casual. Any code that works up to 100Hz will do.
- Print contents of data to screen whenever it is requested.
- Allow the user to “inject” predefined commands to the Arduino via the means of pressing a key on the host machine’s keyboard.
As the programmer, I have a couple preferred ideas for how to go about this:
- Write a single threaded process that merely reads the data, and prints it to a screen. (fine if you don’t ever care to trigger any commands to be sent back to the device itself, ever)
- Write a 2 threaded process in python, using the threading library. One thread will be the “RX Thread,” for reading and processing the data stream. The other thread will be the “main thread,” which takes user input.
- Use Pygame’s event handling tools to do pretty much the same thing, but with some added features in reading/processing the user’s keyboard inputs.
This post focuses on option 2, whereby one can spin up a pretty versatile program fairly quickly using python built in libraries. Below is an example that would be a good starting point for such a program.
Example Program
import termios
import tty
import sys
from time import sleep
import threading
""" -------------------------------------------
---- Datatype representing the Device input ---
-----------------------------------------------
"""
class Data(object):
def __init__(self):
self.x = 0
self.y = 0
self.KILL = 0
def spin(self):
while (self.KILL == 0):
# simulates 100Hz of data input. May be replaced with
# calls to pyserial when it comes to communicating with
# real serial devices.
self.x = (self.x + 1) % 2**16
self.y = (self.y + 200) % 2**16
sleep(0.01)
""" ----------------------------------------
---- python based getch() implementation ---
--------------------------------------------
"""
def _getch():
# save current terminal config
tty_config = termios.tcgetattr(sys.stdin)
ch = None
try:
# set terminal to cbreak mode
tty.setcbreak(sys.stdin)
ch = sys.stdin.read(1)
finally:
# revert terminal back to standard config
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, tty_config)
return ch
""" ----------------------------
---- Main Program Components ---
--------------------------------
"""
def usage():
usagetxt = ("usage:\n"
" a: print contents of RX buffer\n" +
" z: send a (fake) command message\n" +
" k: kill the program.\n" +
" h: print this help text.\n"
)
print(usagetxt)
def demo(data_obj):
i = 0
while (1):
ch = _getch()
# processes the user's keystrokes. Some commands may
# trigger TX packets/commands to send back to the
# microcontroller.
if ch=='a':
# print information contained in RX buffer
print("[Main Thread]: i=%d, x=%d, y=%d" % (i, data_obj.x, data_obj.y))
elif ch=='z':
# send a fake command message
msg = "\x59\x00\xDE\xAD"
print("[Main Thread]: sending '%s' cmd to the arduino" % hex(ord(msg[0])) )
elif ch=='k':
# exit program
break
elif ch=='h':
# print usage table
usage()
# post-command processing
i += 1
def main():
# print usage table
usage()
# define instantiate core variables and objects
data = Data()
RX_THREAD = threading.Thread(target=data.spin)
try:
# start the RX thread
RX_THREAD.start()
# run the demo. Processes user input
demo(data)
finally:
# gracefully kill program in the event of ^C or appropiate signal.
data.KILL = 1
RX_THREAD.join()
print("[Main Thread]: Exiting program")
main()
Note that there are a couple gotchas that will ultimately be up you as to decide how much code you want to write against it:
- Thread locks should be used to make reading and writing RX data more atomic before any kind of additional processing is embedded in the RX thread. Otherwise, your main thread could end up reading and making decisions off of stale (or even incorrect!) data.
- Do you care if your main thread has access to data older than whatever just came in the latest packet? Should you build a queue of recorded data? You may find python’s queue library useful if so.
- Additionally, if you care about missed packets all together, you may need to embed a message counter on the microcontroller, and have it send the the message # along with the other sensor data.
That’s it! To cap it off, here’s an example of using the program:
bash-3.2$ python test_embedded.py usage: a: print contents of RX buffer z: send a (fake) command message k: kill the program. h: print this help text. [Main Thread]: sending '0x59' cmd to the arduino [Main Thread]: i=1, x=378, y=10064 [Main Thread]: i=2, x=395, y=13464 [Main Thread]: i=3, x=411, y=16664 [Main Thread]: i=4, x=446, y=23664 [Main Thread]: i=5, x=461, y=26664 [Main Thread]: i=6, x=476, y=29664 [Main Thread]: sending '0x59' cmd to the arduino [Main Thread]: sending '0x59' cmd to the arduino [Main Thread]: sending '0x59' cmd to the arduino [Main Thread]: sending '0x59' cmd to the arduino [Main Thread]: sending '0x59' cmd to the arduino [Main Thread]: i=12, x=634, y=61264 [Main Thread]: i=13, x=693, y=7528 [Main Thread]: i=14, x=708, y=10528 [Main Thread]: i=15, x=723, y=13528 [Main Thread]: sending '0x59' cmd to the arduino [Main Thread]: sending '0x59' cmd to the arduino [Main Thread]: sending '0x59' cmd to the arduino [Main Thread]: Exiting program bash-3.2$
Other recommendations
This code merely helps you get the most simple kind of terminal-based RX interface on the ground. This may be great if you need something prototyped, but if your ambitions take you further, you can try ncurses to make a fancy GUI inside the terminal window. This is sometimes ideal if your working over an SSH connection.