Serial Sensor Data - Eliminate Unwanted Strings
I am using python code to select a sensor connected to a serial port.
This works very well in most cases. However, sometimes the output from the sensor is incorrect:
['\x80', '19.38', '22.7', '649', 'w\\'] - correct
['\x80', '19.46', '22.7', '648', 'wZ'] - correct
['\x80', '19.49', '22.7', '650', 'w'] - correct
['w', '\x80', '19.45', '22.7', '650'] - error
This is causing a lot of problems with my code. I'm just wondering if anyone knows how to start splitting a string with a value after "\ x80" or some other way around this?
Example:
If the string I receive from the sensor is called "receive" then my code is
Var1,Var2,Var3,Var4,Var5 = receive.split()
Now I need to convert Var2 to float so that I can do some math data on it
Var2float = float(Var2)
This works most of the time when Var2 is a value like 19.38 until the encoder output moves and Var2 becomes "/ x80" which cannot be converted to float
Any ideas on how to filter or work around this sensor output error?
source to share
A simple sentence try except
will solve the problem:
my_buffer = ['\x80', '19.38', '22.7', '649', 'w\\']
converted_buffer = []
for v in my_buffer:
try:
converted_buffer.append(float(v))
except ValueError:
print "{} cannot be converted, skipping to the next".format(v)
print converted_buffer
Output
cannot be converted, skipping to the next
w\ cannot be converted, skipping to the next
[19.38, 22.7, 649.0]
You can iterate over the buffer and when ValueError
only raises the question of what to do with the value. In this case, we just print it out, you can register it or do whatever you want with it.
Note . I think that sync frame
from the device, the signifier \x80
says that this is the beginning of the transmission. Many sensors have start and end frames for synchronization.
source to share
The easiest way I can think of is to use a regex to find the end of the substring \x80
and split the string starting at that.
Here's an example
import re
data = [
r'\x80 19.38 22.7 649 w\\',
r'\x80 19.46 22.7 648 wZ',
r'\x80 19.49 22.7 650 w',
r'w \x80 19.45 22.7 650',
]
for record in data:
m = re.search(r' \\x80 \s+ ', record, flags=re.X)
if m:
values = record[m.end():].split()
print(values)
Output
['19.38', '22.7', '649', 'w\\\\']
['19.46', '22.7', '648', 'wZ']
['19.49', '22.7', '650', 'w']
['19.45', '22.7', '650']
source to share
The general concept would first involve buffering the data, a second check if you have a whole sequence, and a third retrieving the sequence. The next class makes buffering and reading and viewing of single bytes possible.
class MySerialBuffer:
def __init__(self):
self.buffer = []
def peek(self):
return self.buffer[0]
def read(self):
return self.buffer.pop(0)
def count(self):
return len(self.buffer)
def fill(self, new):
self.buffer = self.buffer + new
def extract(self):
while self.count() > 0 and self.peek() != '\x80':
self.read()
if self.count() < 4:
return None
self.read()
return (self.read(), self.read(), self.read())
def __iter__(self):
while True:
out = self.extract()
if out is None:
return
yield out
msb = MySerialBuffer()
msb.fill(['\x80', '19.38', '22.7', '649'])
msb.fill(['\x80', '19.38', '22.7', '649'])
msb.fill(['w', '\x80', '33.4', '19.38', '22.7', '649', 'f', 'w'])
msb.fill(['\x80', '19.38', '22.7', '649', 'w'])
for reading in msb:
print(reading)
This can significantly improve performance. Also, depending on how you read the data, this can be greatly simplified, and performance has increased by handling your data differently, since I wrote a class to work with what you stated in your question. You can read actual integer values, and you can process them as you read them into the buffer. Also you can use a more efficient system than list
if you are processing many samples per second to help. The list is ineffective and adds new items and also the operation is pop
ineffective. However, there are other primitive collections that can handle these types of operations more efficiently, eg Queue
.
source to share