Scapy python script consumes too much CPU while sniffing
I have coded a wireless probe using python + scapy. I want to use this script in openwrt routers.
Every time it captures a probe request from neighboring devices, the information is sent to the web service. (mac, power and probe).
My problem is high CPU consumption. The script works pretty well in my laptop (requires 50-70% of the cpu) but when I run it in the openwrt router (400MHz cpu, 16 rams) it takes 99%. This is a well known bug in scapy of lost packets with heavy loads (I was testing the script at the same time on my laptop and in my router and the router was missing all available packets in the air)
I've already done some optimizations for the code, but I think there is more room for improvement.
This is a script.
#!/usr/bin/python
from scapy.all import *
import time
import thread
import requests
from datetime import datetime
PROBE_REQUEST_TYPE=0
PROBE_REQUEST_SUBTYPE=4
buf={'arrival':0,'source':0,'dest':0,'pwr':0,'probe':0}
uuid='1A2B3'
def PacketHandler(pkt):
global buf
if pkt.haslayer(Dot11):
if pkt.type==PROBE_REQUEST_TYPE and pkt.subtype == PROBE_REQUEST_SUBTYPE:
arrival= int(time.mktime(time.localtime()))
try:
extra = pkt.notdecoded
except:
extra=None
if extra!=None:
signal_strength = -(256-ord(extra[-4:-3]))
else:
signal_strength = -100
source = pkt.addr2
dest= pkt.addr3
pwr=signal_strength
probe=pkt.getlayer(Dot11).info
if buf['source']!=source and buf['probe']!=probe:
print 'launch %r %r %r' % (source,dest,probe)
buf={'arrival':arrival,'source':source,'dest':dest,'pwr':pwr,'probe':probe}
try:
thread.start_new_thread(exporter,(arrival,source,dest,pwr,probe))
except:
print 'Error launching the thread %r' % source
def exporter (arrival,source,dest,pwr,probe):
global uuid
urlg='http://webservice.com/?arrival='+str(arrival)+'&source='+str(source)+'&dest='+str(dest)+'&pwr='+str(pwr)+'&probe='+str(probe)+'&uuid='+uuid
try:
r=requests.get(urlg)
print r.status_code
print r.content
except:
print 'ERROR in Thread:::::: %r' % source
def main():
print "[%s] Starting scan"%datetime.now()
sniff(iface=sys.argv[1],prn=PacketHandler,store=0)
if __name__=="__main__":
main()
[UPDATE]
After much reading and deep searching (it seems not many people have found a complete solution to the same problem or something similar). I found that you can filter directly from the sniff function, so I added a filter to just catch the probe requests.
def main():
print "[%s] Starting scan"%datetime.now()
sniff(iface=sys.argv[1],prn=PacketHandler, filter='link[26] = 0x40',store=0)
It runs very smoothly on my laptop, using between 1% -3% CPU and catching most of the available packets in the air.
but when I run this on the router the script throws an error and crashes.
Traceback (most recent call last):
File "snrV2.py", line 66, in <module>
main()
File "snrV2.py", line 63, in main
sniff(iface=sys.argv[1],prn=PacketHandler, filter='link[26] = 0x40',store=0)
File "/usr/lib/python2.7/site-packages/scapy/sendrecv.py", line 550, in sniff
s = L2socket(type=ETH_P_ALL, *arg, **karg)
File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 460, in __init__
attach_filter(self.ins, filter)
File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 132, in attach_filter
s.setsockopt(SOL_SOCKET, SO_ATTACH_FILTER, bpfh)
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 99] Protocol not available
I have tried using the bpf filter syntax (same as used in tcpdump http://biot.com/capstats/bpf.html ) and it is assumed that you can also use it in scapy but I am getting filter syntax error.
Supply:
def main():
print "[%s] Starting scan"%datetime.now()
sniff(iface=sys.argv[1],prn=PacketHandler, filter='type mgt subtype probe-req', store=0)
Mistake:
Traceback (most recent call last):
File "snrV2.py", line 66, in <module>
main()
File "snrV2.py", line 63, in main
sniff(iface=sys.argv[1],prn=PacketHandler, filter='type mgt subtype probe-req', store=0)
File "/usr/lib/python2.7/site-packages/scapy/sendrecv.py", line 550, in sniff
s = L2socket(type=ETH_P_ALL, *arg, **karg)
File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 460, in __init__
attach_filter(self.ins, filter)
File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 120, in attach_filter
raise Scapy_Exception("Filter parse error")
NameError: global name 'Scapy_Exception' is not defined
In the router, I have installed the latest scapy and tcpdump. Now I really don't know what to do.
source to share
I encountered a similar error (socket.error: [Errno 99] Protocol not available) when I tried to use sniff()
with a filter on my NETGEAR WNDR4300.
After doing a lot of googling, I found that the reason is that my router's Linux kernel doesn't include CONFIG_PACKET. In the Scapy installation guide :
- Make sure packet socket is selected in your kernel (CONFIG_PACKET)
- If your kernel is <2.6 make sure Socket CONFIG_FILTER filtering is selected)
If you set CONFIG_PACKET=y
when compiling the kernel then it will use BPF for the underlying socket.
source to share