How to count bytecodes in Python so that I can change sys.setcheckinterval accordingly
I have a port scan application that uses work queues and threads.
It uses simple TCP connections and spends a lot of time waiting for packets to return (up to half a second). This way, threads don't have to completely execute (i.e. the first half sends a packet, a context switch, does things, returns to a thread that has network data waiting for it).
I suspect I can improve performance by changing sys.setcheckinterval
the default of 100 (which allows up to 100 bytecodes to execute before switching to another stream).
But, not knowing how many byte codes are actually executed in a thread or function, I flew blind and just guessed the values, testing and relying on testing shows a measurable difference (which is difficult, since the amount of executable code is minimal, simple socket connection, so network jitter will probably affect any measurements more than changing sys.setcheckinterval).
Thus, I would like to know how many bytecodes are in certain code executions (i.e. the sum for a function or when executing a stream), so I can make smarter guesses about what to set sys.setcheckinterval to.
source to share
For a higher level (method, class) the wise dis module should help.
But if a fine grain is needed, tracing will be inevitable. Tracing works linearly, but the one described here is a great way to dive deeper into the bytecode layer. Hats by Ned Batchelder.
source to share
"I suspect I can improve performance by changing sys.setcheckinterval"
This rarely works. Correct behavior cannot be time-dependent - you cannot control time. Small changes in OS, hardware, Python patch level, or moon phase will change the behavior of your application.
The select module is what you use to wait for I / O. Your application can be structured as a main loop that makes select and queues to work with other threads. Other threads are waiting for entries in the request queue for processing.
source to share
Reasoning about a system of this complexity will rarely give the correct answer. Measure results and use the fastest setting. If, as you say, testing cannot measure the difference in different setcheckinterval settings, then why change it? Only measurable differences are interesting. If your test run is too short to provide meaningful data, make the run take longer than it will.
source to share