How can I read arguments with a program running in the background?
Example: a simple program that prints a list value every 10 seconds
import argparse
import time
import sys
myList = []
def parseArguments():
parser = argparse.ArgumentParser(description="example")
parser.add_argument('-a', '--addElement', help='adds an element to the list')
args = parser.parse_args()
if args.addElement:
myList.append(args.addElement)
def main():
parseArguments()
while(True):
print(myList)
time.sleep(10)
The problem is that the program only reads the arguments passed at the beginning, I want it to read the arguments passed at any time while running.
I want to run a program in the background as a service and pass arguments to the program every time.
source to share
I understand that what you are asking looks like a service (or daemon process) capable of accepting asynchronous commands.
Front end:
prog foo
=> ok re-prints ['foo']
later:
prog bar
=> the second instance outputs and the first instance re-prints ['foo', 'bar']
Interior design
This is far from easy! You need to set up the IPC mechanism so that the second instance can communicate with the first, with non-blocking IO (or multithreading) in the first instance. On Unix you can use os.mkfifo
, but if you want a portable solution, you will have to use IP sockets on localhost
Structure in high level pseudocode
get argument via argparse
bind to a fix port on localhost, in UDP protocol
if success:
# ok it is the first prog
initialize list from argument
loop:
get command from UDP socket, with timeout = 10s
if cmd is add param:
add parameter to list
elif cmd is exit: # not asked in question but should exist
exit
print list
else:
# another prog has taken the socket, pass it the arg
send the arg to the UDP port with proper protocol
Caveats for this simple design: there is a race condition, there is already a program waiting for a socket that exits from the first bind and send attempt. To deal with this, you must use TCP with select
a timeout on the listening socket and graceful shutdown to ensure that the message is received on the other side. On error, you iterate (maximum amount of time) since the first server may have logged out.
Here's an example implementation:
import socket
import select
import argparse
import time
import sys
TIMEOUT=10
IFACE='127.0.0.1'
PORT=4000
DEBUG=False
myList = []
old = ""
def parseArguments():
parser = argparse.ArgumentParser(description="example")
parser.add_argument('-a', '--addElement',
help='adds an element to the list')
parser.add_argument('-q', '--quit', action='store_true',
help='closes main service')
parser.add_argument('-d', '--debug', action='store_true',
help='display debug information')
args = parser.parse_args()
if args.quit:
senddata("QUIT\n")
sys.exit(0)
if args.debug:
DEBUG=True
if args.addElement:
myList.append(args.addElement)
def read(s):
global old
data = old
while True:
block = s.recv(1024)
if len(block) == 0: return data
if b'\n' in block:
block,o = block.split(b'\n', 1)
old = o.decode()
data += block.decode()
return data
data += block.decode()
def gracefulclose(s, msg):
s.send(msg.encode())
s.shutdown(socket.SHUT_WR)
try:
read(s)
finally:
s.close()
def server(s):
if DEBUG:
print("SERVER")
s.listen(5)
while True:
sl = select.select([s], [], [], TIMEOUT)
if len(sl[0]) > 0:
s2, peer = s.accept()
try:
data = read(s2)
print(data)
gracefulclose(s2, "OK")
finally:
s2.close()
if data.startswith("QUIT"):
return
elif data.startswith("DATA:"):
myList.append(data[5:])
print(myList)
def senddata(data):
s = socket.socket(socket.AF_INET)
try:
s.connect((IFACE, PORT))
s.send(data.encode())
data = read(s)
if (data.startswith("OK")):
return True
except:
pass
finally:
s.close()
return False
def client():
return senddata("DATA:" + myList[0] + "\n")
def main():
end = False
MAX = 5
while not end and MAX > 0:
s = socket.socket(socket.AF_INET)
try:
s.bind((IFACE, PORT))
except Exception:
s.close()
s = None
if s:
try:
server(s)
finally:
s.close()
return
else:
if DEBUG:
print("CLIENT", " ", 6 - MAX)
end = client()
MAX -= 1
time.sleep(1)
if __name__ == "__main__":
parseArguments()
main()
source to share
import argparse
import time
import sys
myList = []
def parseArguments():
parser = argparse.ArgumentParser(description="example")
parser.add_argument('-a', '--addElement', help='adds an element to the list')
args = parser.parse_args()
if args.addElement:
myList.append(args.addElement)
def main():
parseArguments()
import select
while(True):
while select.select([sys.stdin], [], [], 0)[0]:
myList.append(sys.stdin.readline().strip())
print(myList)
time.sleep(10)
If you are passing more arguments at runtime, you should read them from stdin. Using the select module you can check if there is a newline in stdin and then add them to myList.
source to share
Basically what you are asking is how to do Inter-process communication (IPC).
Why did I say that? Well, answer yourself: how would you like to pass these arguments to your background service? By hand? I don't think so (because you will have a simple interactive program that should just wait for user input). You probably want another script / program that dispatches these arguments via some kind of commands on demand.
Typically, there are several ways to link two or more programs, the most popular of which are:
Shared File - You can just check the contents of the file on your disk. The advantage of this solution is that you can probably edit this file with your favorite text editor without having to write a client application.
Pipes - One program reads its input, which is the output of another program. You should just read sys.stdin.
# receiver
def read_input():
for l in sys.stdin:
yield l
Sockets are a stream of data sent over a network interface (but it can be sent locally on the same computer). The Python docs have a very nice introduction to socket programming.
Shared Memory - Your programs read / write the same block of memory. In Python, you can use mmap for this .
Whichever way you define your chosen processes, you must establish some kind of interface between them. It could be a very simple text-based interface like this:
# command syntax
<command> SPACE <parameter> NEWLINE
SPACE := 0x20 # space character
NEWLINE := 0x0A # '\n' character
# a command adding element to receiver list
ADD SPACE <element> NEWLINE
# a command removing element from receiver list:
REMOVE SPACE <element> NEWLINE
# examples:
ADD first element\n
REMOVE first element\n
So, for example, if you are sending a message over a socket (which I recommend), your receiver (server) should read the buffer until the newline character, then check if the first word is "ADD" and then add the remaining characters (minus newline) to your list. Of course, you must be prepared for some kind of "attack" - as you must indicate that your messages cannot be longer than, for example, 4096 bytes. This way, you can cancel your current buffer after reaching its limit, which means you won't allocate memory indefinitely while waiting for a newline. This is a very important rule of thumb: don't trust user input.
Good luck! :)
source to share