[Answer]-Python parallel processing from client-server application

1👍

In general, it’s best to do stuff like this using a single thread when possible. You just have to make sure your functions don’t block other functions. The builtin lib that comes to mind is select. Unfortunately, it’s a bit difficult to explain and I haven’t used it in quite some time. Hopefully this link will help you understand it http://pymotw.com/2/select/.

You can also use the multiprocessing lib and poll each pid in a separate thread. This can be very difficult to manage if you plan to scale out further! Use threads only as a last resort (this is my usual rule of thumb when it comes to threads). https://docs.python.org/2/library/multiprocessing.html#module-multiprocessing

from multiprocessing import Process

def askforprocess(processpid):
    #Create TCP/IP socket
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    # Connect on host and port provided in command line arguments
    server_address = ('172.16.1.105', int('5055'))
    sock.connect(server_address)

    # Send the data
    try:
            message = processpid
            sock.sendall(message)
            data = sock.recv(2048)
    finally:
            sock.close()
    return data

if __name__ == '__main__':
    info('main line')
    p = Process(target=askforprocess, args=(processpid,))
    p.start()

Lastly, there’s Twisted library which is probably the most difficult to understand, but defiantly makes concurrent (not necessarily parallel) functions easy to write. Only bad thing is you’d probably have to rewrite your entire app in order to use Twisted. Don’t be put off by this fact, try to use it if you can.

Hope that helps.

👤notorious.no