[Django]-Google Translation API id blocking ip address for too many requests

1πŸ‘

βœ…

I suggest you read this article from MDN: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429

if this is the response you get so try and look at the header Retry-After in the response object.

so adding a sleep or other delay method, with the value of that header might fix your problem.

πŸ‘€ddor254

1πŸ‘

Try adding delays between consecutive queries(using sleep) and play with the numbers to see what works for you. 2s delay after every pair of translation and 15s after every 10 to works fine for me.

πŸ‘€Abbas

1πŸ‘

I have been blocked too because of many concurrent requests. Usually always gets blocked after 500 concurrent requests. What I did was to put a timeout of 60 seconds after every 100 concurrent requests. It may seem long, but it works. You could also achieve that with a 45 seconds timeout, but I set it to 60 just to make sure.

Here’s an example

class GoogleAPI():

    def __init__(self):
        self.limit_before_timeout = 100
        self.timeout = 60

    def translate(self, source):
        translation = translator.translate(source, dest="ar")
        translation = translation.__dict__['text']
        if translation != "" and translation is not None:
            return translation

    def process(self):
        i = 0
        print("initiation")
        for t in list_of_data:
            if i < self.limit_before_timeout:
                i += 1
                self.translate(t)
            else:
                i = 0
                print("100 words added")
                time.sleep(self.timeout)
        print("All done")
πŸ‘€coredumped0x

0πŸ‘

My IP is blocked after ~450 concurrent connections. I am using php for loop to translate my text array.

So, I changed my IP Address and and changed my code for waiting after every x seconds.

My Codes in For loop ($i is value from for loop):

if ($i % 100 == 0 && $i!=0) {
    //wait 60 seconds every 100
    usleep(60000000);   // 60 seconds
    echo str_pad("XX--> WAITING 60 SECONDS<br>",4096);
}               
else 
if ($i % 10 == 0  && $i!=0) {
    //wait 15 seconds every 10
     usleep(15000000); // 15 seconds
     echo str_pad("XX--> WAITING 15 SECONDS<br>",4096);
}
else    
if ($i % 2 == 0  && $i!=0) {
    //wait 2 seconds every 2
     usleep(2000000); // 2 seconds
     echo str_pad("XX--> WAITING 2 SECONDS<br>",4096);
}
πŸ‘€Rahat

Leave a comment