1π
I suggest you read this article from MDN: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429
if this is the response you get so try and look at the header Retry-After
in the response object.
so adding a sleep or other delay method, with the value of that header might fix your problem.
1π
Try adding delays between consecutive queries(using sleep) and play with the numbers to see what works for you. 2s delay after every pair of translation and 15s after every 10 to works fine for me.
- [Django]-A correct case to use Django signals
- [Django]-Deploying a Geodjango Application on AWS Elastic Beanstalk
- [Django]-Can not assign None to Django DateTimeField()
- [Django]-Mercurial: Exclude django settings.py after intitial commit
1π
I have been blocked too because of many concurrent requests. Usually always gets blocked after 500 concurrent requests. What I did was to put a timeout of 60 seconds after every 100 concurrent requests. It may seem long, but it works. You could also achieve that with a 45 seconds timeout, but I set it to 60 just to make sure.
Hereβs an example
class GoogleAPI():
def __init__(self):
self.limit_before_timeout = 100
self.timeout = 60
def translate(self, source):
translation = translator.translate(source, dest="ar")
translation = translation.__dict__['text']
if translation != "" and translation is not None:
return translation
def process(self):
i = 0
print("initiation")
for t in list_of_data:
if i < self.limit_before_timeout:
i += 1
self.translate(t)
else:
i = 0
print("100 words added")
time.sleep(self.timeout)
print("All done")
- [Django]-CSRF token missing or incorrect β Django
- [Django]-Amazon ELB + Django HTTPS issues
- [Django]-Render to response to a redirected url in Django
- [Django]-Problem with Python modules
- [Django]-Django: How to dynamic filter foreignkey choices by Customizing the Django_Admin
0π
My IP is blocked after ~450 concurrent connections. I am using php for loop to translate my text array.
So, I changed my IP Address and and changed my code for waiting after every x seconds.
My Codes in For loop ($i is value from for loop):
if ($i % 100 == 0 && $i!=0) {
//wait 60 seconds every 100
usleep(60000000); // 60 seconds
echo str_pad("XX--> WAITING 60 SECONDS<br>",4096);
}
else
if ($i % 10 == 0 && $i!=0) {
//wait 15 seconds every 10
usleep(15000000); // 15 seconds
echo str_pad("XX--> WAITING 15 SECONDS<br>",4096);
}
else
if ($i % 2 == 0 && $i!=0) {
//wait 2 seconds every 2
usleep(2000000); // 2 seconds
echo str_pad("XX--> WAITING 2 SECONDS<br>",4096);
}
- [Django]-Use Django+Redis+Socket.io to build chat room, where to start?
- [Django]-Python chained comparison