python - How to scrap data using google api -


import requests  def search(query, pages=4, rsz=8):     url = 'https://ajax.googleapis.com/ajax/services/search/web'     params = {         'v': 1.0,     # version         'q': query,   # query string         'rsz': rsz,   # result set size - max 8     }      s in range(0, pages*rsz+1, rsz):         params['start'] = s         r = requests.get(url, params=params)         result in r.json()['responsedata']['results']:             yield result 

at first 2,3 attempts retrieving required pages, after 2,3 attempts not getting result. returning "none" or []. google blocking ip after few attempts? solution?

i not sure work or not, way avoid getting blocked sites not encourage scraping use proxy while retrieving web pages. please check how proxy can used in code.


Comments

Popular posts from this blog

Capture and play voice with Asterisk ARI -

c++ - Can not find the "fiostream.h" file -

java - Why database contraints in HSQLDB are only checked during a commit when using transactions in Hibernate? -