FRIHOST FORUMS SEARCH FAQ TOS BLOGS COMPETITIONS
You are invited to Log in or Register a free Frihost Account!


Scraping google





manoj2593
i want to scrape google but dont know how to do it
i have a code i.e
from lxml import html
import requests
page = requests.get('http://www.google.com')
tree = html.fromstring(page.text)
result = tree.xpath('//div[@class="urls"]/text()')
print result

this is a example code now what should i do do scrape google urs for a particular result..it would help if you would please post the code and explain so that i can iscrape google which i dont know how to do.thank you..
Marcuzzo
what do you want to do? scrape results to display them on your website? if this is the case, why don't you use the custom search API

EDIT: Is this python code?

anyway, if you are asking for 'help' then you don't ask for code but post your code and ask for help or tips.
- you need to provide as much info as you can, eg: is this server sided ( php ) or client js?
- I don't think there's a lot to scrape from http://google.com because it will only get the input field, the buttons and other stuff like the login button.
Remote_desktop_app
If you can use python for the work it will be perfect "Selinium" and "butifulSoup" python libraries will be helpfull for you just load the url by using "urllib" and pass it to the above libraries according to the neediness. Good luck
marcorele
Why you don't use Beautifull Soupe? Amazing tool!
draight
Very cool scraper that has a host of tools for scraping and ways to integrate it with any langugue flavor you prefer: www.import.io
Related topics
Google's Video Sharing
Google gets license to operate China office
Google buys social-networking service
Google Hacked?
how about put google ad in cpanel?
Beware How You Google !!!
3 things about google
Google Goes Bust!!
Google Spoofs
Problam about google ad code
Google in your scripts
Yahoo! publisher, an alternative to Google AdSense
Google Ads help fund Spyware ?
google adsense
Reply to topic    Frihost Forum Index -> Scripting -> Others

FRIHOST HOME | FAQ | TOS | ABOUT US | CONTACT US | SITE MAP
© 2005-2011 Frihost, forums powered by phpBB.