Can someone tell me how i would go about getting google bots to crawl my forums?? ive heard about robots.txt but i dont really get it.. so....
Google Bots
Robots.txt does not impact the crawling
It is used only to prevent the bot from crawling but cannot be forced to make it crawl
What you do is submit your site to google and wait for it to crawl thats all.
Use some sensible meta tags to help in the crawling
It is used only to prevent the bot from crawling but cannot be forced to make it crawl
What you do is submit your site to google and wait for it to crawl thats all.
Use some sensible meta tags to help in the crawling
The more backlinks you create the more likely you are to get Googlebot or any other search engine bot to crawl your website. In addition to the increased likelyhood of a bot comming to your site you will also get a higher amount pages in total indexed. Take some of the large forums, for example www.phpbb.com and www.area51.phpbb.com/phpBB/ the major search engine crawlers basically live on these forums.
Check the screenshot that I just took
Making and submitting a sitemap to Google and Yahoo will also help in getting your site indexed.
Google - www.google.com/webmasters/sitemaps/login
Yahoo - http://search.yahoo.com/info/submit.html
Finally as scorpio said you can submit your site to google and other major search engines.
Check the screenshot that I just took
Making and submitting a sitemap to Google and Yahoo will also help in getting your site indexed.
Google - www.google.com/webmasters/sitemaps/login
Yahoo - http://search.yahoo.com/info/submit.html
Finally as scorpio said you can submit your site to google and other major search engines.
 |