|
|||||||||||||||||||||||||||||||
get to top in SEO ranking
edit robot.txt
edit robot.txt to enable all robots of SEO can visit anywhere in the site for higher ranking!
you may be familiar with it but its for the beginners.
Best Regards
Pratik Lalwani
edit robot.txt to enable all robots of SEO can visit anywhere in the site for higher ranking!
you may be familiar with it but its for the beginners.
Best Regards
Pratik Lalwani
Whell, edit the robots.txt, but with what?
Please help me to edit create the robots.txt! I whant to learn how to.....
Please help me to edit create the robots.txt! I whant to learn how to.....
| undoweb wrote: |
| Whell, edit the robots.txt, but with what?
Please help me to edit create the robots.txt! I whant to learn how to..... |
Well robots.txt is used when you want search engines not to crawl specific files or directories like admin directory etc.
Use this online robots.txt generator to generate robots.txt file
http://www.webtoolcentral.com/webmaster/tools/robots_txt_file_generator/
where do you upload the txt file to? public_html?
i can't seem to make a .txt, it keeps coming out as robots.lnk when i upload it
| NjRocket wrote: |
| where do you upload the txt file to? public_html? |
yes you should upload it to your root directory such that it should becomes yourdomain.com/robots.txt or username.frih.net/robots.txt
| NjRocket wrote: |
| i can't seem to make a .txt, it keeps coming out as robots.lnk when i upload it |
First open notepad or any other text editor put the code into it and save its as robots.txt and then upload it to your root directory.
For more information about robots.txt file visit www.robotstxt.org
Put what code in robot.txt? I'm confused.
| Hobbit wrote: |
| Put what code in robot.txt? I'm confused. |
robots.txt is usually used if you want certain part of your website not to indexed by search engines.
The code can be obtained by http://www.webtoolcentral.com/webmaster/tools/robots_txt_file_generator/ this onlint robots.txt generator.... You will specify which directory should not be indexed by search engines and it will create the code for you.
Please ask again if this is not clear. www.robotstxt.org is great resource about robots.txt
Hi,
AFAIK, robots.txt is sought after by crawlers when they arrive on your site. Thus the 404 file not found that appears on your log if you don't have one (same for favicon.ico)
But a missing robots.txt won't penalize your site at all, because it won't stop bots to crawl your page. Simply because it works this way :
If you have something that you want to keep secret, write a robots.txt file that will raise "frontiers"/dircetories not to crawl beyond.
If you don't have anything to hide from bots, let the door open, and then having a robots.txt (that only says disallow : ") or nothing is the same (apart from the reports of the missing file in your logs)
Well, this point alone is worth putting a robots.txt on your site, but I just wanted to clear a bit of all that fuss around robots.txt, as if it had a major role to play on your site's rankings.
And don't forget that there's a big drawback : robots.txt files are visible by anyone. Putting boundaries in your directories through the robots.txt may excite visitor's curiosity : make sure you've got .htaccess well-configured !
AFAIK, robots.txt is sought after by crawlers when they arrive on your site. Thus the 404 file not found that appears on your log if you don't have one (same for favicon.ico)
But a missing robots.txt won't penalize your site at all, because it won't stop bots to crawl your page. Simply because it works this way :
If you have something that you want to keep secret, write a robots.txt file that will raise "frontiers"/dircetories not to crawl beyond.
If you don't have anything to hide from bots, let the door open, and then having a robots.txt (that only says disallow : ") or nothing is the same (apart from the reports of the missing file in your logs)
Well, this point alone is worth putting a robots.txt on your site, but I just wanted to clear a bit of all that fuss around robots.txt, as if it had a major role to play on your site's rankings.
And don't forget that there's a big drawback : robots.txt files are visible by anyone. Putting boundaries in your directories through the robots.txt may excite visitor's curiosity : make sure you've got .htaccess well-configured !
Related topics
 |