FRIHOST • FORUMS • SEARCH • FAQ • TOS • BLOGS • COMPETITIONS
You are invited to Log in or Register a free Frihost Account!


trafic and robot





yvanmarl
Hi all!

I'm new to web hosting. What is the trafic (percentage) generated by robot (Yahoo, Googlebot and such) on a "regular" web site?

Any tips to reduce this?

I'm setting a web site about trains. Is there any website subject more "appreciated" by robot than other?

I'm asking the question because I heard some people telling me that some subjects may "attract" robot more easily... True?
Martin. Cool
n0obie4life
The first one has something to do with robots.txt. Google that.

The second on subject, err, no, google/any other bots does not favour anything.
yvanmarl
Thank for the reply.

I think I was not clear with the "generated trafic" statement.
I'll try to be more specific!! Wink

Some website allow a specific bandwidth. If, for example, bots and spiders count for 500 Mb/month (I know this number is unreal, but it is only for "educational" pripose!! Laughing ), so a 1Gb/mo bandwidth will be use more quickly than one may think... (50% by bots, 50% by real visitors)

So, is there statistics showing how much bandwitch for a typical website is eated by bots and other "unsollicited" trafic?

Martin Very Happy
n0obie4life
Very little.

It depends on your site.

First, the robot will check for a file called "robots.txt". If its there, I'll download it (normally less than 2kb). Googlebot only downloads it once per week or day, for the others, I'm not sure.

Then, it'll broswer your homepage. If your homepage takes up 500kb of bandwidth / real visitor visit. Googlebot/otherbots would take up 500kb (Bondings will have to confirm that).
Bondings
n0obie4life wrote:
Then, it'll broswer your homepage. If your homepage takes up 500kb of bandwidth / real visitor visit. Googlebot/otherbots would take up 500kb (Bondings will have to confirm that).

Not really. It only reads the text on your pages. Normally it doesn't download the images or style sheets. There is a separate google image bot, but I really doubt you'll see it often.
Related topics
new technology!DR ROBOT
robot rage
Ben, Robot
Audi TT Made to Like 'I, Robot' Car
a lego robot can beat the first level of super mario bros
Robot.txt and Meta tags
What are your views on Peter Crouch and his robot dance?
Robot Chicken is the smartest show on TV
Whats the Cost of building a robot ?
2. Robot Yarışması
ROBOT RACE
İşte Geleceğin Robot Insanı
[Videogames] Attack of the 50ft Robot!
İnsansı Robot Astronot Eylülde Uzay Yolunda
Reply to topic    Frihost Forum Index -> Support and Web Hosting -> Web Hosting Support

FRIHOST HOME | FAQ | TOS | ABOUT US | CONTACT US | SITE MAP
© 2005-2011 Frihost, forums powered by phpBB.