FRIHOST FORUMS SEARCH FAQ TOS BLOGS COMPETITIONS
You are invited to Log in or Register a free Frihost Account!


Retrieving Loads of Info Efficiently





polly-gone
So, I am developing an Uber Advanced web based game. It will have TONS of information and I need to know how to store/retrieve this information efficiently.

One example of something I want to do:

A commander has 6 000 soldiers. He wants to divide them into 1 000 ration groups with 6 soldiers per. Each group has a totally different set of rations.

Is there any way to store all these efficient and retrieve them without exploding a CPU/having a web host kill you.

-Nick Shocked Shocked Shocked
Gushe
I guess this won't be a problem for any CPU. Wink

Testing my MD5 Bruteforces, it could generate, hash & check over 100.000 strings per second. I think some simple mathematics and stings won't kill your CPU. Smile
Agent ME
Make your PHP code use a database, like MySQL.
polly-gone
So having that massive of a database and running tons of queries wont explode Frihost's CPUs? I do use PHP and MySQL.

-Nick Smile Smile Smile
swizzy
Some of the basic feasible performance tips (will be updated):

Have a DB with lots of tables, separating as much content as possible.
Use persistent connections.
kv
As far as storage is concerned, definitely database is the way to go, as it is meant for storage and retrieval of large data in efficient manner. Before you start, do a little bit of research on normalization (splitting tables to avoid duplication of data), denormalization (reverse of normalization -- duplication of little data, but improved performance), indexing the tables (for fast search/retrieval of data), etc. Once you are familiar with those, you will be able to create a datamodel which is efficient for your use case.
polly-gone
Okay.... I think you guys are totally missing the point. This is in the PHP and MySQL forum, so I am obviously using those. And I split up all my data (even a little too much sometimes) and have every little thing (like usernames) identified by a number.

What I want to know is if I have a game (that is never ending like World of Warcraft) and there are 1000 players and each players has 100,000 soldiers and each soldiers has its own individual attributes, is that too much for a RDMS or should I find a different method.

Thanks,

-Nick Smile Smile Smile
Peterssidan
I'm afraid that if the game gets a lot of players you will get problems.

Is it really necessary to that each soldier occupy one row in the database? If each soldier has it's own individual attributes they will need one row each, but no player will be able to know all his 6000 or 100,000 soldiers. Is it realistic that a player will manage to take care of that huge number of soldiers and ration groups? If we pretend my favourite soldiers are Erik, Daniel and Dave. Brilliant! I have some friends that I can get to know but I will never be able to get to know the whole army. just a very small part so there is a huge amount of data I never will care about.

Will the number of soldiers in each ration group always be 6? It would be much more efficient if you only have to store the number of soldiers and the number of ration groups. or at least make some simplifications. Maybe you can store the individual attributes for the whole group? like: 40% have no injuries, 30% have small injuries, 20% have lost their legs and 10% are seriously ill. Then they are not individual attributes any more but we have gain efficiency. I think the most important is that you try to look at the soldiers as big groups of soldiers in some way and avoid handle them individually.
polly-gone
Okay, I am a little confused by your response, so I think I should clarify. Every troops is not an actual person. It is created in barracks and you can send whatever troops you want into battle.

I want to have troops broken down as much as possible because I am going to have a very, very, very ridiculously complex battle system. All battles are going to be done automatically, but the way the winner, losers, casualties, etc is calculated is very detailed. When a soldier is created, the will be given random characteristics like height and weight, and even things like those will have an effect on their battle performance.

People aren't going to memorize the stats of every soldier off-hand or anything like that, but when planning a battle, they will be able to pull up lists of soldiers to use. They could go onto the page, and select something like [Optimal Height, Overweight, >50% Comfort, >75% Morale, >500 Strength].

I want to know if MySQL can handle managing this.

-Nick
rvec
maybe you should make one row for each possible type in one table and have one other table to combine the id's with the squad/army/...
you could just store the id's in a comma separated list and use an explode to get out a nice array of all types.
polly-gone
Well, see that is the thing. Everyone can have a huge variety of troops. And another thing with that is that the combination's are nearly infinite. Strength, Weight, Energy, Morals, Happiness, etc can change on a day to day basis.
leontius
polly-gone wrote:
Well, see that is the thing. Everyone can have a huge variety of troops. And another thing with that is that the combination's are nearly infinite. Strength, Weight, Energy, Morals, Happiness, etc can change on a day to day basis.


Here is my recommendation: maintain only the average value of strength, weight, energy etc. For each new troops, modify the average accordingly. When splitting armies (or you are in need of getting a status for an individual in the army), put random things based on the average and fixed standard deviation. If a person needs to have status that changes everyday, use a randomizer that takes "the name of the person" and "date" as seed.
polly-gone
Okay. See, here is the thing. You guys are TOTALLY missing the point. My question is not whether or not a database can store this information. I have seen message boards with 100,000 members and over 150 Million Posts. Storage is not an issue for me. The amount of information stored is small.

My question is, will a database be able to handle a lot of queries.

-Nick Shocked Shocked Shocked
sonam
Ones when you will get 100,000 members I am pretty sure you will leave Frihost. Wink In this moment your problem is not DB handling because SQL can handle milions of query. The real problem will occur when you will get your bandwidth qouta.

Sonam
polly-gone
Okay. So then it is not a problem. The bandwidth needed is actually very very small.

Thanks,

-Nick Smile Smile Smile

***CLOSE***
sonam
polly-gone wrote:
Okay. So then it is not a problem. The bandwidth needed is actually very very small.

Thanks,

-Nick Smile Smile Smile

***CLOSE***


That is true, but don't forgot, any query to mysql is also calculated in bandwidth.

Sonam
leontius
polly-gone wrote:
Okay. See, here is the thing. You guys are TOTALLY missing the point. My question is not whether or not a database can store this information. I have seen message boards with 100,000 members and over 150 Million Posts. Storage is not an issue for me. The amount of information stored is small.

My question is, will a database be able to handle a lot of queries.

-Nick Shocked Shocked Shocked


I don't think I miss the point... because the less / simpler data you have, the less processing / queries you need to make, no? Randomizer is a great way to store seemingly many things without actually store the data...
polly-gone
But a randomizer would ruin the gameplay. The whole point of actual data is so that if you have good strategy at managing your troops, you will win battles. If it is done randomly, you could have the best strategy in the world and it wouldn't matter.

And the whole point was is there anything special I can do to STORE these without exploding a CPU.

-Nick Confused Confused Confused

***SOMEONE CLOSE PLEASE***
rockacola
To answer your original question:

Quote:
...I need to know how to store/retrieve this information efficiently.

...Is there any way to store all these efficient and retrieve them without exploding a CPU/having a web host kill you.


IMO, PHP+MySQL isn't the tool for heavy duty web application like a web-based game, simply because MySQL isn't optimized for complex query and does not come with query operators like query stack, object-relational mapping.

Existing PHP web-based games tend to distribute data into different database sits in seperate machine to balance the work load.
rvec
-close- on request
Related topics
css + html
xbox 360, ps3 or revolution?
FTP problems --> Use Filezilla and/or active mode
Memberships to MyEzy
AJAX tutorial
easyplay.info
Xbox Softmod Tutorial
Google's Secret Revealed
[PHOTOSHOP TUTORIAL LINKS]
Top 10 web design mistakes!..
foobar or winamp?
Programming links, info, and tutorials
The Perfect Audio Rips
should we care about antartic ice melting?
This topic is locked: you cannot edit posts or make replies.    Frihost Forum Index -> Scripting -> Php and MySQL

FRIHOST HOME | FAQ | TOS | ABOUT US | CONTACT US | SITE MAP
© 2005-2011 Frihost, forums powered by phpBB.