Project Paradigm

[v6v]

(In response to my previous post)

From the looks of things, it seems the only way  to actually only include sprite properties in saved map data without saving actual tile data to the file itself is to modify mapbase, or to just create a new system for saving data.

Very difficult agh, I might be forced to rewrite MapBase.cs...

[bluemonkmn]

I see your point. I could have both input and updates sent unreliably, updates are important, but as long as the server ultimately holds the information, it's alright. The problem that I had with that was, what if the update was a huge gap from where the sprite was? (from what the client believed) which is where durund was going.

Happens all the time. Sometimes my character dies while waiting for the server to respond in Diablo III. Sometimes the other player playing with me ends up in a very different place on the map.

The system I use works with deltas. It only sends data when an input is pressed differently from the last frame (Look at me thinking about optimization when I just started working on the game!   I guess it was because I was tired of playing laggy games already) It sends a binary value. "0000000000" The first two digits correspond to the player number, say, player 1 is 01, player 2 is 02, etc.

The next 4 digits correspond to input data (Up, Down, Left, Right)
The 4 after that correspond to input button data (Button1, Button2, Button3, Button4)

0 Represents that a button is not pressed
1 Represents that a button IS pressed

Why not just sent Sprite.inputs, which includes all that information (except the player number) in a single 4-byte integer?

Then this is game breaking because say the remote client was attacking an enemy and had defeated the enemy, but your client didn't get the data telling you about the attack.. Since the clients execute sprite rules by themselves to minimize data, then an update sent telling your client that the client that was attacking is now FINISHED attacking which you did receive (When really the server and that client know that the attack defeated the enemy so the enemy is dead on their ends, but since you didnt recieve an update telling you that it was attacking, on your client, the enemy is still alive) would make your client just see the other player casually strolling off with a false sense of accomplishment, as the enemy was still alive in your game.

I've seen that happen too. I'll be playing merrily along in Diablo III and my wife is desperately trying to survive a huge monster on her screen in the same place. (Maybe Diablo III isn't the best model. ) But the server has the authoritative answer and decides whether that monster really exists and whether it has really killed one of us.

This creates an entire mind-jar where the client thinks that it has an item, when on the server, and to other clients, it doesn't. I can't send the entire inventory data for each sprite (Their inventory data for each sprite is colossal!! It's one of the largest arrays in the game!)

I guess that's why most games don't allow clilents to pick up items nor affect their own inventory at all. You can waltz over that item all you like, but nothing's going to happen until the server knows what your doing and send you your inventory update (just one item, not the whole inventory if that's suitable).

Item Pickups are only recognized by the server, right? Well I'll keep it that way. Clients don't have the power to pick up items on their own then. The server sends the data to the client about the pickup when a sprite on the server picks up an item, telling it that Player<X> should get Item <ID>. THIS however, HAS to be reliably sent. The server sends all of the clients on that map the data about the pickup, and when it does it also sends the data to the client logged in with the sprite that picked up the item. When the data is received by that client, that data also holds a confirmation ("Coin collected") that tells it that it not only that it should have the item, but it also indicates that the item was successfully received. This way people aren't confused and they don't think they've picked up an item that they don't actually have.

Yes, unless you're regularly updating all of inventory, that's one update you'd want to make reliable/acknowledged.

I know this isn't how most MO's would do it, because they have item shops that people spend real money for and cant have players compromising data this way. But it works. I think this is an area that'll take care of itself though. Even if a player attempts to hack by giving themselves an item that they aren't supposed to have by changing the data the server tells them that they're supposed to have, things reset when they change maps or log back in. So I don't have much to worry about. Their game will be bugged and out of sync for attempting to hack, and it won't mess up anyone else. (It's their fault and their problem for trying to hack anyways, right?  ) So who cares about them. If it's even an extra healing item, their health will be updated by what I'm planning to do. (read below)

Hacking simply wouldn't work because say, for example, they give themselves a suit of heavenly armor that prevents all damage. Well, when the enemy hits them on the server, it's going to assign damage, and there's nothing the client can do about it. Or say they give themselves a key to open a door. The client might open that door, but the server isn't going to, and all their position updates will leave them standing at the door unable to get through. (And if the server is the one responsible for opening all doors, the door won't even open on the client.)

I'll send this from the server to each client as a positional update.
00000000000000000000

Where each digit isn't binary, but, instead decimal, each 4 digits represents what needs to be updated. The first 4 are X the second 4 are Y, the third 4 is the new HP value, the next is the new MP value, and the next is the SP value. Since I know that HP, MP, SP, and X and Y all have a max value of 9999, I don't have to use variable length values. It ranges from 0000 to 9999. And these are the most important values of the sprite-

Why not just serialize the sprite objects (including "inputs")? Binary serialization is much more efficient than decimal serialization. That's what the save/load game function does. The sprites have all this info. Do your sprites not have HP/MP/SP on them? If not, serialize whatever structure it is that contains that info.

From the looks of things, it seems the only way  to actually only include sprite properties in saved map data without saving actual tile data to the file itself is to modify mapbase, or to just create a new system for saving data.

Very difficult agh, I might be forced to rewrite MapBase.cs...

You saw how easy serialization was. Why not just write a simple function that returns all the sprites of a map in serialized form?

[v6v]

Why not just serialize the sprite objects (including "inputs")? Binary serialization is much more efficient than decimal serialization. That's what the save/load game function does. The sprites have all this info. Do your sprites not have HP/MP/SP on them? If not, serialize whatever structure it is that contains that info.

At the moment I'm just sending the properties that I need. By only sending the input state, it's only 0000000000.

If I'm sending all serializable data in SpriteBase, I'm sending more data than I need to.

Normally there's only two things I need to send:
Input Data (I'm going to look into Sprite.inputs)
I don't want to send anything BUT Sprite.inputs for each frame, I'm looking on MSDN about Serialization, is there a way for me to Serialize ONLY THAT field on SpriteBase? It seems that on MSDN to exempt fields from serialization, I have to use attributes set on my class. However, there's times I'm going to want to serialize certain fields and other times where I'm going to need to serialize other fields.
I'd love to send only a serialized version of the input data for SpriteBase, but I'm unsure if my method results in much less to send, or if serialization results in much less to send..

Here's another thing about serialization I'm wondering about..
I know serialized objects are stored as bytes, but is the output format bytes or a byte array? Lidgren can easily send byte arrays. But here's my other question-

Is this serialized data extremely large? Or reasonable to send to all clients every 10 seconds? If I'm sending all clients a serialized version of a SpriteBase object for positional updates, and there's about 4 objects on the map, would it be too much to handle?


X/Y/HP/SP/MP/ well, come to think of it- I'm going to need to send everything BUT the array that holds inventory data. So I'm going to be forced to serialize that..

So for positional updates I understand that I'll need to send serialized fields for SpriteBase, but for optimization's sake, will I need to serialize for input data?

Huh..

Edit Ignore my silly question about the byte array format. I've solved half of it I guess. I can send the serialized byte array..

http://www.digitalcoding.com/Code-Snippets/C-Sharp/C-Code-Snippet-Object-to-byte-array.html

but I'm unsure about how to load data from a file that stores the byte array back into the Map to tell it how many sprites are in the Map's SpriteCollection, and then load the sprites' individual parameters and "instances" for lack of a better term, back into SpriteCollection.

I'm going to have to look into LoadGame. If I have any more issues I'll try and post them here

Edit Ignore that one too, Deserialization is extremely simple. (Insanely simple)

Ok, but this all doesn't solve my problem for the SaveFiles. It stores the current map to a hashtable in saveunit, but by storing the map, it's also storing the map's tile data automatically...
I can't be selective about this. I'd probably have to go and find the map that was saved in the SaveUnit, and then find out how MapBase handles it's tile data through individual instances of LayerBase, which I'd have to go into individually and use a method from GeneralRules.MapBaseObject.AllInstancesOfLayerBaseforThisMapBaseInstance to clear their tile data.

Then there's the matter of sending serialized sprite data. I'd have to find some way to tell the client which serialized data belonged to which sprite when the server sends serialized data to the clients, which is another mind-jar within itself.

The one way of doing things is having the server handle XML files alongside the save files for each map. Through each of these XML files I could have tags such as <Players> <Player 01> *Serialized Data </Player01> <Player 02> *Serialized Data </Player 02> etc.

(I never mentioned this, Lidgren.cs and NetController, the two objects that I use for Netgames at the moment, keep an array of players for each map, assigning each of these players's sprites a unique incremental ID from 00 to 99 *It increments every time a new player joins and fills in gaps whenever a player leaves, or that was my recollection of how it worked when I last tested it almost 8 months ago.. *  I can just use these unique values to determine serialization data. I was already using it for input data.)

Then I would have to send the XML file to the client for each positional update and let the client handle deserialization and then the processing of each field from the data by reading the XML file.

Come to think of it, this could work for inputs as well. But instead, I could serialize SpriteBase.inputs and send that as a byte[SerializedDataByte[]] instead of using the XML file where byte[] holds PlayerNumber at index 0 and serialized data at index 1. It would be much faster than having to wait and process an XML file.

Or I could use the above system and just exclude XML files altogether. What do you think?

(It looks like I'm going to be spending more time on optimization again. Just when I thought I was finished. But then again, isn't most time usually spent on optimization?)

[bluemonkmn]

OK, I don't know if I can address everything you're bringing up but some of the basics:
1. To serialize Sprite.inputs (a single scalar value) all you need to to is convert it (cast it) to an integer and then call BitConverter.GetBytes() on it.
2. I estimate a sprite by itself could serialize to about 100 bytes. I estimate you could transmit about 1 Mbps, which is 125,000 bytes per second. Given that the average network latency I see is about 104 milliseconds (1 tenth of a second) I estimate you could send about 5 updates per second, which would amount to 25,000 bytes per update, which means you could send about 250 whole serialized sprites per update (five times per second) if that's all you were sending.
3. However, serializing a sprite is not quite as straightforward as I wish I could tell you it is. I'd like to tell you, just take the map's sprite array, call serialize on it, and send it over the wire and you're done. But each sprite has a reference to the layer it's on and some other objects. Because of the way the .NET serializer works, this will cause it to automatically serialize the layer and other objects linked to the sprite as part of the serialization process. And I know you don't want to include all the tile data from the layer. The next best thing to to create a structure like SaveUnit. SaveUnit is really quite a simplifying structure -- it basically exists solely to bundle together for the serializer all the objects that we want to serialize. So you can just make up any object/class you want (your own version of SaveUnit... maybe call it NetUnit) that stores all the information you want in a friendly format. Put arrays of integers and floating point numbers and hashtables in there. So long as everything you put in there is serializable (which all scalar and array types are) the binary serializer will serialize it very efficiently. So you could just make a hashtable where the key of each element is a sprite name or identifier of some sort and the value is an array of numbers that represent copies of that sprite's important properties. Then you just serialize it, transmit it, decode it and copy those values into the client's copy of the sprites. I wish I could go on, but I don't have the time or energy to write a book right here in this post . I hope this is making sense to you. .NET really can work some miracles for you here if you know how to use it.

[v6v]

OK, I don't know if I can address everything you're bringing up but some of the basics:
1. To serialize Sprite.inputs (a single scalar value) all you need to to is convert it (cast it) to an integer and then call BitConverter.GetBytes() on it.
2. I estimate a sprite by itself could serialize to about 100 bytes. I estimate you could transmit about 1 Mbps, which is 125,000 bytes per second. Given that the average network latency I see is about 104 milliseconds (1 tenth of a second) I estimate you could send about 5 updates per second, which would amount to 25,000 bytes per update, which means you could send about 250 whole serialized sprites per update (five times per second) if that's all you were sending.
3. However, serializing a sprite is not quite as straightforward as I wish I could tell you it is. I'd like to tell you, just take the map's sprite array, call serialize on it, and send it over the wire and you're done. But each sprite has a reference to the layer it's on and some other objects. Because of the way the .NET serializer works, this will cause it to automatically serialize the layer and other objects linked to the sprite as part of the serialization process. And I know you don't want to include all the tile data from the layer. The next best thing to to create a structure like SaveUnit. SaveUnit is really quite a simplifying structure -- it basically exists solely to bundle together for the serializer all the objects that we want to serialize. So you can just make up any object/class you want (your own version of SaveUnit... maybe call it NetUnit) that stores all the information you want in a friendly format. Put arrays of integers and floating point numbers and hashtables in there. So long as everything you put in there is serializable (which all scalar and array types are) the binary serializer will serialize it very efficiently. So you could just make a hashtable where the key of each element is a sprite name or identifier of some sort and the value is an array of numbers that represent copies of that sprite's important properties. Then you just serialize it, transmit it, decode it and copy those values into the client's copy of the sprites. I wish I could go on, but I don't have the time or energy to write a book right here in this post . I hope this is making sense to you. .NET really can work some miracles for you here if you know how to use it.

Right. I see serializaition of inputs to be the more rapid update, sending those each time a new input is pressed. I don't think I'll be sending anything each and every frame, I only send data on state changes (and after thinking heavily: NPC's and enemies have unique timers that they count down on that if it ends, they execute an action such as attacking or walking away and restart). And since sending serialized sprites only occurs every 10 seconds (And with that I'm going to be forced to send the server's current timer values and server's overall IsActive state for each of the enemies and NPC's to sync everything on all of the clients but I can contain both of those values in a single byte per NPC/enemy)

But if I serialize a SaveUnit-ish object containing the sprites rather than serializing each sprite individually, it won't serialize the other "objects" and "attached layers" assosiated with the sprites, right?

Right?

I see a breakthrooooouuuugggghhh

EDIT: After re-reading your post I see what you mean now. Fair Enough. I can simply copy these parameters across, rather than saving the actual SpriteBase objects themselves . Wheeeeeeee

I have approximately everything that I need to complete it now. Wish me luck and thank you for the help.

[v6v]

Working through this now and (destroying my old Lidgren.cs and NetController.cs classes entirely to re-write things in a much more efficient format)

You said I could send 250 whole serialized sprites per second. (Thank God I'm only sending serialized sprite data as a single one frame update every 10 seconds because I'm definitely planning on having more than 250 players + NPC's and entity sprites combined)

With that being said I guess I'll have to send updates every 10 seconds relative to the clients on a certain map, but not at the same time as I'm sending updates to another map. Since I'm not serializing SpriteBase (at 100 bytes an instance), and my new SaveUnit class won't be too large because it only stores important properties, I can make this work.


I'm going to use this thread to sort of just devlog and publish my silent achievements and thoughts. The following comments wont be important unless they're a question (I don't think I'll have any for a while) so happy reading I guess...  

TODO in the far future: (Create ProjectileSprite.cs to handle projectiles)

Might as well document this just so I remember

When the sprite is attempting to use an item it has, or equip armor it has, it sends the request to the server. If the sanity check ran on the server says that this is possible, it uses the item or equips the armor on all instances of that player's sprite for all clients including the server. Like other methods dealing with items, the data sent from the clients and server is reliably sent.

I might be forced to stick with ASCII for the chat messages... It's no big issue at all though.
(its kind of inconsiderate to those who don't chat with english, but maybe I can just have the server decide the language of whatever's going on and the clients determine which font to use according to that. and maybe I'll replace characters not used often with grave and acute accents)

[durnurd]

Do you plan on having 250 sprites on one screen on one server all at once?  Because every client doesnt need to know the location of every sprite, just the near by ones.

Also, it's highly unlikely that you will be able to have hundreds or even dozens of real-time, persistent connections into one server at a time.  Even large scale MMOs try to keep the load on any one server down, and unless you have your own rack of dedicated servers with a high-density connection to the Internet backbone, the server would probably lag immensely or perhaps just melt into a sad puddle of silicon and wires on the floor.

Moral of the story: 250 or more sprites is fine, but not all on one screen simultaneously.

[v6v]

I guess I worded that wrong. I plan on having more than 250 sprites on a server, not all on the screen at once. These sprites aren't going to be users. Some will be user sprites, but others will be other significant sprites such as NPC's enemies, etc. (This is actually one of the things I hated about most 2D ORPG's. They had so many players on the screen that it was impossible to distinguish your own sprite, or even read any chat messages)

I'm starting out with a 32 player connection limit per server. That's the default connection limit for the Skulltag game engine, even Doom. I highly doubt servers are going to reach 32 players before someone attempts to host their own, because even Doom and Skulltag engines begin to lag at around 20 players (Which usually happens in games that allow you to host your own server)

But that's the beauty in allowing users to host their own servers. Instead of having a string of dedicated servers, from the start of developing this game, I knew that this would be impossible for me. I'm hoping that a large group of smaller servers could work much better. Most servers will probably start lagging at around 16-20 players. So having more servers by allowing any player with a port forwarded connection to host and allowing the hosts of those servers to moderate through kicks and bans is probably best. (The only real centralized server I plan on having is a simple MasterServer that shows all other servers that are online)

The clients only receive updates about other clients on the map. If things get worse, I'll look into limiting it to only sprites within certain pixels of other sprites.

Also, it's highly unlikely that you will be able to have hundreds or even dozens of real-time, persistent connections into one server at a time.  Even large scale MMOs try to keep the load on any one server down, and unless you have your own rack of dedicated servers with a high-density connection to the Internet backbone, the server would probably lag immensely or perhaps just melt into a sad puddle of silicon and wires on the floor.

Can you go further with this?

EDIT: I'm not quite understanding how the InputBits works. Can I only get it? Or can I also set it? I want to serialize an integer casted InputBits and then set it on the client's sprite by having the client cast it back to InputBits.

Then again I can't just explicitly cast an integer to InputBits. Agh please, don't let this be an impasse...

EDIT: I asked the previous question wrong. How exactly does the enumeration handle multiple key-presses?
I see that inputs (the enumeration) has a value for each keypress, but what if I'm pressing 2 or more inputs at once? Will explicitly integer casted data for that frame hold both keypresses? If I cast it back with (InputBits)castedData; will it hold both keypresses? And can I set inputs on a SpriteBase instance? Or is the input enum only used for retrieval?

EDIT: I see how it works, I think. SetInputState and IsInputPressed demonstrate it. It looks like my extremely and unnecessarily complicated way of doing it before can be scrapped now.

I'm anxious to see how far I can get with this.

Oh and hey look what I found: *cough*

http://gamedev.enigmadream.com/index.php?topic=1568.0

Guess I should ask questions about things that I won't decide to finish later.

[v6v]

Some art?

I realize that I'll divide things into multiple game modes.

Adventure mode will prove to be the most taxing on the server because that's the mode that handles multiple maps simultanously.

For most of the other game modes, I'll try and contain things on the same map.

So Adventure Mode will be the last mode I'll work on.

For now I'll start on a Lobby Mode. One map that displays all of the users waiting in a Purgatory-Like Map.

With the exception of Adventure Mode, I've decided to make most online game modes constrain all users to the same map.


Thanks for reading!

[Vincent]

Well, thanks for the news!

[v6v]

Thanks Vincent, and I probably won't post anything else news related here unless it's a download. 

In the meantime, I'm working on the lobby.

The system I'm using right now assigns new players as Anons until they change their name in the lobby or in Character Creation.

Anons might only be able to spectate and watch other players.

[v6v]

Just so people don't think that this is dead, I'll post some pretty motivating updates.

There is no download in this post (yet) I apologize for that.

I'm switching to protobuf as my serializer, I've spoken to some other people who have done online gaming in C# and they recommend this.

I've done a huge resources overhaul. Music and multimedia work well.

If anyone knows of a way for me to switch away from SGDK2's usage of OpenGL in Immediate Mode, please share, I'd love to use vertex buffers, because I have a good amount of geometry that needs to be drawn (I've added an option to turn off fancy graphics) I can't say I know anything about OpenGL past GL.Begin(BeginMode.Triangles)

On my system, this means the difference between 45FPS and 18 FPS.

 
Here are some motivating screenshots. I need to get things done.




I have scrolling text made with adjustable text speed, and the item system works: since players can only hold a certain number of items at any given time, the game prioritizes itself on searching for areas to hide and store items. Checking and examining everything leads you to some pretty interesting discoveries, as most storage areas may blend in the background and yield some interesting items.




The Status area GUI is pretty simple on the eyes, and is nostalgic to the GBC era (My era of gaming) . I don't want something that will force the player to have to strain themselves too much looking at the detail in something so unimportant.
Finally, the game starts off from the start of my first game: formerly Eureka Star, now called Child of Streams 1. You're instantly faced with an onslaught of alien enemies and a boss as soon as you start the game.


It's still going, I'll never quit this.

[Vincent]

Very nice!  Keep going!