Project Paradigm

[bluemonkmn]

I just noticed you asked with port, not which protocol. Based on the list at http://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers 24886 seems to be available, picking kind of at random and googling the results.

True, UDP isn't reliable, but remember that TCP is based on UDP, and you can make an un-reliable protocol reliable with confirmation messages, and then you have control over when you add the overhead of confirmation/reliability. You can add them only in places where you *care* about the message getting to all clients or to the server. And you can skip confirmations in those cases that aren't critical like a status update that can easily be superseded by a subsequent update.

Edit: See also http://stackoverflow.com/questions/133879/how-should-one-go-about-choosing-a-default-tcp-ip-port-for-a-new-service.

[v6v]

I just noticed you asked with port, not which protocol. Based on the list at http://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers 24886 seems to be available, picking kind of at random and googling the results.

True, UDP isn't reliable, but remember that TCP is based on UDP, and you can make an un-reliable protocol reliable with confirmation messages, and then you have control over when you add the overhead of confirmation/reliability. You can add them only in places where you *care* about the message getting to all clients or to the server. And you can skip confirmations in those cases that aren't critical like a status update that can easily be superseded by a subsequent update.

Edit: See also http://stackoverflow.com/questions/133879/how-should-one-go-about-choosing-a-default-tcp-ip-port-for-a-new-service.

Thankfully Lidgren supports confirmation messages and I can determine what data needs to be sent reliably and in order, so I won't have to get my hands dirty.. I think I'll use Lidgren.Network, as Michael Lidgren said on google groups, if it's a one time only thing, you can simply just send it as a Message (Lidgren's Unit of Transferred Object Management) Right now I'm not using reliable transmission for input data, but I have a method called every now and again from the server that sends the accurate positional data, that's sent reliably.

Sav files aren't too large either. At least I'm not trying to send 50MB so things should be fine.

Anything to prevent game breaking lag.

[v6v]

Get ready for a long post, and if you read the entire thing I will reward you with whatever you want that's game related, everyone

It turns out Lidgren has a special class for just this, that actually downloads a file to the current working application directory. I tested it on my local machine (localhost) with one of the LoK Savefiles that was about 367KB.

1.4MB per second. It took 2 seconds. That might sound like nothing as seconds, but with networking I'm measuring everything in ms and that's around 2000ms.

I'm not pleased with that. (You'll all see why later when multiple players are on a server)

2 entire seconds on localhost for a 367KB file isn't good. I'm not sure if there's a mathematical rule for this, but anything which takes a certain amount of time through localhost will definetely take longer through networking outside of the local machine, especially outside of the local network.

It could even take 5 seconds.. *shudders*

The one thing I've been neglecting to account for is that the host will have to process all rules on multiple maps, which is another reason for me to decide against using more maps. (Of course I won't expect the host to Draw everything from those maps)
I already asked a question about this on the forums and it was answered a while back.

As I've said before multiple times on other threads on the forum, I'm saving each map with it's own save file.

I'm going to be forced to save the map and quickly freeze the game when a new player is downloading a save file, to keep the games consistent. Since Lidgren runs in it's own seperate non-gamebreaking thread, I'm hoping I can just use Thread.Sleep() on SGDK2 (Unless SGDK2 already has a method that freezes game logic on a certain map (Or even a method that processes game logic on a certain map that I can turn off for just one second). That would be even better because I could only use it to halt whatever map's save file was being downloaded and allow the other maps to continue unaffected.) until the files are downloaded from the server to the client. Then I can handle things from there.

Everything below only has relevance if I'm not a complete idiot who didn't realize that SGDK2 has a way to halt game logic for a certain map and leave other maps unaffected

The thing is though, if the server is calling Thread.Sleep to prevent current players from changing anything while a new player is downloading the savefile, I would be forced to halt the entire game, even for players that aren't on that map, because the server would be busy trying to make sure that the other clients "remain quiet, still, and well behaved *Frozen Thread*" while the new player is getting their information.

This is why I fear the 2 seconds that it takes to download the data. That means 2 seconds of stop-frame. 2 seconds of a halted game. Eventually it will aggravate players when new players are constantly switching maps or joining. Doom is famous for this (It freezes the game temporarily for new players to join and download the .Sav file to make sure that everything is consistent. It's even more noticeable if you're not using LAN)

(Of course, I just got the very clever idea of making it harder to switch maps by making longer, more difficult to traverse maps. I could use a value that waits a certain amount of frames before letting the player go to another map. It could also double as temp invulnerability flash that the old games gave you whenever you started a level or got hurt from an enemy. )

I was already planning on keeping my maps small and having multiple maps anyways, so there's no reason for a my game's savefiles to be large unless they're laden with sprites and players. (Well now I have to reconsider the amount of sprites on a map. Maybe I can exclude non-important sprites from the savefile.)

I'm getting ahead of myself.

But against what I said above about the size of the maps I'm changing my mind.
I'd rather have slightly longer download times and slightly longer game halts with fewer actual game halts than more frequent shorter game halts and shorter download times.

Ergo

I'm going to just create lesser, larger maps. Before I kept running into OutOfMemoryExceptions, but I'll try not to make the maps 14MB. anything smaller than 1MB will suffice.
This is also more efficient because the poor host is having to process all of the map's plans and sprites on that map's plans at the same time and then sending that data to the clients. Mapsize won't have an effect on this though because I'll only have the host use drawing operations on the map that the player who's hosting is playing. But this will cause the game to have less maps to be processed. That's a good thing. (Really it only sends save files and occasional reliable (checked to make sure that they were recieved correctly) coordinate updates for players. It relays unchecked input data from clients and sends it to all other clients.)

Tell me, if I extend the size of my maps to about 500 by 500 tiles for 5 layers each, which would bring the resources file to about 1 or 2 MB, would this necessarily cause a sharp increase on the size of a savefile?

[bluemonkmn]

1. By simply skipping the "CurrentMap.ExecuteRules" step of the main Run function in GameForm.cs you can effectively pause all processing on the current map (you may also want to skip OverlayMap.ExecuteRules). On the server side I assume you will have some custom code that is calling ExecuteRules on all active maps and not drawing anything. Of course there you would simply stop calling ExecuteRules on the paused maps.

2. I think there is a *lot* of room for optimization in what you are transferring when downloading a save.  If you have 500x500 tiles and 5 layers, and 2 bytes per tile (that's up to 65536 tiles, which I can't imagine you'd need to exceed) that's 2.5 million bytes.
a. But you probably don't need 2 bytes per tile on all 5 layers. Aren't there some layers that have a limited variety of tiles? You could drop those to 1 byte per tile.
b. And you may not need each of those layers to fill the entire map. A background layer, that scrolls more slowly, for example, could be 100x40 tiles if it scrolls more slowly and mainly only horizontally.
c. If some layers only need to have data for part of the map you can use the offset properties of the layer to set where it is and it won't have to store tile data for the whole area.
d. If the layer is repetitive, you can make the virtual size 500x500, which determines how large the layer appears, and the actual size much smaller. This will cause the data to loop. For example if I have a background of just blue sky, my actual size will be 1x1 tile (1 byte) and the virtual size can be as big as I want. Sprites don't loop like the tile data does, so you could even put some sprites on this layer to make it vary if you only have a few features to put on a large layer.
e. Layer data may be highly compressible in some instances. It might be worthwhile running some simple compression algorithms on it before transmitting. I think .NET has some built-in compression streams.
f. It's highly likely that some of your layers are not changing that much. Aren't there some layers that don't change? You wouldn't need to send those at all since all clients would have the data for those already, knowing that they never change.
g. The parts of the layers that change are much smaller than the whole layers. Maybe you could write your own compression algorithm (or I could help) that only collects the tiles that are different than the initial state of the layers and sends just those tiles. Sending that information would be much smaller than sending the whole layer.
h. You might not need to send the whole layer in one frame. Maybe you could sync just the visible part of each layer. Then proceed to download the rest in subsequent frames, always making sure that at least the visible part of the layers are synced first.

I suspect you could eliminate 95%+ of what you need to transfer by following those optimizations. In fact if you made those last two optimizations, you could probably let the server handle all the rule processing and have the clients just download and draw the results on every frame, thereby eliminating the chance of cheats due to altered clients. Or to make the clients more responsive, you could do what I think I see in other games: allow them to process the rules and draw the results while they wait for the real updated data from the server, which will provide the actual state of the game and replace whatever happened on the client since the last update. But then you'd need to do one of 3 things: 1) Make sure the client's rule processing doesn't alter things that aren't being replaced by the server update and make sure the server update sends everything that the client might have altered; 2) Keep 2 copies of data on the client, one for local operations and one for server updates, and whenever a server update arrives, copy all the data from the server copy (after applying the update) to the client copy; this would effectively "undo" everything the client has done while waiting for the server update; 3) Not care about the things that might get out of sync between the client and the server because all the substantial things that matter are being processed on the server and the client is just drawing what it sends anyway. All client's games are essentially being played on the server and anything out of sync would just be a "drawing glitch", and might even be temporary.

[v6v]

1. By simply skipping the "CurrentMap.ExecuteRules" step of the main Run function in GameForm.cs you can effectively pause all processing on the current map (you may also want to skip OverlayMap.ExecuteRules). On the server side I assume you will have some custom code that is calling ExecuteRules on all active maps and not drawing anything. Of course there you would simply stop calling ExecuteRules on the paused maps.

2. I think there is a *lot* of room for optimization in what you are transferring when downloading a save.  If you have 500x500 tiles and 5 layers, and 2 bytes per tile (that's up to 65536 tiles, which I can't imagine you'd need to exceed) that's 2.5 million bytes.
a. But you probably don't need 2 bytes per tile on all 5 layers. Aren't there some layers that have a limited variety of tiles? You could drop those to 1 byte per tile.
b. And you may not need each of those layers to fill the entire map. A background layer, that scrolls more slowly, for example, could be 100x40 tiles if it scrolls more slowly and mainly only horizontally.
c. If some layers only need to have data for part of the map you can use the offset properties of the layer to set where it is and it won't have to store tile data for the whole area.
d. If the layer is repetitive, you can make the virtual size 500x500, which determines how large the layer appears, and the actual size much smaller. This will cause the data to loop. For example if I have a background of just blue sky, my actual size will be 1x1 tile (1 byte) and the virtual size can be as big as I want. Sprites don't loop like the tile data does, so you could even put some sprites on this layer to make it vary if you only have a few features to put on a large layer.
e. Layer data may be highly compressible in some instances. It might be worthwhile running some simple compression algorithms on it before transmitting. I think .NET has some built-in compression streams.
f. It's highly likely that some of your layers are not changing that much. Aren't there some layers that don't change? You wouldn't need to send those at all since all clients would have the data for those already, knowing that they never change.
g. The parts of the layers that change are much smaller than the whole layers. Maybe you could write your own compression algorithm (or I could help) that only collects the tiles that are different than the initial state of the layers and sends just those tiles. Sending that information would be much smaller than sending the whole layer.
h. You might not need to send the whole layer in one frame. Maybe you could sync just the visible part of each layer. Then proceed to download the rest in subsequent frames, always making sure that at least the visible part of the layers are synced first.

I suspect you could eliminate 95%+ of what you need to transfer by following those optimizations. In fact if you made those last two optimizations, you could probably let the server handle all the rule processing and have the clients just download and draw the results on every frame, thereby eliminating the chance of cheats due to altered clients. Or to make the clients more responsive, you could do what I think I see in other games: allow them to process the rules and draw the results while they wait for the real updated data from the server, which will provide the actual state of the game and replace whatever happened on the client since the last update. But then you'd need to do one of 3 things: 1) Make sure the client's rule processing doesn't alter things that aren't being replaced by the server update and make sure the server update sends everything that the client might have altered; 2) Keep 2 copies of data on the client, one for local operations and one for server updates, and whenever a server update arrives, copy all the data from the server copy (after applying the update) to the client copy; this would effectively "undo" everything the client has done while waiting for the server update; 3) Not care about the things that might get out of sync between the client and the server because all the substantial things that matter are being processed on the server and the client is just drawing what it sends anyway. All client's games are essentially being played on the server and anything out of sync would just be a "drawing glitch", and might even be temporary.

Come to think of it, I'm not making a minecraft game (For the most part, but hey lets see where this game takes us), so the tiles on my layer are going to be static. Most of the obstacles (Locked Doors, Unpassable Walls) are going to be sprite based. This eliminates the need for me to even send tile data. Since this is true, the only thing that savegames need to store is everything BUT tile data. Thank god I'm not using 2 Bytes per tyle (I found a much better way. I have 3 tilesets. One is the normal tileset, one is the normal tileset flipped tileset across the Y axis, and the other is a tileset that's invisible that handles solidity. The sprite is on the invisible tileset. The other two are drawn in front of it. Each layer uses 1 byte per tile. I figured at 2 bytes per tile, I was using more that I even needed. My math here was that a 256 tile layer with 2 bytes per tile equaled 256 ^ 256 tiles. 3 tiles with 1 byte per tile equal 256*3 bytes. Significantly less than what 2 bytes per tile offered, but just enough for me to work with.)

I'm not sure how the current savegame system works, but I think I can just exclude tile data by changing a line or two somewhere.

There's 3 crucial layers that are important in my game, as stated above. I think what happens is that I just need to send positional data, as you've said, then since the rest of the layer doesn't change, I can just use the Map and Layer Data on the client to render the game and ignore that (Hey maybe that might reduce my sav files to 10KB an Ideal size). There's still a couple more values I need to send though.

I've given mapbase and layerbase a few extra properties. These properties are important to the game (They handle weather, and what texture is needing to be drawn to the skydome to represent day and night) I think in my version of SaveFiles, I can allow it to save the properties on MapBase and LayerBase, exclude the tile data for every one of the Layerbases and go from there. So my new version of Saves only have Map and Layer properties including sprite data, without the tile data.

That map I mentioned earlier, the one that only holds sprites that have the User and Password data, I think it will be an initial save only sort of thing. You download it alongside the starting map when you join the game, and after that, since the Host knows that you've logged in, I can just associate that username with an IP, and even if a malicious client attempts to change the value of the username at runtime (Which is nearly impossible thanks to DotPeek but anyways..) the server keeps a memory of the actual username associated with the IP. So when the IP of the client requests to change a map, the server looks at it's own information and not the information of the client before creating a new sprite on that map with the given properties.

Really and truly the server is running the game logic. The Clients are running their own game logic so things don't seem choppy while the server is updating all clients. But this game logic is for them and them only. Even if they change something on their end, ultimately, it's the server that handles things. I have a method that updates everything with the right coordinates and parameters every 10 seconds (Sounds reasonable, no?)

I'm thinking that NPC's and enemies should use a different system (I'm getting ahead of myself again but this needs to be planned for)
For NPC's I wanted to have a property called Action, which is an integer that usually ranges from 1-5 or 1-10. The server calls GetRandomNumber and stores it to that instance of the NPC's Action Integer. When the Integer is recieved, the NPC executes an action (Move Left) (Move Right) (Attack Left) and as soon as the server gets it, it's all a matter of sending the value to the client. Since this is a one integer value that updates every 3-7 seconds, I'm hoping that this won't be too halting.

So apart from client side things such as the processing of random numbers to predict the position of raindrops and snowflakes, I'm going to have to generate all random numbers in the server and send it to the client.


Again, I just repeated what you said  

I was thinking of a more efficient way though. I could have a RandomNumberTable (We used these in Statistics class in High School, it's essentially just a 2D array with randomly generated numbers) which was one giant array of integers randomly generated. I was thinking of having 10 embedded spreadsheets or comma seperated databases of these, as a start. When the client joins the server, it downloads the value that tells it which spreadsheet the server is currently (The client and server both have the same spreadsheets locally embedded) using of random numbers and the server sends the client the positional data of the spreadsheet/array (AKA array

• where x is the value being received) Then random number generation is done on the server by just moving to the next value, and done on the client  by just moving to the next value, separately from each other, so the client isn't waiting for the server's random number, and the method that I mentioned that executes every 10 seconds performs the checks required to see if the client has everything the server has.

I might have to call a method that kicks or bans if the spreadsheets aren't alike. But with the DotPeek obfuscator (I'm not putting all of my trust in it) this prevents anyone from decompiling and changing the data of the spreadsheet before recompiling.

Maybe I can even check to see if something in the server doesn't match the client, and kick or ban automatically. If the usernames no longer match, that's a good reason to ban, right? The one thing I don't want to end up doing is what DOOM does, it kicks players that are inconsistent with the netgame through it's Consistency Kicks. Consistency can be fixed without kicks from what I'm seeing.

I think I just repeated the last part of your message right there. It's only logical NOT to trust the client.

I noticed that you said you could help with compression, it should be even easier now that I'm excluding tile data from my save files (One I figure our how to do this). That would be wonderful. The only thing though is I'm afraid how long the decompressing process will take in the game.

Which method do you prefer of random number generation?

Come to think of it, if tile data isn't being saved, no longer will I be forced to optimize (Or even care about the filesize of) my maps. The only real effect it will have is on the size of the Executable (I created a map with one layer of 5000 by 5000 tiles with one byte per tile on the SGDK2 sample project. The map resources file was 14.2MB. It's amazing what effect maps can have on the filesize)

I suspect you could eliminate 95%+ of what you need to transfer by following those optimizations. In fact if you made those last two optimizations, you could probably let the server handle all the rule processing and have the clients just download and draw the results on every frame, thereby eliminating the chance of cheats due to altered clients.

Cloud gaming? In that case the clients could just render a video file of what was going on on the server. But it would force the clients to wait for the server.... Ack.
  _{plus I'd have to completely rewrite my current networking architecture}

EDIT: Depending on what response I get to this message, I'm going to try and implement this in the duration of the next 4 days. Professors are bombarding me with tests this week.

[bluemonkmn]

1. For random numbers, you don't need to use a table. Computerized random number generators can use a "seed". If both the server and client use the same random seed, then they will follow the same series of random numbers:http://en.wikipedia.org/wiki/Random_seed.
2. I think if you have managed to eliminate the need to communicate all the tile data, you probably won't need my help on compression.
3. I'm surprised the resources file was only 14 MB with a 5000x5000 tile map. 5000x5000 = 25,000,000. You'd think you'd need to store 25 million bytes at least. I don't find it surprising, but I do agree that it's amazing or remarkable. I don't know if you realize how big 5000x5000 is. If you worked 24 hours a day and filled in an average of 1 tile per second, it would take you 9 months to fill that size map. If you worked 8 hours a day, it'd take you more than 2 years. And if you were playing the game and could explore 1000 tiles per second, it would take you 7 hours just to look at all the tiles. So you can see that 5000x5000 tiles is probably big enough for a whole game, not just one map (In RPG games I'm sure I look at far less than 1000 tiles per second). 2-dimensional expansion is a fast-growing "function". That's why I like scrolling games. You can have such a huge detailed world in such a simple space.
4. Of course if you were to "render a video from the server" I wouldn't transfer it as traditional video, which would be terribly inefficient. I would just transfer it as a set of visible tiles/frames, which would be much smaller. But I don't think you need to do that now that you have found a way to eliminate so much data from your communication needs.

[v6v]

When I was starting to use SGDK2 I honestly had no idea that 5000 by 5000 was that large. I actually disregarded filesize and made maps with so much extra space that I didn't even need.
I think this is the first game where I've calmed a bit towards using high values.

I haven't even touched the code yet. So I'm actually relieved that you haven't said that creating saves that excluded tile data isn't impossible.

1. For random numbers, you don't need to use a table. Computerized random number generators can use a "seed". If both the server and client use the same random seed, then they will follow the same series of random numbers:http://en.wikipedia.org/wiki/Random_seed.
2. I think if you have managed to eliminate the need to communicate all the tile data, you probably won't need my help on compression.
3. I'm surprised the resources file was only 14 MB with a 5000x5000 tile map. 5000x5000 = 25,000,000. You'd think you'd need to store 25 million bytes at least. I don't find it surprising, but I do agree that it's amazing or remarkable. I don't know if you realize how big 5000x5000 is. If you worked 24 hours a day and filled in an average of 1 tile per second, it would take you 9 months to fill that size map. If you worked 8 hours a day, it'd take you more than 2 years. And if you were playing the game and could explore 1000 tiles per second, it would take you 7 hours just to look at all the tiles. So you can see that 5000x5000 tiles is probably big enough for a whole game, not just one map (In RPG games I'm sure I look at far less than 1000 tiles per second). 2-dimensional expansion is a fast-growing "function". That's why I like scrolling games. You can have such a huge detailed world in such a simple space.
4. Of course if you were to "render a video from the server" I wouldn't transfer it as traditional video, which would be terribly inefficient. I would just transfer it as a set of visible tiles/frames, which would be much smaller. But I don't think you need to do that now that you have found a way to eliminate so much data from your communication needs.

Using seeds is nice, but lets say that the server is using a seed, and has been using said seed for almost 10 minutes now, and a client joins and downloads the seed and begins using it. Although the client and server share the same seed for *Random(seed);* , the server would be at a different place in the series than a client beginning to generate numbers in the same series, right?

This is what I was afraid of when using seeds. Even if the numbers are in the same series, there isn't a way to tell the client where in the series to start generating numbers from when it joins. (Wait... is there?)

[v6v]

Judging from this thread,

http://gamedev.enigmadream.com/index.php?topic=1564.0

I'll need to call GetMap() on every map to load every(Networking) map into memory (Not all of my maps are networking maps. Some are local maps. (Save Rooms, Shops) for the server and then call ExecuteRules on all loaded Maps (While exempting the Host from Drawing Operations and other Operations such as FmodBase, etc). What concerns me is, is it inefficient to load every map into the memory while having to load and keep up with the TileData (for lack of a better term) for every loaded map? I don't need to update and check for updates to Tile data but I'll need to Execute Tile Based functions called by Sprites and Plans (TouchTiles, If isOnTile, etc)

What I'm really asking is I'm going to need tile data, but is there any way to just load Tile Data once, and keep it loaded without checking for Updated or Changed Tiles each iteration?

I was wondering if I could exclude the part of MapBase.ExecuteRules that checks for updated tile data each frame (Or even updates it). I want to optimize it to only focus on Sprites and Plans as well as Counters.

[durnurd]

You can make reliable connections on UDP the same way TCP does, using ACKs and packet numbers to ensure full packet delivery in the correct order.  The difference is that you can make it more efficient than TCP because TCP must ipso facto be generalized for any purpose and therefor is inefficient when you know more specifically what you're sending.

There are plenty of things you can do with this in mind.  For example:

If you can get away with it, only sending deltas can additionally speed up data transfers considerably.  By that I mean determine only what changed from the previous transmission to this one and send only that data.  For example, the player moves horizontally X pixels.  This is all the remote client needs to know.  Other data, such as Y position, are not necessary

You can keep track of remote data loss and account for that by sending bigger deltas instead of re-sending dropped packets.  For example, the player moves X pixels horizontally.  It then moves Y pixels vertically for the next transmission, and finally Z pixels vertically for the next.  The naive approach sends each packet (X,Y,Z) sequentially.  If any packet is not acknowledged, it, and each packet after, are re-sent.  Rather than simply sending the three packets with deltas X, Y, and Z, hoping each arrives, you could send each packet as (X), (X,Y), (X,Y+Z), accumulating deltas until you receive acknowledgement of receipt of any of those packets, then reset the delta from there.  For example, if you receive acknowledgement of receipt of the packet containing the data X, then your next packet only needs to send (Y+Z) instead of (X,Y+Z).  With this approach, you definitely need to keep track of packet numbers.  That would be how you specify what the delta refers to.

In our previous example, packet 1 would send X, referring to a base NULL packet.  Without receiving acknowledgement of receipt of packet 1, packet 2 would send (X,Y), referring to the same base NULL packet.  Then you receive acknowledgement of packet 1.  Then packet 3 would send (Y+Z), referring to a base packet 1 instead of NULL.

You can use interpolation on the remote client.  For example, we know the player's previous velocity.  Assume they keep going on that trajectory until we are told otherwise.  Then update with the actual position based on the data received.  You would also need to make sure your interpolation doesn't get skewed by a quick change of position due to an updated actual position.

[bluemonkmn]

I haven't even touched the code yet. So I'm actually relieved that you haven't said that creating saves that excluded tile data isn't impossible.

It's not directly possible, but if you look at the save game function, you will see it's relatively simple. It just tells the other objects to convert themselves into savable content (by "serializing" then). So you can serialize whatever objects you want into bytes and save them or send them directly over the network or whatever. Should be relatively easy.

Using seeds is nice, but lets say that the server is using a seed, and has been using said seed for almost 10 minutes now, and a client joins and downloads the seed and begins using it. Although the client and server share the same seed for *Random(seed);* , the server would be at a different place in the series than a client beginning to generate numbers in the same series, right?

This is what I was afraid of when using seeds. Even if the numbers are in the same series, there isn't a way to tell the client where in the series to start generating numbers from when it joins. (Wait... is there?)

Hmm... Interesting thought. *Experimenting...* Okay, if you take an existing random number generator and create another using the next random number as a seed, that doesn't synchronize the random number generators. Peeking into the internals of the random number generator, it appears it actually has 50 integers worth of seeds, which would all have to be synchronized. I think this helps to ensure that if you get the same integer generated twice, you don't end up in an endless loop going through the same series of integers. (If the current Integer result represented the whole seed then every time you arrived at that number, you'd be able to predict the same sequence coming up again. Not very random.). But I just verified there *is* a way to synchronize random number generators by serializing them and de-serializing them. I can show you the code if you like. The serialized random number generator is only 512 bytes, so that's all you'd need to transmit to synchronize the client's generator with the server's. Of course an easier option might be to re-seed all the random number generators, *including* the one on the server with a new value when someone joins. This would also synchronize them. You could just use the next random number from the server as the new seed for all the generators.

Ah, I might as well put the other solution's code in here so I don't lose it:

Code: [Select]

Random generator = new Random(1234567);
for (int i = 0; i < 10; i++)
   Console.WriteLine(generator.Next());
Random generator2;

System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bf = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
System.IO.MemoryStream ms = new System.IO.MemoryStream();
bf.Serialize(ms, generator);

byte[] buf = ms.GetBuffer();
Console.WriteLine("The serialized random number generator is {0} bytes.", buf.Length);

ms.Dispose();
ms = new System.IO.MemoryStream(buf);
generator2 = (Random)bf.Deserialize(ms);

for (int i = 0; i < 10; i++)
{
   Console.WriteLine(generator.Next());
   Console.WriteLine(generator2.Next());
}


Judging from this thread,

http://gamedev.enigmadream.com/index.php?topic=1564.0

I'll need to call GetMap() on every map to load every(Networking) map into memory (Not all of my maps are networking maps. Some are local maps. (Save Rooms, Shops) for the server and then call ExecuteRules on all loaded Maps (While exempting the Host from Drawing Operations and other Operations such as FmodBase, etc). What concerns me is, is it inefficient to load every map into the memory while having to load and keep up with the TileData (for lack of a better term) for every loaded map? I don't need to update and check for updates to Tile data but I'll need to Execute Tile Based functions called by Sprites and Plans (TouchTiles, If isOnTile, etc)

What I'm really asking is I'm going to need tile data, but is there any way to just load Tile Data once, and keep it loaded without checking for Updated or Changed Tiles each iteration?

I was wondering if I could exclude the part of MapBase.ExecuteRules that checks for updated tile data each frame (Or even updates it). I want to optimize it to only focus on Sprites and Plans as well as Counters.

I'm not quite clear on what you're referring to about checking for updated tile data. SGDK2 code doesn't check for updated tile data during ExecuteRules, and you said your maps don't change, so why would you be adding code to check for updated tile data? SGDK2 framework only looks at tile data when it needs to draw the tile or when a sprite is interacting with the tile... or when it's loading or saving the tile to a file. That's all I can think of.

[bluemonkmn]

If you can get away with it, only sending deltas can additionally speed up data transfers considerably.  By that I mean determine only what changed from the previous transmission to this one and send only that data.  For example, the player moves horizontally X pixels.  This is all the remote client needs to know.  Other data, such as Y position, are not necessary

I suppose you're just using an example here, but it seems to me that it would be a lot simpler to just send all the properties for each sprite that has moved since the last acknowledged packet (I hope data for each sprite isn't *that* big). Then you don't have to track deltas at such a meticulous level or message numbers, you just have to reset a "dirty" flag for each sprite when an acknowledgement is received, right? Furthermore if off-screen sprites aren't having significant effects on the game, maybe you only worry about those that are visible. Or following from the previous idea, maybe you make sure that off-screen sprites don't move/change as often so they don't get "dirtied".

[durnurd]

My example was contrived, yes.  Your expansion on the idea is a better example: Only send data for sprites that moved; in the case of simple x,y coordinates, it's okay to send the complete state every time it changes.  But if you're relying on specific animation sequences or hit boxes, or more data than just x,y coordinates, you will want to send deltas only, accumulating the data you need to send until packets are acknowledged.  If the lag is considerable, it may make sense to eventually just refresh everything, fetching the world state from the server (or "authoritative client") and resetting everything from there.

[v6v]

I'm not expecting servers to be large anyways, so I'll only send inputs for sprites that share a map. (I'm starting with a 32 player max per server, and I don't think 32 people will share a map at any given time)

Since ExecuteRules doesn't do anything with tiles as you've said, I guess I don't have much to worry about.

EDIT: Durund posted as soon as I did :-/

@Durund I'm sending input data only, I'm most definetely not sending x and y coordinates (Could you imagine? :-/ Not only would things be choppy, but someone could EASILY manipulate the data..)

There's a method that the server calls every few seconds that sends the updated map data (The actual coordinates and properties of all sprites on the layer)

The inputs sent regulate animation sequences and whatnot.

EDIT: Whoa, Durund posted another message earlier that I didn't read. *Reads*

You can use interpolation on the remote client.  For example, we know the player's previous velocity.  Assume they keep going on that trajectory until we are told otherwise.  Then update with the actual position based on the data received.  You would also need to make sure your interpolation doesn't get skewed by a quick change of position due to an updated actual position.

Well, thank God that I don't have to worry about the bare technical logic behind this. Lidgren.Network shields me from that. I have many modes of how I send data and input data is sent unreliably. However, the actual updates, as well as the download of the map is reliable and is checked to make sure that each packet arrived to it's destination. But I will have to worry about that last thing you said- but it shouldn't be much of an issue at all (The distances between actual positions and client believed positions shouldn't be game breaking *But they will be slight*)

If push comes to shove I'll be forced to look into interpolation..

That's what the testing is for, right? I can't wait until I start!

EDIT: After reading through your comment bluemonkmn about random numbers, I'm liking the final option the most. Reseeding would be the most efficient way to go about things. Whenever someone joins, everyone gets a new seed and that's transmitted. You agree, right?

By the way, Serialization is something I should read into. I didn't know it was so easy to send things across a network. For the longest of time I just hoped that Kaleta.Network would recieve an update that would make it compatible with Lidgren Generation 3. That would have simplified nearly everything.

Edit: (Again) Everything below isn't important at all. (Why am I posting it?) But everything below is related to the game but off topic to the more important discussion

Off topic, but I haven't posted any screenshots here recently so I'll post some showing the registration interface. I'd like to show many of the bizarre combinations that exist so far of the human physiognomy ;P

(This part is actually done ^^)

















At the current moment, hair and skin color are selectable though modifying the RGB value directly, similar to Terraria. Should I keep this system? Or should I limit hair and skin colors to a selection based selection so that only natural hair and skin colors are selectable?

I've been doing some graphical overhauls as well. I'm trying to achieve the graphical style Daft Punk uses in their Anime themed music videos while preserving graphical quality noticed on the Sega Genesis/Megadrive. Trying and failing at least. The composer who's helping me with my game is an orchestral composer... perhaps I can just ask him to use the MegaDrive soundfont for the music he's composing for my OST ( omg I actually have someone making me an OST already?!)

To any artists on this forum tired of hearing about network mumbo jumbo and want to give me input in an area where you can help me:

I realized that Daft Punk has blue aliens the EXACT hue of the Aliens in my game. (AND extremely SIMILAR costumes) notice the jeweled headband... This was purely coincidental.



Hm. I'll have to recolor then. What's a good alien color then...? Magenta? ForestGreen?
If there aren't any better colors and you all truly like blue, I can just tribute this game to one of the greatest genre defying musical groups of France..

The shojo anime style Daft Punk uses focuses on elongated heads and proportionate faces. The blue lady's face in the background doesn't follow this, because that graphic was made before I performed the Daft Punk Graphical Overhaul.

I don't want to let this distract myself from the topic at hand, gameplay and decent internet values. As other indies tell me, focusing on graphics will force you to abandon your project eventually (I can attest to this) so the graphics are purely motivational. I'm going to eventually do a complete overhaul, hire another pixel artist, or create a system that allows the importing of graphic files so users can at least pretty things up. I'm proficient at pixel art at least. This game should serve as a nice fit in my portfolio for the future.

[bluemonkmn]

I'm not expecting servers to be large anyways, so I'll only send inputs for sprites that share a map. (I'm starting with a 32 player max per server, and I don't think 32 people will share a map at any given time)
...
@Durund I'm sending input data only, I'm most definetely not sending x and y coordinates (Could you imagine? :-/ Not only would things be choppy, but someone could EASILY manipulate the data..)
...
Well, thank God that I don't have to worry about the bare technical logic behind this. Lidgren.Network shields me from that. I have many modes of how I send data and input data is sent unreliably. However, the actual updates, as well as the download of the map is reliable and is checked to make sure that each packet arrived to it's destination. But I will have to worry about that last thing you said- but it shouldn't be much of an issue at all (The distances between actual positions and client believed positions shouldn't be game breaking *But they will be slight*)

I think you and durnurd are a bit out of sync in what you're talking about. I think durnurd is talking about what the server sends to the client and you're talking about what the client sends to the server. Naturally you wouldn't want to be sending sprite positions from the client to the server because that's too easily hacked. So yes, you only send the inputs from client to server. But the server has to process everything and send information back about the results of those inputs. This is where you send sprite positions and such, and try to optimize what you are sending. I don't see why this has to be any more reliable than what the client sent to the server. If the client doesn't get the update, the server can send it in a later update. It's not like the server is going to forget where the sprites are. And this is where allowing un-reliability is important for optimization because the server has so much to send.

I realized that Daft Punk has blue aliens the EXACT hue of the Aliens in my game. (AND extremely SIMILAR costumes) notice the jeweled headband... This was purely coincidental.
...
Hm. I'll have to recolor then. What's a good alien color then...? Magenta? ForestGreen?
If there aren't any better colors and you all truly like blue, I can just tribute this game to one of the greatest genre defying musical groups of France..

Would it be practical to add texture or something? Like make the aliens metallic or irridescent or furry :alien:? Or use a combination of colors like fractals or swirls or ... can you imagine an alien with skin like this?


:surprise:

(I wonder what else they have at http://garmahis.com/reviews/top-10-free-stock-textures-resources-for-designer/)

[v6v]

Sorry for another long post...

I'm not expecting servers to be large anyways, so I'll only send inputs for sprites that share a map. (I'm starting with a 32 player max per server, and I don't think 32 people will share a map at any given time)
...
@Durund I'm sending input data only, I'm most definetely not sending x and y coordinates (Could you imagine? :-/ Not only would things be choppy, but someone could EASILY manipulate the data..)
...
Well, thank God that I don't have to worry about the bare technical logic behind this. Lidgren.Network shields me from that. I have many modes of how I send data and input data is sent unreliably. However, the actual updates, as well as the download of the map is reliable and is checked to make sure that each packet arrived to it's destination. But I will have to worry about that last thing you said- but it shouldn't be much of an issue at all (The distances between actual positions and client believed positions shouldn't be game breaking *But they will be slight*)

I think you and durnurd are a bit out of sync in what you're talking about. I think durnurd is talking about what the server sends to the client and you're talking about what the client sends to the server. Naturally you wouldn't want to be sending sprite positions from the client to the server because that's too easily hacked. So yes, you only send the inputs from client to server. But the server has to process everything and send information back about the results of those inputs. This is where you send sprite positions and such, and try to optimize what you are sending. I don't see why this has to be any more reliable than what the client sent to the server. If the client doesn't get the update, the server can send it in a later update. It's not like the server is going to forget where the sprites are. And this is where allowing un-reliability is important for optimization because the server has so much to send.

I realized that Daft Punk has blue aliens the EXACT hue of the Aliens in my game. (AND extremely SIMILAR costumes) notice the jeweled headband... This was purely coincidental.
...
Hm. I'll have to recolor then. What's a good alien color then...? Magenta? ForestGreen?
If there aren't any better colors and you all truly like blue, I can just tribute this game to one of the greatest genre defying musical groups of France..

Would it be practical to add texture or something? Like make the aliens metallic or irridescent or furry :alien:? Or use a combination of colors like fractals or swirls or ... can you imagine an alien with skin like this?


:surprise:

(I wonder what else they have at http://garmahis.com/reviews/top-10-free-stock-textures-resources-for-designer/)

I see your point. I could have both input and updates sent unreliably, updates are important, but as long as the server ultimately holds the information, it's alright. The problem that I had with that was, what if the update was a huge gap from where the sprite was? (from what the client believed) which is where durund was going.

@durund - I've been sending data sequentially  and I'm guessing that's a bad thing.. (I've tested networking with Lidgren once and it worked extremely well, but the person who helped me isn't contactable right now)

The system I use works with deltas. It only sends data when an input is pressed differently from the last frame (Look at me thinking about optimization when I just started working on the game!   I guess it was because I was tired of playing laggy games already) It sends a binary value. "0000000000" The first two digits correspond to the player number, say, player 1 is 01, player 2 is 02, etc.
The next 4 digits correspond to input data (Up, Down, Left, Right)
The 4 after that correspond to input button data (Button1, Button2, Button3, Button4)

0 Represents that a button is not pressed
1 Represents that a button IS pressed

For the longest of time I was afraid to change that, because I thought that if I did, it would slow down things to a gamebreaking level. Thankfully to the CS courses I'm taking, I'm learning about different kinds of numbers, more specifically, Base 16.

I'm eventually going to convert input data to Base16 so I can add more input buttons later on. 4 isn't proving to be enough buttons..

But I'm getting off topic

It sends the delta each time a new input is pressed, or if the inputs pressed on the last frame AREN'T pressed on this frame. Sequentially this wouldn't be smart, because if something about another client isn't recieved by your client from the server, your client will think that another client is either standing still, not attacking, or, still pressing a button.

Then this is game breaking because say the remote client was attacking an enemy and had defeated the enemy, but your client didn't get the data telling you about the attack.. Since the clients execute sprite rules by themselves to minimize data, then an update sent telling your client that the client that was attacking is now FINISHED attacking which you did receive (When really the server and that client know that the attack defeated the enemy so the enemy is dead on their ends, but since you didnt recieve an update telling you that it was attacking, on your client, the enemy is still alive) would make your client just see the other player casually strolling off with a false sense of accomplishment, as the enemy was still alive in your game.

I know you both could probably say: "Well then send updates about enemies and NPC's- that way when the client receives the positional update, it'll get information saying that the enemy will be dead, right?"

While that would solve the problem, the game implements the item drop system I mentioned earlier, where a defeated enemy drops an item. If an enemy sprite has 0 HP, it creates an item sprite before Deactivating that can be picked up.

Say another player has already picked up the dropped item. (Item pickups are only recognized by the server. Only the item picked up by the client's sprite on the server is recognized to have the item) Your (Late)client recieves the update telling you that the enemy just died (And an item should be dropped)
But another player has already picked up that item, right?

This creates an entire mind-jar where the client thinks that it has an item, when on the server, and to other clients, it doesn't. I can't send the entire inventory data for each sprite (Their inventory data for each sprite is colossal!! It's one of the largest arrays in the game!)

One solution I have is to just do this:

Item Pickups are only recognized by the server, right? Well I'll keep it that way. Clients don't have the power to pick up items on their own then. The server sends the data to the client about the pickup when a sprite on the server picks up an item, telling it that Player<X> should get Item <ID>. THIS however, HAS to be reliably sent. The server sends all of the clients on that map the data about the pickup, and when it does it also sends the data to the client logged in with the sprite that picked up the item. When the data is received by that client, that data also holds a confirmation ("Coin collected") that tells it that it not only that it should have the item, but it also indicates that the item was successfully received. This way people aren't confused and they don't think they've picked up an item that they don't actually have.

I know this isn't how most MO's would do it, because they have item shops that people spend real money for and cant have players compromising data this way. But it works. I think this is an area that'll take care of itself though. Even if a player attempts to hack by giving themselves an item that they aren't supposed to have by changing the data the server tells them that they're supposed to have, things reset when they change maps or log back in. So I don't have much to worry about. Their game will be bugged and out of sync for attempting to hack, and it won't mess up anyone else. (It's their fault and their problem for trying to hack anyways, right?  ) So who cares about them. If it's even an extra healing item, their health will be updated by what I'm planning to do. (read below)



I've found something else that I'm also going to have to send Unreliably. HP, MP, and SP. Say by some error or complication, all the sprites have different values for a certain client's HP, MP, and SP. This is bad because it's game breaking. I'm going to have to incorporate it inside of positional updates. I can't send HP to the clients everytime a person gains or loses HP, because that would be terribly inefficient. (Say someone just walks into a pit of spikes?   )

So now I have to devise another system to send positional updates.

I can't send this as a variable length value, (Ex: 1 instead of 0001, because I know that each map will be 9999 by 9999 pixels of scrollable size.)

Here's my new way, that I'm hoping for:

I'll send this from the server to each client as a positional update.
00000000000000000000

Where each digit isn't binary, but, instead decimal, each 4 digits represents what needs to be updated. The first 4 are X the second 4 are Y, the third 4 is the new HP value, the next is the new MP value, and the next is the SP value. Since I know that HP, MP, SP, and X and Y all have a max value of 9999, I don't have to use variable length values. It ranges from 0000 to 9999. And these are the most important values of the sprite-

think of it as 0000-0000-0000-0000-0000

HP MP and SP are all values that the sprites rely on.

I'm going to try and build everything from this system of reliable and unreliable updates, so I don't have to tear apart my code later to fix things.

I could actually optimize that value above even more actually. I'll figure out how to later.

This is why I was so concerned on making positional updates reliable. (And by positional updates, I mean the one that occurs every 10 seconds) But if I use the system I'm using now- it should work with unreliable transmission.



as for using a texture instead of a solid color..



T'was the old title screen to my game. (I didn't like it for being an still image, so I replaced it with something more attention grabbing) I used carpet, a purple metallic bowling ball, and aluminum foil for the texture.

Strangely genius using a texture instead of a solid color for aliens. I love the idea. Metal would be a nice fit too. Maybe even vast, endless stars.

I thing I'll go with a combination of metal and stars for the texture.