Author Topic: Master Server  (Read 7827 times)

v6v

  • Clever
  • Fanatic
  • ***
  • Posts: 500
  • Has renamed his project to Galaxy!
    • View Profile
    • My Developer Page!
    • Email
Master Server
« on: 2012-03-05, 05:25:18 PM »
So, on the development of my game, I've been working on the database as much as I've been working on the client. I've been taking time to work on each separately.

I've concluded that, since I cannot run my own computer 24/7, that I have a Globalized Server that lists all of the Master Servers running at any given time.

A MasterServer, in essence, is a Server that tells the Client who is hosting, their Address, and other information about the server.

The users host their own master servers, and it reduces (ALOT) of the work having to be done and bandwidth by my rails application on my slow web host.

Debating with myself, I chose RoR3.1 for development (It was remarkably easy, almost like programming a TI-84, I picked it up in less that a day)
Finally, my web host will run my rails app. Due to Subdomain and Rails version errors, there was a 2 month delay that I couldn't run it from my host.

I need help. The master servers are C# binaries. The users can download a binary, run it, and then request a server for themselves on my List Server. In my rails environment, I check from a list of Registered IP addresses could be potentially running a Master Server, query each address, and then return the result to the browser (I'm planning to implement it in the game.)

But again, I need to find a way to run a MasterServer, because I can't keep mine up. Is there any way to reliably run C# applications on a remote computer, for at least 24/7? (For free, also, I can't pay for things over the internet due to the "Lack of Paypal and Credit Cards" and the fact that any money I get it backdrip from my mother and she's having to deal with alot at the current moment..)

So again, is there anyone who knows of a website, or even, who can themselves run a server binary for my server? Again, it's just a binary that tells a client what servers are running..

http://ms.paradigm.heliohost.org/ <- ListServ

Although the game is still in the beta stage, I'll need to test these things. If you choose to run it yourself, it won't be until next week or so until I need it to be hosted.
« Last Edit: 2012-03-05, 05:34:36 PM by #Sharp »

durnurd

  • Lead Lemming
  • Expert
  • Fanatic
  • *****
  • Posts: 1234
  • Games completed so far: 0
    • MSN Messenger - durnurd@hotmail.com
    • View Profile
    • Find My Ed
Re: Master Server
« Reply #1 on: 2012-03-05, 10:42:08 PM »
If it is indeed just a service that reports a list of servers (presumably ip addresses, and perhaps some metadata) is there a reason that must be a C# executable?  That sounds like something that could be handled by a PHP script running on a web server.
Edward Dassmesser

v6v

  • Clever
  • Fanatic
  • ***
  • Posts: 500
  • Has renamed his project to Galaxy!
    • View Profile
    • My Developer Page!
    • Email
Re: Master Server
« Reply #2 on: 2012-03-05, 11:21:58 PM »
I considered that, but I wasn't sure if PHP had TCPClient functionality, (I've never dove that far into it!:P)

The other thing is though, the server itself isn't reliable (Due to a disclaimer on the websie)
I didn't want the webhost to be completely responsible for Netgames- (It's down during the "peak" hours of internet surfing on certain days)
It's (The Webhost) also extremely slow.. and with multiple users trying to access it, I was afraid that it might bog down the rest of the site.

I chose the unstable virtual server because of the software I could use (The unstable server has more server development software and environments, ironic, right?) but I'm actually considering going with the more reliable server with PHP support being it's other best aspect.
After looking into PHP more, after reading this post, I think it's safe to say that I can use a TCPClient with it, I may end up scrapping this scrapheap of directories and erb files for PHP files.

I think that the more reliable server on my host can handle the PHP. I have to be careful though, as some websites (VistaPanel.Net for instance) Terminate accounts due to high traffic with PHP files. I've had bad experiences where my entire site was commandeered and replaced with "MyBookFace.com" (Or some other generic benefactor website) and my files unreachable with my FTP account disabled.
That may actually be my worst fear, and why I never considered PHP files having to be loaded in the first place.
With RoR, I could limit exactly when the server application allowed access, or when to take a breather and cool down for a moment.

Back to Notepad, I suppose..

bluemonkmn

  • SGDK Author
  • Administrator
  • Fanatic
  • *****
  • Posts: 2761
    • ICQ Messenger - 2678251
    • MSN Messenger - BlueMonkMN@gmail.com
    • View Profile
    • http://sgdk2.sf.net/
    • Email
Re: Master Server
« Reply #3 on: 2012-03-06, 06:19:38 AM »
I don't have time to put a lot of thought into this a the moment, but if you have a relatively simply PHP page, I would consider hosting it as a sub-domain of my enigmadream.com domain.  Dreamhost just reboots the server if the load is too heavy.

Edit: Now that I have a moment to think, I would add, for something as simple as maintaining a list of server names, I would think that it would be quite difficult to put such a significant load on it (if it's well written) that the average host could not handle it.  Some design factors that could ensure high performance:
1. The page that lists the servers should be able to be a simple HTML or XML file which has no server side logic except to deliver its content.
2. The HTML or XML file could be generated by a PHP (or other) server side script that verifies, on average, no more than 1 server in the list every X (5?) seconds.  Maybe (due to limitations on fine-grained server-side scheduling) that just means it verifies 12 servers from the list every minute.  This helps to ensure that the load is never unreasonable, even if it means that a highly loaded system will take quite a while to get around to validating your server.  If possible, you may want to make sure to start the timer after the previous verification has completed (and have a timer that doesn't consume CPU).  I'm not sure whether cron would be appropriate for this or if there is a better way.  But if you use a scheduler like cron, the alternative would be to somehow disable the process while it is already/still running.
3. When someone wants to add their server to the list, it is simply enqueued into the list of servers that need verification.  Very little server side logic except to verify that it's not already in the list, and perhaps that some hard-limits haven't been hit (maximum number of servers to list; maximum number of submissions from one IP address within the past hour)
4. The verification process, actually, should be separate from the HTML/XML generation process.  The HTML generation process could run once per minute (and only if the verification process has indicated that there are changes).  The verification process would:
   a. Mark un-validated submissions as validated as they are validated
   b. Possibly re-check already validated entries in the list if there are no un-validated items in the list
   c. Remove invalid entries (those that failed validation) from the list
   d. Limit the number of entries it is checking to no more than an average of 12 per minute.
Then the generator just needs to output all the validated servers in the list.

Seems like that could limit reasonably well the potential load on the globalized server.
« Last Edit: 2012-03-06, 10:15:49 AM by bluemonkmn »

v6v

  • Clever
  • Fanatic
  • ***
  • Posts: 500
  • Has renamed his project to Galaxy!
    • View Profile
    • My Developer Page!
    • Email
Re: Master Server
« Reply #4 on: 2012-03-06, 03:29:01 PM »
That was my main issue, Heliohost only allows 2 Cron jobs a day, so requesting a server's alive status from the server would be impossible in PHP without cron.


(Coming back to me now)
I was looking into that, which was another reason I chose RoR- I think it would be reasonable- since the rest of the server won't do much else but simply show information about the game, and I would need a backend that could continuously run- if I ever chose to run a single MS on my host (Anything else, like MediaWiki, I would run from a different host.) But in the end-  I made the masterservers Binaries- so that the players would deal with the load, and all rails had to do was direct a new user to the servers by 'pinging' them on request.

http://www.igvita.com/2007/03/29/scheduling-tasks-in-ruby-rails/

My original plans(Before I turned my host server from a MasterServer to a ListServer) were to just remove a server from the MS after 3 minutes, if the server (The one hosted directly by the game) didn't send a certain request to the Listserv after 2 minutes and 30 seconds.
It seemed reasonable, to have that on a seperate thread.

WebClient.DownloadString inside of the game would download a string from a HTML file with the lists, and process that, when a client wants to see what hosted servers are alive.

It seems similar to what you're mentioning. 2 processes- one that verifies, and another that displays the servers.

Looking back- I think I can eliminate the C# binaries and have the whole thing in RoR. If you say that it shouldn't be too much- I'll go for it (Although I was skeptical when I began development,)

bluemonkmn

  • SGDK Author
  • Administrator
  • Fanatic
  • *****
  • Posts: 2761
    • ICQ Messenger - 2678251
    • MSN Messenger - BlueMonkMN@gmail.com
    • View Profile
    • http://sgdk2.sf.net/
    • Email
Re: Master Server
« Reply #5 on: 2012-03-06, 03:57:11 PM »
I wonder if it would make sense to have another level of server... or a non-hard-coded "global server".  Is the "global server" address configurable?  I'm just thinking ahead to the possibility that the global server might have to change host & address.  Maybe there should be a static list of level 1 servers (formerly "global servers") on some level 0 server (like the place where one downloads your software from).  This is a list of servers to probe when looking for the server that will tell you what other servers are available.  There would be no need for this list (possibly of just 1 entry) to change frequently.  And it might never be pinged automatically.  Just something users can refer to if their software isn't working because the global server moved.

v6v

  • Clever
  • Fanatic
  • ***
  • Posts: 500
  • Has renamed his project to Galaxy!
    • View Profile
    • My Developer Page!
    • Email
Re: Master Server
« Reply #6 on: 2012-03-07, 12:49:58 AM »
I wonder if it would make sense to have another level of server... or a non-hard-coded "global server".  Is the "global server" address configurable?  I'm just thinking ahead to the possibility that the global server might have to change host & address.  Maybe there should be a static list of level 1 servers (formerly "global servers") on some level 0 server (like the place where one downloads your software from).  This is a list of servers to probe when looking for the server that will tell you what other servers are available.  There would be no need for this list (possibly of just 1 entry) to change frequently.  And it might never be pinged automatically.  Just something users can refer to if their software isn't working because the global server moved.

I couldn't quite follow you, but I think you're mentioning the possibility that the ListServ might have to change addresses? The MasterServer sits on my hard drive as a tar.gz file and can be easily transfered from any host to any host. (Given the presence of the RoR3.1 framework)

If you're mentioning the ability to change the global server address to what the game itself looks to, I've accounted for that. I'm working on an ingame console, that can show me what's going wrong inside of my game, rather than using a "LogDebugLabel" or my own custom method, "DrawString", to the display. It would allow the user to type "/masterserver <address> <portnum>"to set the globalized server.
It seems entirely easy, as the address can be fed into my custom code object that downloads the list from the server.

I think I see what you're saying. Instead of one "global" server, I could have multiple global servers.  That's the way it works, with (If I follow) the level 0 server being the #masterserver, and the Level 1 servers being (Ross, Yuri, Kirk, Sydney) These servers were created to redirect a user to the IP address of the C# Binary Server. (Let me diagram)

*=New Player +*
(Requests...)
Global #masterserver
(which Requests...)
Yuri... Teddy... Kirk... etc by pinging each address
Example: In my Database, Kirk has the IP address of -blah-
The #masterserver looks at this and pings that address on port 1493.
If there's the correct response, the server tells the New player that the server is alive.

(If one of those level 1 servers is alive, and player chooses it then...)
C# binary which would store the list of servers to an SQLLite database would send the list (As a formattable string) to the requesting player.

The C# binary would do most of the updating. I'm not sure if this is what you meant, but that was the original system.


Thanks for the offer above, I know you've rethought it, but it means a lot that you would have (..hastily :)) considered something that generous.

Most of this is modified server logic from another game I had frequently played- their problem however was, what would happen when the Level0 server was down? :D
« Last Edit: 2012-03-07, 12:56:15 AM by #Sharp »

durnurd

  • Lead Lemming
  • Expert
  • Fanatic
  • *****
  • Posts: 1234
  • Games completed so far: 0
    • MSN Messenger - durnurd@hotmail.com
    • View Profile
    • Find My Ed
Re: Master Server
« Reply #7 on: 2012-03-07, 07:48:33 AM »
Hopefully, the level 0 servers aren't needed unless the level 1 server's address cannot be verified (i.e. cannot be reached). For the most part, it wouldn't need to be queried at all unless something is going wrong down the pipeline. It's basically how DNS servers work.  DNS subdomain servers don't often look up the IP address of the .com authoritative server, because that doesn't change all that often.  They just use a locally cached value.  This whole thing sounds like DNS services, with the addition that you are providing a list of all entries as well.
Edward Dassmesser

v6v

  • Clever
  • Fanatic
  • ***
  • Posts: 500
  • Has renamed his project to Galaxy!
    • View Profile
    • My Developer Page!
    • Email
Re: Master Server
« Reply #8 on: 2012-03-07, 03:24:27 PM »
I follow- instead of having the game query the Level 0 server, I would have it query all of the Level 1 servers. But if none of the level 1 servers could be reached, then it would query the level 0 server to find out what's going wrong, and to potentially get new information (Like for instance- if the level 0 server's pre-mentioned database of IP addresses referred to completely new addresses for the game to query.)

That would take a large amount of the load off of my server! Instead of having the Level 0 server look for what's alive, it only stores the addresses to the MySQL database (Or XML file), and if all of said addresses were unreachable, then and only then would the game send a request to the #masterserver to find the new addresses.

I'm thinking, that I'll try to host one level 1 server myself during the hours I'm home.. but have the level 1 server combined into the same binary as a game server itself- so if someone hosts a level 1 server, they also host a level 2 server simultaneously (Which is the actual game server in Lidgren) on a different port and thread. However, since I know not everyone can do this, I'll allow those who choose to host a level 2 server directly within the game's interface (Through the Party Host option) to only host the level 2 server, but register it at the level 1 server.

The game has 2 gameplay options on it's start. "Global Play" and "Party Host". "Global Play" contacts the servers to see what's online and join-able (Displaying which Level 1 servers are alive, then allowing the user to browse each to find out what's alive), then joins a server. "Party Host" allows players to host their own game, whilst allowing players to join the game.

The locally cached addresses could save my server a large amount of useless and rather avoidable work. :D

This still raises one final issue (again), and I think I've overcome some issues but, where would I run the Level 1 servers? Are there any hosting services that would let me run a binary on a reliable server?
« Last Edit: 2012-03-07, 04:03:39 PM by #Sharp »

bluemonkmn

  • SGDK Author
  • Administrator
  • Fanatic
  • *****
  • Posts: 2761
    • ICQ Messenger - 2678251
    • MSN Messenger - BlueMonkMN@gmail.com
    • View Profile
    • http://sgdk2.sf.net/
    • Email
Re: Master Server
« Reply #9 on: 2012-03-07, 07:24:37 PM »
Why does does your level 1 server have to be an executable? It's just validating the responsiveness of level 2 servers and listing the results, right? Can't that be done with a simple php script?

v6v

  • Clever
  • Fanatic
  • ***
  • Posts: 500
  • Has renamed his project to Galaxy!
    • View Profile
    • My Developer Page!
    • Email
Re: Master Server
« Reply #10 on: 2012-03-07, 08:07:26 PM »
Right- I'll try it on my server as a PHP and hope for the best. I'll use RoR to wipe the Database contents every x amount of minutes. It shouldn't be too much, like you mentioned.

bluemonkmn

  • SGDK Author
  • Administrator
  • Fanatic
  • *****
  • Posts: 2761
    • ICQ Messenger - 2678251
    • MSN Messenger - BlueMonkMN@gmail.com
    • View Profile
    • http://sgdk2.sf.net/
    • Email
Re: Master Server
« Reply #11 on: 2012-03-08, 08:27:02 AM »
Right- I'll try it on my server as a PHP and hope for the best. I'll use RoR to wipe the Database contents every x amount of minutes. It shouldn't be too much, like you mentioned.

You should be able to achieve this without scheduling if you use requests and submissions as the trigger:
1. If the request is a submission and the previous validation cycle was less than X minute(s) ago, add the submission to the list as un-validated
2. If the request is a submission and the previous validation cycle was at least X minute(s) ago, validate this submission and up to N other un-validated submissions that have been submitted since that last validations; update the last validation cycle time; update the validation time of each entry that was validated.
3. If the request is a request for the list of validated servers, and the previous validation cycle was less than X minute(s) ago, return the cached list of currently validated servers
4. If the request is a request for the list of validated servers, and the previous validation cycle was at least X minute(s) ago, validate up to N un-validated submissions (if any exist) and then return the current list of validated servers; update the last validation cycle time; update the validation time of each entry that was validated.
5. After processing any request, if there are no un-validated entries remaining, mark the oldest N entries as un-validated.
« Last Edit: 2012-03-08, 08:31:58 AM by bluemonkmn »

v6v

  • Clever
  • Fanatic
  • ***
  • Posts: 500
  • Has renamed his project to Galaxy!
    • View Profile
    • My Developer Page!
    • Email
Re: Master Server
« Reply #12 on: 2012-03-08, 03:15:40 PM »
That should do it. Thanks! I guess I'll have to wait for a new user to change something. It should be easy.

v6v

  • Clever
  • Fanatic
  • ***
  • Posts: 500
  • Has renamed his project to Galaxy!
    • View Profile
    • My Developer Page!
    • Email
Re: Master Server
« Reply #13 on: 2012-03-11, 04:17:16 PM »
Hey I'm wondering- I want the game to send a message to the Server- but I don't want any malicious users to be able to forge requests. If I use some form of encrypted string- a user could infact use some form of Proxy to see what's being sent and duplicate it in the form of an attack. (Security is at the utmost importance here.)

Is there any possible way that I could verify that the game and only that, were sending requests? So far I'm considering using a Webclient object that posts to a hidden form in order to send to the PHP file..

bluemonkmn

  • SGDK Author
  • Administrator
  • Fanatic
  • *****
  • Posts: 2761
    • ICQ Messenger - 2678251
    • MSN Messenger - BlueMonkMN@gmail.com
    • View Profile
    • http://sgdk2.sf.net/
    • Email
Re: Master Server
« Reply #14 on: 2012-03-12, 05:39:33 AM »
I'm not sure there is a totally sure way to secure the client software.  If you are distributing compiled executables for the client, you are distributing "code" that knows how to make a proprietary request to the server.  And if you are distributing that, you are opening up the possibility that someone could make an altered copy of that.  The only sure way of securing your interface is to maintain all the critical data in the server and not in the client.  Then you don't have to worry about forged requests because the server knows what's real and not real.  For example, if you had your client software running on the server for each player, and made the real clients just be "dumb" displays of what the "clients" running on the server are doing, you would have total control.

That said, the next best thing I can think of is to use cryptography (included in the .NET framework - see System.Security.Cryptography), and allow each legitimate user to "sign up" for their own unique private/public key pair.  The client would generate a private key (or the server would generate one on the client's behalf, but this is less secure once you transmit it over the network) and submit its public key to the server.  Then if you can (manually?) verify that this client is legitimate once, you can use the public key to verify that every request from that client is "legitimate" (is actually from someone who holds that client's private key).  Only the holder of the private key will be able to generate requests that can be verified by the related public key.  *However* this still doesn't prevent someone from creating another version of the client that anyone could use with their private key instead of the legitimate client.  It just makes it more difficult because the forged client must have a legitimate private key.  But if legitimate users want to use the forged client with their private key, there's nothing stopping them unless you revoke their public key access when you detect forgery on your end.

I recommend you post a detailed question on http://stackoverflow.com/ to find the range of options available to you in securing your client/server interface (and/or more details about how to do what I described above, if you like that solution).  I'm not sure I have all the details for you, but StackOverflow is a great place to get programming questions answered.