Large number of frames causes Stack Overflow

[bluemonkmn]

Apparently when a project has too many frames, a stack overflow error can occur. I am considering rewriting the code generator for framesets so that they are not all hard-coded into a single function. My first inclination is to convert the parameters of all the frames in each frameset into a binary file and store that in an embedded resource (a .bin file in a "Framesets" folder in the generated project), which will be parsed when the frameset is constructed. But that means you will not be able to easily see the details of the framesets in the generated code just by looking at it. But it will be much more compact. Another alternative would be to store it in XML instead of binary format. What do you think?

I hope that constructing framesets this way uses heap memory instead of stack memory, but I'm a little fuzzy on what was going on the stack.

[Vincent]

I'm surprised!  I thought that LoK:Revival was the biggest project around, there's a lot of frames and a lot of framesets too in my project.  I don't remember getting a stack overflow error...  Just how many frames do you need in a frame set to get such an error?

[bluemonkmn]

I guess about 500 frames per frameset... and lots of framesets. Let's see if the user who found the problem will provide more detailed info.

[v6v]

My project had a total of 256 * 60 added to 320

Scratch that:

15,680 + 510 frames.

:/

To be honest, .bin sounds nice. I rarely will need to modify framesets outside of SGDK2 in a text or XML editor. That may be different for other users though. The main question is, I'm still far from being a computer programmer, everything should still be in a binary file, right (With the exception of libraries such as OpenTK.dll and Lidgren.Network.dll)

[Vincent]

Yikes!

[v6v]

Yikes!

How much more was that from LoK? When I was submitting the issue to bluemonkmn last night I had doubts that the issue was from the stack, because I figured your project probably had twice the frames that I had.

[Vincent]

Good question.  I don't have the project with me, so I can't be sure.  I suppose I have more than 1,000 frames but certainly less than 10,000.  I'll check the exact number weekend. 

[bluemonkmn]

To be honest, .bin sounds nice. I rarely will need to modify framesets outside of SGDK2 in a text or XML editor. That may be different for other users though. The main question is, I'm still far from being a computer programmer, everything should still be in a binary file, right (With the exception of libraries such as OpenTK.dll and Lidgren.Network.dll)

When compiled the frameset info will be embedded in the EXE. But the source code of the project generated by SGDK2 would represent the data in binary files instead of as source code. The format of the SGDK2 file itself would not change; everything including Framesets would still be XML in that file. Does that answer your question?

[Vincent]

LoK Revival has:
5378 frames (I counted manually so I may be a little off)
226 framesets
the biggest frameset has 293 frames

Wow, your project is huge: 3x more frames than mine... 

[v6v]

Even mine was an approximation. Where I said 256, I should have said 360.

It's mainly just the fact that each limb, body part, shirt, hair, hat, etc, has 360 frames because of 360 degrees of motion.

I'm concerned. The EXE is looking to be around 170MB at the moment and I only have enough for a demo- I want people to download this online. From the looks of things I'll probably have a //REQUEST GAME CD// link, and mail people the CD's.


Even with compression it's still going to be ridiculously large

(Well actually most games of my game's genre that play like my game max up to 2GB)

What do you recommend, having the users download a much smaller ParadigmInstaller file, that installs the EXE, can be paused, and resumed?

[bluemonkmn]

170 MB isn't too big to simply post as a file for download. And usually level design is not the largest piece of a game, and I assume that's most of what remains in your project. But it does seem excessive for a scrolling game created by one person. I find it hard to believe that one person could have created that much content without having some unnecessary overhead. So here are things to check:
1. Zip the EXE of course. That should cut down its size significantly.
2. Eliminate unused graphics.
3. Use more efficient mechanisms for storing sound (I guess I can imagine 170 MB of sounds accumulating from one person easily). Music could be MOD or MIDI, which is much smaller than MP3 or OGG. And use lower bitrate compression on other sounds.
4. Use 2 bytes per tile instead of 4 bytes per tile. 4 bytes per tile is only necessary if you have more than 65,536 frames/tiles.
5. If you have many layers in addition to the background and player's layer on your maps, and there are large unused portions of those layers, consider breaking up the layer into smaller layers that are offset so they appear where they should and only cover areas where you have tiles on those layers.

The last two probably don't affect EXE size much, but would affect memory usage at runtime.

Hopefully the new frameset mechanism will improve things a bit, but I suspect the actual code part of the project is still under 5 MB, and not a large portion of your project's size.

[v6v]

170 MB isn't too big to simply post as a file for download. And usually level design is not the largest piece of a game, and I assume that's most of what remains in your project. But it does seem excessive for a scrolling game created by one person. I find it hard to believe that one person could have created that much content without having some unnecessary overhead. So here are things to check:
1. Zip the EXE of course. That should cut down its size significantly.
2. Eliminate unused graphics.
3. Use more efficient mechanisms for storing sound (I guess I can imagine 170 MB of sounds accumulating from one person easily). Music could be MOD or MIDI, which is much smaller than MP3 or OGG. And use lower bitrate compression on other sounds.
4. Use 2 bytes per tile instead of 4 bytes per tile. 4 bytes per tile is only necessary if you have more than 65,536 frames/tiles.
5. If you have many layers in addition to the background and player's layer on your maps, and there are large unused portions of those layers, consider breaking up the layer into smaller layers that are offset so they appear where they should and only cover areas where you have tiles on those layers.

The last two probably don't affect EXE size much, but would affect memory usage at runtime.

Hopefully the new frameset mechanism will improve things a bit, but I suspect the actual code part of the project is still under 5 MB, and not a large portion of your project's size.

There was the thing that my first game ever made was around 256 MB.

Almost 7 years later, I still don't understand why that was. Shame I can't find it.

I keep everything in OGG, which is about 1MB each for music, that's good, right?

[bluemonkmn]

I guess 1 MB is good for a music file, but MOD or MIDI could be much smaller. I guess you want to focus on what is taking the largest percentage of your whole project size.

[v6v]

Graphics. Not graphic sheets mainly, mainly Skysphere textures. 2048 by 2048 is the preferred size, because anything less has horrible scaling.

PNG seems to be a small enough format as it is, it's a shame I can't have a button called "Download Graphics" which downloads a separate resources directory from the usable by my game for graphics. I wouldn't want the images to be editable though, because of collision rectangles and such, but something that held the images but was safeguarded from user editing. It would chop it down by nearly half its amount.

Code doesn't seem to take up much at all, which is strange. I have more code objects than anything else..

[bluemonkmn]

Well, it's true what they say; a picture is worth 1000 words (at least). Images do take a lot more space than text.
I have a new version of the SGDK2 IDE for you to test:
http://sgdk2.enigmadream.com/ben/SGDK2IDE.7z
It's just the EXE file, so don't delete what you have. Just rename it and drop this one in.

Info & how to use:
With the new version, if you have a file called "Frameset.cs" in your SourceCode folder, SGDK2 will not generate Frameset.cs in the old format. Instead it will generate a Framesets directory filled with bin files representing frameset details, which will get embedded in the EXE when compiled. Of course you don't need to create Frameset.cs yourself. If you create a new sample project, for example, you will notice that the SourceCode folder now contains Frameset.cs whereas it did not before. If you delete the file, it will generate the project in the old format. If you keep it, the new mechanism will be used. If you don't have Frameset.cs in your project, there are two ways to get it:
1. Reset source code
2. Create a new project, which will include the file from the template. Then copy the content of Frameset.cs into a file named Frameset.cs in your own project.

Good luck! If it works, I will release 2.2.6 with this feature as-is.